[Gluster-devel] Gluster internals

[Ian Latter]

> >     0) Is there any developer docco?  I've just done
> >         another quick search and I can't see any.  Let
> >         me know if there is and I'll try and answer the
> >         below myself.
> 
> Your best bet right now (if I may say so) is the stuff I've posted on
> hekafs.org - the "Translator 101" articles plus the API overview at
> 
> http://hekafs.org/dist/xlator_api_2.html

You must say so - there is so little docco.  Actually before I
posted I went and re-read your Translator 101 docs as you 
referred them to me on 10 May, but I hadn't found your API
overview - thanks (for both)!


> >     2) Is there a way to write linearly within a single
> >         function within Gluster (or is there a reason
> >         why I wouldn't want to do that)?  
> 
> Any blocking ops would have to be built on top of async ops plus semaphores
> etc. because (unlike e.g. an HTTP server) the underlying sockets etc. are
> shared/multiplexed between users and activities.  Thus you'd get much more
> context switching that way than if you stay within the async/continuation style.

Interesting - I haven't ever done semaphore coding, but it may not
be needed.  The syncop framework that Krish referred too seems to 
do this via a mutex lock (synctask_yawn) and a context switch (synctask_yield).  What's the drawback with increased context
switching?
  After my email thread with Krish I decided against syncop, but the
flow without was going to be horrific.  The only way I could bring it
back to anything even half as sane as the afr code (which can
cleverly loop through its own _cbk's recursively - I like that, whoever
put that together) was to have the last cbk in a chain (say the
"close_cbk") call the original function with an index or stepper 
increment.
  But after sitting on the idea for a couple of days I actually came
to the same conclusion as Manu did in the last message.  I.e.
without docco I have been writing to what seems to work, and
in my 2009 code (I saw last night) a "mkdir" wind followed by "create" 
code in the same function - which I believe, now, is probably a 
race condition (because of the threaded/async structure forced
through the wind/call macro model).
  In that case I *do* want a synchronous write - but only within
my xlator (which, if I'm reading this right, *is* what syncop does)
- as opposed to an end-to-end synchronous write (being sync'd
through the full stack of xlators: ignoring caching, waiting for 
replication to be validated, etc).
  Although, the same synchronous outcome comes from the 
chained async calls ... but then we get back to the readability/
fixability of the code.
 
> Some day in the distant future, I'd like to work some more on a preprocessor
> that turns linear code into async code so that it's easier to write but retains
> the performance and resource-efficiency advantages of an essentially async
> style.  I did some work (http://pl.atyp.us/ripper/UserGuide.html) in this area
> several years ago, but it has probably bit-rotted to hell since then.  With
> more recent versions of gcc and LLVM it should be possible to overcome some of
> the limitations that version had.

Yes, I had a very similar thought - a C pre-processor isn't in my 
experience or time scale though; I considered writing up a script that 
would chain it out in C for me.  I was going to borrow from a script that I 
wrote which builds one of the libMidnightCode header files but even 
that seemed impractical .. would anyone be able to debug it?  Would I
even understand in 2yrs from now - lol


So I think the long and the short of it is that anything I do here won't be
pretty .. or perhaps: one will look pretty and the other will run pretty :)





--
Ian Latter
Late night coder ..
http://midnightcode.org/