[Gluster-devel] Gluster internals
ian.latter at midnightcode.org
Mon May 21 23:02:44 UTC 2012
> > 0) Is there any developer docco? I've just done
> > another quick search and I can't see any. Let
> > me know if there is and I'll try and answer the
> > below myself.
> Your best bet right now (if I may say so) is the stuff I've posted on
> hekafs.org - the "Translator 101" articles plus the API overview at
You must say so - there is so little docco. Actually before I
posted I went and re-read your Translator 101 docs as you
referred them to me on 10 May, but I hadn't found your API
overview - thanks (for both)!
> > 2) Is there a way to write linearly within a single
> > function within Gluster (or is there a reason
> > why I wouldn't want to do that)?
> Any blocking ops would have to be built on top of async ops plus semaphores
> etc. because (unlike e.g. an HTTP server) the underlying sockets etc. are
> shared/multiplexed between users and activities. Thus you'd get much more
> context switching that way than if you stay within the async/continuation style.
Interesting - I haven't ever done semaphore coding, but it may not
be needed. The syncop framework that Krish referred too seems to
do this via a mutex lock (synctask_yawn) and a context switch (synctask_yield). What's the drawback with increased context
After my email thread with Krish I decided against syncop, but the
flow without was going to be horrific. The only way I could bring it
back to anything even half as sane as the afr code (which can
cleverly loop through its own _cbk's recursively - I like that, whoever
put that together) was to have the last cbk in a chain (say the
"close_cbk") call the original function with an index or stepper
But after sitting on the idea for a couple of days I actually came
to the same conclusion as Manu did in the last message. I.e.
without docco I have been writing to what seems to work, and
in my 2009 code (I saw last night) a "mkdir" wind followed by "create"
code in the same function - which I believe, now, is probably a
race condition (because of the threaded/async structure forced
through the wind/call macro model).
In that case I *do* want a synchronous write - but only within
my xlator (which, if I'm reading this right, *is* what syncop does)
- as opposed to an end-to-end synchronous write (being sync'd
through the full stack of xlators: ignoring caching, waiting for
replication to be validated, etc).
Although, the same synchronous outcome comes from the
chained async calls ... but then we get back to the readability/
fixability of the code.
> Some day in the distant future, I'd like to work some more on a preprocessor
> that turns linear code into async code so that it's easier to write but retains
> the performance and resource-efficiency advantages of an essentially async
> style. I did some work (http://pl.atyp.us/ripper/UserGuide.html) in this area
> several years ago, but it has probably bit-rotted to hell since then. With
> more recent versions of gcc and LLVM it should be possible to overcome some of
> the limitations that version had.
Yes, I had a very similar thought - a C pre-processor isn't in my
experience or time scale though; I considered writing up a script that
would chain it out in C for me. I was going to borrow from a script that I
wrote which builds one of the libMidnightCode header files but even
that seemed impractical .. would anyone be able to debug it? Would I
even understand in 2yrs from now - lol
So I think the long and the short of it is that anything I do here won't be
pretty .. or perhaps: one will look pretty and the other will run pretty :)
Late night coder ..
More information about the Gluster-devel