[Bugs] [Bug 1649037] Translators allocate too much memory in their xlator_mem_acct_init()

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1649037



--- Comment #2 from Yaniv Kaul <ykaul at redhat.com> ---
(In reply to Vijay Bellur from comment #1)
> (In reply to Yaniv Kaul from comment #0)
> > Description of problem:
> > If I'm looking at most xlators, they use it with something like (random
> > example):
> > gf_mt_jbr_end
> > 
> > which is defined in xlators/experimental/jbr-server/src/jbr-internal.h as:
> > enum {
> >     gf_mt_jbr_private_t = gf_common_mt_end + 1,
> >     gf_mt_jbr_fd_ctx_t,
> >     gf_mt_jbr_inode_ctx_t,
> >     gf_mt_jbr_dirty_t,
> >     gf_mt_jbr_end
> > };
> > 
> > 
> > What is the value of gf_common_mt_end ? Some number, defined in
> > libglusterfs/src/mem-types.h as 150 or so (did not really count or looked,
> > but seen it's quite a large list)
> > 
> > So gf_mt_jbr_end ends up being 155 or so.
> > Then in xlator_mem_acct_init(), I see these:
> >    xl->mem_acct = MALLOC(sizeof(struct mem_acct) +
> >                           sizeof(struct mem_acct_rec) * num_types);
> > 
> > Which seems to me that we are allocating plenty of mem_acct_rec structs,
> > even if we only need 5 or so? We are clearly allocating and then memsetting
> > and probably not using way too many mem_acct_rec records.
> > 
> 
> All memory accounting happens per xlator. When a xlator invokes a
> libglusterfs function, any memory allocation happening there leverages a
> common memory type and the accounting happens in xl->mem_acct[common_mt].
> Hence it is not very easy to determine which memory type record would not be
> used and so the allocation in init() looks ok to me.

I think the real issue comes from the fact that common is not really common.
If we had real 10 or 100 common types, which was actually shared by many
xlators, then fine, I can understand that. But the reality is that there is an
abuse of the common types.
Completely random examples:
gf_common_mt_tw_ctx - used exactly once, in ctx.c
gf_mt_sql_cbk_args_t - not used AT ALL?
gf_common_mt_rdma_arena_mr - used exactly once, in rdma.c

So I think the real issue is the abuse. If we had a real concise list, then the
overuse wouldn't have been a big deal (of course, we are talking here on few
KBs or 10's of KBs, so it's not such a big deal anyway, but it just looks wrong
to me)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.