[Gluster-devel] Adding xxhash to gluster code base

[Aravinda]

regards
Aravinda VK

On 06/27/2017 01:38 PM, Niels de Vos wrote:
> On Tue, Jun 27, 2017 at 12:25:11PM +0530, Kotresh Hiremath Ravishankar wrote:
>> Hi,
>>
>> We were looking for faster non-cryptographic hash to be used for the
>> gfid2path infra [1]
>> The initial testing was done with md5 128bit checksum which was a slow,
>> cryptographic hash
>> and using it makes software not complaint to FIPS [2]
>>
>> On searching online a bit we found out xxhash [3] seems to be faster from
>> the results of
>> benchmark tests shared and lot of projects use it. So we have decided to us
>> xxHash
>> and added following files to gluster code base with the patch [4]
>>
>>      BSD 2-Clause License:
>>         contrib/xxhash/xxhash.c
>>         contrib/xxhash/xxhash.h
>>
>>      GPL v2 License:
>>         tests/utils/xxhsum.c
>>
>> NOTE: We have ignored the code guideline check for these files as
>> maintaining it
>> further becomes difficult.
>>
>> Please comment on the same if there are any issues around it.
> How performance critical is the hashing for gfid2path?
For this we need non-crypto hashing. This hash calculation will be in 
the I/O path(Affected FOPs are create, mknod, link, symlink, rename, 
unlink)
>
> What is the plan to keep these files maintained? At minimal we need to
> add these files to MAINTAINERS and the maintainers need to cherry-pick
> updates and bugfixes from the original project. The few patches a year
> makes this a recurring task that should not be forgoten. It would be
> much better to use this as an external library that is provided by the
> distributions. We already rely on OpenSSL, does this library not provide
> an alternative 'FIPS approved' hashing that performs reasonably well?
I think, this library is not available in distribution packaging yet. 
Please suggest if you know any non-crypto hashing lib which is already 
available in distros.
>
> Some distributions are very strict on bundling external projects, and we
> need to inform the packagers about the additions so that they can handle
> it correctly. Adding an external project to contrib/ should be mentioned
> in the release notes at the very least.
>
> Note that none of the symbols of any public functions in Gluster may
> collide with functions in standard distribution libraries. This causes
> for regular problems with gfapi applications. All exposed symbols that
> get imported in contrib/ should have a gf_ prefix.
>
> Thanks,
> Niels
>
>
>> [1] Issue: https://github.com/gluster/glusterfs/issues/139
>> [2] https://en.wikipedia.org/wiki/Federal_Information_Processing_Standards
>> [3] http://cyan4973.github.io/xxHash/
>> [4] https://review.gluster.org/#/c/17488/10
>>
>>
>>
>> -- 
>> Thanks and Regards,
>> Kotresh H R and Aravinda VK
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20170627/ba8fde9f/attachment-0001.html>