[Gluster-users] GlusterFS and Kafka
Christopher Schmidt
fakod666 at gmail.com
Mon May 29 13:24:16 UTC 2017
I was stupid enough to copy an additional newline from email. So, sorry for
the noise.
Works so far.
Thanks for getting that solved, best Chris
Raghavendra Talur <rtalur at redhat.com> schrieb am Mo., 29. Mai 2017 um
13:18 Uhr:
>
>
> On 29-May-2017 3:49 PM, "Christopher Schmidt" <fakod666 at gmail.com> wrote:
>
> Hi Raghavendra Talur,
>
> this does not work for me. Most certainly because I forgot something.
> So just put the file in the folder, make it executable and create a
> volume? Thats all?
>
> If I am doing this, there is no /var/lib/glusterd/hooks/1/create/post/log
> file and the Performance Translator is still on.
>
> and idea?
>
>
> Most certainly selinux. I think you will have to set context on the file
> to same as others in hooks dir.
>
> You can test temporarily by
> setenforce 0
>
>
> Raghavendra Talur <rtalur at redhat.com> schrieb am Fr., 26. Mai 2017 um
> 07:34 Uhr:
>
>> On Thu, May 25, 2017 at 8:39 PM, Joe Julian <joe at julianfamily.org> wrote:
>> > Maybe hooks?
>>
>> Yes, we were thinking of the same :)
>>
>> Christopher,
>> Gluster has hook-scripts facility that admins can write and set those
>> to be run on certain events in Gluster. We have a event for volume
>> creation.
>> Here are the steps for using hook scripts.
>>
>> 1. deploy the gluster pods and create a cluster as you have already done.
>> 2. on the kubernetes nodes that are running gluster pods(make sure
>> they are running now, because we want to write into the bind mount),
>> create a new file in location /var/lib/glusterd/hooks/1/create/post/
>> 3. name of the fie could be S29disable-perf.sh , important part being
>> that the number should have capital S as first letter.
>> 4. I tried out a sample script with content as below
>>
>> ```
>> #!/bin/bash
>>
>>
>>
>> PROGNAME="Sdisable-perf"
>> OPTSPEC="volname:,gd-workdir:"
>> VOL=
>> CONFIGFILE=
>> LOGFILEBASE=
>> PIDDIR=
>> GLUSTERD_WORKDIR=
>>
>> function parse_args () {
>> ARGS=$(getopt -l $OPTSPEC -name $PROGNAME $@)
>> eval set -- "$ARGS"
>>
>> while true; do
>> case $1 in
>> --volname)
>> shift
>> VOL=$1
>> ;;
>> --gd-workdir)
>> shift
>> GLUSTERD_WORKDIR=$1
>> ;;
>> *)
>> shift
>> break
>> ;;
>> esac
>> shift
>> done
>> }
>>
>> function disable_perf_xlators () {
>> volname=$1
>> gluster volume set $volname performance.write-behind off
>> echo "executed and return is $?" >>
>> /var/lib/glusterd/hooks/1/create/post/log
>> }
>>
>> echo "starting" >> /var/lib/glusterd/hooks/1/create/post/log
>> parse_args $@
>> disable_perf_xlators $VOL
>> ```
>> 5. set execute permissions on the file
>>
>> I tried this out and it worked for me. Let us know if that helps!
>>
>> Thanks,
>> Raghavendra Talur
>>
>>
>>
>> >
>> >
>> > On May 25, 2017 6:48:04 AM PDT, Christopher Schmidt <fakod666 at gmail.com
>> >
>> > wrote:
>> >>
>> >> Hi Humble,
>> >>
>> >> thanks for that, it is really appreciated.
>> >>
>> >> In the meanwhile, using K8s 1.5, what can I do to disable the
>> performance
>> >> translator that doesn't work with Kafka? Maybe something while
>> generating
>> >> the Glusterfs container for Kubernetes?
>> >>
>> >> Best Christopher
>> >>
>> >> Humble Chirammal <hchiramm at redhat.com> schrieb am Do., 25. Mai 2017,
>> >> 09:36:
>> >>>
>> >>> On Thu, May 25, 2017 at 12:57 PM, Raghavendra Talur <
>> rtalur at redhat.com>
>> >>> wrote:
>> >>>>
>> >>>> On Thu, May 25, 2017 at 11:21 AM, Christopher Schmidt
>> >>>> <fakod666 at gmail.com> wrote:
>> >>>> > So this change of the Gluster Volume Plugin will make it into K8s
>> 1.7
>> >>>> > or
>> >>>> > 1.8. Unfortunately too late for me.
>> >>>> >
>> >>>> > Does anyone know how to disable performance translators by default?
>> >>>>
>> >>>> Humble,
>> >>>>
>> >>>> Do you know of any way Christopher can proceed here?
>> >>>
>> >>>
>> >>> I am trying to get it in 1.7 branch, will provide an update here as
>> soon
>> >>> as its available.
>> >>>>
>> >>>>
>> >>>> >
>> >>>> >
>> >>>> > Raghavendra Talur <rtalur at redhat.com> schrieb am Mi., 24. Mai
>> 2017,
>> >>>> > 19:30:
>> >>>> >>
>> >>>> >> On Wed, May 24, 2017 at 4:08 PM, Christopher Schmidt
>> >>>> >> <fakod666 at gmail.com>
>> >>>> >> wrote:
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > Vijay Bellur <vbellur at redhat.com> schrieb am Mi., 24. Mai 2017
>> um
>> >>>> >> > 05:53
>> >>>> >> > Uhr:
>> >>>> >> >>
>> >>>> >> >> On Tue, May 23, 2017 at 1:39 AM, Christopher Schmidt
>> >>>> >> >> <fakod666 at gmail.com>
>> >>>> >> >> wrote:
>> >>>> >> >>>
>> >>>> >> >>> OK, seems that this works now.
>> >>>> >> >>>
>> >>>> >> >>> A couple of questions:
>> >>>> >> >>> - What do you think, are all these options necessary for
>> Kafka?
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >> I am not entirely certain what subset of options will make it
>> work
>> >>>> >> >> as I
>> >>>> >> >> do
>> >>>> >> >> not understand the nature of failure with Kafka and the
>> default
>> >>>> >> >> gluster
>> >>>> >> >> configuration. It certainly needs further analysis to identify
>> the
>> >>>> >> >> list
>> >>>> >> >> of
>> >>>> >> >> options necessary. Would it be possible for you to enable one
>> >>>> >> >> option
>> >>>> >> >> after
>> >>>> >> >> the other and determine the configuration that ?
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >>>
>> >>>> >> >>> - You wrote that there have to be kind of application
>> profiles.
>> >>>> >> >>> So to
>> >>>> >> >>> find out, which set of options work is currently a matter of
>> >>>> >> >>> testing
>> >>>> >> >>> (and
>> >>>> >> >>> hope)? Or are there any experiences for MongoDB / ProstgreSQL
>> /
>> >>>> >> >>> Zookeeper
>> >>>> >> >>> etc.?
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >> Application profiles are work in progress. We have a few that
>> are
>> >>>> >> >> focused
>> >>>> >> >> on use cases like VM storage, block storage etc. at the moment.
>> >>>> >> >>
>> >>>> >> >>>
>> >>>> >> >>> - I am using Heketi and Dynamik Storage Provisioning together
>> >>>> >> >>> with
>> >>>> >> >>> Kubernetes. Can I set this volume options somehow by default
>> or
>> >>>> >> >>> by
>> >>>> >> >>> volume
>> >>>> >> >>> plugin?
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >> Adding Raghavendra and Michael to help address this query.
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > For me it would be sufficient to disable some (or all)
>> translators,
>> >>>> >> > for
>> >>>> >> > all
>> >>>> >> > volumes that'll be created, somewhere here:
>> >>>> >> >
>> https://github.com/gluster/gluster-containers/tree/master/CentOS
>> >>>> >> > This is the container used by the GlusterFS DaemonSet for
>> >>>> >> > Kubernetes.
>> >>>> >>
>> >>>> >> Work is in progress to give such option at volume plugin level. We
>> >>>> >> currently have a patch[1] in review for Heketi that allows users
>> to
>> >>>> >> set Gluster options using heketi-cli instead of going into a
>> Gluster
>> >>>> >> pod. Once this is in, we can add options in storage-class of
>> >>>> >> Kubernetes that pass down Gluster options for every volume
>> created in
>> >>>> >> that storage-class.
>> >>>> >>
>> >>>> >> [1] https://github.com/heketi/heketi/pull/751
>> >>>> >>
>> >>>> >> Thanks,
>> >>>> >> Raghavendra Talur
>> >>>> >>
>> >>>> >> >
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >> -Vijay
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >>
>> >>>> >> >>>
>> >>>> >> >>>
>> >>>> >> >>> Thanks for you help... really appreciated.. Christopher
>> >>>> >> >>>
>> >>>> >> >>> Vijay Bellur <vbellur at redhat.com> schrieb am Mo., 22. Mai
>> 2017 um
>> >>>> >> >>> 16:41
>> >>>> >> >>> Uhr:
>> >>>> >> >>>>
>> >>>> >> >>>> Looks like a problem with caching. Can you please try by
>> >>>> >> >>>> disabling
>> >>>> >> >>>> all
>> >>>> >> >>>> performance translators? The following configuration commands
>> >>>> >> >>>> would
>> >>>> >> >>>> disable
>> >>>> >> >>>> performance translators in the gluster client stack:
>> >>>> >> >>>>
>> >>>> >> >>>> gluster volume set <volname> performance.quick-read off
>> >>>> >> >>>> gluster volume set <volname> performance.io-cache off
>> >>>> >> >>>> gluster volume set <volname> performance.write-behind off
>> >>>> >> >>>> gluster volume set <volname> performance.stat-prefetch off
>> >>>> >> >>>> gluster volume set <volname> performance.read-ahead off
>> >>>> >> >>>> gluster volume set <volname> performance.readdir-ahead off
>> >>>> >> >>>> gluster volume set <volname> performance.open-behind off
>> >>>> >> >>>> gluster volume set <volname> performance.client-io-threads
>> off
>> >>>> >> >>>>
>> >>>> >> >>>> Thanks,
>> >>>> >> >>>> Vijay
>> >>>> >> >>>>
>> >>>> >> >>>>
>> >>>> >> >>>>
>> >>>> >> >>>> On Mon, May 22, 2017 at 9:46 AM, Christopher Schmidt
>> >>>> >> >>>> <fakod666 at gmail.com> wrote:
>> >>>> >> >>>>>
>> >>>> >> >>>>> Hi all,
>> >>>> >> >>>>>
>> >>>> >> >>>>> has anyone ever successfully deployed a Kafka (Cluster) on
>> >>>> >> >>>>> GlusterFS
>> >>>> >> >>>>> volumes?
>> >>>> >> >>>>>
>> >>>> >> >>>>> I my case it's a Kafka Kubernetes-StatefulSet and a Heketi
>> >>>> >> >>>>> GlusterFS.
>> >>>> >> >>>>> Needless to say that I am getting a lot of filesystem
>> related
>> >>>> >> >>>>> exceptions like this one:
>> >>>> >> >>>>>
>> >>>> >> >>>>> Failed to read `log header` from file channel
>> >>>> >> >>>>> `sun.nio.ch.FileChannelImpl at 67afa54a`. Expected to read 12
>> >>>> >> >>>>> bytes,
>> >>>> >> >>>>> but
>> >>>> >> >>>>> reached end of file after reading 0 bytes. Started read from
>> >>>> >> >>>>> position
>> >>>> >> >>>>> 123065680.
>> >>>> >> >>>>>
>> >>>> >> >>>>> I limited the amount of exceptions with the
>> >>>> >> >>>>> log.flush.interval.messages=1 option, but not all...
>> >>>> >> >>>>>
>> >>>> >> >>>>> best Christopher
>> >>>> >> >>>>>
>> >>>> >> >>>>>
>> >>>> >> >>>>> _______________________________________________
>> >>>> >> >>>>> Gluster-users mailing list
>> >>>> >> >>>>> Gluster-users at gluster.org
>> >>>> >> >>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>> >>>> >> >>>>
>> >>>> >> >>>>
>> >>>> >> >
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Cheers,
>> >>> Humble
>> >>>
>> >>> Sr.Software Engineer - Red Hat Storage Engineering
>> >>> website: http://humblec.com
>> >
>> >
>> > --
>> > Sent from my Android device with K-9 Mail. Please excuse my brevity.
>> >
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users at gluster.org
>> > http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170529/fbff0687/attachment.html>
More information about the Gluster-users
mailing list