[Gluster-devel] Reducing the size of the glusterdocs git repository

Tue May 17 12:44:23 UTC 2016

On Tue, May 17, 2016 at 6:02 PM, Nigel Babu <nigelb at redhat.com> wrote:

> We could potentially setup travis-ci to do builds that'll fail loudly if
> we commit something that throws a warning. I've tried out the possibility
> here:
>
> https://travis-ci.org/nigelbabu/glusterdocs/jobs/130816121
>
> I've purposefully made it fail. Success looks like this:
>
> https://travis-ci.org/nigelbabu/glusterdocs/jobs/130815368
>
> We can, in the future, add stuff so that documentation has working links
> and there are no large files checked in. If there's interest happy to send
> a pull request for this.
>

I like this a lot.
It's a way to make sure w'ere not putting in things that haven't been
thoroughly checked.

PR welcome.

- amye

>
> On Tue, May 17, 2016 at 4:55 PM, Amye Scavarda <amye at redhat.com> wrote:
>
>>
>> On Tue, May 17, 2016 at 3:59 PM, Amye Scavarda <amye at redhat.com> wrote:
>>
>>>
>>>
>>> On Tue, May 17, 2016 at 3:56 PM, Niels de Vos <ndevos at redhat.com> wrote:
>>>
>>>> On Tue, May 17, 2016 at 02:42:27PM +0530, Amye Scavarda wrote:
>>>> > Hi all,
>>>> >
>>>> > So we have a new slideshare.net account, GlusterCommunity (
>>>> > http://www.slideshare.net/GlusterCommunity/) that connects with the
>>>> > Gluster.org G+ community - and it'll even connect with the YouTube
>>>> channel!
>>>> >
>>>> > I've submitted a PR to the glusterdocs repo that will need some
>>>> review: it
>>>> > removes all of the presentations from the repo and links to
>>>> slideshare. (
>>>> > https://github.com/gluster/glusterdocs/pull/109)
>>>>
>>>> Cool, but note that the size of the repository does not decrease with
>>>> that commit. The git repository will still contain all the presentations
>>>> in the history/log. But not adding any more presentations is a good step
>>>> already :-)
>>>>
>>>> You are correct, but it will not make the current issue worse. It would
>>> help if I actually hit 'reply all'.
>>>
>>>
>>>> > In no way does this mean that anyone needs to use Slideshare to host
>>>> PDFs
>>>> > of slides, you can use whatever you want. I chose slideshare because
>>>> there
>>>> > was an older Gluster account that had some Gluster.com presentations
>>>> and it
>>>> > links with YouTube.
>>>> >
>>>> > Thoughts?
>>>>
>>>> Looks good to me, but maybe you can address this comment in the GitHub
>>>> pull request:
>>>>   https://github.com/gluster/glusterdocs/pull/109/files#r63498585
>>>>
>>>> That's why I have you all to proofread.
>>>
>>>
>> One thing I'm noticing, we don't have any sort of CI on Read The Docs.
>> Let me see if there's not an easy way to fix that and have TravisCI tell us
>> if we're about to merge something with a bunch of borked links.
>> -- a
>>
>>
>>>  - amye
>>>
>>>
>>>> Thanks,
>>>> Niels
>>>>
>>>> > - amye
>>>> >
>>>> >
>>>> >
>>>> > On Thu, May 12, 2016 at 7:49 PM, Niels de Vos <ndevos at redhat.com>
>>>> wrote:
>>>> >
>>>> > > On Thu, May 12, 2016 at 03:55:23PM +0530, Kaushal M wrote:
>>>> > > > On Thu, May 12, 2016 at 1:25 PM, Niels de Vos <ndevos at redhat.com>
>>>> wrote:
>>>> > > > > On Thu, May 12, 2016 at 02:56:52AM -0400, Prashanth Pai wrote:
>>>> > > > >>
>>>> > > > >>
>>>> > > > >> > > Right now, even cloning the main docs branch is a huge
>>>> pain due
>>>> > > to the size
>>>> > > > >> > > of the repo.
>>>> > > > >> > > I think that branching will solve not this problem, and
>>>> might
>>>> > > make the
>>>> > > > >> > > problem worse.
>>>> > > > >> >
>>>> > > > >> > Branching would not increase the size of the repository
>>>> itself.
>>>> > > Only the
>>>> > > > >> > size used on RTD will be bigger as the HTML for different
>>>> branches
>>>> > > will
>>>> > > > >> > be generated (so contents is there 2x). Cloning the
>>>> repository is
>>>> > > not
>>>> > > > >> > affected.
>>>> > > > >> >
>>>> > > > >> > Deleting files (like the presentations) will also not remove
>>>> them
>>>> > > from
>>>> > > > >> > the git repository. It will stay possible to checkout an
>>>> older
>>>> > > version
>>>> > > > >> > of the docs from the same repository, all of the history is
>>>> > > downloaded
>>>> > > > >> > once the repository is cloned.
>>>> > > > >> >
>>>> > > > >> > In order to reduce the size of the repository, you need to
>>>> create a
>>>> > > new
>>>> > > > >> > one, and import the changes without the big files. While
>>>> importing
>>>> > > > >> > changes from an other (the current) repository, it is
>>>> possible to
>>>> > > modify
>>>> > > > >> > the changes on the fly and prevent importing the big files.
>>>> This
>>>> > > keeps
>>>> > > > >> > the history and the credits for the contributors.
>>>> > > > >>
>>>> > > > >> This is an alternative solution:
>>>> > > > >> https://rtyley.github.io/bfg-repo-cleaner/
>>>> > > > >
>>>> > > > > Right, I was thinking about git-filter-branch. In the end, I am
>>>> pretty
>>>> > > > > sure that the old/original repository is not valid anymore. I
>>>> expect
>>>> > > > > that 'git rebase' is used for the cleaning, and that will
>>>> change the
>>>> > > > > commit-ids of patches that follow after a 'cleaned' patch.
>>>> > > > >
>>>> > > > > Mu recommendation for a seperate repository, is only for
>>>> preventing
>>>> > > > > inconsistencies between the upstream repository (after
>>>> cleaning) and
>>>> > > the
>>>> > > > > previously cloned/forked repositories that contributors have.
>>>> > > > >
>>>> > > > >> > Where would you suggest the presentations (and other files?)
>>>> should
>>>> > > get
>>>> > > > >> > located?
>>>> > > > >>
>>>> > > > >> May be an official Gluster community slideshare or speakerdeck
>>>> > > account ?
>>>> > > > >
>>>> > > > > Possibly something like this. But we should have a plan for the
>>>> > > existing
>>>> > > > > presentations too. And we have to accept that not everyone
>>>> presenting
>>>> > > > > about a Gluster (related) topic will use 'our' SaaS instance.
>>>> > > > >
>>>> > > > >> Git LFS is also also an option but we don't really need
>>>> versioning for
>>>> > > > >> presentation files. Git LFS will keep large files in a separate
>>>> > > location
>>>> > > > >> and keep a "pointer" to those in the repo.
>>>> > > > >
>>>> > > > > I'd prefer something like this. Most of my presentations are
>>>> written
>>>> > > > > while I'm travelling, so a connected service is not really an
>>>> option
>>>> > > for
>>>> > > > > me in any case.
>>>> > > >
>>>> > > > The docs repo should just have links to the presentations.
>>>> > > > They could be hosted on slideshare/speakerdeck, google drive or
>>>> they
>>>> > > > could be hosted html5 presentations.
>>>> > > > If required we could just host the presentations on
>>>> download.gluster.org
>>>> > > .
>>>> > > > I've seen it being used to host resources for tutorials previously
>>>> > > > (like disk images),
>>>> > > > so hosting the actual presentations shouldn't be too hard.
>>>> > >
>>>> > > I really do not care where they are hosted. We just can not demand
>>>> the
>>>> > > use of a SaaS for them. We can offer the option of course, but still
>>>> > > allow presenters to use the tool of their preference.
>>>> > >
>>>> > > Niels
>>>> > >
>>>> > > _______________________________________________
>>>> > > Gluster-devel mailing list
>>>> > > Gluster-devel at gluster.org
>>>> > > http://www.gluster.org/mailman/listinfo/gluster-devel
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Amye Scavarda | amye at redhat.com | Gluster Community Lead
>>>>
>>>
>>>
>>>
>>> --
>>> Amye Scavarda | amye at redhat.com | Gluster Community Lead
>>>
>>
>>
>>
>> --
>> Amye Scavarda | amye at redhat.com | Gluster Community Lead
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>

-- 
Amye Scavarda | amye at redhat.com | Gluster Community Lead
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160517/fc682a7d/attachment.html>