Re: Splitting the security-tracker repo [was Re: [PATCH 00/12] Plannings for secure-testing repository migration to git]
Hi Guido,
On Thu, Dec 28, 2017 at 09:11:35PM +0100, Guido Günther wrote:
> Hi Salvatore,
> On Thu, Dec 28, 2017 at 09:02:56PM +0100, Salvatore Bonaccorso wrote:
> > Hi Guido,
> >
> > On Thu, Dec 28, 2017 at 06:59:11PM +0100, Guido Günther wrote:
> > > Hi,
> > > On Thu, Dec 28, 2017 at 12:33:27PM +0100, Guido Günther wrote:
> > > > Hi,
> > > > On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote:
> > > > [..snip..]
> > > > > have that already :). But I'm unahappy with my current versions, thus
> > > > > I have not sent those yet. Basically: I'm struggling with a proper
> > > > > replacement of such constructs:
> > > > >
> > > > > cd /srv/security-tracker.debian.org/website/secure-testing/data && svn update && ../bin/update && svn commit -qm "automatic update" CVE && cd CVE && ../../bin/bts-update list
> > > >
> > > > cd /srv/security-tracker.debian.org/website/security-tracker/data && git pull origin master && ../bin/update && git commit -qm "automatic update" CVE && git push origin master && cd CVE && ../../bin/bts-update list
> > > >
> > > > git commit might fail if there are no changes so we need to either guard
> > > > for that or use --allow-empty. git push might fail as well if there were
> > > > pushes in between. Since there are several commands that
> > > >
> > > > pull, modify, commit push
> > > >
> > > > should we move this to a script that
> > > >
> > > > pulls, invokes the modification command, commits when there are
> > > > changes, pushes. If that fails merges again && pushes again.
> > > >
> > > > That won't save you from merge conflicts but that's the same with SVN a
> > > > assume?
> > >
> > > Salvatore pointed out to me that it would be good to not mix tracker
> > > code updates with data updates so that the above commands that pull and
> > > push data to/from git would only operate on the data part but not on the
> > > tracker itself. One possible solution would be to split the current
> > > secure-testing repository into two:
> > >
> > > security-tracker: containing the data that is the data/ packages/ org/
> > > doc/ and conf/ directories.
> > > security-tracker-bin: containing the security tracker service and other
> > > scripts and python modules: check-external, bin, tools, lib,
> > > templates, website
> > >
> > > security-tracker-bin could then become a submodule of security
> > > tracker. To ease migration we would create symlinks so the directory
> > > layout stays the same. The top level of the secure-testing repo would
> > > look something like this:
> > >
> > > -rw-r--r-- 1 agx agx 3449 Dez 28 11:56 TODO.gitmigration
> > > drwxr-sr-x 9 agx agx 4096 Dez 28 11:56 data
> > > drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 packages
> > > drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 org
> > > drwxr-sr-x 5 agx agx 4096 Dez 28 11:56 doc
> > > -rw-r--r-- 1 agx agx 8450 Dez 28 12:01 Makefile
> > > drwxr-sr-x 2 agx agx 4096 Dez 28 13:30 conf
> > > -rw-r--r-- 1 agx agx 179 Dez 28 17:20 .gitmodules
> > > lrwxrwxrwx 1 agx agx 35 Dez 28 17:20 check-external -> security-tracker-bin/check-external
> > > lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 bin -> security-tracker-bin/bin
> > > lrwxrwxrwx 1 agx agx 26 Dez 28 17:20 tools -> security-tracker-bin/tools
> > > lrwxrwxrwx 1 agx agx 30 Dez 28 17:20 templates -> security-tracker-bin/templates
> > > lrwxrwxrwx 1 agx agx 27 Dez 28 17:20 static -> security-tracker-bin/static
> > > lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 lib -> security-tracker-bin/lib
> > > lrwxrwxrwx 1 agx agx 28 Dez 28 17:20 website -> security-tracker-bin/website
> > > drwxr-sr-x 10 agx agx 4096 Dez 28 17:20 .
> > > drwxr-sr-x 9 agx agx 4096 Dez 28 17:25 security-tracker-bin
> > > drwxr-sr-x 2 agx agx 4096 Dez 28 17:26 stamps
> > > drwxr-sr-x 9 agx agx 4096 Dez 28 17:26 .git
> > > drwxr-sr-x 4 agx agx 4096 Dez 28 18:17 ..
> > >
> > > I have scripts¹ that produce such a layout and pushed temporary repos
> > > here
> >
> > That looks good to me, can you push your conversion scripts to
> > tools/git-migration?
>
> Comitted to the security-testing svn repository.
Thank you. There is one further change needed I think, and one
question/concern.
The Makefile must be as well not in the security-tracker.git, but
linked from there. But now this is still not safe to have
security-tracker-bin git as submodule, because one can just replace
the symlinks we created.
So in the long run we would better of to decouple the data part and
code part defintively, but I wonder if for now the safest road would
be to go back one step, do the svn to git migration in one repository
and for the autmoatic updates triggeres, operate in a separate
workdir.
Thoughs?
Regards,
Salvatore
Reply to: