Splitting the security-tracker repo [was Re: [PATCH 00/12] Plannings for secure-testing repository migration to git]
Hi,
On Thu, Dec 28, 2017 at 12:33:27PM +0100, Guido Günther wrote:
> Hi,
> On Wed, Dec 27, 2017 at 11:32:53PM +0100, Salvatore Bonaccorso wrote:
> [..snip..]
> > have that already :). But I'm unahappy with my current versions, thus
> > I have not sent those yet. Basically: I'm struggling with a proper
> > replacement of such constructs:
> >
> > cd /srv/security-tracker.debian.org/website/secure-testing/data && svn update && ../bin/update && svn commit -qm "automatic update" CVE && cd CVE && ../../bin/bts-update list
>
> cd /srv/security-tracker.debian.org/website/security-tracker/data && git pull origin master && ../bin/update && git commit -qm "automatic update" CVE && git push origin master && cd CVE && ../../bin/bts-update list
>
> git commit might fail if there are no changes so we need to either guard
> for that or use --allow-empty. git push might fail as well if there were
> pushes in between. Since there are several commands that
>
> pull, modify, commit push
>
> should we move this to a script that
>
> pulls, invokes the modification command, commits when there are
> changes, pushes. If that fails merges again && pushes again.
>
> That won't save you from merge conflicts but that's the same with SVN a
> assume?
Salvatore pointed out to me that it would be good to not mix tracker
code updates with data updates so that the above commands that pull and
push data to/from git would only operate on the data part but not on the
tracker itself. One possible solution would be to split the current
secure-testing repository into two:
security-tracker: containing the data that is the data/ packages/ org/
doc/ and conf/ directories.
security-tracker-bin: containing the security tracker service and other
scripts and python modules: check-external, bin, tools, lib,
templates, website
security-tracker-bin could then become a submodule of security
tracker. To ease migration we would create symlinks so the directory
layout stays the same. The top level of the secure-testing repo would
look something like this:
-rw-r--r-- 1 agx agx 3449 Dez 28 11:56 TODO.gitmigration
drwxr-sr-x 9 agx agx 4096 Dez 28 11:56 data
drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 packages
drwxr-sr-x 2 agx agx 4096 Dez 28 11:56 org
drwxr-sr-x 5 agx agx 4096 Dez 28 11:56 doc
-rw-r--r-- 1 agx agx 8450 Dez 28 12:01 Makefile
drwxr-sr-x 2 agx agx 4096 Dez 28 13:30 conf
-rw-r--r-- 1 agx agx 179 Dez 28 17:20 .gitmodules
lrwxrwxrwx 1 agx agx 35 Dez 28 17:20 check-external -> security-tracker-bin/check-external
lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 bin -> security-tracker-bin/bin
lrwxrwxrwx 1 agx agx 26 Dez 28 17:20 tools -> security-tracker-bin/tools
lrwxrwxrwx 1 agx agx 30 Dez 28 17:20 templates -> security-tracker-bin/templates
lrwxrwxrwx 1 agx agx 27 Dez 28 17:20 static -> security-tracker-bin/static
lrwxrwxrwx 1 agx agx 24 Dez 28 17:20 lib -> security-tracker-bin/lib
lrwxrwxrwx 1 agx agx 28 Dez 28 17:20 website -> security-tracker-bin/website
drwxr-sr-x 10 agx agx 4096 Dez 28 17:20 .
drwxr-sr-x 9 agx agx 4096 Dez 28 17:25 security-tracker-bin
drwxr-sr-x 2 agx agx 4096 Dez 28 17:26 stamps
drwxr-sr-x 9 agx agx 4096 Dez 28 17:26 .git
drwxr-sr-x 4 agx agx 4096 Dez 28 18:17 ..
I have scripts¹ that produce such a layout and pushed temporary repos
here
https://salsa.debian.org/agx/security-tracker-with-submodule
https://salsa.debian.org/agx/security-tracker-bin
So if you want to see how it looks to a
git clone --recursive https://salsa.debian.org/agx/security-tracker-with-submodule
With some follow up work we can get rid of most of the symlinks but
this could happen after the git migration. So soriano would only see an
update of security-tracker-bin code if we bump the submodule revision
(end even then only if we do a "git submodule update").
Does this make sense or does it introduce too much complexity? Is the
above split about right or should some of the directories move between
security-tracker and security-tracker-bin (e.g. website or static).
Feedback would be welcome.
Cheers,
-- Guido
¹: https://salsa.debian.org/agx/security-tracker/commit/4a4a83117e3d6791fbe091a4de2a2bc1ad0abf98
Reply to: