Re: GSoC project: make the Sage build system more distribution friendly
Le 17/03/2013 15:22, leif a écrit :
thansen@debian.org wrote:
Am Samstag, 16. März 2013 20:57:13 UTC+1 schrieb leif:
But if we switch to git, improve Sage's package management (as a first
step, split vanilla upstream sources off the spkgs :P ), ...
Is splitting the vanilla upstream sources off planned? That would be
very helpful for distributions. Another helpful thing would be a clear
distinction between fixes/adjustment to the library and Sage glue,
because we need all the Sage glue, but want to decide ourselves which
other patches to apply.
We discussed this many times, and hopefully due to switching to git this
will finally happen, because the src/ (upstream) dirs in spkgs aren't
tracked, should always be vanilla (modulo fixing weird file permissions
in a few cases), and hence do not change the often many times we bump an
spkg's patch level (making just small changes to the Sage part which is
under our revision control, including patches to upstream).
Here is what is used when managing a debian package with git, to store
upstream sources in a branch and get a unpatched (pristine) source
tarball when needed :
http://joeyh.name/code/pristine-tar/
when a new upstream gets out, a script makes it easy to update the
upstream source by just pointing to the new tarball:
http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.html
As is, any one-character change to an spkg's spkg-install script, say,
means you have to again download the whole compressed archive (often 12
MB+) just to check the changes or reinstall the package.
Although we have an "upgrade path" (untarred sage source dist accessible
via http or rsync) in addition to the source distribution tarball, the
former still just contains the compressed, "self-contained" spkgs. (Even
worse, sometimes "unmodified" spkgs get repackaged from release to
release such that not just their file modifications times but also their
md5sums differ, for no real reason.) I.e., you currently cannot simply
pull changes to (Sage's part of) spkgs.
A branch for upstream, a branch for packaging, and scripts which make it
easy to work with both ; see for example:
Another thing are patch headers. Sometimes I can't find out what a patch
in a spkg does, for example I found patches that just move a few lines
of code somewhere else. I can't apply a patch when I don't know what it
does. In Debian we have this specification for patch headers to document
what they do: http://dep.debian.net/deps/dep3/
Well, every spkg contains an SPKG.txt file with a changelog and --
hopefully -- a "Patches" section (if appropriate), which in a perfect
world precisely documents what's done why (including what patches are
applied for what reason, whether they come from or went upstream etc.).
Each patch should be its own documentation -- even if it's just to say
"Look in SPKG.txt, silly!".
(Each entry of the changelog should also contain a Sage trac ticket
number -- unfortunately sometimes more or less just this, which means
"further reading".)
There's usually also a "Special Update/Build Instructions" section,
which for example states "patch foo can and should be removed once we
upgrade to upstream version x.y", or "check whether patch foo is still
required". Unfortunately the distinction between generic fixes to
upstream and Sage-specific modifications isn't always that clear; a
single patch may even include both.
In addition, there's the Mercurial history (but the repo is again only
in the spkg itself), although few people provide detailed commit messages.
This is perhaps suboptimal, but similar to the patches to the Sage
library (see below), which hardly ever contain details, but just the
corresponding trac ticket numbers in their commit messages.
I'd be happy if Sage in whole was more modular / less monolithic; I'm
not very optimistic regarding the Sage /library/ though.
What do you mean with the Sage /library/?
The "Sage library" is contained in a sage-x.y.z spkg, has its own repo
(one can pull from for "official"/final releases, hg.sagemath.org), and
contains "the heart" of Sage, i.e., genuine Sage code, as opposed to
upstream components. In a Sage installation, it lives in
$SAGE_ROOT/devel/sage/ (meant to get [cloned and] modified by pretty
ordinary users as well; "every Sage user is [also] a Sage developer").
Hmmmm... there is no .hg in $SAGE_ROOT/devel/sage/, but there is one in
$SAGE_ROOT/, another in $SAGE_ROOT/local/bin/, another in
$SAGE_ROOT/devel/sage-main/ and a fourth in
$SAGE_ROOT/devel/ext-main/... [looking at a freshly compiled sage 5.7 on
my ARM box]
(There are also the Sage scripts, Sage "root", and "extcode" spkgs /
repositories; the former two less interesting to a typical user.)
What I meant is that it's IMHO unlikely you'll one day be able to pick
only the parts you want / need from it; there are optional Sage *spkgs*
(interfacing with the Sage library), but everything else which is
shipped in the huge Sage spkg is closely tied together, i.e., not really
modular (although it perhaps could be [more] -- there's e.g. also Purple
Sage -- purple.sagemath.org).
Well, the huge number of packages in any linux distribution shows that
it's really possible to have a big dependency tree with complex
relations. The software only needs to be written in a flexible way.
P.S.: IIRC, it took ages to get the obsolete (and broken) Sage 3.0.x
package out of Debian's / Ubuntu's package repositories... ;-)
Well, if everybody knows it's broken but nobody reports a grave-level
bug asking for its removal, it will just bitrot...
Snark on #debian-science and #sagemath
Reply to: