On Wed, Sep 24, 2003 at 10:42:15PM -0500, Terry Hancock wrote:
> On Wednesday 24 September 2003 07:13 pm, Zed Pobre wrote:
> > This is only correct for third-party packages. Packages that are
> > part of a distribution belong in /usr
>
> Um, basically, I think we're going to get into a serious conflict here,
> or else this just means IRAF will remain a "3rd party package."
Well a 3rd party package is by definition one that isn't
distributed by the vendor, so that leaves you making a package that
you'll have to distribute yourself.
> Please remember that I have time to package these things because
> I'm paid by users to keep it working, and packaging it for Debian
> happens to be a good way to do that. But I can't betray their
> interests just to satisfy some procrustean 3rd party policy.
*sigh* You asked me for my comments on your packaging, and I gave
them to you. For all the things in Debian Policy there are to quibble
over, there are very good reasons not to touch things in /usr/local or
/opt, which I gave you before, and I will reiterate in a moment. I'm
not detailing these issues to torture you; I'm trying to give you
pointers on what is necessary to get a package into Debian. Debian
Policy is what allows so many thousands of packages to interoperate
cleanly and *not* surprise an admin on an install or an upgrade. It's
this attention to detail that makes Debian the most reliable GNU/Linux
distribution for servers, putting it on par with the BSDs.
> IRAF has a particular structure that it needs to keep to satisfy its
> users, and to make it network-installable, etc. I can't just tear
> it apart and expect that all to keep working, and the users wouldn't
> trust it if I did.
Well, to be blunt, most users generally wouldn't know the
difference. But I don't think the issues are as insurmountable as you
seem to believe.
> Besides, IRAF is older than Debian, I think its design has seniority. ;-)
With all due respect to your smiley, being older doesn't make it
any less wrong.
That's a bit harsh, but I'm absolutely exhausted right now and I
can't come up with a nicer way to say it, and it's a fairly important
point. The FHS is there for a reason.
> > The /opt and /usr/local areas are for the system administrator to
> > play around with, and even if he or she has "apt-get dist-upgrade" in
> > a cron job, nothing that Debian packages do should ever affect the
> > files stored there.
>
> Well, the FHS says /usr/local is for the sysadmin, and opt is for
> 3rd party packages.
Specifically, this means that /usr/local is for stuff that the
sysadmin writes, overrides, or adds on piecemeal, and /opt is for
stuff that the sysadmin adds in large packaged or semi-packaged
chunks. In both cases, however, the sysadmin has personally placed
files into known locations, and has every right to expect them to stay
there undisturbed until he personally does something else to them.
As a very concrete example, if a sysadmin downloads his own
package of IRAF to play with (because for instance the Debian version
is somewhat out of date), or possibly even downloads some other
IRAF-related software that is convenient to stick in /opt/iraf, then
if at some point he installs the Debian package of IRAF or upgrades an
existing Debian system, and Debian clobbers his personal build, he has
every right to be extremely upset. Debian just destroyed his personal
data on an upgrade, and that's completely unacceptable.
The stuff in /usr, excluding /usr/local, is vendor territory, and
it's entirely expected that things there will be shifted around by
package upgrades or installs. The stuff in /usr/local and /opt
(ignoring for a moment the fact that Solaris uses /opt as a dumping
ground, since we're not dealing with Solaris) is very much the domain
of the admin, and admins will be surprised if someone usurps that.
Neither of us are in a position to have any effect at all on that
expectation, and neither of us has the right to tell them they have to
live with it just because IRAF has a monolithic and inflexible design.
> IRAF is not alone in being package-oriented, either: AIPS, AIPS++,
> and CIAO will all insist on their own structure (and I get to do them
> just as soon as I finish with IRAF). I think there are enough such
> packages in the world, that Debian policy ought to have room
> for them.
KDE tried that too, even, but all of the Debian KDE packages
conform to the FHS. Was it a lot of work? Well, yes, but it's not
impossible, and someone unfamiliar with KDE but familiar with the FHS
will find all the parts in the expected locations.
But if you want to argue this point, you're doing it with the
wrong person. I don't define Debian Policy. If you want to argue
with the project over this, there's a debian-policy mailing list just
for that, and I can send you information on how to subscribe if you
are interested.
> I'll think about it some more, but I'm not convinced that the concept
> of breaking it up into /usr/share, /usr/lib, /usr/bin can work. At best,
> we might be able to put everything under /usr/share/iraf and use
> symlinks to satisfy the policy.
Unless everything there could be NFS-mounted on a completely
different architecture and still work, it needs to go in /usr/lib/iraf
instead. Doing this and symlinking the binaries is a passable, if
ugly, way of dealing with this, and if you're pressed for time, what I
recommend. The separation of architecture-independent data out into
/usr/share isn't really enforced -- the only important thing is to
make sure that nothing in /usr/share is actually
architecture-dependent.
> > If the stsci is under a free software license, I would suggest
> > merging it, unless it is useful in its own right. If it is, then it
> > can be packaged separately, and the resulting binary package listed as
> > a Build-Depends entry for the IRAF source packages that need it.
>
> I don't think that's really such a great idea. IRAF has dozens of packages,
> these are just a few. What really bothers me about this is that STScI and
> NOAO have separate release schedules. So I think they should remain
> as separate as possible.
Circular build-dependencies are very ugly. You can get away with
them, but they need to be documented. Can STScI be built without
IRAF? Is it useful without IRAF? I'm not familiar with it, I'm
afraid. If it is a complete product in itself, then you'll have to
package it first, then use it as a build-dependency for IRAF.
Actually, if it's not a very large or complex package, I might be
able to find time to do that part myself. It's just IRAF that gives
me a headache every time I start to work with it.
Thanks for taking this on, by the way. IRAF is an important bit
of software, and I've felt lousy about letting it languish, but I can
never find enough time in one chunk to be able to do what I want with
it.
> You should think of this as being analogous to Python w.r.t. to Python
> modules, for example -- a core package with lots of add-on 3rd-party
> components.
Well, yes, but Python doesn't require a 3rd-party component to
build, which is the problem here.
> That one of them, "noao" happens to be maintained
> by the same folk who maintain the core is kind of coincidental (even
> though they've long been bundled). Actually things didn't get complicated
> until tables was used in the noao package -- and I think Mike was
> telling me that this is a source-dependency, but not a binary one.
As long as nothing in STScI actually depends on something in IRAF
to either to build or to install, this actually isn't too bad. Even
if it isn't particularly *useful* without IRAF, you can leave the IRAF
packages at the Recommends: level, build STScI, install STScI, build
IRAF, then install IRAF, and everything is happy.
> Okay. Maybe a "Build-Depends" on Python would be okay, too?
> I know Python better than either shell language, and the STScI
> packages will depend on it anyway, as they provide a python shell
> for IRAF called "pyraf".
If you're going to have a run-time dependency on Python at the
end, it isn't going to noticably hurt you to have a build-time
dependency on it. I ended up taking the Perl route myself some time
back, but I have nothing against Python.
> > If you create a properly named orig.tar.gz and use the debian
> > tools to create a diff and a dsc file, you can cut it down to one
> > step: dpkg-source -x iraf_2.12-1.dsc. Check the man pages for
> > dpkg-source, dpkg-buildpackage, and debuild. Since one of your later
> > steps actually uses dpkg-buildpackage, you should have ended up with a
> > source, diff, and dsc at the end anyway.
>
> I wasn't sure how complete .dsc is -- I wanted to show you the
> iraf.debian, iraf.diffs, and iraf.overlay directories which separate
> out my custom work from the main distribution. Other than unpacking
> into the right directories and making sure I'm using the right versions,
> the orig.tgz file is pretty much straight from upstream (although not
> "pristine").
Well, the dsc is nothing more than a list of build-dependencies
and the md5sums of the diff and orig.tar.gz. Otherwise, dpkg-source
will generate a diff.gz from everything you changed against the
orig.tar.gz file. If you want to keep a number of individual patches
organized, you should probably look into using either dpatch or dbs,
though they both (dbs especially) have a bit of a learning curve. If
you have the time, this might not be a bad thing to do, however,
especially if you intend to send patches upstream. My own noble
intentions long ago of cleaning up the IRAF sources got run over by
the size of the changes I was making, and eventually came to nothing
because the monolithic diff wouldn't apply to the next release of
IRAF. This was the major thing that prevented me from moving forward
-- untangling all of that requires a large uninterrupted block of time
so I can keep it all straight, and I couldn't bring myself to just
give it up.
Either dbs or dpatch probably would have helped a lot with that,
but neither of them were really usable at the time. Now, both of them
are fine; dbs is more powerful, but dpatch is easier to use.
> > If you require environment variables to be set before running
> > debian/rules, you're going to get into fairly serious trouble. Make
> > sure that whatever you need gets set from there, and that it will
> > build from an arbitrary environment.
>
> Hmm. They are used to remember targets used by th BUILD_TARGET.csh
> script. This seems to work. What would tend to break this system?
The fact that an autobuilder is going to automatically unpack the
source from the orig.tar.gz, apply the diff, then run debian/rules
without knowing anything about the special script that set up the
environment. If the environment isn't set up from debian/rules, you
can pretty much count on a serious bug being filed on the package
about it not building properly from source.
As a side note, you might want to be careful about building in
anything other than a very standard environment for other reasons --
there was a case where someone attempted to subvert a Debian package
by adding code to the upstream source that specifically checked for a
known environment variable on the maintainer's build system so that
when the maintainer tested the package, nothing would happen, but the
code would take effect anywhere else. Fortunately, it was caught
anyway.
> > > 3) cd iraf-2.12
> > > dh_make -e <your-email-address> --single
> > > This does some housekeeping and creates a debian subdirectory
> > > (but we're about to clobber that directory, so don't worry about
> > > it).
> >
> > It will also generate an iraf-2.12.orig directory one level up,
> > which is going to take a whole lot of space. Since all that dh_make
> > does is make that and the debian/ directory, you're better off simply
> > including the debian/ directory in a patch.
>
> Yeah, it's pretty big, but don't we need that to make the diff.tgz?
Nope. The diff is made against the orig.tar.gz file in any case
-- the iraf-2.12.orig directory is only used to automatically generate
such a file in the case where it doesn't already exist.
Incidentally, I keep seeing you use ".tgz" as an extension, and it
may eventually bite you. For starters, the diffs are straight
compressed diffs (.diff.gz), and never pass through tar. If you try
to tar it up, you'll get into trouble. Also, the Debian packaging
tools are hardwired to look for .tar.gz, not .tgz, so it really does
have to be "iraf_2.12.orig.tar.gz", not "iraf_2.12.orig.tgz". The
latter won't be found correctly.
> > If you just want it to build the debs, but not generate a diff or
> > dsc file or bother with signing, you can run "debian/rules binary"
> > instead.
>
> Thank you for mentioning this. I suspect I can use this approach to
> shorten my development cycle. Building takes about 10-15 minutes
> even on a very fast computer, so testing the final install steps by
> just running dpkg-buildpackage was getting pretty tedious. I guess
> I could just leave the build alone and test with "debian/rules install",
> huh?
Yes, and if you're correctly generating a build-stamp on build
completion, repeating a "debian/rules binary" will notice that the
build has already been done and won't rebuild, either. If you use
dpkg-buildpackage or debuild, however, it will do a debian/rules clean
as it starts and rebuild everything.
Thanks again,
--
Zed Pobre <zed@debian.org> a.k.a. Zed Pobre <zed@resonant.org>
PGP key and fingerprint available on finger; encrypted mail welcomed.
Attachment:
pgp9MD7ac58gm.pgp
Description: PGP signature