Re: What should we do now?
* Anthony Towns <aj@azure.humbug.org.au> [011023 17:22]:
> On Tue, Oct 23, 2001 at 02:42:42PM +0200, Gregor Hoffleit wrote:
> > Just to make the discussion a little bit more focussed: I think several
> > issues were mixed up in my original mail:
>
> Sounds like a plan.
>
> > (1) For once, #!/usr/bin/env has it's problems. Scripts that use
> > "#!/usr/bin/env python" are more fragile than scripts without.
> >
> > (2) Another issue is "python" vs. "python1.5". It's a separate question
> > whether scripts need to be strictly coupled to a specific Python
> > version.
> >
> > Regarding (1): If you ask me how common the situation is that people
> > install local Python versions in /usr/local, then I will ask you how
> > common it is that it's reasonable that a script provided by a Debian
> > package will benefit from using #!/usr/bin/env ?
>
> Well, how about you answer the question instead?
>
> It's reasonable for Debian scripts to use #!/usr/bin/env because any
> harm that could be caused is both rare and easily avoidable, and because
> that's what many upstream scripts use. AFAICS.
Certainly that's not a fair statistic, but if I grep through my
/usr/bin, there are 12 scripts using some kind of "#!/usr/bin/env
python", but 79 scripts using something like "#!/usr/bin/python" (in
/usr/local/bin, the ratio is 2 for /usr/bin/env and 4 for
/usr/bin/python).
That's consistent with my observations: The common idiom in the Python
community is "#!/usr/local/bin/python", /usr/bin/env is only rarely used.
That means most Python scripts have to be modified anyway. It's just the
question whether we should replace #!/usr/local/bin/python with
"#!/usr/bin/env python" or with "#!/usr/bin/python".
> (I don't really care if you want to encourage people to not use
> /usr/bin/env, though)
> > Regarding (2): Making the dependency explicit (by using
> > /usr/bin/python1.5) is just playing safe.
>
> It's not merely playing safe though. It's also creating a strict
> dependency that will bite you later (by making transitions to python X.Y++
> require significantly more changes, by having packages break needlessly
> when those changes aren't made).
>
> > As far as I can see, if we hadn't the legacy of the existing packages
> > and installations, and if versioned dependencies would work on all
> > systems in all situations, we could come up with a quite clean setup:
> > python-base et al. could be empty packages (virtual packages wouldn't
> > work!), and other packages could depend on them (e.g. "Depends:
> > "python-base (>= 2.0), python-gtk" for a package that need at least
> > Python 2.0 or better. Still in this case, what could you do if Python
> > 2.3 broke that script ?)
>
> There're two possible solutions, obviously: either you assume in advance
> that scripts will work with new versions of python and allow them to be
> installed together, or you assume they don't and forbid them from being
> installed together.
>
> I think that's a correct statement: I can't see any other possibilities
> being even logically consistent.
>
> If you assume that scripts will work with new versions of python,
> you end up with the result that occassionally your assumption will be
> wrong for some scripts, and you'll have some possible partial upgrades
> that'll break. This isn't a cardinal sin.
>
> OTOH, if you assume that scripts *won't* work, you're committing
> yourselves and a bunch of other people to going through whole bunches of
> Python code checking that it works, and twiddling bits to note that fact.
>
> The only question is whether the (presumable) handful of bugs every year or
> two due to the former is worth the (presumable) whole lot of effort involved
> in the latter.
>
> The easiest way to handle the latter, IMO, is to add a:
>
> Depends: python-base (<< <X>.<Y+1>)
>
> to the package, where <X>.<Y> is the most recent version of python that
> it works with.
>
> Note that this allows you to use a package on multiple versions of python
> (so if woody were to release with python 1.5 and 2.0, your package could
> conceivably Depend: python-base (>= 2.0), python-base (<< 2.3), and you
> won't have to make further changes when Debian packages for 2.1 or 2.2 come
> out), unlike #!pythonX.Y, which is much more restrictive.
Just to make sure we have the same thing in mind: At this point,
python-base is no virtual package provided by some real pythonX.Y-base
package, but it's a real package, right ?
This would work indeed, provided a single, canonical Python version was
an option (see below).
> > As far as I can tell our principal problem is that there's a large set
> > of existing packages (from slink, potato) that have wrong dependencies
> > (like unversioned "Depends: python-base" for packages that depend on
> > Python 1.5.2) that would be (erronously) satisfied if we just continued
> > to use python-base and so on.
>
> Well there's two questions; one is working out what we want, and the other
> is working out how to get there.
>
> > One way to resolve this is to include a huge list of conflicts in
> > python-base et al.
> > Another way to resolve this is to drop these package names, perhaps in
> > favour of some other package names.
>
> The easiest would be to say "Packages that have an unversioned dependency,
> or just a >> dependency and expect version 1.5 (but not 1.6 or 2.0 or
> so on) are broken", and not care.
>
> Depending on how many packages actually expect python 1.5, this might
> even be a reasonable thing to do (if you go the route of buring the future
> backwards incompatabilities bridge when you come to it, this only matters
> for modules, which can probably be pretty easily replaced in one shot).
> Switching the package name from "python-base" to "python" would be relatively
> easy too, if the dependencies aren't maintainable.
That's a solution that came up to me yesterday night, laying in bed ;-).
Earlier, python-base was just a minimal set of Python. During the last
two years, we have merged in many things so that by now, python-base is
essentially everything from Python that doesn't depend on some exotic
stuff (like Tk, expat). Therefore, I think it would now be fair to call
this package "python".
This is something we should really consider.
> > It's much the same reasoning why libraries have a number in them that
> > will be increased for incompatible API changes. AFAICS, with the setup
> > you are suggesting you can never warranty that kind of robustness.
>
> Bumping library revisions usually just requires a recompile though,
> with absolutely no source changes at all. Changing #!/usr/bin/python1.5
> to #!/usr/bin/python2.0, OTOH, doesn't. One can be done trivially by an
> autobuilder, the other can't.
That's a valid point.
> And again: the focus here (IMAO) should be on maintaining a single,
> canonical version of Python in the distribution. Having older versions
> and brand new beta releases around is nice, but having the main version
> be simple to handle and robust seems a lot more important. (Saying "Hey,
> Debian's Python is 2.1.1! But you can't install 1.5 in /usr/local without
> taking some care" is way better than saying "Debian's Python is 1.5! Yeah,
> we know that sucks, but hey, you can install 2.2 and 1.6 and 2.0 all at
> once into /usr/local and nothing will ever break no matter how screwed
> up it is!", eg)
Sorry, but maintaining a single canonical version of Python in the
distribution won't work. That's not my opinion, that's what the Python
developers crew at python-dev told me.
Until recently, Mailman didn't work with Python >> 1.5.2. Zope 2.2
didn't work with Python >> 1.5.2. Zope 2.4 doesn't work with Python <<
2.1.
There's a very big chance that at the release date we don't have all
those applications in sync wrt. the necessary Python version. We have
then the alternative to either stick with an old Python version until
*all* Python packages are compatible with the new version, or we *have*
to support multiple Python versions in a single distribution.
Again, I wouldn't support this scheme if the Python cabal hadn't told me
that it's necessary from their point of view:
To quote just a few:
* Fred L. Drake, Jr. <fdrake@acm.org> [010703 16:04]:
>
> Gregor Hoffleit writes:
> > Seriously, does anybody (besides the Python developers) feel a need to
> > have multiple Python versions on the same system ?
>
> Absolutely! Anyone that wants to write cross-version Python code
> needs to be able to have multiple versions available. I'd even like
> to be able to have both Python 2.0 and Python 2.0.1 available on the
> same $prefix/$exec_prefix -- that can't be done currently. This kind
> of thing is pretty important when you want to take cross-version
> compatibility seriously.
>
> Barry A. Warsaw writes:
> > Yes, definitely as both a Zope and Mailman developer <wink> I need
> > multiple Python versions. But I suspect even normal users of the
> > system will need multiple versions. Different Python-based apps are
> > requiring their users to upgrade Python on their own schedule, so
> > multiple versions will still be required.
>
> Another excellent reason to support multiple versions! As more
> widely distributed applications are written using Python and don't
> want to include the interpreter, this becomes a more noticable issue.
* Guido van Rossum <guido@digicool.com> [010703 16:36]:
> > > What about when you want to have multiple python versions, like
> > > python 1.5.2, 2.0.1, 2.1.1 and 2.2-CVS-snapshot ? :-)
> >
> > You've hit the forbidden question ;-)
> >
> > Seriously, does anybody (besides the Python developers) feel a need
> > to have multiple Python versions on the same system ?
>
> I've had enough requests over the years for this, so it is indeed
> supported, and I believe there is a need. Quite often people have
> important programs that for some minor reason don't work on a newer
> version yet and they can't find the person or the time to fix it.
>
> Python's standard installation makes this possible. You can have only
> one "python" but you can request a specific version by appending the
> "major dot minor" part of the version number, e.g. python1.5,
> python2.0, python2.1, python2.2. "python" is a hard link to one of
> these. You can't (easily) have multiple version with the same
> major.minor, but that should never be needed. I've heard though that
> some Linux distributors break this versioning scheme in favor of their
> own.
>
> > If there's a real world need for this, then, yes, we had to come up
> > with a completely different setup. I guess this setup might involve
> > symlink farms (urghh).
>
> Ugh maybe, but it's the only thing that scales.
So specifically developers of Python scripts have a legitimate need for
multiple concurrent Python versions.
And, even for the simple deployment of Python applications, it seems to
be a common understanding that it's not possible to find a singular
common version at any place in time.
A whole lot of points depend on the question whether providing a single,
canonical Python version is possible and viable for the distribution.
If it's possible, all other issues have a straightforward solution.
If that's not viable, we have to use a completely different scheme.
Do we agree about this ?
Gregor
Reply to: