Re: Challenges packaging Python for a Linux distro - at Python Language Summit
Hi Luke,
First, I'd like you to know I feel sorry to read how much you seem
affected by what you describe below. Hopefully, you'll find a viable
solution soon, and hopefully, we may help.
However, please try to understand what others are telling. The solution
you're looking for is probably not the one you expect, but I'm convince
it does exist (ie: venv, probably, or better dependency management). See
below.
On 5/17/21 8:10 AM, Luke Kenneth Casson Leighton wrote:
>> All the horrors that you are painting after this paragraph, are due to
>> the fact that you aren't doing "apt-get dist-upgrade". I'm having a hard
>> time understanding why you're both:
>> - not doing "apt-get dist-upgrade"
>
> because precisely the difficulties encountered, and due to software development
> that critically requires more recent variants of what is typically in
> debian/stable
> i.e. usually a minimum of one even three years out-of-date.
>
> a dist-upgrade to debian / testing - a way to obtain the latest
> variants of critical
> software - frequently resulted in massive breakage.
Massive breakage of what? Your own Python code? Well, it's up to you to
stay current to whatever version is in Testing/Unstable. If you don't do
it, then it's your responsibility, and you must live with the
consequences of it.
Yes, it's a massive undertaking to always stay up-to-date, and drains a
lot of energy, though no choice if you're serious with what you're doing...
> i quickly learned never to attempt that again given the massive disruption
> it caused to me being able to earn money as a software engineer.
So, to "earn money as a software engineer", you prefer your software to
rely on older, probably buggy, and not security supported, dependencies?
I'm sorry, but that's far from best practice, and it will bite hard anyways.
Yes, dependency management is a hard work. Yes, it takes some of your
time (sometimes massively). No, you can't avoid it...
>> - complaining that it's breaking your system
>
> ah, you're missing the point by focussing on the background and context.
Are you *SURE* that you're the only person that's right, and that the
dozen experienced Debian Developers saying the opposite way are the
wrong persons? I would strongly advise you to reconsider.
Do you completely understand how "library transitions" are working in
Debian? If not, I would strongly advise you to read about it.
> i would prefer to solicit some
> responses that acknowledge that this is an actual problem that needs
> solving.
Well, it feels like the only thing you accept, is the answer that *YOU*
want, and not the one that others are telling you. Chances that it's
going to happen are very thin, since we all know that what you're
discussing is not the way Debian works.
> Bryan's message unfortunately is a repeat of prior experiences
> in reporting this issue over the years.
Looks like a lot of people told you the same thing...
Do you sometimes listen to what others tell? :)
> Brian May wrote:
>> A rolling type update might be convenient, but it is not supported by
>> Debian, and has not been supported by Debian in sometime. There are
>> complexities in such an arrangement, and it is difficult to test such
>> arrangements will work as expected. Such an arrangement is not
>> guaranteed or tested to work.
>
> Brian: as an experienced debian systems administrator and python
> developer I'm not asking for help with either, and over the past 20 years
> have successfully learned and deployed the techniques required to
> keep a rolling system operational
Apparently, you haven't...
> like Thomas, you are missing the point by focussing on the context
> and background material rather than focussing on the problem.
We aren't at all missing the point that doing the way you're doing is
often breaking your system.
The problem: you're expecting that never doing "apt-get dist-upgrade"
will handle library transitions, when this isn't the way Debian works.
We're all telling you, but you do not want to listen, and insist that
what you've been doing (wrongly) for 20 years is the correct way of
doing things. Sorry, but it's not... You *will* experience breakage, and
there's nothing we can do to save you.
> this was also the tack taken by others when i explained the problem:
> "it's your own fault, this isn't supported, therefore we don't have to
> do anything, therefore it's not a bug, therefore the bugreport is
> summarily dismissed".
Which is 100% correct.
> in my report, if you read it again carefully, i specifically stated that
> *multiple debian maintainers* are receiving *multiple complaints* of
> breakage and disruption due to the standard debhelper-python
> build system not following the "safe" practice outlined by
> numpy and sci-py.
The problem isn't the build system, which is doing things the correct
way. Any "arch: any" package linked against Python *must* be upgraded
together with the interpreter itself when it transition from Unstable to
Testing. That's a fact, and how Debian is designed.
By *policy* Debian doesn't support multiple versions of a library on a
single system. The only thing that is allowed, is to have 2 versions
*during a transition*. What we hope is to have such thing happen (ie: 2
versions of a single library) only in Unstable, though sometimes, this
also migrates to testing, but that's not what we desire.
If you wish Debian to support multiple versions of a library, you should
first attempt to discuss this feature at large with the Debian community
because *by design* (and therefore, written in the stones of the Debian
Policy Manual) that isn't how we want Debian to work, and how the Debian
community wants it to work.
If you wish to have such a feature, then yeah, probably you should go
and use another distribution, because I don't see Debian changing on the
direction you describe anytime soon.
> i can only surmise that they probably don't want to say anything
> because the message being sent to them on summary closure
> of bugreports is that, well, "nobody cares".
I don't think that's right. They just think you are wrong, because you
believe Debian should support multiple versions of a library (here:
Python), when it's not the case *by design*, because we (ie: everyone
contributing to Debian) want Debian to be this way. This isn't specific
to the Python world, this is how Debian *at large* is designed.
> once this is accepted as actually being a problem that needs solving,
> rather than being avoided because "it's not supported, you're on your
> own, not our problem", i am happy to help advise and discuss
> solutions.
>
> given that this has been ongoing for 10 years now i have been giving
> it some thought for a long, long time.
>
> however before getting to a discussion of solutions i would prefer
> to see acknowledgement that it is recognised as a problem that
> people actively wish to see fixed.
As I wrote earlier, before discussing if this is a problem or not, you
will need to convince a majority of Debian Developers that having 2
versions of a shared library at the same time on a given Debian system
is something we desire. Good luck doing that, as this is a very strong
feature of Debian. I don't know any Debian Developer that believes this
is something desirable.
> otherwise i will have to wait patiently
> for the next disaster situation to occur, or, sadly, after 20 years of
> using debian, and remaining loyal to it despite maltreatment, i will
> have to find an alternative distro.
There's no maltreatment. Just a misunderstanding from your side of what
Debian does, IMO.
Finding another distro may be one solution indeed.
Doing what Jeremy suggests (ie: running your builds in a chroot, or with
venv, etc.) may work for some times, though I would strongly recommend
the best practices in terms of dependency management, which means:
always try to stay current with all of your dependencies.
> what is stopping me from doing that is the severe financial consequences
> involved and risk to myself and my family due to my income being
> far below average. the downtime that would result is a huge risk,
> plus learning a new distro also compromises my effectiveness which
> also results in further lost income.
Then learn how to reproducibly build chroot and venvs.
> what i am saying is: this is serious - i am effectlvely financially
> trapped and critically dependent on the decisions that you make,
> even though i am not paying you money for the work that you do.
IMO, you've trapped yourself here... but there's exist strategies. :)
I sincerely hope you will find a solution that matches your need,
hopefully by continuing to use Debian.
Cheers,
Thomas Goirand (zigo)
Reply to: