Re: python devs complaining about debian packaging
On Sunday, 26 May 2024 03:33:09 CEST Ian Norton wrote:
> I am puzzled about some of the responses there, how can anyone expect to
> randomly update packages on the system using pip and not have it go wrong
> on any distribution? This is why things like pipenv exist.
Or whatever today's tool is called. That's just one of the problems that the
core Python packaging scene has. To be fair, though, I found myself using pip
in a job a couple of years back just to "go with the flow", and it mostly
worked. My expectations were not particularly high, however.
Where I was just installing things casually, I made pip install things using
the --user option, which employs that unpleasant but pervasive approach of
stashing huge volumes of software inside a hidden directory (something things
like Plasma also do), where one wonders where all the disk space has gone.
There's also the issue of uninstalling unwanted software, which probably still
isn't solved.
Where I was installing things in a Web application environment, I made pip
install in a designated directory, but I seem to remember having to do a fair
amount of environmental configuration to make sure that Python and Apache were
able to find all the modules. I guess I could have used virtualenv, venv,
pipenv (see what I mean?), but one is left wondering how all of these things
interact with the rest of the system. How does one make Apache "enter" the
virtual environment of whichever flavour, for instance?
(Yes, "containers" exist, but I am reminded of the saying about solving
problems with regular expressions: now you have two problems. Particularly
with the inelegant way Linux provides support for containers.)
With the fundamental Python packaging technology, the one thing that seems to
have improved somewhat is the availability of wheels - whose implementation is
probably heavily underappreciated - which avoids the issue I previously had
where a source package is downloaded by pip and then fails to compile in a bad
way. Twenty or so years ago, people wanted to replicate the CPAN experience
for Python, but I doubt that this was what they meant.
The Python core developers have always complained about distribution packaging
and distributions "ruining the experience for users", all reported rather
selectively, but this is a situation much of their own making. They have
always wanted to do everything themselves, traditionally with the
justification that Windows and the Mac have to be catered for. People used to
band around the term "stop energy", but pandering to Microsoft and Apple
seemed to be a significant source of that energy.
Meanwhile, there are competing but incompatible tools like Conda that do
everything themselves, too. When deploying things conservatively using pip, I
had it suggested to me that I should use Conda. Conda's shared libraries are
incompatible with the way Apache is built on Red Hat systems, so the proposed
"solution" was to run the Web application using the built-in Python Web server
and to proxy it using mod_proxy in Apache. What a joke!
So, the core developers make their own package repository, downloader,
dependency resolver, installer, environment manager. At many an opportunity,
they had the opportunity to leave this particular highway of madness, but they
raised the stakes and now are in the business of effectively running their own
"app" store, all without Big Tech's legions of low-cost workers trying to
prevent malware being part of everyone's next invocation of "pip install".
I could be rather cynical about some of the motivations of people in the core
development community. I see that a few of the discussion participants have
worked for Red Hat and at least one of them was involved in the enthusiastic
deprecation of Python 2 on the basis that people could always pay for
continued support. No coincidence that RHEL is where people will go for that
support, I suppose.
And these discussions have always had the tone of people pontificating about
what "the users" want or do, even speaking for them, and yet never really
engaging with the actual users. Combined with a lack of appreciation that most
users run other software that isn't written in (or for) Python, probably
because Windows doesn't provide such software out of the box. Some actual
users are still running Python 2, believe it or not, regardless of whether
that can be seen from the ivory tower.
In case anyone thinks this rant was a bit unfair, I followed the Python
mailing lists for almost a couple of decades and saw plenty of discussions on
such matters on python-dev. Then again, I think the stewardship record of the
Python language over the last twenty years rather speaks for itself. That is a
statement left for one to interpret in the way one feels most comfortable
with.
Paul
Reply to: