Re: PyCon 2013 -- tentative title/abstract/outline -- feedback plz - v.2
ok -- here is my next take trying to make at least the title and introduction
more user oriented and mention those aspects which might be be of interest for
developers... As a result it probably became even less English and now
exercises working memory even harder ;)
Propelling Python to the masses with the universal OS
Python has found appreciation not only among professional developers
but also among students, scientists and programming novices due to its
scripting nature, "batteries included", good collection of 3rd party
libraries, and ability to interface to libraries written in other
languages and computing environments (e.g. R). To conveniently
deliver such a versatile Python platform to users (and their humble
system administrators), the community have been distilling the
ultimate Python distribution utilities and bundling pre-built Python
and core 3rd party libraries and modules for the distribution on
proprietary systems. Meanwhile nearly for two decades Python has been
a part of the largest community-driven software distribution platform
-- Debian. The Debian project delivers a complete operating system
with tens of thousands of FOSS projects available on 11 hardware
architectures and 3 different kernels (Linux, HURD, kFreeBSD). Being
a binary distribution Debian guarantees safer -- free of build-errors
-- installations and seamless upgrades. Coupled with the standardized
specification of build and run-time dependencies, it made it easy to
build, verify, or simply deploy projects of nearly arbitrary
complexity of inter-dependencies and varying implementation origins.
Such agnosticism to the origins of the software made Python-based
products a 1st citizen in this heterogeneous distribution ecosystem,
assuring that Python works well with the rest of it. Recent advances
in hardware virtualization support, followed in tandem with the
explosion of cloud solutions, made Debian systems popular not only
among Linux "fan-boys" but for various, especially scientific and
community-driven, deployments. The ease with which thousands of
Python-based FOSS became available and maintainable made Debian the
Python distribution with "**all** batteries (and virtualenv)
included".
In this talk I would like to briefly present the history of Python in
Debian (which can be traced to nineties with Python 1.4), outline
benefits Debian provides for Python users/developers and present what
to expect in upcoming stable (wheezy) release of Debian. To
familiarize listeners with Python-in-Debian ecosystem I will then
overview core package naming, versioning, and modularization
conventions in Debian and ongoing QA efforts (build-time testing,
full-archive rebuilds, etc). I will briefly present the "Debian
packaging" helper tools, including recent GSOC project aiming to
provide automatic packaging of the packages on PyPI. To facilitate
the synergy between Python and Debian communities, I will accent on
common sense practices (following PEPs, clean and exhaustive legal
terms, CI, etc.) which would make any Debian packaging and
maintainership more efficient and benefit upstream developers. I am
planing to conclude by presenting few easy ways on how to start using
Debian.
As the outcome of the talk, I expect listeners to become more familiar
with the Debian project goals, standards and principles, become aware
of integration aspects involved in delivering such plethora of Python
FOSS solutions, and become intrigued enough to try Debian on their
systems or in the cloud.
On Fri, 28 Sep 2012, Paul Boddie wrote:
> On Friday 28 September 2012 00:23:10 Yaroslav Halchenko wrote:
> > Thank you Paul ;-)
> > Good comments -- once again, arguments seems to be oriented mostly
> > toward developers... I guess I should explicitly guide the
> > abstract more toward 'user-' and "sysadmin-" use cases: people
> > in need to have easy and uniform paths for software installation and
> > maintenance of the heterogeneous system. In scientific users domain it
> > becomes even more fun with heavier reliance on computational and I/O
> > libraries (blas/atlas, hdf5, etc) building and maintaining which might
> > be quite a bit of a hassle.
> Yes, I had cause to build NumPy from scratch recently, and it was quite
> intimidating. It did happen to involve a fairly low-performance device with
> fairly severe memory constraints, and the experience really pushed me towards
> looking harder at Debian and the packaging work that I knew would have been
> done to potentially solve some of the issues I was experiencing, one of which
> being modularisation of the library, although I'm not sure how much this is
> actually done with NumPy in Debian.
> > Few inline comments.
> > > I was going to give some feedback more as the kind of person who has gone
> > > to Python conferences, and certainly, if you want a native speaker to
> > > give feedback on the phrasing of your proposal, I'd be happy to make some
> > > suggestions.
> > I would appreciate "native speaker" feedback! since "accepting all
> > types of proposals through September 28, 2012", I guess I have the whole
> > tomorrow to revise and submit. I hope to find some time later today to
> > revise my abstract and will post it again for further phrasing
> > suggestions
> I'll try and follow the list and get back to you.
> > That is true... Somewhat offtopic -- that is why with neuro.debian.net
> > we pretty much serve an unofficial backports repository for a good
> > portion of Python modules we maintain. Besides immediate benefit for
> > users, benefit from backporting for developers has been build-time
> > testing across various releases of Debians and Ubuntus, picking out
> > problems with specific versions of the core libraries... So, may be I
> > should add an accent that availability in Debian doesn't only guarantee
> > ease of installation (for users) but provides a good test bed for the
> > developers to preclude problems with future deployments on Debian-based
> > platforms... ?
> Having gone through the packaging process, I know that a lot of the hurdles
> actually do help packagers improve the reliability of the eventual package,
> even though the process can be drawn out and frustrating. Even non-technical
> stuff like auditing the licences is a valuable activity, however, that gives
> users confidence that the end product is of high quality and not just zipped
> up and uploaded to some site or other.
> The Python conference scene seems to love testing, so if you can make a case
> for Debian and quality assurance, and Debian has done things popular with
> this crowd for years like automated builds and the use of very strict package
> building tools that won't let you build without a precise specification of
> the build dependencies, then that may appeal to some people.
> > > Python packaging has become somewhat insular over the years with
> > > Python-centric solutions that work across different systems rather than
> > > solutions that work well with the rest of the software on particular
> > > systems. However, people appear to like things like virtualenv,
> > > especially the Web crowd that makes up a lot of the audience at events
> > > like this, because it lets them set up relatively cheap configurations
> > > for separate Web applications or for experimenting.
> > virtualenv is indeed great for the reasons you guys point out AND
> > indeed, it is very Python-centric and maintenance of a configured
> > virtualenv might become cumbersome for projects with lots of 3rd party
> > dependencies and for regular users who would not want to care to switch
> > among different virtualenvs etc.
> It's a Python tool for Python developers, primarily. If you're running a Web
> operation with lots of Python developers and a commitment to Python then it
> may make sense, but that sort of builds a wall that deters people from
> adopting Python, too.
> > I guess I should revise abstract to aim a listener wondering "why should
> > I care about Debian if there is virtualenv" WITHOUT explicitly pointing
> > to its pros to not cause any flames. And not sure I would be able to
> > convince hard-core Python-ians, so I might not even try and orient
> > it more toward users/admins.
> I don't think you should worry too much about flames. My impression is that
> the packaging people are trying to scale back their ambitions and just get
> everyone to do the basic things like write decent metadata, mostly because I
> think the process of delivering a comprehensive solution is deadlocked once
> again, and I think that people do see the need to hear from distributions and
> to try and get as much input from them as possible.
> > > I have advocated solutions based on fakechrooted debootstrapped
> > > installations
> > btw -- how is it working out for you? i.e. are you still pushing it
> > forward?
> Not really, mostly because I was frequently using them for accessing newer
> distributions, and then fakechroot wasn't quite capable enough to avoid
> low-level library conflicts. So I now routinely use these installations as
> genuine chroots instead, and I can convert them to run as User Mode Linux
> installations, but I suppose the principle still stands. :-)
> (For my packaging activities, I actually used squeeze under UML to launch a
> sid chroot, since my host kernel is not modern enough for sid. I could
> probably just launch sid under UML from an installer image, but that's
> something for another time.)
> > > if only because you can manage the libraries below the Python modules and
> > > extensions as well as the stuff that supports things like distutils and
> > > setuptools. However, the people who can change this situation don't see
> > > the need or the point: it's either "but I have root!" or "they can always
> > > build
> > many (users on managed boxes) -- don't, so I would have pushed these
> > approaches for them as well ;)
> You can certainly make the case to anyone working in a large organisation.
> [...]
> > > The one case that many language-focused groups ignore, and where
> > > distributions do well, is the case where a range of different
> > > technologies needs to be managed and where administrators just wouldn't
> > > be able to keep up with Python eggs, Ruby gems, CPAN, and the
> > > language-specific technology of the week. Persuading the Python community
> > > to feed packages into Debian so that they become a safer choice for
> > > people who routinely use or know other technologies is definitely a
> > > worthwhile cause.
> > indeed safer and more accessible choice.
> I actually have to do this with RHEL at work, but the point stands: if you
> can't rely on a stream of packages maintained by someone else, your ability
> to deploy and manage a hetergeneous suite of applications is impaired. And
> the result of that may involve you deploying a bunch of not so great
> applications instead. That latter part is arguably a difficult point to make
> to an audience who may be convinced that all the best tools are written in
> Python and work perfectly with virtualenv, pip, easy_install or whatever, but
> it matters to potential users of Python software, and it may end up mattering
> to them if their bosses actually prefer Perl, Ruby or PHP.
> So you have something to really scare people with if you wanted to. ;-)
> Paul
--
Yaroslav O. Halchenko
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
Reply to: