[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#488753: Boost bundling


Thanks again for your prompt and complete response to this. I'd like to
respond to one or two points you brought up.

* Hongli Lai <hongli@phusion.nl> [2009-03-05 04:06-0500]:
> - The best argument for wanting to depend on Boost dynamically, is to  
> make it easier to solve security problems. However, upgrading the Boost  
> library will only partially fix security problems. That's because most  
> Boost code live in C++ header files, which get inlined directly by the  
> compiler into the executable. If a security flaw was found in a header  
> then you'd have to recompile the executable that uses Boost even if  
> Boost is a shared library.

Of course this is true, there is no dispute that you would have to
recompile the exeuctable that uses Boost in this case. However, in
Debian this issue is actually best handled through the distribution's
dependency system. In otherwords, an application, such as Passenger,
would be packaged by a Debian developer to have certain meta-data
detailed in its debian/control file. The important bits in this case are
the Build-Depends line, which specifies what libraries, development
headers and associated applications are necessary to properly build the
final application.

When we find a security issue in headers, such as the Boost library, we
can easily find those applications that have properly listed those
Build-Depends and rebuild them against the fixed header files. If the
application has a hidden set of vulnerable files tucked somewhere deep
in its distributed directory structure, we will only be able to rely on
either our hand constructed list of embedded code copies, the maintainer
being diligent, or some other external mechanism for notification that
there is a security issue that might also affect Passenger, due to the
embedded code. 

In this case, someone from the security team will need to tear apart the
Passenger source package, dig around until they find the Boost embedded
code and then compare the published vulnerability (which is otherwise
clearly associated with either Boost version numbers or source code
revision numbers for patches) and compare, side-by-side the affected
code region and try to make a vulnerability assessment. This is not fun
work, and is prone to errors. Its much simplier to get the CVE id from
Mitre that details that Boost version X.Y.z has a buffer overflow issue
in whatever chunk of code, the fix is in this patch, and we can quite
simply determine which packages in our archive need to be rebuilt as a
result of the clear dependency declarations. 

This is all of course if we find out that an application has this
embedded copy of the code and that this code has an issue. We of course
now know that Passenger has this, but in many situations we find out the
hard way, or not at all.

> - Most people don't have Boost installed, or don't have the right  
> version of Boost installed. By far and large, most of our users are  
> _not_ Debian users, and installing Boost is a huge huge pain for 80% of  
> our user base. By _not_ bundling Boost we'll alienate most of our users.  
> I have a different software program which does not bundle Boost, and the  
> #1 support question by users is related to installing Boost.

As a Debian developer, where making this process easy for the user is
one of our primary goals, I understand what you are trying to do. Most
people don't have pretty much everything installed that every
application needs to run, and if they do, do not have the right version
installed at all. This is exactly what distributions, such as Debian,
manage for the users. When you install a package on Debian, you
automatically get all the right dependencies, at the right version,
needed to properly run that package. Due to the fact that Passenger is
not currently being distributed by distributions like Debian, Ubuntu, or
any of the other myriad of other distributions, then people who are
running these distributions (and estimates of numbers for Debian alone
are well over a million users and this doesn't include Ubuntu which
pulls its packages from Debian, or any of the other many distributions
which base themselves also on Debian packages) will come to you to get
the code and install it, and will appreciate the reduction of depedency
resolution that you provide by giving them everything they
need. However, if we could get this into Debian, where we have over a
decade of experience managing dependencies, then this very large
percentage of your user-base now doesn't care at all that you are
bundling Boost as a convenience copy, in fact they now are annoyed that
you do because of the security problems previously outlined. So when we
are talking about transitioning from you playing the role of the
distributor, to Debian playing the role of the distributor, this
convenience copy of Boost looses its relevance (for these people) and
instead becomes a negative, rather than a positive, addition to the
package as a whole.

Of course, not everyone of your users is a Debian user, or a user of any
other distribution that has a dependency system (are there any Linux
systems out there that do not declare dependencies still?!). So you are
talking about those users who need a convenience copy of other software
packages that they need to compile your code without having to download
multiple packages. Of course you don't bundle things like Apache, or a
binary compiler, because you can make certain assumptions about what
stuff people typically have available. I completely understand that you
want to help these folks out, in whatever way you can, and I do not want
to discourage you from continuing to do so. I think its perfectly fine
for you to do this, although its my opinion that if you are going to be
shipping a fork of Boost, you should provide a *separate* download of
the Boost pieces for people, rather than bundling them all together. If
you did this, then issues in Boost could be handled separtely and Debian
and other distributions could ship your version of Boost as a
dependency, while handling the conflicts with other versions of Boost
that an admin might want to have installed.

> Even Debian users will have a difficult time. We depend on a very  
> specific version of Boost, one that hasn't been packaged by Debian yet.

It is Debian's policy to use its system to bring in the proper
dependencies, instead of using convenience copies, especially if the
included code is already in the Debian archive in the form of a library,
and if that included copy is not in Debian, it should be packaged as a
prerequisite. Its not a difficulty, its what we do, we want to do this
because it makes everyone's life easier.

> I don't think that telling our Debian users "what? don't have the right  
> version of Boost installed? then wait x months/years until Debian has  
> packaged it, then upgrade your distro" is an acceptable answer to our  
> users, don't you agree?

This part is very interesting to me. Have you tried to push your
specific Boost modifications upstream to the Boost project? Have you
tried and had difficulties, what sort of difficulties? 

It seems like managing the code deltas between your modified version and
theirs will over time become an unnecessary burden that Phusion will
have to carry. Getting these modifications (which likely would benefit
many people and projects) into Boost would reduce that workload on you
folks, and if they were upstream in the original Boost we wouldn't be
having this discussion :)

> The Fedora guys have tried to patch Phusion Passenger to get rid of the  
> Boost bundling, but after they're done they found out that Fedora ships  
> the wrong version of Boost, putting them back at square 1.

It seems like you are having this problem with other distributions as
well. If Fedora was able to patch Passenger to get rid of the bundling,
and all they needed to do was to then get the right version of Boost
available (or perahps apply your Boost patch set to the version of Boost
that they distribute, if it were not too intrusive), then this issue
would be solved.

I'd be very interested in seeing what they needed to do to patch
Passenger, if there was a way to make such a patch clever enough so that
Passenger could include it so that regular users could use the bundled
Boost, and distributions could have their policies satisfied, that would
be even better.

> - We have made all kinds of modifications to Boost, which are specific  
> Phusion Passenger. If you try to compile Phusion Passenger against a  
> stock Boost it wouldn't succeed

If your specific version of Boost was available to Debian users, as a
separate package, this wouldn't be such an issue. Perhaps your
modifications to Boost could be included in the Debian package of Boost,
even if the upstream Boost maintainers weren't including them (this
would serve as an additional source of pressure on the upstream Boost
folks to include your code as the Debian folks would also be urging for
integration to reduce the code delta). Are these modifications available
somewhere for review?

> If you have more questions I'd be glad to answer them. But please  
> understand that we bundled Boost for good reasons, not because we like  
> imitating Windows.

I don't believe you specifically are imitating Windows, I just believe
that the Rails community as a whole doesn't understand how this
mechanism of doing things is a path towards DLL hell. If you will hear
me out on my rant for a second... :)

Most scripting languages have their own infrastructure for managing the
modules/libraries/pacakges/whatevers dependencies. Perl has CPAN, PHP
has PEAR, Ruby has Gems... Although the Rails crowd started off being
Unix-centric, the Windows and MacOS world has exerted significant
pressure on the community. Gems cater very well to their needs, but on
the distribution side, we suffer greatly due to the integration
issues. The only sane way that people have managed to stay clear of the
"DLL hell" is to do what you folks do, ship everything a given program
requires, bundled all together. This results in serious application
bloat, sloppy security support, and a total nightmare for upgrades. How
many copies of Rails do you have installed on your system? In each of
those Rails copies, how many copies of other ruby libraries are bundled
in there that you have installed elsewhere (as gems, etc.)? How many of
those Rails application trees have Rails itself bundled underneath them?
In your gems directory, how many different versions of the same gem do
you have installed? How many of those do you actually need?

If one has two different gems that use the same binaries, how does that
work? Typically, the last installed gem wins. This sounds very much like
Windows DLL hell, and this is why I got into the Debian game, I didnt
want to waste my life playing dependency resolution games.

Playing these games is the path to total madness. This is why Debian,
and I am sure other distributions, are begging the Ruby/Rails community
to realize this problem and begin supporting simultaneous versions of
libraries on a system, this is how most other programming languages

At one time, this is how C was, but it became totally unmanageable and
it *had to change*, I believe that for any programming language, or
framework, to succeed over the long run is to figure out how to become
friendly to distributions that will bring their amazing applications,
their language, and their framework to the larger world in a convenient
distribution format, coupled with ease of management and security and
piece of mind that they do not have random stuff tucked away in weird
places that could result in their system being vulnerable. That
essentially requires doing the only sane solution to this problem that
has been develped in the last 40 years: get a system that will let a
binary, or package use the installed system libraries through a
mechanism of stable library versioning, and stop trying to work around
an existing package management system.


Attachment: signature.asc
Description: Digital signature

Reply to: