Hi, Thanks again for your prompt and complete response to this. I'd like to respond to one or two points you brought up. * Hongli Lai <hongli@phusion.nl> [2009-03-05 04:06-0500]: > - The best argument for wanting to depend on Boost dynamically, is to > make it easier to solve security problems. However, upgrading the Boost > library will only partially fix security problems. That's because most > Boost code live in C++ header files, which get inlined directly by the > compiler into the executable. If a security flaw was found in a header > then you'd have to recompile the executable that uses Boost even if > Boost is a shared library. Of course this is true, there is no dispute that you would have to recompile the exeuctable that uses Boost in this case. However, in Debian this issue is actually best handled through the distribution's dependency system. In otherwords, an application, such as Passenger, would be packaged by a Debian developer to have certain meta-data detailed in its debian/control file. The important bits in this case are the Build-Depends line, which specifies what libraries, development headers and associated applications are necessary to properly build the final application. When we find a security issue in headers, such as the Boost library, we can easily find those applications that have properly listed those Build-Depends and rebuild them against the fixed header files. If the application has a hidden set of vulnerable files tucked somewhere deep in its distributed directory structure, we will only be able to rely on either our hand constructed list of embedded code copies, the maintainer being diligent, or some other external mechanism for notification that there is a security issue that might also affect Passenger, due to the embedded code. In this case, someone from the security team will need to tear apart the Passenger source package, dig around until they find the Boost embedded code and then compare the published vulnerability (which is otherwise clearly associated with either Boost version numbers or source code revision numbers for patches) and compare, side-by-side the affected code region and try to make a vulnerability assessment. This is not fun work, and is prone to errors. Its much simplier to get the CVE id from Mitre that details that Boost version X.Y.z has a buffer overflow issue in whatever chunk of code, the fix is in this patch, and we can quite simply determine which packages in our archive need to be rebuilt as a result of the clear dependency declarations. This is all of course if we find out that an application has this embedded copy of the code and that this code has an issue. We of course now know that Passenger has this, but in many situations we find out the hard way, or not at all. > - Most people don't have Boost installed, or don't have the right > version of Boost installed. By far and large, most of our users are > _not_ Debian users, and installing Boost is a huge huge pain for 80% of > our user base. By _not_ bundling Boost we'll alienate most of our users. > I have a different software program which does not bundle Boost, and the > #1 support question by users is related to installing Boost. As a Debian developer, where making this process easy for the user is one of our primary goals, I understand what you are trying to do. Most people don't have pretty much everything installed that every application needs to run, and if they do, do not have the right version installed at all. This is exactly what distributions, such as Debian, manage for the users. When you install a package on Debian, you automatically get all the right dependencies, at the right version, needed to properly run that package. Due to the fact that Passenger is not currently being distributed by distributions like Debian, Ubuntu, or any of the other myriad of other distributions, then people who are running these distributions (and estimates of numbers for Debian alone are well over a million users and this doesn't include Ubuntu which pulls its packages from Debian, or any of the other many distributions which base themselves also on Debian packages) will come to you to get the code and install it, and will appreciate the reduction of depedency resolution that you provide by giving them everything they need. However, if we could get this into Debian, where we have over a decade of experience managing dependencies, then this very large percentage of your user-base now doesn't care at all that you are bundling Boost as a convenience copy, in fact they now are annoyed that you do because of the security problems previously outlined. So when we are talking about transitioning from you playing the role of the distributor, to Debian playing the role of the distributor, this convenience copy of Boost looses its relevance (for these people) and instead becomes a negative, rather than a positive, addition to the package as a whole. Of course, not everyone of your users is a Debian user, or a user of any other distribution that has a dependency system (are there any Linux systems out there that do not declare dependencies still?!). So you are talking about those users who need a convenience copy of other software packages that they need to compile your code without having to download multiple packages. Of course you don't bundle things like Apache, or a binary compiler, because you can make certain assumptions about what stuff people typically have available. I completely understand that you want to help these folks out, in whatever way you can, and I do not want to discourage you from continuing to do so. I think its perfectly fine for you to do this, although its my opinion that if you are going to be shipping a fork of Boost, you should provide a *separate* download of the Boost pieces for people, rather than bundling them all together. If you did this, then issues in Boost could be handled separtely and Debian and other distributions could ship your version of Boost as a dependency, while handling the conflicts with other versions of Boost that an admin might want to have installed. > Even Debian users will have a difficult time. We depend on a very > specific version of Boost, one that hasn't been packaged by Debian yet. It is Debian's policy to use its system to bring in the proper dependencies, instead of using convenience copies, especially if the included code is already in the Debian archive in the form of a library, and if that included copy is not in Debian, it should be packaged as a prerequisite. Its not a difficulty, its what we do, we want to do this because it makes everyone's life easier. > I don't think that telling our Debian users "what? don't have the right > version of Boost installed? then wait x months/years until Debian has > packaged it, then upgrade your distro" is an acceptable answer to our > users, don't you agree? This part is very interesting to me. Have you tried to push your specific Boost modifications upstream to the Boost project? Have you tried and had difficulties, what sort of difficulties? It seems like managing the code deltas between your modified version and theirs will over time become an unnecessary burden that Phusion will have to carry. Getting these modifications (which likely would benefit many people and projects) into Boost would reduce that workload on you folks, and if they were upstream in the original Boost we wouldn't be having this discussion :) > The Fedora guys have tried to patch Phusion Passenger to get rid of the > Boost bundling, but after they're done they found out that Fedora ships > the wrong version of Boost, putting them back at square 1. It seems like you are having this problem with other distributions as well. If Fedora was able to patch Passenger to get rid of the bundling, and all they needed to do was to then get the right version of Boost available (or perahps apply your Boost patch set to the version of Boost that they distribute, if it were not too intrusive), then this issue would be solved. I'd be very interested in seeing what they needed to do to patch Passenger, if there was a way to make such a patch clever enough so that Passenger could include it so that regular users could use the bundled Boost, and distributions could have their policies satisfied, that would be even better. > - We have made all kinds of modifications to Boost, which are specific > Phusion Passenger. If you try to compile Phusion Passenger against a > stock Boost it wouldn't succeed If your specific version of Boost was available to Debian users, as a separate package, this wouldn't be such an issue. Perhaps your modifications to Boost could be included in the Debian package of Boost, even if the upstream Boost maintainers weren't including them (this would serve as an additional source of pressure on the upstream Boost folks to include your code as the Debian folks would also be urging for integration to reduce the code delta). Are these modifications available somewhere for review? > If you have more questions I'd be glad to answer them. But please > understand that we bundled Boost for good reasons, not because we like > imitating Windows. I don't believe you specifically are imitating Windows, I just believe that the Rails community as a whole doesn't understand how this mechanism of doing things is a path towards DLL hell. If you will hear me out on my rant for a second... :) Most scripting languages have their own infrastructure for managing the modules/libraries/pacakges/whatevers dependencies. Perl has CPAN, PHP has PEAR, Ruby has Gems... Although the Rails crowd started off being Unix-centric, the Windows and MacOS world has exerted significant pressure on the community. Gems cater very well to their needs, but on the distribution side, we suffer greatly due to the integration issues. The only sane way that people have managed to stay clear of the "DLL hell" is to do what you folks do, ship everything a given program requires, bundled all together. This results in serious application bloat, sloppy security support, and a total nightmare for upgrades. How many copies of Rails do you have installed on your system? In each of those Rails copies, how many copies of other ruby libraries are bundled in there that you have installed elsewhere (as gems, etc.)? How many of those Rails application trees have Rails itself bundled underneath them? In your gems directory, how many different versions of the same gem do you have installed? How many of those do you actually need? If one has two different gems that use the same binaries, how does that work? Typically, the last installed gem wins. This sounds very much like Windows DLL hell, and this is why I got into the Debian game, I didnt want to waste my life playing dependency resolution games. Playing these games is the path to total madness. This is why Debian, and I am sure other distributions, are begging the Ruby/Rails community to realize this problem and begin supporting simultaneous versions of libraries on a system, this is how most other programming languages work. At one time, this is how C was, but it became totally unmanageable and it *had to change*, I believe that for any programming language, or framework, to succeed over the long run is to figure out how to become friendly to distributions that will bring their amazing applications, their language, and their framework to the larger world in a convenient distribution format, coupled with ease of management and security and piece of mind that they do not have random stuff tucked away in weird places that could result in their system being vulnerable. That essentially requires doing the only sane solution to this problem that has been develped in the last 40 years: get a system that will let a binary, or package use the installed system libraries through a mechanism of stable library versioning, and stop trying to work around an existing package management system. micah
Attachment:
signature.asc
Description: Digital signature