Package, a future replacement for setup.rb and mkmf.rb

[Christian Neukirchen <chneukirchen@gmail.com>]

Hello,

During a discussion on IRC, I started to wonder if Ruby's install
scripts are state of the art, what could be done better and how.

Ruby's mkmf.rb and Aoki's setup.rb probably have their roots in the
oldest pieces of Ruby source still in use.  While setup.rb had some
changes in the latter time, mkmf.rb more or less stayed the same.

I have looked into how other languages install source and compile
extensions, and the library I liked best so far is Python's distutils.
I'm not very familiar with Python, but I like the general approach and
the essence of API.  Basically, you create a file, setup.py, like
this:

  from distutils.core import setup

  setup (name = "Distutils",
         version = "0.1.1",
         description = "Python Module Distribution Utilities",
         author = "Greg Ward",
         author_email = "gward@python.net",
         url = "http://www.python.org/sigs/distutils-sig/";,

         packages = ['distutils', 'distutils.command'])

In Ruby, this would maybe look like that:

  require 'package'

  setup {
    name "Distutils"
    version "0.1.1"
    description "Python Module Distribution Utilities"
    author "Greg Ward"
    author_email "gward@python.net"
    url "http://www.python.org/sigs/distutils-sig/";

    packages ['distutils', 'distutils/command']
  }

Given this file, we can simply run:

  python setup.py install

and the files will get installed where they belong to.  distutils can
also handle different prefixes, installing into home directories, and
complex cases like putting scripts to /usr/bin, but libraries to
/opt/local and whatever.

Python's distutils also handles compiling extensions:

  name = 'DateTime.mxDateTime.mxDateTime'
  src = 'mxDateTime/mxDateTime.c'
  setup (
    ...
    ext_modules =
      [(name,
       { 'sources': [src]
         'include_dirs': ['mxDateTime'] }
      )]
  )

Here, something like this would be possible in Ruby (I'm not yet sure
about exact semantics of the Python version):

  setup {
    # ...
    extension("DateTime/mxDateTime/mxDateTime") {
      sources "mxDateTime/mxDateTime.c"
      include_dirs "mxDateTime"
    }
  }

Of course, more complex build descriptions can be represented too:

  extension("foolib") {
    sources "foo.c", "bar.c"
    if have_library("foo", "fooble")
      define "HAVE_FOO_H"
      cflags << `foo-config --cflags`
      ldflags << `foo-config --libs`
    else
      fail "foolib is needed"
    end
  }

Whether this will generate a Makefile (like mkmf.rb), a Rakefile
or compile directly (like distutils) is still an open question.

To allow for an easy conversion of setup.rb usage, Package will
provide convenience methods that will make it behave like setup.rb
with respect to the directory structure.

Package doesn't try to conquer the world, however, it aims to be just
a tool that would be useful if it was standard and everyone could
build on due to it's policy-neutrality

What advantages will Package have over setup.rb and mkmf.rb, as
they are now?

* simple, clean and consistent working
* unified library to handle both extensions and libraries
* lightweight approach (if included in the standard library)
* easy adaption
* more flexible directory layout: especially small projects
  profit from this, as setup.rb's directory layout is quite
  bulky by default and not very customizable
* easier packaging by third-party packagers due to simple
  but flexible and standardized invocation

What do we need to get a wide adoption of Package?

* inclusion in the standard library so it doesn't need to be
  shipped with every package (as setup.rb unfortunately is).
* backing from the community to make use of Package.
* acceptance from packaging projects like RPA, RubyGems and
  distributions like Debian, FreeBSD and PLD.

Coding of Package has not started yet (the name is also not set into
stone yet, so if you have better ideas, please tell me) because it
would be pointless without a strong feedback from the community.  I
expect to get a first version done rather quickly, possibly borrowing
code from setup.rb and mkmf.rb, but Package will not depend on these
both.  If anyone is interested in helping development, please mail me;
helpful hands are always of use.  Also, there will be need for testers
on all and even the most weird platforms.

But now, I'll ask you: Are you satisfied with the way installing Ruby
extensions and libraries works?  Do you think there is a place for
Package?  Do you have further improvements or can provide alternative
ideas?

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org