[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Proposal: New source format (was Re: [Fwd: Re: dpkg question])

> Hi,
> 	I like the sdeb idea, but I would like to have byte-for-byte
>  verifiability of upstream sources, (I think we should not rename
>  upstream sources as far as possible).

I don't think everyone understands my idea yet.  Maybe some examples would 
(I apologize if the file sizes don't add up)

--- .sdeb's and .upsdeb's    vs.   .src.rpm's ---

We would have two source packages (.upsdeb and .sdeb) vs. Red Hat's one
package (.src.rpm).

The .sdeb would contain the Debian modifications.  The .upsdeb (for lack of
a better name) would contain the image of the upstream source tarfile (or
whatever) byte-for-byte.  This is similar to how Red Hat's .src.rpm's 
(SRPM's) work, for example:

$ rpm -q -p -l jdk- 

--- .upsdeb's ---

For a .upsdeb (upstream source package), the equivalent would be something
like (to use a real-life example):

$ sdpkg -c jdk_1.0.2.2-2.upsdeb
-rw-r--r-- jim/jim     2102371 1996-12-06 21:16 linux.jdk-1.0.2-pl2.common.tar.
-rw-r--r-- jim/jim      445072 1996-12-06 21:15 linux.jdk-1.0.2-pl2.shared-moti
-rw-r--r-- jim/jim     1475541 1996-12-06 21:16 linux.jdk-1.0.2-pl2.static-moti
-rw-r--r-- jim/jim        1076 1996-12-06 21:16 md5-10-17-96

Notice that it contains an md5 checksum from off of the web site.  In
this case, it hasn't been PGP signed though.

(actually this is too real-life - the jdk-1.0.2.orig.tar.gz off of master 
 contains these files.  Unfortunately, the .diff.gz that goes along with 
 them patches them after they've been unzipped -- which means that 
 dpkg-source -x can't patch them, and it has to be done manually.)

It might also contain a small control file:

$ sdpkg -I jdk_1.0.2.2-2.upsdeb
 debian upstream source package, version 3.0.
 size 3577288 bytes: control archive= 918 bytes.
     447 bytes,    12 lines      upstreamcontrol              
     710 bytes,    11 lines      upstreammd5sums              
     103 bytes,     3 lines      upstreamsites
     156 bytes,     2 lines      upstreamcertificates
     876 bytes,    20 lines   *  upstreamfetch        #!/bin/sh
 Package: jdk
 Architecture: i386
 Maintainer: Jim Pick <jim@jimpick.com>
 Description: Binary distribution of Sun's Java programming language
  and virtual machine.  

So, the .upsdeb would simply wrap the upstream sources, and provide
a place to stick things like md5 checksums and upstream digital
signatures + a list of sites where the identical upstream sources
could be downloaded from (in the sites control file).  Additional
control files may be added.

-- .sdeb's --

There would be an additional package, a .sdeb file, which contains the
Debian packaging code + patches - but not the upstream source:

$ sdpkg -c jdk_1.0.2.2-4.sdeb 
-rw-r--r-- jim/jim        2438 1997-05-08 19:42 patches/patch-common.1
-rw-r--r-- jim/jim        2438 1997-05-08 19:42 patches/patch-shared.1
-rw-r--r-- jim/jim        2438 1997-05-08 19:42 patches/patch-static.1
drwxr-xr-x jim/jim           0 1997-05-10 19:49 debian/
-rw-r--r-- jim/jim         436 1997-05-07 20:18 debian/README.debian
-rw-r--r-- jim/jim        2438 1997-05-07 20:18 debian/changelog
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/control
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/srccontrol
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/srcpostinst
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/srcpostrm
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/builddeb
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/buildsrc
-rw-r--r-- jim/jim        1800 1997-05-07 20:18 debian/buildrpm
-rw-r--r-- jim/jim         176 1997-05-07 20:18 debian/copyright
-rw-r--r-- jim/jim         802 1997-05-07 20:18 debian/java
-rw-r--r-- jim/jim       11749 1997-05-07 20:18 debian/javac.properties
-rw-r--r-- jim/jim         764 1997-05-07 20:18 debian/jdk-common.postinst
-rw-r--r-- jim/jim          53 1997-05-07 21:06 debian/substvars
-rw-r--r-- jim/jim         302 1997-05-07 20:18 debian/jdk-common.postrm
-rw-r--r-- jim/jim          90 1997-05-07 21:01 debian/jdk-static.substvars
-rw-r--r-- jim/jim        1196 1997-05-07 20:18 debian/jdk-shared.postinst
-rw-r--r-- jim/jim          90 1997-05-07 21:05 debian/jdk-shared.substvars
-rw-r--r-- jim/jim         353 1997-05-07 20:18 debian/jdk-shared.prerm
-rw-r--r-- jim/jim        1196 1997-05-07 20:18 debian/jdk-static.postinst
-rw-r--r-- jim/jim         353 1997-05-07 20:18 debian/jdk-static.prerm
-rwxr-xr-x jim/jim        7211 1997-05-07 20:18 debian/rules
-rw-r--r-- jim/jim          27 1997-05-07 20:18 debian/shlibs.local

$ sdpkg -I jdk_1.0.2.2-4.sdeb
 new debian source package, version 3.0.
 size 16624 bytes: control archive= 316 bytes.
     251 bytes,     9 lines      srccontrol              
      70 bytes,     2 lines   *  srcpostinst          #!/bin/sh
      70 bytes,     2 lines   *  srcpostrm            #!/bin/sh
      70 bytes,     2 lines   *  builddeb             #!/bin/sh
      70 bytes,     2 lines   *  buildrpm             #!/bin/sh
      70 bytes,     2 lines   *  buildsrc             #!/bin/sh
 Package: jdk
 Architecture: i386
 SrcDepends: jdk.upsdeb (= 
 Depends: debmake (>=1.95), dpkg-dev (>=1.4.07)
 Builds: jdk-common, jdk-static, jdk-shared
 Installed-Size: 20077
 Maintainer: Jim Pick <jim@jimpick.com>
 Description: Debian source package for Sun's JDK 1.0.2 
  This package will take the 3 original source tar files from Sun,
  patch them, and produce either a .deb file, an .rpm file, or
  this .sdeb file.

Some highlights:  we could have a srcpostinst routine.  This would
be very nice for source packages that require certain things to 
happen when they are first unpacked (like add actions to menus).  
We could also have control file scripts to automatically build the
binary (.deb/.rpm) or source (.sdeb) packages.  For building 
.deb's, it would typically just invoke "dpkg-buildpackage -rsudo" 
(or "build" if using debmake).  This also allows to use 
un-conventional methods to build packages other than just
the debian/rules makefile (think of Java).

Again, an important concept is that the upstream source files 
would be stored in a separate package (or multiple packages).  A
control file field like "SrcDepends:" in the example above
would force sdpkg to ensure that the upstream source package
was installed on the system at the same time as the .sdeb was.
Then the debian/rules script could simply unpack the upstream
sources into the appropriate temporary directory, apply the
patches in the /patches directory, and proceed normally.

Another cool feature is that it can enforce dependencies on
certain packages like "debmake" or "dpkg-dev" (on the developers
machine) before unpacking the .sdeb file.

With this structure, there's no reason the upstream source has
to be a .upsdeb file.  In many cases, it might be easier to
have the source be a Red Hat SRPM (.src.rpm).  In this case,
the "SrcDepends:" field might look like:

 SrcDepends: jdk.src.rpm (=
In this case, the scripts might apply the Red Hat patches, in
addition to the patches in the .sdeb file.

In the future, we might even support something like:

 SrcDepends: java-application.jar (=3.0.1-1)

Where "java-application.jar" is a digitally-signed JAR file
following Sun's specification.

-- .deb's --

The above .sdeb's and .upsdeb's would build .deb's that are in
every sense identical the ones we are using today.

-- .rpm's --

Why not target these too?  We can currently do this even with the
current packaging system, but it makes even more sense when
we can use existing .src.rpm's as our upstream sources.

(in the future, we could even consider targetting "ActiveX"
 controls, since they are essentially packages too, and they
 aren't required to use the Win32 API)

-- Summary --

Let's do it.

It shouldn't be too hard.  Since the .upsdeb's and .sdeb's are
essentially the same thing as .deb's, we can re-use almost all
of the code from dpkg.

Also, it should be trivial to fix the scripts on master for 
putting the source packages into the archive, since Guy is
already doing the same thing with .deb files.  Actually, it
should make the job easier.  We would need an "overrides"
mechanism and a Packages file for the source packages, but
this should essentially be the same as what is going on
for the binary packages right now.

I hope that these examples make my proposal a bit clearer.  It's
pretty tough explaining it since I have such a clear idea in
my head -- I just assume everyone understands it as well as
I do.  :-)


 - Jim

Attachment: pgpPg8irf3ows.pgp
Description: PGP signature

Reply to: