[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[REPORT BACK] Backing up system customization: Is Debian packaging better than Remastersys?

Dear list members

Some months ago I asked the list about my idea to package all sorts of
system customization (programs, scripts, configuration, installed
packages) using the Debian packaging system [1]. I have implemented this
idea and tested it -- now I thought it might be a good idea to share the
results for everyone interested in setting up a similar system.

My use of Debian packaging to distribute system customization

I started with [2] and packaged some applications which are a
"classical" case for packaging: They are built using the normal "make"
and "make install" invocations. Such programs only make a small fraction
of my system customization which is why I quickly started to package
something different: A large collection of HTML documentation was to be
packaged. The problem with "debuild" (which was the recommended
packaging program when following [2]) was that it checked all files with
"lintian" but as the documentation consisted of about 60,000 files this
took too much time. Therefore I used another technique to package
architecture-independent files: I directly created .deb files out of a
prepared directory structure. These packages are not really ready to be
uploaded by a Debian Developer (missing copyright files and such) but
they are way faster to create. In the end, I created six directories for
different types of packaging:

        For binary packages which should contain different files
        depending on the architecture (i386 or amd64). Example: nanozip
        Packages to be compiled with debuild. Of course, the binaries
        vary on different architectures but it is easy to compile first
        on native amd64 and then in an i386 chroot. Example: dhex
        Packages in .deb form from various sources. Example: virtualgl.
        Packages which only consist of dependencies.
        Example: mdvl-base-minimal (depends on what I consider a minimal
        useful Debian installation for desktop and development usage)
        For packages which consist of configuration files and other
        architecture independent data. These are directly compressed to
        .deb files. Example: mdvl-ial (HTML documentation collection)
        A few things that do not fit into any other category. MDPC has
        to be edited in order to do anything with "special" packages.
        Example: dxx-rebirth (a game for which I want to compile
        mutliple binaries for the same architecture).

In the course of setting up the packaging system I decided to use
reprepro to manage the repository: It could be used with simple commands
and there were good usage examples available. Also, I created several
scripts to automate the process, start applications I used in
conjunction with the packaging and setup a suitable chroot to
cross-compile for i386.

At first, these scripts were loosely connected and could be invoked via
several Makefiles distributed over various directories. To further
improve the situation I started to merge all scripts into one central
interface I called MDPC (MDVL Debian Packaging Control Script). This
script can be used to newly create packages, update them or to enter a
specially created chroot and all the other functions which were
previously managed by separate scripts. With MDPC it is not necessary to
remember any specific commands of the Debian Packaging system (like
debuild, dpkg-deb or reprepro) -- all commands are simply invoked with
$ mdpc and suitable arguments.

In case anybody wants to implement a similar system, I have attached
MDPC [3] to this mail. To use it, it is required to generate
archive-signing keys, change the configuration and probably adjust some
settings I made directly in the source code (like the server's
IP-address -- if you want to grep for it). Also, many
"Copyright (c) 2013 Ma_Sys.ma" lines are hard-coded because MDPC was
developed for a specific need. Therefore MDPC can not serve as a
solution to setup an own Debian packaging infrastructure with
zero-effort -- it's use for the community should rather be the source
code which shows how to generate (minimal and simple) Debian packages.

Advantages of using Debian packaging to backup system customization

 * The system can be re-created by installing a single "root" package
   which pulls in the whole system as dependencies on a minimal Debian
 * It is not required to backup machines by copying their whole HDD. The
   repository, user data and a Debian netinst CD are all that is
 * Systems' configuration and software (selections) can be upgraded via
   the normal "aptitude update && aptitude full-upgrade".
 * Customization integrates well with the rest of the system. Everything
   is at it's standard location.
 * System-specific configuration can be managed by if-conditions in
   installation and configuration scripts. Also, there can be specific
   packages which are only installed on special systems (e.g.
   mdvl-special-client for all clients, mdvl-special-server for the main
 * A management scripts automates most interactions with the packaging
   system: I.e. Commands one might foreget are usually automated. Adding
   new packages is rather simple.
 * The "source-code" of the repository is a great documentation of /all/
   system configuration which needs to be installed on systems. Unlike
   package-lists it can never be outdated because all systems are
   upgraded from it and as long as all changes to configuration files
   are recorded in the packagaging system it is not necessary to keep
   track of them by different means (e.g. lists of changed configuration
   files which often become outdated).
 * Ideally, no old/unused/irrelevant configuration files are backed up
   and distributed which can often happen when updating to newer
   releases and doing "full" backups of e.g. /etc.
 * Packages outside the repositories can be seamlessly integrated once
   an own repository was set up.


 * Upgrading to a new Debian version is difficult. It might take very
   long to resolve the dependencies and the
   package-management-utilities' suggestions are not always the way to
   go. Sometimes, packages have to be temporarily removed and installed
   afterwards again.
   This is the mayor and really annoying disadvantage. Especially when
   using aptitude, it can be a challenging game to satisfy the
   dependencies eventhough a test in a VM has shown that the
   dependencies /can/ be resolved. apt-get on the other hand often
   forces some packages to be removed which can be installed afterwards
 * When packages are changed, the changing person has to keep oddities
   of the package-management in mind: E.g. when changing a package with
   symlinks problems might arise that require the package to be removed
   with dpkg --force-depends --purg PACKAGE and installed back with
   aptitude -f afterwards.
 * Moving files between packages is a minor issue: The changed packages
   need to get a suitable "Conflicts:" field -- otherwise the upgrade
 * If a single user is required to remove a package which was part of
   the "root" package's dependencies the package-management-software
   suggests the user to remove almost the whole system because the root
   package is removed and all "automatically installed packgages"
   (dependencies of the root package) are removed with it. However, it
   is also possible to mark the packages as manually installed in such


I am proud of having all system customization in packages: It simplifies
things a lot and whenever I am at a machine which is not the main system
a simple package upgrade installs all the newest configuration files and

But I would not recommend such a setup for everyone: It does probably
not make sense when there are very few machines (eventhough with four
computers it already pays off in my opinion). Also, if each user does
his/her own heavy system customization the package-based setup becomes
hard to manage because to simplyfy installation of a new client there
are many dependencies on a single "root" package.

Finally, the setup of such a system is a lot of work. Even though my
script might help one or the other to get started it sometimes requires
accessing the packaging system directly and it was not designed to be
sort of "user-friendly". Invalid actions might just yield no output
without further notice. For most users who do not regularily setup new
systems (e.g. in virtual machines) the best solution is a normal
HDD-backup by either rsyncing (or something more advanced of that sort)
or even imaging which has the advantage of being easily and safely
restored without any additional thoughts on a complex packaging system.
For those whith less customization, just backing up /etc and /home might
be enough.

For those interested in learning about their system, tinkering with a
lot of scripts or those who need to manage many (might also be virtual)
computers or frequently setup new systems I can definitly recommend a
solution based on or built around Debian packaging.

The "work once, enjoy afterwards" ideal is almost implemented but still
there can be some work once the system is up (especially upon release
upgrade / a test with at least one VM is absolutely required).


[1] Original thread on debian-user

[2] Introduction to Debian Packaging

[3] Ma_Sys.ma Development Linux Debian Packaging Control Script
	Attachment "mdpc_1.0.1.7.tar.gz" (19 KiB)

I hope that this mail was useful to read for at least some members of
the community and that the code of MDPC is also useful for someone apart
from myself.

Thank you for reading that (hopefully not too longish) story.


Attachment: mdpc_1.0.1.7.tar.gz
Description: GNU Zip compressed data

Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: