[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bits from the dpkg project: 1.17.x series, general news


This is an update on dpkg development, with general updates, and a broad
summary of new features (mostly interface additions; see the man pages
for further information), user visible or very significant changes since
the last announce [A] covering the dpkg 1.16.x series.

[A] <https://lists.debian.org/debian-devel-announce/2015/03/msg00011.html>

General News

* Raphaël Hertzog has stepped down as maintainer.

* The debsig-verify and dpkg-repack packages have been taken over under
  the “Dpkg Developers” umbrella, to be hopefully assimilated into dpkg
  proper in the future.

* The wiki [W] has been revamped; there's a FAQ [F] now too, and pages
  listing source [S] and binary [B] package format support, among others.

  [W] <https://wiki.debian.org/Teams/Dpkg>
  [F] <https://wiki.debian.org/Teams/Dpkg/FAQ>
  [S] <https://wiki.debian.org/Teams/Dpkg/DscSupport>
  [B] <https://wiki.debian.org/Teams/Dpkg/DebSupport>

* Commit message mailing list back alive: debian-dpkg-cvs@lists.debian.org

  The list has been silent (due a change in the commit notification
  scripts not matching the list setup) for the duration of the 1.17.x
  series, it should be now back to normal.

  I'm undecided if bouncing missing mails would be useful, as it's unlikely
  anyone would go through the flood? Please let me know. Alternatively you
  can also subscribe for VCS updates through the PTS/tracker.

* Branches now named after the series versions.

  For downstream distributors it's not always easy to know which Debian
  release matches what dpkg series, so now the branches are named based
  on the series versions, backward compatibility refs have been created.

  The new names are 1.13.x (etch), 1.14.x (lenny), 1.15.x (squeeze),
  1.16.x (wheezy) and 1.17.x (jessie).

* Porting system access or automated builds sought.

  Portability in dpkg is very important, to be able to support downstreams
  and people who use it on other systems, or even package it in other
  (non-GNU/Linux) distributions. But for many such systems I'm currently
  porting purely through documentation. And as such, subsequent build and
  run-time issues are clearly reactive, but I'd like to switch to a more
  proactive model. So I'd very much appreciate if either interested
  parties could provide access to such systems, or setup some kind of
  continuous integration system from git. I'm thinking specifically of
  systems such as non-glibc based Linux, FreeBSD, NetBSD, OpenBSD, Minix,
  Solaris, Mac OS X, HP-UX and AIX.

1.17.x (Debian Jessie)


* Many memory and file descriptor leaks, out-of-bounds, non-security
  sensitive TOCTOU races, and use-after-free fixes.
* Many error messages have been improved.
* Major formatting and text organization cleanup of man pages.
* All databases with user data are now backed up by the dpkg cron job.
* Some non-controversial changes have been merged that remove
  unreproducibility from the source and binary packages build process.
* Add support for versioned Provides [!]:
  - Packages can provide a specific version, “virtual (= 1.0)” which will
    be honored, previously it would just be accepted when parsing.
  - Non-versioned virtual packages will not satisfy versioned dependencies.
  - Versioned virtual packages will satisfy non-versioned dependencies.
* Switch the dpkg files database string hashing function from what appears
  to be a custom hash function to the libdpkg FNV-1a implementation.
* Double the dpkg files database hash table size to the closest 2^18 prime.
* Improved test suite setup and coverage [C] (although dpkg-test.git does
  not yet have a coverage report).

[C] <https://dpkg.alioth.debian.org/coverage/>


* Improvements to the dpkg build system:
  - Check availability of warning flags at build time.
  - Support non-GCC compilers by default.
  - Allow to override the run-time GNU tar binary to use, at build time.
* Do not assume sensible-editor is present in «dpkg-source --commit».
* Use makedev(3) when extracting .deb archives rather than ad-hoc
  computations, to be able to support large major/minor device numbers,
  supported on at least Linux, NetBSD and OpenBSD based systems.
* Fixes for Mac OS X, BSD, Android and uclibc based systems, to reduce
  the delta they are carrying.

Changes in dpkg.deb


* Bump GNU triplets: any-i386 now maps to i586-*-gnu.
* New CPUs: any-or1k, any-mips64, any-mips64el, any-powerpcel, any-ppc64el.
* New GNU/Linux systems: mipsn32 and mipsn32el.
* Remove GNU/Linux systems: lpia.
* New Linux systems: musl-linux-any.
* New BSD systems: dragonflybsd-any.
* Drop the archtable file, it was Debian-archive specific and serves no
  purpose anymore.


* Activate file triggers on removal and disappearance more accurately,
  only when we know we are inevitably removing things.
* Add support for new «interest-await» and «activate-await» trigger
* The recently introduced deferred trigger processing when packages do
  not fulfill dependencies has been disabled (only in the 1.17.x series)
  as it was still giving upgrade problems so close to the Debian release.


* New --ctrl-tarfile command.
* More strict checking of non-conformant .deb archives.
* New --deb-format option that replaces the --new and --old options.
* Switch default compressor from gzip to xz. (Downstreams can select a
  different default at dpkg build time using --with-dpkg-deb-compressor.)
* Deprecate bzip2 as a compressor.
* Add support for gzip compression strategies («filtered», «huffman»,
  «rle» and «fixed»).
* Add support for .deb archives with a control member not compressed
  (control.tar) or compressed with xz (control.tar.xz).
* Add support for uniformly compressed .deb archive members, with new
 --uniform-compression option (this might become the default in 1.18.x).
* Change «-Zgzip -z0» to generate non-compressed members (instead of
  uncompressed members with a .gz extension), as documented.
* Reset environment variables affecting compressor commands when not using
  the shared library implementations. Namely XZ_DEFAULTS, XZ_OPT, BZIP and
* Change --field to use libdpkg's deb822 parser.
* Change conffile name length warning into an error, as dpkg will reject
  those packages at install time anyway.
* Remove arbitrary filename limits from dpkg-deb when processing .deb
  archive contents.
* Do not warn anymore on user-defined field names.


* Correctly support multibyte strings on --list output, fixing unaligned
  columns and multiple mojibake issues.
* New virtual fields db:Status-Want, db:Status-Status and db:Status-Eflag.


* New --await option (which was and is currently the default).


* Change default implicit package name arguments to be arch-qualified,
  which works better for Multi-Arch:same packages, but can cause problems
  for non-Multi-Arch:same ones.
* New dir_to_symlink and symlink_to_dir commands.


* Improve SE Linux support:
  - The label database is reloaded if it changes during a package upgrade.
  - Maintainer scripts get their own «dpkg_script_t» execution context.
* Installation, removal and purging now always reset the want status, so
  this now resets the holds.
* Control file parser is more strict and does not allow several broken
  constructs or operations, like:
  - Empty field names.
  - Fields starting with a hyphen (which do not mix well with OpenPGP
  - Matching partial field names.
* On removal, check Depends and Pre-Depends fields for packages in
  unpacked and half-configured states too.
* Support a read-only root directory with a read-write overlay or a
  symlink farm, by not removing it or its backups.
* Remove arbitrary package conflictor limits (was 20, now “unlimited”).
* Try to preallocate disk size for extracted files on unpack. This should
  help avoid filesystem fragmentation and possibly improve performance on
  “new” filesystems.
* Improvements to the progress reporting messages.
  - Now all progress messages always print the package versions.
  - Now the “Preparing to unpack” and “Unpacking …” messages are symmetric.
* Pass DPKG_MAINTSCRIPT_PACKAGE_REFCOUNT to maintainer scripts.
* Change directory to «/» before executing maintainer scripts.
* Add invoke hooks for dpkg --add-architecture and --remove-architecture.
* New --assert-versioned-provides command.
* New --verify command, and --verify-format option, with ‘rpm’ as the only
  current value supported, but the default might change in the future.
* Change --audit command to allow per-package checks.
* Change --update-avail and --merge-avail commands to allow getting
  Packages-files from standard input if the argument is omitted or is ‘-’.
* Make --set-selections warn that it needs an up-to-date «available»
  database when it gets passed unknown packages.
* Do not write to the «available» database anymore during unpack.


* Remove this transitional wrapper. Systems should have been switched
  to the GNU install-info package implementation by now.


* More strict parsing of command-line arguments (e.g. fatal errors on out
  of range priorities).
* Use the current alternative link as the first best value to avoid
  flip-flops of alternatives with equal priority.


* New --pid and --ppid match options.
* New --remove-pidfile option.
* Use /proc/PID/status instead of /proc/PID/stat when using Linux procfs.
* Do not require /proc to be mounted on kFreeBSD systems, as linprocfs is
  not the native procfs on kFreeBSD, and programs on FreeBSD do not expect
  any procfs to be present anyway.
* Use a native kFreeBSD sysctl(3) method instead of using KVM to check for
  the executable.
* Add support for DragonFlyBSD.

Changes in dselect.deb

* New architecture columns in package list view. The new columns, shown
  by default, can be turned off with the new ‘A’ key, or bound to another
  key via the new “archdisplay” keybinding.

Changes in libdpkg-perl.deb

Perl Modules

* Drop obsolete DM-Upload-Allowed support.
* Warn on usage of deprecated Source-Version substvar.
* Add support for GnuPG 2.x, gpg2/gpgv2 are preferred if installed,
  otherwise gpg and gpgv are used instead.
* Deprecate lowercase and exported by default variables from all modules.
* All public modules (ones with a version >= 1.0 in the CHANGES section)
  are documented now. Private modules that have documentation should
  have a version 0.xx in the CHANGES section.
* If a vendor does not have a Dpkg::Vendor module, try loading one of
  its ancestors, before falling back to Dpkg::Vendor::Default.

Changes in dpkg-dev.deb


* Now -O option in dpkg-genchanges, dpkg-gensymbols and dpkg-shlibdeps
  always takes an optional filename argument.
* New -g and -G build type options, and only one different build type
  option is allowed now in dpkg-genchanges and dpkg-buildpackage.

Build Profiles

This [P] will allow to conditionally restrict build-time dependencies and
build parameters, with expressiveness similar to Gentoo USE flags.

  [P] <https://wiki.debian.org/BuildProfileSpec>

* Add support for build-time restriction formulas in dependencies
  enclosed in “<>”, as a disjunctive normal form expression:
    “pkgname (>= version) [arch-list] <foo> <bar baz> <!blub>”.
* New -P option in dpkg-checkbuilddeps and dpkg-buildpackage.
* Add support for DEB_BUILD_PROFILES environment variable.
* New Built-For-Profiles output field in .deb and .changes files.


* Add support for cross-compiler target system information via the new
  DEB_TARGET_ family of variables, and new -A and -T options to override
  defaulting to the host system.
* Add architecture restriction options for -L command. This allows to
  select specific subsets of all valid known architectures, matching by
  wildcard (-W), endianness (-E) or bits (-B). The restricting options
  can be combined, or omitted altogether.
* Now has saner command-line parsing so that «--option=value»,
  «--option value», «-ovalue» and «-o value» will all be accepted.
* Add long option names for all short options.


* Change default source package compressor for new formats (>= 2.0) to xz.
* Source package building is more strict for 3.0 formats, the version has
  to match the format (native vs non-native).
* Source package parsing is more strict, disallowing several attack
  vectors, this might break packages with bogus patches.
* Use «tar --format=gnu» when creating source archives.
* New --ignore-bad-version extraction option.
* Add --build and --extract command aliases (to the existing -b and -x).
* Allow detached upstream signatures for upstream orig.tar files in the
  .dsc file.
* Add support for Testsuite source field (it is added automatically to
  .dsc if missing from debian/control and a debian/test/control is found).


* New --show-field option.
* Add support for reading from standard input when using «-l-».
* Add support for parsing compressed changelog files.


* New -l option to replace having to abuse LD_LIBRARY_PATH to pass
  build-time paths.
* Honor Build-Depends-Arch for minimal version checks.


* Add support for Ignore-Blacklist-Groups field in symbols files, with
  the two available group values «aeabi» and «gomp».
* Turn the ARM Embedded ABI symbols blacklist into a regex, to stop having
  to keep up with the GNU toolchain, or other toolchains emitting different
* Blacklist GOMP critical section symbols.


* Warn when using the deprecated -is/-ip/-isp/-ips options.


* Add Architecture and Build-Profiles information to Package-List field,
  as key-value entries.


* Rename --export «configure» argument to «cmdline» (but preserve
  «configure» as a legacy alias).
* Mask «fortify» hardening option from output on «noopt» (glibc 2.16 and
  later issue a warning on this condition).
* Add support for «pie» and «stackprotector» to FFLAGS.
* Add support for GCJFLAGS, FCFLAGS, OBJCFLAGS and OBJCXXFLAGS build flags.
* Add support for new hardening feature «stackprotectorstrong».
* Add support for new qa feature area, with features «bug» and «canary».
* Add support for new reproducible feature area, with feature «timeless».


* New --force-sign option.
* UNRELEASED uploads are not signed by default anymore.
* Honour DEB_SIGN_KEYID environment variable.
* Move signing to the end of the build.
* Detect a missing gain-root-command even if running as root.
* Detect a missing sign-command before starting the build, to avoid a
  failure at the end of the process.
* Add shell hooks support, based on the debuild implementation in
  devscripts 2.13.9.
* New --check-command and --check-option options, and use
  DEB_CHECK_COMMAND environment variable as a default value, to specify
  a package checker (e.g. lintian) to use before the signing process.
* Add support for automatic parallel job selection with «-jauto»,
  matching currently active processors.
* New --host-arch, --host-type, --target-arch and --target-type options,
  which will be passed through to dpkg-architecture.


* New --hash option, to enable generating only specific file checksums.

1.18.x (Debian Stretch)

It is opening up in experimental or sid soon. Exciting things (at least
to people that can get excited by package management stuff :) will be
appearing in these series, but I'd rather not give spoilers just yet.

I'll also start sending some proposals and RFCs to the debian-devel and/or
debian-dpkg mailing lists shortly.

[!] This one got a 4-digit bug number (#7330)!

Attachment: signature.asc
Description: Digital signature

Reply to: