Re: debtags support proposed for xcontrol
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wed, Apr 09, 2008 at 10:28:45AM +0200, Andreas Tille wrote:
> On Tue, 8 Apr 2008, Jonas Smedegaard wrote:
>> Really I would recommend to just use the classic "dist" target that
>> it wraps around anyway.
>
> Well, this is quite simple to change. If it helps understanding
> things I have no problem.
Indeed I believe it does. I believe it could have avoided the recent
confusion Holger had understand problems in Debian Edu.
Sure it wouldn't have avoided the _problems_ (which was related to wrong
use of cdd-dev) - but the confusion _understanding_ those problems.
>> Sorry for my bad description. I agree with your description. Only it
>> seems to me that your current design requires a "make dist" after a
>> "configure".
>
> This would be equal to autotools: You have always to call "configure"
> before "make dist" because there exists no Makefile before.
Yes, as developer.
As user, you should *not* need to run "make dist" at all.
>> That you do not "finalize", and after that allow treating the package
>> as any other package (like being able to run "debuild" even if distro
>> field is "UNRELEASED").
>
> This does inspire me to an unrelated thought: Perhaps we should always
> use UNRELEASED inside debian/changelog and ask the user to say:
>
> make dist <dist>
>
> where <dist> might be "unstable", "terra" or whatever. So we also
> "autogenerate" the debian/changelog to make sure the user is
> explicitely aware to build the soure vor a certain target
> distribution. What do you think about this?
I would prefer to define and use a make variable in your makefile, like
cdbs do it:
DEB_AUTO_UPDATE_DEBIAN_CONTROL=unstable
Then it can be overrided either in command line:
DEB_AUTO_UPDATE_DEBIAN_CONTROL=terra make dist
or in environment:
export DEB_AUTO_UPDATE_DEBIAN_CONTROL=terra
make dist
The benefit of this approach is that most (if not all) build systems
supports preserving and/or centrally defining DEB_* environment
variables. So even if Debian officially does not support the dynamic
update (for good reasons), you can easily enable it locally, even when
using automated build daemons as part of the development.
>> It is sane as upstream to use autotools, but only if finishing before
>> passing on the code to distributors:
>>
>> This I find sane:
>>
>> / write code
>> upstream < setup build hints (Makefile.am + configure.ac)
>> \ generate build system (autoreconf)
>> / configure (add changelog entry)
>> distro < use build system (make + make install)
>> \ package (debhelper + friends)
>>
>> This, however, I dislike:
>>
>> / write code
>> upstream \ setup build hints (Makefile.am + configure.ac)
>> / generate build system (autoreconf)
>> / configure (add changelog entry)
>> distro < use build system (make + make install)
>> \ package (debhelper + friends)
>
> If you _really_ want to separate upstream and distro (which I don't
> because it is plain nitpicking) it is rather like this in our case:
>
> / write code
> upstream < setup build hints (Makefile.am + configure.ac)
> \ generate build system (autoreconf)
> \ configure (add changelog entry)
> distro / use build system (make + make install)
> \ package (debhelper + friends)
It is more than nitpicking: You must allow distro to configure.
BinNMUs automatically change version number. That is a change to
configuration. Not one that hurts your tool, but it is wrong of you to
assume that the distribution field is static. Imagine semi-automated
stable-proposed-updates in the future.
> I see no other chance to respect Debian policy than to configure
> (= build a valid fixed debian/control file) as "upstream".
That is because you see "make dist" as the autotools "configure".
I see it as the autotools "autoreconfig".
With autotools you release unconfigured, forcing the user to configure.
With dpkg you release configured (you must provide a working debian
subdir, including a sane changelog), but user can still reconfigure (add
a new changelog entry on top of the old one, possibly changing the
distribution field).
> But in my eyes those who build the debian/control file are not
> upstream but Debian and that's why the whole concept of upstream and
> distro becomes blured and this is by design because the whole point
> just is to generate a debian/control file.
The tasks are different, but the separation of _kind_ of tasks are
similar:
* A software developer does whatever is needed beyond
invoking "[configure && ] make && make install"
* A package developer does whatever is needed beyond
invoking "[dch && ] debuild && dput"
>> You might then argue that designing a CDD is all packaging, no upstream.
>> And you are free to see it that way - I just disagree.
>
> Ahhh, I would be astonished if you would agree that easy. ;-)
> But your arguments are just not valid to me and I would like to
> see a reference design that follows your upstream / distro
> separation. The whole point is about generating a valid debina/control
> file when being offline. Just tell me how this can be done in
> a policy compliant way by separating upstream and distro.
Packages using CDBS with auto-updated build-dependencies:
* development build:
dch [any options]
DEB_AUTO_UPDATE_DEBIAN_CONTROL=yes debuild
* releasing to users:
(nothing)
* user build:
dch [any options]
debuild
Dynamically resolvable parts (build-depends) are replaced (not created)
when a special environment flag is set during normal package build.
Packages using cdd-dev:
* development build:
make dist
dch [limited options]
debuild
* releasing to users:
(nothing)
* user build:
dch [any options]
debuild
Dynamically resolvable parts (control file) are created with a custom
command.
Since the file is created (not just updated - the update is triggered by
the file not there), the command must run even before development build,
unlike autotools "make dist" which only creates a tarball of the
(separately finalized through autoreconf) source.
Is my description of cdd-dev usage above correct?
Does it make sense now how it makes sense to chance?
>> When I develop code targetted specifically for Debian (as was the case
>> with lessdisks[1]), I keep code and packaging striclty separate:
>
> IMHO this example just does not fit. Meta packages are packages with
> no content. So upstream code is just empty.
Lessdisks also contained virtually empty packages. But whatever. It
was an example - you found it irrelevant so it is irrelevant.
>> Similarly, when I do "magic tricks" to debian/control, I separate my
>> _pakaging_ routines themselves in "devel" and "distro" parts. It must
>> be possible to repackage my package without knowledge of the magic part.
>> With my "upstream" hat on, all files may be dynamically generated from
>> outside sources or relying on the phase of the moon. But whatever I do,
>> I must freeze it, so that my potent tricks does not explode in the hands
>> of some backporter or build daemon later on.
>
> I don't think that this is the case. IMHO the build system is safe
> regarding backporting or rather the whole staff was invented to be
> safe about backporting because the control file will be adapted to
> the packages that exist in the older distribution.
I don't get it. Do you say that the control file does or does not
change based on changelog file?
...even on build daemons, or what triggers an update?
>> I mentioned 2 classic types of badly released upstream code - unclean
>> and half-baked. I only find that your code is half-baked.
>
> And I continue to fail to see _why_ you regard it half baked. It is
> rather the contray (well done or even over done if such term exists).
>
>> Continuing the autotools analogy, I find that your tarballs lack
>> Makefile.in files:
>
> BTW, have you ever looked into the tarball that is created by "make
> dist"?
>
>> I consider packaging development finalized, when it contains a static
>> debian/control, and can be built using the following command:
>>
>> debuild
>
> But it does!!!!
You are right, it does.
Your "make dist" does the equivalent of these autotools commands:
autoreconf
configure && make dist
make maintainer-clean
That is a very unusual "make dist" routine IMO. Don't you agree?
Would you be interested in cleaning that up, or was that on purpose?
> You obviosely never inspected the released tarball.
I sure did. Your debian/control target is declared as .PHONY, and
debian.control file is indeed touched at build time.
I have not fully understood the inner workings of cdd-gen-control but it
seems invoked always, also on build daemons. That worries me.
Oh, and you have a typo in there: s/lover/lower/
If what you say is that you only want to discuss further if I can
demonstrate a situation that actually causes Policy violation, then fair
enough - then I shall not waste your time any longer... :-/
>> I believe that you consider packaging development finalized when it can
>> be built using this:
>>
>> make -f debian/rules get-orig-source && debuild
>
> No. Exactly this would fail. This is what I explained Frederic in
> Merida. debuild will fail here because the debian dir does not contain
> a control file.
...which is similar to autotools source not containing a configure
script. Possible to build, but only when you do development tasks like
generate autotools files with autoreconf or similar.
> This control file is only inside the tarball that was created and
> which has to be unpacked in some other place. There you can run
> debuild (and this is described in the docs). I don't say that
> unpacking the source at another place is very convinient but it is the
> only means I know to avoid everything that you are critizising above.
And I am offering a solution - if only we can agree what is the problem.
See further down...
>> In the development of the debian-edu CDD, José joined the upstream part
>> and worked on composing task files. Recently the build routines broke
>> due to wrong "configuration" (package was built with distro field in
>> changelog still the bogus UNRELEASED) and Holger wondered how cdd-dev
>> was really supposed to work.
>
> This is not true. The build system was not broken at all. Holger
> _thought_ it would be broken but it was perfectly fine. What was
> broken was _another_ _external_ tool that was not able to cope with
> a feature that Holger introduced into the changelog. If you would
> have tried to build the tarball using "make dist" this would have
> been failed which is perfectly correct behaviour because if there
> is no target distribution specified no tarball should / can be
> released, because no dependencies can be resolved for the distribution
> "UNRELEASED".
I don't care who or what wrote UNRELEASED into the changelog, or who or
what chose to build the package while in that state. But I do care
where is this distribution field limitation?
* Debian Policy?
* Some unnamed other external debian edu specific tool?
* cdd-dev?
I suspect cdd-dev - wich is _exactly_ my concern here.
>> I want my final source package to be easy to repackage by others.
>
> In this case you surely need the tasks files (and that's why they
> are inside).
No. I mean that I want it easy for others to throw the source package
into their automated build daemon, which might automatically change both
version number, distribution and other fields in the changelog, but
STILL the package should preserve the old control file!
Only if they wanted to *derive* their copy of the package into having
different content in the control file based on their environment should
they do a make dist.
>> A backport should be just changes to to packaging, not to the apckage
>
> Backporting means in this case: Just add a new changelog entry
> containing a new version number and the target distribution and than
> say
>
> make dist
>
> which creates a source tarball for the backported distribution which
> that way gets the right information for the debian/control file.
Yes, I am aware of that.
I try to explain to you that I dislike being forced to dig into the
inner workings of a package in order to backport it.
>> itself (unless of course the package needs patching to work in the
>> alien environment, but that's another story). With native packages
>> there is only a single version number, and a single tarball, so you
>> cannot avoid messing with the "source" of the package, and must
>> either introduce a new packaging version, or bogusly bump upstream
>> version number.
>
> Well, I can see the point that this might be a reason to adopt
> the concept of native package.
s/native/non-native/
Or if not then I do not understand at all.
* single version, only tarball: native
* dual version, tarball + diff: non-native
> So you would like to let "make dist" create a
>
> debian-edu_1.x.orig.tar.gz
>
> and use a changelog entry like
>
> debian-edu (1.x-1) unstable
>
> if Debian is the target distribution and
>
> debian-edu (1.x-1terra) terra
>
> for instance if it is targetted to terra?
Agreed.
> If you agree with this I tend to my suggestion above which would work out
> like this but leads to severe Problems:
>
> 1. The SVN contains a changelog which first entry is:
> CDDNAME (<version>) UNSTABLE; urgency=low
> 2. make dist <dist>
> --> If dist == unstable the the resulting changelog entry becomes
> CDDNAME (<version>-1) unstable; urgency=low
> otherwise it becomes
> CDDNAME (<version>-1<dist>) <dist>; urgency=low
> 3. We probably need a means to define under which circumstances the
> debian packaging version should be increased and how this should
> be specified.
> 4. The resulting tarball will be called orig.tar.gz
>
> Problems:
>
> 1. Interestingly enough the diff.gz file that is created by debuild
> will contain nothing because of the special nature of the meta
> package thingy
> 2. Every other debian packaging version than "-1" should provide a
> diff.gz against the original tarball. I have no idea how to
> implement this because THIS way to backport would bring you
> a real lot of problems to solve. It definitely needs a lot of
> enhancements to cdd-dev and I personally am not willing to
> do this work because of some nitpicking about native and upstream
> packages. For sure I adopt patches that solve this problem.
This is all handled by dpkg-buildpackage. Why on earth o you want to
reimplement those core tools?
If empty tarball is a problem, then throw in a COPYING and a README,
just for the fun of it.
dpkg-* automagically generates a diff. Or do you mean something else?
Here's a recipe for changing to cdbs-style update of control file:
1. Drop "make dist" - attach it to "make clean" instead, wrapped in
an ifdef so it is only ever invoked when
DEB_AUTO_UPDATE_DEBIAN_CONTROL=yes
3. Rewrite to always update (not only when controlfile is missing)
That's it.
The hard part (it seems) is for you to understand how it works... :-P
> So I would see a general agreement in both opinions. I also agree
> that regarding backports / other distributions some orig.tar.gz
> approach might be more precise. Here we have to decide between a
> precise naming convention which is really hard to implement and the
> current implementation which was *designed* *to* *work* with other
> dists (like the SkoleLinux dists) and just works.
Sorry, I can't parse.
As I understand it, you have consistently claimed that current approach
works fine and that you cannot see any need to change, but now suddently
you _do_ agree that non-native packaging is better, or do you somehow
argue that non-native packaging won't work with Skolelinux?!?
>>> We found means to create a source tarball which enables us to build
>>> Debian packages in a policy compliant way. So in my eyes splitting
>>> this up to stick to the narrow meaning of "get-orig-source" is extra
>>> owrk for no profit at all and I personally refuse to change a
>>> running system but I'd consider working patches (to code _and_ doc).
>>
>> Ok.
>
> This final OK seems like my interpretation inside the my last
> paragraph was basically right.
No, I do not agree that it is extra work for no profit at all.
I is fine with me that you are blind to the benefit of supporting a
"development mode" (where dynamic updates are allowed and happens
automatically when running normal build commands) and a "user mode"
(where Deian policy is obeyed, not touching debian/control at all).
I am very confused if you are at all interested in this correspondence.
I Feel like we are going in circles here :-(
- Jonas
- --
* Jonas Smedegaard - idealist og Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/
- Enden er nær: http://www.shibumi.org/eoti.htm
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFH/pWWn7DbMsAkQLgRAghLAJ0fKW8lY86PN8Cnr3Xx3Wlg1oXIdwCgjkqo
kZFADOKu2a3fJoYjdO3l0WM=
=1jJr
-----END PGP SIGNATURE-----
Reply to: