Hello, Since my last report [1], not a lot has changed with respect to my project, this is because all last week I have had exams and have not been able to work. However due to the tendency of the mind to wonder when attempting to revise colour changes in ligand substitution reactions and the equations for the scalar product of vectors, I have been thinking about my project and the apt code in general. One thing that I have done since my last report is some further investigation in to the ordering failure Michael showed me. I have included some of the output from my testing (… are lines removed for clarity). I will apologise to anyone in advance if the following attempt at code annotations goes wrong. SmartConfigure openoffice.org-common ← SmartConfigure is a method used in APT to perform configuration on a package and all of its dependants. DepAdd: openoffice.org-common ← DepAdd recursively (it calls itself) adds all the dependants of the package to the configuration list. You can see the iteration as it quite nicely indents the output. DepAdd: openoffice.org-style-oxygen ... DepAdd: libcurl3-gnutls ← Here is one of the packages I am interested in, its been added to the configure list, the DepAdd routine is now looking at its dependencies. DepAdd: libgcrypt11 ... DepAdd: libldap-2.4-2 ← Its found the other package of interest, as you can see libcurl3-gnutls depends on libldap-2.4-2, this means that libldap-2.4-2 must be configured before libcurl3-gnutls. DepAdd: libsasl2-2 DepAdd FAILS on: openoffice.org-core ← Something has gone wrong DepAdd FAILS on: openoffice.org-style-oxygen DepAdd: openoffice.org-style-human ... Conf libsysfs2 ... ← But even thought there has been a error, the configuration of the list begins ... Conf libcurl3-gnutls ... ← Now here is the problem, not only is libcurl3-gnutls being configured before libldap-2.4-2, libldap-2.4-2 is not even configured with this group of packages! Conf libxml2 (2.7.6.dfsg-1ubuntu1.1 Ubuntu:10.04/lucid-updates [amd64]) [] … Something must have happened, or not happened in the Dep add routine to cause the routine to ignore libldap-2.4-2. At the moment I have not had the opportunity to discuss this with my Mentor, this is something I will hopefully learn more about and do further investigation in to the code this coming week. Currently I have written a small piece of code to intercept configure calls from the package manager in APT and check for any unconfigured dependencies (This is currently not in any repository). This currently correctly detects the error above, but as the following example shows it does not cope with dependency loops. Here is the relevant output from APT: Conf firefox (3.6.17+build3+nobinonly-0ubuntu0.10.04.1 Ubuntu:10.04/lucid-updates [amd64]) Conf firefox-branding (3.6.17+build3+nobinonly-0ubuntu0.10.04.1 Ubuntu:10.04/lucid-updates [amd64]) … E: Package firefox should not be configured because package firefox-branding is not configured Now firefox depends on firefox-branding (seems sensible enough), but firefox-branding also depends on firefox (this does not seem as sensible). This results in a dependency loop. To configure these packages, the loop needs to be broken, the Debian policy manual covers the exact mechanics of how to do this [2], but as far as apt is concerned, dpkg will handle this. The way this works is the Conf statements that appear in the output above are grouped together by APT, and inserted in to one call to dpkg. So the dpkg invocation will look like: dpkg --configure A B C dpkg --unpack D dpkg --configure D Not, dpkg --configure A dpkg --configure B dpkg --configure C dpkg --unpack D dpkg --configure D Thanks go to David Kalnischkies for the above example. So the initial outcome of this is that my code is detecting these breakages as errors where they are actually none. So I could add to my check to see if there is a dependency loop between the two packages concerned, but while this is simple enough for the example above there could be any number of packages involved in this loop, so I am not sure if this is the best approach. There is also another possible problem though, as I said above APT passes packages to dpkg in batches, now if there is a large run of packages being configured (which does happen) this call will get split in to many (because of limitations outside APT's control). Now imagine what would happen if for the above example the large list of packages was split between firefox and firefox-branding, dpkg would not be able configure the packages properly. Now this might or might not impact firefox (depending on the way the package works), but for some packages this could be problematic. One solution I can think of for this problem, is before ordering packages, split the list of packages in to batches (groups of packages). Now this has two advantages: - By limiting the size of batches, you can prevent any splitting of dpkg calls (without doing any complex analysis of dependencies for a second time). - Batches could be processed in parallel, allowing packages to be installed and fetched at the same time. There are some practical considerations about the process needed for separating packages in to batches e.g. criteria for separating packages and how do deal with dependency loops and large batches, but from thinking about the problems at hand, this seems to me to be a good and flexible approach. I am now just days from my final exam (on Tuesday), and from then I will be free to work on APT full time (which I am very much looking forward to). Also if anyone has any advice, comments, questions or corrections, feel free to voice them. Chris 1. http://lists.alioth.debian.org/pipermail/soc-coordination/2011-June/001005.html | http://lists.debian.org/deity/2011/06/msg00005.html 2. http://www.debian.org/doc/debian-policy/ch-relationships.html#s-binarydeps 3. http://lists.debian.org/deity/
Attachment:
signature.asc
Description: This is a digitally signed message part