[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Ordering

Hi,
>>"Jason" == Jason Gunthorpe <jgg@gpu.srv.ualberta.ca> writes:

>>  I don't understand this. What are the two passes again?

Jason> Unpacking and Configuration, at this point I am resonably
Jason> convinced that you cannot mix them :<

	Well, pkg order, and dftp, have always worked with the
 assumption that one does it together. The combination now handles all
 the upgrade from bo to hamm apart from the case of Perl; and that
 particular brand of conflict and depend loop is looked for and
 special cased.

	I never though of doing the ordering in two phases anyway.

	
>> Eiher the packages are ordered for install, or they are not. If
>> they are not yet ordered, do no unpack anything.

Jason> They are ordered for UNPACK which means that conflicts and
Jason> predepends are valid at all points during the install.

	Which really says little. All you are sayin is that I had a
 random list pf packages, and I have minimally rearranged it such that
 the pre-depend targets always occur before the dependent packages.


	Is that good enough to install anything? As you have
 discovered, no, since any configure now packages can not be handled
 since the list is not ordered at all. I mean, even though the
 pre-depend targets come before, their dependencies have not yet been
 satisfied, 

	This is not good enoough to act on. You cannot configure
 anything at this point, and unpacking packages when nothuing can be
 configred is bad.

	The way pkg-order does it, you can install the list with
 breaks at almost any point (our goal was to be able to install one
 package at a tie, though we were quite paranoid enough about not
 breaking essential, which I shall address in pkg-order next.

>> If they are indeed ordered, then yuo can stop at any point and
>> configure everything upto that point; with the exception of a
>> conflicts depends loop, which requires forcing.

Jason> No, because you might have had to re-oder things that cause a
Jason> depends+predepends+conflicts loop to junk the depends loop (ie
Jason> your loop detection)

	Auugh. Think about this. when you ordered before, we put all
 pre-dependency targets before the pre-depended packages. Now that
 should *never* change. 

	You can't just brush dependency loops under the carpet (which
is what you are doing by re-arranging on subsequent passes) and hope
not to have problems.

	If there are no loops, there is one true order which shall
 work for both unpack and configure.

	If there are loops, then things aren't too nice: but no amount
 of rearranging solves that: you have to snip edges; and let
 the least important edge be snipped.

Jason> This will create a situation where an unpacked package cannot
Jason> be immedately configured - even your proprosed algorithm has to
Jason> have this problem.
 
	Nope, since I do not get into unpacked but unconfigurable
 stage ever. When I unpack things, they can be configured right then
 and there; except in a few dependency loop cases.

Jason> I wouldn't be sure how to go about finding them to decide using
Jason> your 2nd algorithm.
>>  You do this implicitly, by descending the children in the order of
>> importance of the dependency. So you try descending predepends
>> first, then you decend depends links, and optionally the weaker
>> relationships can be descended.

Jason> Hm, interesting idea, might have to do that, trouble is that
Jason> pre-depends+conflicts are ALWAYS important, they cannot be
Jason> viloated, dpkg will not allow it.
  
	Fine. Never let predepends+conflicts be an ignored
 revisit. Fail hard before taking any action at that point then. But
 look at *ALL* the relationships that you shall consider at one go, or
 else your topological sort has not happened, and your ordering is
 incorrect. (stop thinking package for a moment, think about nodes in
 a graphs and edges, you can't just look at *some* edges)

	So, descend the Pre-depends children first. get the final
 unpack/configure order. (A topological sort means that both should be
 the same). Any dependency loop breakage occurs by snipping a *less*
 important link (since we descend the less important children later).

	Culus: maybe we should talk. 

	This is the relevant discussion from pkg-order, along with the
 final code: we decided to order the packages pretending that the
 conflict with older versions was equivalent to a new ``dependency'',
 and that works for us in dftp+pkg-order

	manoj

______________________________________________________________________
For the following, assume a typical example:

  Package: libfoo1
  Version: 1.0
  Depends: libc5

is installed, and the following are candidates:

  Package: libfoo1
  Version: 1.1
  Depends: libc5
  
  Package: libfoo1g
  Version: 1.1
  Depends: libc6
  Conflicts: libfoo1 (<< 1.1)

  Package: libfoo1g-dev
  Version: 1.1
  Depends: libc6-dev, libfoo1 (= 1.1)

Since libfoo1g conflicts with earlier versions of libfoo1, the upgrade
for it must be installed before libfoo1g. This is two-part: libfoo1
must not be after a break between it and libfoo1g, and it must be
before libfoo1g if they're installed in the same dpkg run. Means: It
is sufficient if we pretend that libfoo1g depends on libfoo1.The faked
dependency is introduced for all xx (<< v) type conflicts, not only
for those where the conflict is really only avoided by the upgrade of
package xx. But for that, I'd have to search the installed list, too,
if there's a package xx with version << v, and xx >= v is on the
candidates list. This seems a bit much of trouble... :-) (I'd need the
installed list as parameter, and I'd have to do two times dpkg
--compare-versions.)

On the other hand, it doesn't hurt much if the dependency is faked
though it isn't necessary. Or does it hurt? Don't think so...

The bad news: For some packages, I now get dependency cycles :-( This
is due to the fact that some libc5 compat packages depend on their
libc6 variant! Examples are xview[g], libpaper[g], and e2fsprogs&Co.
Here are the relevant excerpts from the Packages file:

  Package: libpaper
  Version: 1.0.3-6
  Depends: libc5 (>= 5.4.0-0), libpaperg

  Package: libpaperg
  Version: 1.0.3-6
  Depends: libc6
  Conflicts: libpaper (<< 1.0.3-4)
  
I think these two can never be installed (if an old libpaper is
present) without some kind of errors... :-( You can't install
libpaperg without an appropriately new libpaper, and that not without
libpaperg... some kind of catch-22, isn't it :-)

  Package: xview
  Version: 3.2p1.4-2
  Depends: libc5 (>= 5.4.0-0), xlib6 (>= 3.2-0), xviewg (= 3.2p1.4-2)
  
  Package: xviewg
  Version: 3.2p1.4-2
  Depends: libc6, xlib6g (>= 3.3-5)
  Conflicts: xview (<< 3.2p1.4-1)
  Replaces: xview


This is basically the same as above, but with an additional Replaces:.
Think this is for taking over some files of old xview, not related to
libc5/libc6 issues.

  Package: e2fslibsg
  Version: 1.10-10
  Depends: comerr2g, libc6
  Conflicts: e2fsprogs (<= 1.10-7)
  
  Package: comerr2g
  Version: 1.10-10
  Depends: libc6
  Conflicts: e2fsprogs (<< 1.10-6), comerr2
  
  Package: e2fsprogs
  Version: 1.10-10
  Depends: comerr2g, e2fslibsg, libc6
  Conflicts: e2fsprogsg

Here it seems a bit more difficult, also because three packages are
involved. But e2fsprogs depends on both, comerr2g and e2fslibsg, and
those again conflict with earlier versions of e2fsprogs, which gives
the cycler later.

Are you the same opinion that in cases of libpaper and xview the
additional dependencies are bogus and should be removed? If yes, I'll
report bugs against those packages. Or do I miss something?

But in the case of e2fsprogs is more complicated. If you have an old
(<= 1.10-6) e2fsprogs installed, that should be updated first.
Otherwise, the libs should be installed first to satisfy dependencies
of e2fsprogs. 

And: It did everthing correct (libpaper, libjpeg, libelf, zlib,
xlib6), but with one exception: perl and perl-base. Here we have the
same situation as usually with the libs:

  Package: perl-base
  Version: 5.004.04-4
  Pre-Depends: libc6, libgdbmg1
  Suggests: perl
  Conflicts: perl (<<5.004.04-2)
  Replaces: perl
  
  Package: perl
  Pre-Depends: perl-base (>=5.004.04-2)
  Depends: perl-base (=5.004.04-4)
  Suggests: perl-suid, perl-debug

i.e.: perl-base conflicts with older versions of perl (creating the
implicit dependency), but on the other hand perl depends and
pre-depends on perl-base. The current ordering decides that perl is to
be installed first (with --force-depends), and perl-base comes later.
But this is incorrect! Forcing installation of new perl leaves the
perl installation unusable (e.g. Exporter.pm missing). So I think we
should never drop Pre-Depends: due to a Conflicts. That means another
test before inserting the Conflicts edge in the ordering graph.

I've looked up what autoup.sh does in this case:

  dpkg -i --auto-deconfigure perl-base_*.deb || true
  dpkg -i perl_*.deb || true
  dpkg --pending --configure

I think --auto-deconfigure is the only solution for this special
case... And I think it's possible to handle this in dftp, too:
pkg-order should add marks '--auto-deconfigure --ignore-errors' to
perl-base and '--ignore-errors --pending-configure' to perl.
--ignore-errors is a special mark that dftp can interpret as "ignore
an error returned from dpkg", and '--pending-configure' stands for
"run dpkg --pending --configure" after installing this package. It
should be no big trouble to handle those pseudo dpkg flags in dftp.


 $pkg_name holds the current install candidate, $target is the package we
 conflict with; 

      if ($dep_type eq 'Conflict'){
        if (exists($params->{'New'}->{$target}->{' _Pre-Depends'}->{$pkg_name}))
          {
            $params->{'New'}->{$target}->mark(Mark=>'--pending-configure');
            $params->{'New'}->{$target}->mark(Mark=>'--ignore-errors');
            $params->{'New'}->{$pkg_name}->mark  (Mark=>'--auto-deconfigure');
            $params->{'New'}->{$pkg_name}->mark  (Mark=>'--ignore-errors');
            # do not add an ordering edge for this conflict
            return;
          }
        if (exists($params->{'New'}->{$target}->{' _Depends'}->{$pkg_name}))
          { # Warn the user about this circular dependency
            warn "Circular Depends/Conflicts ordering between $pkg_name " . 
              " and $target\n$target must be installed with --force-depends\n";
            
            # Remove the dependency on our package from the upgrade of a
            # package whose earlier version we conflicted with; even if
            # that package depends on us, the upgrade has to be installed
            # first 
            @{$params->{'New'}->{$target}->{' _Order'}} =
              grep($_ ne "$pkg_name $target",
                   @{$params->{'New'}->{$target}->{' _Order'}} );
            
            $params->{'New'}->{$target}->mark('Mark' => '--force-depends');
          }
        Push(@{$params->{'Package'}->{' _Order'}}, "$target $pkg_name");
      }

______________________________________________________________________  

	manoj

-- 
 "It's better to get mugged than to live a life of fear." Freeman
 Dyson Freeman did indeed say that, but I'm probably the only person
 who was listening to him at the time.  So, you won't find it written
 in any of his books. Russell Nelson <nelson@clutx.clarkson.edu>
Manoj Srivastava  <srivasta@acm.org> <http://www.datasync.com/%7Esrivasta/>
Key C7261095 fingerprint = CB D9 F4 12 68 07 E4 05  CC 2D 27 12 1D F5 E8 6E
Reply to: