Re: PATCH: dpkg: allow missing /etc/dpkg/dpkg.d directory
OK, I was able to revert everything to a version before I applied the
OpenEmbedded/Yocto patch I proposed to them, that contained the dpkg
patch I proposed here and rerun the entire scenario until I was able
to reproduce the issue. It's an even more complicated set of
circumstances than I suspected.
The return value from scandir when it actually fails in the wild is
EACCES, not ENOTDIR. When I was troubleshooting the method I used to
simulate the problem didn't quite match the real scenario, but was
close enough to "fix" (e.g. workaround) both. Sorry for the confusion.
This is made even more weird because when the failure occurs dpkg is
running under apt-get and both are running under a pseudo/fakeroot
where a LDPRELOAD shim tricks the user processes into thinking they
are root (https://www.yoctoproject.org/tools-resources/projects/pseudo).
I'm not sure if this is why we're getting an unusual return value from
scandir or if the manual page is inaccurate (POSIX has EACCES listed),
but at a minimum it is something to be aware of. In this case the
entire directory tree was removed AND some of the higher level
directories did not have group permissions sufficient for the other
user to traverse.
I don't think its fair to complain if dpkg does something wrong when
running in this highly unusual environment, but I really appreciate
the offer to add a feature for our cross-builds to make it more
robust. It will avoid having either OE/Yocto or me carry around a
downstream patch of, as you point out, dubious robustness.
Conclusions:
1. ignore this patch request except for history/discussion purposes
2. the willingness of dpkg to ignore "weird" errors from reading the
config directories is a permissive as it should be. I'm especially
convinced by the argument that you shouldn't continue with a 1/2 baked
configuration. Presumably and empty cfg.d is by design, but
permissions and other errors should cause a failure.
3. rather than a workaround to allow dpkg-native to continue past the
error in our very unusual cross environment, a better approach would
be to fix the root cause and avoid the hard-coded path to another
users or a obsolete or random cfg.d directory in the first place by
adding a command-line option.
I found that the original Yocto list I posted my patch (which
contained this patch) to was the incorrect list as dpkg is pulled in
from OpenEmbedded core
(http://lists.openembedded.org/pipermail/openembedded-core). Now that
I have a basic idea of a direction, I will post the details problem
over there to make sure I'm on the right track with the desired fix. I
did some looking and I think the best approach is more along the lines
of adding a --configdir vs. --sysroot.
On Thu, Dec 15, 2016 at 9:21 AM, Anders Oleson <anders@openpuma.org> wrote:
> On Thu, Dec 15, 2016 at 8:16 AM, Ian Jackson
> <ijackson@chiark.greenend.org.uk> wrote:
>> Anders Oleson writes ("Re: PATCH: dpkg: allow missing /etc/dpkg/dpkg.d
>> directory"):
>>> But they expressed a desire to upstream patches (in general), so I
>>> thought I'd see what you thought. If don't-care/don't fail was
>>> acceptable,
>>> then why not start the discussion there.
>>
>> I wanted to reply to this. I absolutely encourage you to come and try
>> to upstream your patches. You will get the benefit of our orneriness
>> about error handling :-). By which I meank people like me can give
>> you advice on how to solve your practical problems in a more robust
>> way.
> Actually I agree with 100% of all of this. It isn't good for anyone to have
> the baked in paths reused - it's not supposed to be that way.
>
> So the first question was just whether we actually *want* to check the
> errors that way - asked and answered. No surprises really. I independently
> debated nearly all the same points with myself.
>
>>
>> And we will hopefully get the benefit of your code contribution.
>> So, thanks.
>>
>> Regards,
>> Ian.
>
> Either way the best long term option would be to make dpkg support floating
> around in sysroots without hardcoded paths baked in. My thoughts exactly on
> if we had an override, we would set the baked-in paths to something neutral
> and illegal same as you suggested to catch stuff like this. I don't like
> peoples names floating around baked into executables either even if only
> used for the cross-build.
>
> The biggest help would be to get design guidance. I know Yocto passes
> --instdir and --admindir. If we were to add a --sysroot or a --configdir
> option at a high level where would we need to touch? There are multiple
> executables, perl is involved somewhere, etc. etc. What has to be
> considered? What should get touched?
>
> I am not a frequent contributor, but I am an experienced developer and user
> of many open source tools and I maintain, debug and hack on them internally
> quite a lot. And lurk vs. post mostly too. It is rare to actually find
> something that is actually worth sending upstream, e.g. not so specific or
> not already fixed, etc. But the point is that both Yocto and dpkg are of
> huge value to me and my way of saying thanks is to offer to pitch in when I
> do find something. Give me a few hints and I'll take this on.
>
> If that goes OK, in addition, I can probably look at the other patches that
> Yocto is applying. They have a system where the have a list of patches that
> are applied during the build. See:
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/dpkg/dpkg
>
> Thanks!
>
>>
>> --
>> Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
>>
>> If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
>> a private address which bypasses my fierce spamfilter.
Reply to: