Re: Hashsum mismatch prevention strategies
On Sun, May 20, 2012 at 2:35 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Joerg Jaspert <joerg@debian.org> writes:
>> On 12843 March 1977, David Kalnischkies wrote:
> I would like to extend the format of the Index file there so that a
> single entry can list multiple patches like this:
>
> SHA1-Current: ced0270ddecf73e18ed604a632cd201d8006980e 29140592
> SHA1-History:
> 333d3ea2262f49befe07069df22390277587a4fb 29094219 2012-05-06-0214.07 2012-05-06-0813.32 2012-05-07-0813.32
> 5bca4bd997d4466c9f7df3fcf77a21e02a152c12 29094451 2012-05-06-0813.32 2012-05-07-0813.32
> 2fbf8424e650310c7cd9cf20070543ea036a39bf 29094377 2012-05-06-1414.33 2012-05-06-2013.13 2012-05-07-0813.32
> b08a3149b0be7201e98d74c699c0d5a33d493e84 29105545 2012-05-06-2013.13 2012-05-07-0813.32
> d6fb28816fffb9b74ae60c82d503cc79d531185c 29112085 2012-05-07-0213.45 2012-05-07-0813.32
> 359bd6a4bf838c87d9d19b10e41a576784810237 29111799 2012-05-07-0813.32
> 8b7482acdeda7514ad887720bda05ab9cf9d7abc 29111794 2012-05-07-1413.33 2012-05-07-2013.52 2012-05-08-0817.24
> 6d9b7ccb8e2e6c066ac80e779d64d1f523042ae9 29112045 2012-05-07-2013.52 2012-05-08-0817.24
> 12e2733c9a34fff1c86735b266ff315e84a78fc5 29112958 2012-05-08-0213.34 2012-05-08-0817.24
> ef7882176b5277e5be7b38e396c69c8f8167ac35 29117503 2012-05-08-0817.24
> 380c5683e0942266c0570f77c989db2e565ca799 29120576 2012-05-08-1413.33 2012-05-08-2018.17
> cad793bca1f269d5f1ec68df1bc59931d7321098 29125630 2012-05-08-2018.17
> 1799dbc63e7856d42cec3eb6b4192290f007bc9a 29127145 2012-05-09-0213.20
>
> So instead of a single patch file a list of patch files is given. The
> above example would be for the above log(N) method. With the current
> totaly incremental files the lines would have 1-56 patches listed.
The old proposal i remember introduced a new section SHA1-Results mentioning
the sha1sums of the Packages file after the patch is applied. This way a
client can built up it's own path and old clients can continue to function
with this new style (just keep being slower) - otherwise we would need to
wait for all clients supporting this new scheme to reach oldstable or
otherwise stable users would be unable to have unstable in their sources.list.
> Option C)
> =========
>
> The "Mirrors are going to be out-of-sync. Deal with it." option.
>
> Instead of coming up with more and more complex repository layouts or
> mirror scripts why not just accept that sometimes mirrors will be
> out-of-sync and think of a way we can help the client to recover.
>
> To recover the client needs to get hold of the correct files. So lets
> add a location where the client can request them. In InRelease we add
>
> Hash-Service: http://hash.debian.org/hash?h=#
> ftp://hash.debian.org/hash/#2/#
> rsync://hash.debian.org/hash/#2/#
>
> Now when a client needs to download a Packages/Sources/PDiff/Translation
> file with SHA1 863c99eb851479720c8930b31a9e85b6535ddab9 and the mirror
> does not have the correct file the client can download one of
>
> http://hash.debian.org/hash?h=863c99eb851479720c8930b31a9e85b6535ddab9
> ftp://hash.debian.org/hash/86/863c99eb851479720c8930b31a9e85b6535ddab9
> rsync://hash.debian.org/hash/86/863c99eb851479720c8930b31a9e85b6535ddab9
>
> In Hash-Services a '#' means the full SHA1 checksum and #<X> the first X
> characters of the SHA1 checksum.
>
> This service would only be used when a mirror is out-of-sync and only
> for the index files. So traffic should be much less than mirrors usualy
> have.
The problem is, usually the mirror has the new indexes (Packages, …) and an
old InRelease file, hence it has acquired quiet a few MB's of data (in worst
case) which can't be validated. What i need here is a way to get the "new"
InRelease file, not a way to get the old indexes as i already have the new
ones - i would need to download the old indexes completely as there is no
possibility to "downgrade" the new indexes.
If we really get the new InRelease file and old indexes, we still have the
problem that the old indexes can't be validated before we work with them
(= to apply a patch we need to decompress them. A security bug in the used
compression type and we have a problem…).
Beside that it perfectly shows why i dislike using hashes:
SHA1 is not an ideal choice anymore, so at some point we should move to
something else leaving us with a transition and therefore a fallback-
system behind even through any random number would have done the trick.
Best regards
David Kalnischkies
P.S.: Again, really, i am subscribed to dak@ and deity@ …
Reply to: