Hello.
Summary:
* test-packages.pl is slow and depends on volatile hackage index
* new implementation is fast and is not volatile, but requires
accurate synchronizaton of patches from packages to package plan
* synchronization is hard where packages are located in many diffrent
places
* developers, we need to converge. DHG_packages repo is most oblivious
candidate, but any other would do.
Long story:
With introduction of "revision" misfeature of Hackage, package plan
testing becomes much less reliable. Essentially, to revert upstream
updates to cabal files via revisions, we manually maintain directory
of revision-0 cabal files, subsequent versions of which are know to
break check (usually patches no longer apply).
Essentially, any upstream maintainer, creating new revision could cause
two unwanted effects on our development:
* package plan that successfully check would no longer check. To fix it
we would have to add one more revision-0 cabal file into
additional-cabals/
* package plan would succesfully check, while packages from pristine
tarballs are not actually co-installable.
In addition, "test-packages.pl" is quite slow, since it access huge
(~500Mb) tar archive in random-access way, while the only search tar
supports is linear.
I created another, faster implementation of index-patching code in C,
which have following advantages:
* it is faster (no random access). On my laptop, it takes 12 seconds to
process current index. The biggest timehog is call to external
/usr/bin/patch, but I found no library implementation.
* it does not require maintaining ever-increasing collection of
revision-0 cabal files in addition-cabals/ by virtue of more
sophisticated handling of format of index archive.
On other hand, it much less forgiving to incoherences in out package
plan. For example, in packages.txt at master (7fcb08) we have:
cborg-json 0.2.0.0
aeson 1.3.1.1
and check is successfull (see CI). Catch is that pristine cborg-json
have dependency on aeson < 1.3, but since revision-0 cabal file is
missing in additional-cabals/ and no patches for cborg-json are present.
By pure luck two errors canceled each other.
Ideally, all this mess would be handled by stackage, but currently we
have number of key packages that are not on stackage.
So, new implementation primises stability, but requires more thorous
synchonization of patches in source packages into package plan. I wrote
simple script, that syncronize patches from DHG_packages only to
discover that it is enough.
We have pandoc[1], haskell-intern, haskell-swish that are maintained
separately from DHG_packages, we have cborg-json, which don't even have
Vcs-Git field, probably more packages in unexpected places. My point is
that we need more unification.
[1] Jonas, I remember that you dislike team mainenance, but let me try
to convince you.
--
Note, that I send and fetch email in batch, once every 24 hours.
If matter is urgent, try https://t.me/kaction
--
Attachment:
pgpRXQ83lkXjb.pgp
Description: PGP signature