Re: Building/testing on s390x
Le 13/04/2014 17:39, Peter Palfrader a écrit :
> Building shouldn't rely on the network in the first place.
Humm, this is a very good point, and it did give me some headache.
The problem is: how do you *really* test basic functions of a website
copier /without/ network ?
Solution 1: do not run unit tests when building at all
Unfortunately I had a couple of architecture-specific bugs (on
architectures where I never actually tested anything), and having unit
tests on every single archs is probably a sane thing.
Solution 2: do not depend on the network at all
It means installing somehow a local webserver and do tests on this
server. Unfortunately you lose many of the benefits of a remote live
server (a real HTTPd server with all RFC 2616 corner cases and features
such as chunk/compression/keep-alive/Etag update capabilities/etc.),
network subtle differences (data sent in multiple packets unlike on
localhost), and many other little details, such as testing on several
different hosts (including IDNA ones in this case), that will make code
coverage not satisfying.
I could probably add some code to trick the resolver - or bypass it -
but here again I had couple of bugs related to... the resolver itself,
that would not have been detected otherwise (in this case a trivial
reuse-after-free which was bombing on only specific archs)
Solution 3: do not depend too tightly on the network
This was the chosen solution ; basically unit tests aren't fired unless
network connectivity is sane (ie. the first page could be fetched
without any troubles), and only very small pieces of data are used to
validate basic parser/resolver/whatever testing.
I tried to make the tests as robust and fool-proof as possible (with
maximum time limit, reasonable timeout, connection limits) and I only
had one issue so far (this very weird DNS issue)
Maybe an additional protection could be to make tests non-fatal, maybe,
but I would have to post-check every single build.
[ Note that the network overhead is not huge: the complete test suite is
fetching 32268 bytes of data remotely - 12K compressed ; and hosted on
the project servers ]
Reply to: