On 02/06/17 14:47, Carlos Alberto Lopez Perez wrote: >> Also it'd be nice to know what kind of automated testing is happening. I know >> WebKit has an extensive test suite (including layout tests) that upstream >> continuously runs for development series. I don't know if that's the same for >> stable series though. Also can we enable the test suite in Debian? >> > On trunk (master) we have a extensive test coverage per each commit. > We even have a bot testing that WebKitGTK+ always remains build-able > both on Debian stable and Ubuntu LTS. Check our bots here: > http://build.webkit.org/waterfall?category=GTK > > All those bots are running Debian 9, except for the 32-bit and ARM bot > that are running Debian 8 but they will be eventually upgraded to Debian > 9 also. > > (Also except the Debian and Ubuntu bots that are of course running > Debian 8 and Ubuntu 16.04) > > For the stable branch (that is what you are interested about I guess) we > have a bot running all those tests here: > > https://build-webkit.igalia.com/waterfall?category=GTK > > This bot running the tests on the stable branch is currently running > Debian 8 as base distro. > We are happy to upgrade it to Debian 9 if you wish (is something we will > eventually do in any case). > > Note that those 9 failures are not really meaningful. > Current layout test suite of WebKit has 47556 tests > > If there is some more testing we can do to accommodate your needs, we > will be happy to help. > > Regards. > > One note more: The results of the layout tests are highly dependent on very specific versions of the libraries we depend on. For example, different versions of GTK+ or Cairo produce different results (this are usually small pixel differences on the render of the webview). So what we do, is to have the expected results of those tests specially created for very specific versions of those libraries that we build with JHBuild and we use it for testing. If you try to run the layout test suite without using our internal JHBuild you are going to get thousands of tests failures due to this (they are not real failures, but the tooling is unable to detect that as it can't know if a 1-pixel difference is because of a real failure or because your version of GTK+ draws arrows/boxes/etc in a sightly different way than the one was used to produce the expected result) So, I don't think is flexible for Debian to run our layout test suite unless you are willing to maintain your own list of expected results. There are however other tests that you can run that don't depend on the version of the libraries used like the jscore-tests or the API tests. However, the tarballs we release are trimmed to remove all those testing stuff from them. You would have to use the SVN repository for that, which complicates things. However, we are happy to keep our bot that runs all those tests for the stable branch on Debian.
Attachment:
signature.asc
Description: OpenPGP digital signature