Hi Noah, all, On 08-09-2020 20:33, Sudip Mukherjee wrote: > On Tue, Sep 8, 2020 at 5:48 PM Noah Meyerhans <noahm@debian.org> wrote: >> >> On Tue, Sep 08, 2020 at 10:38:18AM +0100, Sudip Mukherjee wrote: >>> The test done in the autopkgtest of 'awscli' does not provide >>> significant test coverage and it should be marked with "Restrictions: >>> superficial". >> >> The current test merely invokes "/usr/bin/aws help". Is there any >> actual value in this test at all? Should we just remove it? > > I think you can keep it and mark it superficial. This will atleast > show you that it has built properly and is executing. That. And that your dependencies don't suddenly break your binary completely. >> Short of communicating with a remote service, I'm not sure how we can >> get more meaningful functional test coverage in this package, but it >> would be nice to do so. > > There is also another "Restrictions: needs-internet" which might be > useful for you. > Please check https://sources.debian.org/data/main/a/autopkgtest/5.14/doc/README.package-tests.rst > which has a section for "Network access". That says: > "But for many packages their main purpose is to interact with remote > web services and thus their testing should actually cover those too, > to ensure that the distribution package keeps working with their > corresponding web service.". Please be aware that relying on remote service may make you test become flaky, depending on the reliability of the service now and in the future. Not saying those tests should be added, but just a thought about them. > Also, Paul might have some more suggestions. Well, there is a bug open for advanced support in autopkgtest itself: #908274 Please add the capability to run tests that require a server. Not sure if that's feasible for awscli if autopkgtest would grow that feature. Paul
Attachment:
signature.asc
Description: OpenPGP digital signature