Bug#966573: progress packaging awscli v2
- To: Ross Vandegrift <rvandegrift@debian.org>, 966573@bugs.debian.org
- Subject: Bug#966573: progress packaging awscli v2
- From: Noah Meyerhans <noahm@debian.org>
- Date: Mon, 31 Oct 2022 22:50:19 -0700
- Message-id: <Y2Czm/Uf6jUXxMI/@doom.morgul.net>
- Reply-to: Noah Meyerhans <noahm@debian.org>, 966573@bugs.debian.org
- In-reply-to: <20211006051043.dlghirp77mxoptta@stgulik>
- References: <159613997435.30364.10545201843971877554.reportbug@bigtour.home> <YMt+qrg8RG0PRkA7@doom.morgul.net> <159613997435.30364.10545201843971877554.reportbug@bigtour.home> <20211006051043.dlghirp77mxoptta@stgulik> <159613997435.30364.10545201843971877554.reportbug@bigtour.home>
Hi Ross,
On Tue, Oct 05, 2021 at 11:10:43PM -0600, Ross Vandegrift wrote:
> > awscli v2 remains quite difficult to package, but it seems that upstream
> > is looking to address this. See
> > https://github.com/aws/aws-cli/issues/6186 for details and tracking.
>
> Using the source dist poc from https://github.com/aws/aws-cli/pull/6352 I've
> made enough progress to get a packaged aws-cli v2 working. There's a lot more
> that needs to be done, but idea of the above linked PR could work for us. I'm
> going to document my findings here.
So, only a year later, I've picked this up and made some additional
progress:
I have no name!@b02f1db79f9e:/src$ aws --version
aws-cli/2.8.7 Python/3.10.8 Linux/6.0.0-2-amd64 source/x86_64.debian prompt/off
I have no name!@b02f1db79f9e:/src$ dpkg -s awscli | grep Version
Version: 2.8.7-1
> aws-crt-python is the hard part. It depends on a bunch of C libraries which
> follow a more modern development style. They:
> - provide no abi/api stability guarantees.
> - have versioned releases, but no one cares about the versions. The intention
> is that everyone should be following the main branch.
> - only nominally support dynamic linking (it can be enabled, but is not used or
> tested by upstream).
> - seem to be mostly used via source inclusion.
Indeed, none of this has improved in the past year. In fact, we've
actually gained a few more such dependencies that didn't exist last year
(aws-c-sdkutils, at least).
We might actually be able to make a case that all the aws-c-* libraries
end up being easier to maintain if we ship them as private internal
components of the aws-crt-python package rather than as first class
packages that could end up becoming dependencies of other downstream
packages.
> - some repos have tests disabled due to failing during builds. So far, I don't
> know if these are real failures, or if upstream's build method.
I think I've got most of these fixed.
> - copyright attribution for aws-lc is very hard. It's a fork of Google's
> BoringSSL, which is a fork of pre-3.0 OpenSSL.
>
> - That also means that aws-lc inherits the openssl gpl incompatibility.
Here's the good news: We don't actually need aws-lc at all. awscli v2
and its various dependencies (including s2n-tls) can build against
OpenSSL 3.
> - the aws-cli2-temp repo is based on upstream, not our awscli repo. I was
> intentionally being sloppy to quickly get through a test.
Same. I essentially Debianized the upstream v2 repo from scratch,
pulling in some of your packaging metadata as it made sense. Given that
v2 is developed on a different branch and by now differs quite
significantly from v1, a case could be made for introducing a new
awscli2 package as a new source package and retiring the original awscli
package. However, the debian package metadata isn't really all that
complex, so it may not actually be necessary.
> - aws-lc and s2n-tls may be hard to maintain. Both are complicated, security
> critical crypto libraries.
Fortunately, aws-lc isn't an issue. But s2n-tls remains one. Not sure
we're going to be able to do anything about that. The difficult thing
is that it's typically expected to be used as a statically linked
library, which means updates end up being tedious.
I haven't pushed my changes anywhere, yet. Once I do, the remaining
tasks will be to any lintian issues or other obvious problems and get
these packages into NEW. I think they're in reasonably good shape, but
we don't have a lot of time before bookworm starts freezing, so I'd love
any help with these steps.
We have to get at least the following packages into the archive:
aws-c-auth_0.6.18-1_amd64.changes
aws-c-cal_0.5.20-1_amd64.changes
aws-c-common_0.8.4-1_amd64.changes
aws-c-compression_0.2.15-1_amd64.changes
aws-c-event-stream_0.2.15-1_amd64.changes
aws-c-http_0.6.23-1_amd64.changes
aws-c-io_0.13.6-1_amd64.changes
aws-c-mqtt_0.7.12-1_amd64.changes
aws-c-s3_0.1.51-1_amd64.changes
aws-c-sdkutils_0.1.4-1_amd64.changes
aws-checksums_0.1.13-1_amd64.changes
aws-crt-python_0.15.1-1_amd64.changes
s2n-tls_1.3.26-1_amd64.changes
Reply to: