[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: golang protobuf status (was Re: Bug#1033839: Packaging docker 24.0.9)





On Tue, Jun 4, 2024 at 8:15 PM Shengjing Zhu <zhsj@debian.org> wrote:
Yes they are not compatible. But for the packaging, there are no
conflicts around gogoprotobuf packages. You probably misread
golang-goprotobuf-dev as golang-gogoprotobuf-dev. (These names are
very similar...). golang-goprotobuf-dev is a transitional package
which ties the runtime dev package and the code generator together.

Thanks for pointing that out, yes, that is indeed contributing to the confusion.


>  - It seems to me that we probably want to move packages away from src:golang-gogoprotobuf as much as possible, in particular because of the interoperability concerns. Is that agreeable?
>

Yes. This package is dead, and we also should convince upstream to
move away from it.

so, here is a todo list for removing gogoprotobuf:

$ build-rdeps gogoprotobuf
[...]
caddy
docker.io
golang-github-centrifugal-centrifuge
golang-github-centrifugal-protocol
golang-github-containerd-cgroups
golang-github-crc-org-crc
golang-github-gogo-googleapis
golang-github-gogo-status
golang-github-googleapis-gnostic
golang-github-hashicorp-go-plugin
golang-github-influxdata-influxql
golang-github-influxdata-yarpc
golang-github-lightstep-lightstep-tracer-common
golang-github-openshift-api
golang-github-opentracing-basictracer-go
golang-github-smallstep-certificates
golang-github-tonistiigi-fsutil
golang-opentelemetry-contrib
influxdb
prometheus

That's... not nothing. Maybe we can focus on docker.io, caddy, influxdb and prometheus first and see how that goes?

>  - It also seems to me that we probably want to avoid having both golang-github-golang-protobuf-1-3 and golang-github-golang-protobuf-1-5 at the same time. Both implement APIv1, and as such should be source-code compatible. We currently seem to have 19 packages build-depending on golang-github-golang-protobuf-1-3-dev. Can we lift-and-shift all of them in one transition to @v1.5?
>

I think so. The runtime library golang-github-golang-protobuf-1-5-dev
should be compatible with golang-github-golang-protobuf-1-3-dev.
Current approach of listing them as alternative dependency seems to be
working more gracefully. I probably should not introduce
golang-github-golang-protobuf-1-3-dev at first, but only keep the
protoc-gen-go-1-3 (the pb.go generator) around.

So here is a list of packages in unstable that produce binary packages that depend on golang-github-golang-protobuf-1-3-dev but not on golang-github-golang-protobuf-1-5-dev:

golang-go.opencensusgolang-go.opencensus-dev0.24.0-1
golang-google-genprotogolang-google-genproto-dev0.0~git20200413.b5235f6-3
notarygolang-github-docker-notary-dev0.7.0+ds1-2

And here is a list of packages that have an alternate dependency on both golang-github-golang-protobuf-1-3-dev | golang-github-golang-protobuf-1-5-dev

etcdgolang-etcd-server-dev3.4.30-1
golang-github-armon-go-metricsgolang-github-armon-go-metrics-dev0.4.1-1
golang-github-census-instrumentation-opencensus-protogolang-github-census-instrumentation-opencensus-proto-dev0.2.1+dfsg1-4
golang-github-go-kit-kitgolang-github-go-kit-kit-dev0.10.0-6
golang-github-golang-groupcachegolang-github-golang-groupcache-dev0.0~git20210331.41bb18b-1
golang-github-golang-protobuf-1-3golang-goprotobuf-dev1.3.5-4+b6
golang-github-google-gnostic-modelsgolang-github-google-gnostic-models-dev0.6.8-3
golang-github-grpc-ecosystem-go-grpc-middlewaregolang-github-grpc-ecosystem-go-grpc-middleware-dev1.3.0-2
golang-github-grpc-ecosystem-grpc-gatewaygolang-github-grpc-ecosystem-grpc-gateway-dev1.16.0-4
golang-github-openzipkin-zipkin-gogolang-github-openzipkin-zipkin-go-dev0.1.5+git20190103.2fd7f4a-2
golang-gocloudgolang-gocloud-dev0.26.0-1
golang-gomegagolang-gomega-dev1.27.10-1
golang-google-appenginegolang-google-appengine-dev1.6.7-2
golang-google-cloudgolang-google-cloud-dev0.56.0-3
golang-google-grpcgolang-google-grpc-dev1.38.0+really1.33.3-1
golang-gopkg-rethinkdb-rethinkdb-go.v6golang-gopkg-rethinkdb-rethinkdb-go.v6-dev6.2.1-3
golang-protobuf-extensionsgolang-protobuf-extensions-dev1.0.4-2
 
I would say that's a significant amount of uploads, but yet doable.

What's the correct order to do those transitions? How about we start with etcd, notary and opencensus? If that works out, the grpc-ecosystem packages might be a good next target.

> - As for packaging docker 2.4 (cf. #1033839, which spun off this conversation), which of the four library implementations above should it be linking against? -- We can savely rule out gogoprotobuf (it's deprecated), and I guess google-protobuf as well, because I don't think all dependencies have moved to the APIv2. That leaves two options: golang-protobuf @v1.3 and @v1.5. Given the compatibility concerns above, I guess @v1.5 is the better choice. That means that at least the 'notary' package needs to be updated (as Arnaud points separately) to pick up protobuf v1.5.
>

I think the notary code doesn't need to change. Just switching the
golang-github-golang-protobuf-1-*-dev in Depends could work. As the
1.5 should be compatible with 1.3.

Indeed, thanks, done so in notary 0.7.0+git20240416.9d2b3b3+ds1-1 which I've uploaded to experimental.
 
> - Currently, packages that use grpc (that is, build-depend on  golang-google-grpc-dev or protoc-gen-go-grpc), contain code that is generated with golang/protobuf @v1.5. Wouldn't we expect applications that use grpc to (possibly silently) break when linked against any other library than protobuf @v1.5? Would it help to reflect this in package relationships (i.e., add appropriate Conflicts or Build-Conflicts relationships)?
>

protoc-gen-go-grpc is a new binary package, which is not used by any
package yet.

In theory, if the pb.go files are generated by protoc-gen-go-1-5 or
protoc-gen-go (the generator without version suffix is from
src:golang-google-protobuf), and the application still uses
golang-github-gogo-protobuf-dev as runtime library, than it will
break.
>From my observation for some upstream projects over these years, I
think the breakage only happens when the pb.go files from
golang-google-genproto-dev are generated by the new generators.
For example, the containerd project pins the version at
https://github.com/containerd/containerd/blob/74a86c30b3fcbb06802f4e3d95f49d0a127d4298/go.mod#L143
but they have bumped protobuf runtime as well as grpc runtime.

Yeah, that matches my observations. I think that's a good example why the new src:golang-google-genproto-apiv1 package I propose in https://lists.debian.org/debian-go/2024/06/msg00008.html makes sense


--
regards,
    Reinhard

Reply to: