Re: [RFC] shirou/gopsutils v2 to v3 migration

[Shengjing Zhu <zhsj@debian.org>]

Please cc lists which have public logs, eg debian-go@lists.d.o or BTS.
See my reply below.

On Sun, Nov 21, 2021 at 08:33:14PM -0500, Nicholas D Steeves wrote:
> Hello Aloïs, DGPT, and gopsutil maintainers.
> 
> This is a gentle ping to please express your position on the correct way
> to migrate Debian's gopsutil from v2 to v3.  The email below contains
> the discussion between Aloïs and myself, and what we've discovered while
> investigating the current situation.
> 
> Alexandre, I hope you don't mind that I also CCed you.  Resolution to
> this issue appears to be required to keep Syncthing from being dropped
> from testing, so I thought it would be a good idea to keep you in the
> loop.
> 
> Aloïs Micard <creekorful@debian.org> writes:
> 
> [snip]
> > On 14/11/2021 01:11, Nicholas D Steeves wrote:
> >>   [process] RLimit is now uint64 (#364)
> >>   [mem] VirtualMemoryStat JSON fields capitalization (#545)
> >>     * various JSON field name and some of Variable name have been
> >>       changed. see v3migration.sh
> >>   [process] process.Status() now returns []string. and status string is
> >>             "Running", not just "R". defined in process.go. (#596)
> >>   [disk] disk.Opts is now string[], not string. (related to #955)
> >>   [disk] GetDiskSerialNumber() is now SerialNumber() and spread to all platforms
> >>   [disk] GetLabel () is now Label() and spread to all platform
> >>   [net] Change net.InterfaceStat.Addrs to InterfaceAddrList (#226)
> >>   https://github.com/shirou/gopsutil/blob/master/_tools/v3migration/v3Changes.md
> >> 
> >
> > This is definitely a good idea. I'll raise an issue upstream to have their feedback
> > and double check to make sure there's no drawback / regression induced by the major
> > downgrade.
> >
> 
> Any news from upstream about if this is viable and not as risky as it
> seems to be?
> 
> >> Other than that, in a Debian context, we want to avoid a scenario where
> >> all Debian rdeps of v3 reject v3 and patch in support for v2, no?  So I
> >> wonder if it would be a good idea to file a bug against lintian to start
> >> checking for this.
> >> 
> >
> > Certainly, in the long/mid term, we want to package v3 in order to have latest
> > features and bugfixes. But I'm not sure if a Lintian warning is needed for this?
> >
> 
> You're right, I don't think it's needed for the existing packages, but
> it seems like it would be a good idea to guard against existing software
> that isn't yet in Debian, but that already bundles v2.  I'm assuming
> this is fairly common but would be happy to be wrong (less work for
> us!).  [edit: it also seems like a good idea to encourage maintainers to
> DTRT when v2 is truly dropped upstream and all the rdeps break at once]
> 
> [snip]
> 
> > # Broken Depends:
> > nomad: golang-github-hashicorp-nomad-dev
> > syncthing: golang-github-syncthing-syncthing-dev
> >
> > # Broken Build-Depends:
> > consul: golang-github-shirou-gopsutil-dev
> > golang-github-satta-ifplugo: golang-github-shirou-gopsutil-dev
> > ncbi-entrez-direct: golang-github-shirou-gopsutil-dev
> > nomad: golang-github-shirou-gopsutil-dev (>= 2.18.02~)
> > packer: golang-github-shirou-gopsutil-dev
> > slinkwatch: golang-github-shirou-gopsutil-dev
> > syncthing: golang-github-shirou-gopsutil-dev
> >
> > Dependency problem found.
> > ```
> >
> > So I guess the job should be doable without doing a 'proper transition'
> > involving the release team & co. The only thing about the rdeps is that
> > they are important packages.
> >
> 
> With a "golang-github-shirou-gopsutil-v3-dev" package in unstable, bugs
> against these packages can be filed today with a clear path forward in
> Debian.  Without one it looks like we need to wait for consensus, and
> then staging the updated packages for v3 in experimental.  Both
> approaches are fine, but the latter seems slower.
> 
> More on this later, I discovered something strange...given that
> strangeness, it might be possible to update the existing package and
> file bugs asking maintainers to switch to the new bin:v3 generated from
> the existing src:gopsutil package...maybe.  I'm stumped because AFAIK,
> XS-Go-Import-Path can take exactly one path, and it would need to
> support two for this "one src package for both v2 and v3" case.  Read on
> for something shocking!
> 
> [snip]
> >> 
> >> Here's where I have a Golang ecosystem question for everyone reading
> >> this:
> >> 
> >>     Is there an obvious correct solution based on XS-Go-Import-Path?
> >>     eg: Upstream migration instructions says the patch changes from
> >>         "github.com/shirou/gopsutil/disk"  // to use v2
> >>     to
> >>         "github.com/shirou/gopsutil/v3/disk"
> >>     thus it seems that XS-Go-Import-Path would need to move from
> >>         "github.com/shirou/gopsutil"
> >>     to
> >>         "github.com/shirou/v3/gopsutil"
> >> 
> >
> > Actually the XS-Go-Import-Path doesn't need to change because it is
> > resolved to the root of the repository. For example:
> >
> > - github.com/shirou/gopsutil/disk
> > - github.com/shirou/gopsutil/foo
> > - github.com/shirou/gopsutil/bar
> >
> > modules would all be provided by the root github.com/shirou/gopsutil.
> >
> 
> Are you sure?  "v3 is in the v3 directory"
>   https://github.com/shirou/gopsutil/blob/master/_tools/v3migration/v3Changes.md
>

In this case, could you try putting v2 and v3 in same package, which is
golang-github-shirou-gopsutil-dev, just like upstream that put v2 and v3 in
same repository.

And try to rebuild the reverse dependencies.  I think it would just be fine,
and no package will be broken.

 
> and
> 
>   import (
>     "fmt"
> 
>     "github.com/shirou/gopsutil/v3/mem"
>     // "github.com/shirou/gopsutil/mem"  // to use v2
>   )
>   https://github.com/shirou/gopsutil/blame/master/README.md#L55
> 
> It looks to me like gopsutil is providing both v2 and v3 in the same
> repository...which confuses me even more about what the correct approach
> in Debian would be.
> 
> >> Then, if https://www.debian.org/doc/debian-policy/ch-sharedlibs.html is
> >> applied to Golang -dev packages, then a trip through NEW cannot be
> >> avoided, because
> >> 
> >>     bin:golang-github-shirou-gopsutil-dev  should become
> >>     bin:golang-github-shirou-gopsutil-v3-dev
> >> 
> >> In light of this, I suspect that packaging NEW
> >> src:golang-github-shirou-gopsutil-v3 is the right way forward.
> >> 
> >
> > Actually, what I think feels the best way (opinionated) would be to update
> > golang-github-shirou-gopsutil to v3 (keeping the same source package)
> > and then doing an exp upload to looks for breakage and then fixing them
> > and doing unstable upload. This way we keep only one package in the archive.
> >
> 
> Will this really work?  If upstream gopsutils provides both v2 and v3 in
> the same repo, then src:pkg could start additionally providing
> bin:golang-github-shirou-gopsutil-v3-dev...but on the surface it looks
> to me like this would break assumptions related to XS-Go-Import-Path.
> It probably won't save Syncthing from temporarily being dropped due to
> the NEW processing queue lag.
> 
> > Why I haven't choose this way? I guess it's a bit of laziness and the fact that
> > syncthing will be soon removed from testing and I'd love to prevent that.
> >
> > But after re-consideration maybe I shouldn't care that much about syncthing being
> > removed and follow what feels best.
> >
> 
> Thank you :-)  We don't need perfection, but we did commit to working
> towards technical excellence.  I'm honestly not sure what the right way
> forward is, and I truly appreciate this discussion!
> 
> > What do others think?
> >
> 
> +1 please reply!
> 
> >> This discussion is also a cool opportunity to figure out how
> >> https://go-team.pages.debian.net/packaging.html should be updated to
> >> deal with this type of case :-D
> >>
> 
> For the record, I'd be happy to summarise this thread for this
> documentation effort once it becomes clear what the right way forward
> is.  I'm sorry I don't have the Golang background to logic my way
> through the problem.
> 
> > Many thanks for your email and involving the team :)
> >
> 
> You're welcome, and thank you for this challenging and stimulating
> discussion :-)
> 
> 
> Cheers,
> Nicholas