Re: Need help with lazy module loading & gvisor

[John Goerzen <jgoerzen@complete.org>]

On Mon, Sep 05 2022, Shengjing Zhu wrote:

> On Sun, Sep 04, 2022 at 05:44:36PM -0500, John Goerzen wrote:
>> Hi folks,
>>
>> At least two Debian packages use https://github.com/inetaf/netstack,
>> which was a popular pure-Go TCP/IP networking stack.  It was a fork of
>> gvisor, pulling out the networking stack from what was a more ambituous
>> pure Go operating system project.
>>
>> gVisor is indeed /huge/.  Before one of the upstreams had switched to
...
>
> For Debian packaging, you can exclude unneeded pkgs with
> DH_GOLANG_EXCLUDES environment in debian/rules file.
> https://manpages.debian.org/unstable/dh-golang/Debian::Debhelper::Buildsystem::golang.3pm.en.html#DH_GOLANG_EXCLUDES

So I'm going down this path.  In debian/rules, I have:

export DH_GOLANG_EXCLUDES := pkg/api pkg/binary pkg/bitmap pkg/bpf pkg/buffer pkg/cleanup pkg/compressio \
        pkg/control pkg/coretag pkg/coverage pkg/crypto pkg/errors pkg/eventchannel pkg/eventfd pkg/fd \
        pkg/fdchannel pkg/fdnotifier pkg/flipcall pkg/fspath pkg/hostarch pkg/hostos pkg/ilist \
        pkg/lisafs pkg/marshal pkg/memutil pkg/metric pkg/p9 pkg/pool pkg/procid pkg/ring0 pkg/safecopy \
        pkg/safemem pkg/seccomp pkg/secio pkg/segment pkg/sentry pkg/shim pkg/sighandling pkg/syncevent \
        pkg/syserr pkg/test pkg/trie pkg/unet pkg/urpc pkg/usermem pkg/xdp tools \
        tools/bazeldefs tools/bigquery tools/checkaligned tools/checkescape tools/checkinfo \
        tools/checklinkname tools/checklocks tools/checkunsafe tools/constraintutil tools/github \
        tools/go_fieldenum tools/go_generics tools/go_marshal tools/go_stateify tools/nogo \
        tools/parsers tools/tracereplay tools/xdp tools/yamltest gvisor/tools gvisor.dev/gvisor/tools \
        src/gvisor.dev/gvisor/tools/xdp gvisor.dev/gvisor/tools/xdp _build/src/gvisor.dev/gvisor/tools/xdp
export DH_GOLANG_EXCLUDES_ALL := 1
export DH_VERBOSE := 1

But during build it is trying to do something with tools:

r.dev/gvisor/tools/go_marshal/test
found packages parsers (go_parser.go) and main (parser_main.go) in /home/jgoerzen/work/nncp-debian/golang-gvisor-gvisor/_build/src/gvisor.dev/gvisor/tools/parsers
_build/src/gvisor.dev/gvisor/tools/xdp/main.go:53:13: pattern bpf/drop_ebpf.o: no matching files found
        cd _build && go install -trimpath -v -p 16
package .: no Go files in /home/jgoerzen/work/nncp-debian/golang-gvisor-gvisor/_build
dh_auto_build: error: cd _build && go install -trimpath -v -p 16 returned exit code 1
make: *** [debian/rules:18: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
debuild: fatal error at line 1182:

(you can see I tried everything I could think of to stop the tools stuff
from being installed)

I've pushed the tree I have so far to
https://salsa.debian.org/go-team/packages/golang-gvisor-gvisor so you
can take a look if you like.

Thanks!

- John

>
> The Go compiler can exclude the unneeded pkgs itself, but when packaging
> for Debian, we don't have such automatic solution, you need to figure out the
> unneeded pkg list yourself, then set DH_GOLANG_EXCLUDES.
>
> One tricky is to run `go mod vendor` in the package that imports the one that
> is to be packaged, and see what Go compiler thinks that should be included
> in vendor directory.
>
> And for gvisor case, another tricky is just to see what inetaf/netstack
> includes.