Bug#1030991: lintian: checking intel-mkl takes 18 hours

[Nilesh Patra <nilesh@debian.org>]

I wanted to renew the discussion around this bug

> my %copy = %{$self->index->elf_storage_by_member->{$self->name} // {} };
>
> So basically copying data in RAM around seems to be the main cause for
> taking so long on these binary packages.
>
> Accordingly I've made this temporary local change to check what it
> would be without copying around that data:
>
> diff --git a/lib/Lintian/Index/Item.pm b/lib/Lintian/Index/Item.pm
> index 7ef6c0994..87d766cbe 100644
> --- a/lib/Lintian/Index/Item.pm
> +++ b/lib/Lintian/Index/Item.pm
> @@ -1434,9 +1434,10 @@ sub elf_by_member {
>         return ();
>     }
> 
> -    my %copy = %{$self->index->elf_storage_by_member->{$self->name} // {} };
> +    #my %copy = %{$self->index->elf_storage_by_member->{$self->name} // {} };
> 
> -    return \%copy;
> +    #return \%copy;
> +    return $self->index->elf_storage_by_member->{$self->name} // {};
> }
> 
> =item pointer
>
> (Patch and all tests based on the 2.116.3 release.) 
>
> Not copying that data and just returning the original cuts the run
> time down from 5m 7s to 1m 56s, i.e. by about 60%.
>
> On the largest package (libmkl-computational-dev) it went down even a
> much bigger percentage, from close to 3 hours (178 minutes) down to only 3
> minutes and 32 seconds.
>
> The question is now: Do we really need a separate copy of that data
> for each call? Sure, if it's modified afterwards, this makes sense.
> But I have no idea where this might happen.

I only skimmed through the code where it is called, and it appears to me
that it is not getting modified anywhere as such.

If it even does change the data in some operations, we can probably copy 
elevant structures only for those sections instead of issuing a copy everywhere.

On top of this, the subroutine is called at multiple places (unless I am mistaken, there are more
than one redundant copies made for checking same elf file) through different
lintian checks.

> So I've run the test suite on the above mentioned modification and to
> my surprise it passed. 😲

May be it makes sense to revert https://salsa.debian.org/lintian/lintian/-/commit/192e15ae2eda7535622d44d2f3f382783b110ee5 altogether?

It was added to mitigate #1003456 and Ben said there that it is still not fixed.
Further, the author of this change commented that the MLDBM database does not
get deleted during global dest https://lists.debian.org/debian-lint-maint/2022/04/msg00013.html

Thoughts?

Thanks
Nilesh