Re: Updates to linux-firmware
Hi Diederik,
On Tue, May 7, 2024, at 5:22 AM, Diederik de Haas wrote:
> On Tuesday, 7 May 2024 08:25:23 CEST Didier 'OdyX' Raboud wrote:
>> What I did back then to try to help getting the firmware package in better
>> shape was to dive in the (complex) packaging and try to produce "ready-to-
>> merge" merge-requests for new upstream releases, fixes, etc, then pinging
>> Ben at (what I considered) reasonable intervals. I see that Diederik
>> (CC'ed) has started to do the same for recent versions.
>
> What I did was indeed my effort to get things into better shape with the hope/
> goal that it would be easier to do going forward. Which in turn hopefully
> means that it would be done more regularly which would not make it a
> monumental task ... which no-one is looking forward to do or has the capacity
> (time/motivation/etc) to do.
>
>> The package is complex and a mined land of legal concerns,
>
> It was a HUGE amount of work, partly because I wanted to get things into what
> I consider a better shape. Another part which made it *look* complex, were
> inconsistencies. In one of my commits I determined in which sequence the
> control fields were going to get placed and then fixed all the config files to
> conform to that sequence. I also ordered the entries alphabetically.
>
> I haven't documented it (yet), but I followed a different procedure
> then 'the
> official one', which I think makes it easier and better ... lowering
> the barrier
> for future contributions/MRs.
>
> It was basically the following:
> 1) Have the upstream repo locally and checkout the next tag/release and open
> that with gitk
> 2) Start a new branch in your local Debian firmware-nonfree repo
> 3) Go through each commit and decide what to do with it
> - Upstream merge commit: ignore it
> - Upstream working on their own infra: ignore it. Unless it has an effect on
> (future) Debian packaging, then add it to d/changelog. And use the
> secondary commit message to properly document it.
> - Updated version of firmware files: if it has a(n explicit) version number,
> update the version in the appropriate ``defines`` file.
> - New links (documented in WHENCE): Add that entry to the ``files:`` section
> of the Debian ``defines`` file.
> - New files: Add an entry to the `files:` section and add a Stanza for it with
> its 'ID'/Name, description and if available the version number
> - Check whether d/copyright covers any new files and if it doesn't expand the
> 'regex' so that it does if the license is already known. Otherwise add an
> entry and the license text to d/copyright*
>
> With updated versions or new links or new files, add an entry to d/changelog
> which is mostly or fully upstream's primary commit message, prepended by the
> Debian package name. This way there's an almost 1-on-1 match between the
> Debian and upstream's commit (message). This should also make reviews easier.
>
> I ordered the entries by Debian package name, so that if someone only has AMD
> graphics firmware, it doesn't have to read the full changelog, only the part
> which is about amd-graphics. Upstream's git commit log sequence is rather
> random, which isn't useful or helpful to our users (IMO ofc)
>
> *) I did improve d/copyright too, but not fully. All the license texts have
> been moved to the bottom, but I didn't fix the sequencing of all the stanzas.
>
>> the upstream repo needs repacking,
>
> Upstream now also produces a deb package per tag/release, but that's 350MB in
> size ... and growing. And it includes firmware which isn't DFSG compliant and
> therefor we shouldn't ship or f.e. can only be shipped as part of the kernel,
> so we can't distribute them.
>
>> and the (upstream & debian/) repositories are split;
>
> I think the split is very useful, especially for our users. Some users only
> need one or a few small firmware files. Not having to download >350MB for that
> for every upstream/Debian release/version is quite useful. Not everyone has
> unmetered GbE internet connection ... or wants to waste 300+MB for files for
> which they have no need/use.
>
>> I have not found this a very pleasant experience, sadly.
>
> I didn't find it pleasant either, but I hope/expect that with my updates and
> restructuring it shouldn't be that bad either.
> When things are kept up-to-date it's quite doable, but when you have to work
> through months of 'backlog' the 'fun' quickly diminishes.
>
>> And again; no-one to blame here: there _is_ tooling to help, and immense
>> work has been put towards making this manageable! I'm merely saying it's not
>> a matter of running "debian/rules get-new-upstream" at all.
>
> The tooling is mostly great.
> *I* do think that the script which imports upstream's changelog should go
> though as it makes 'you' think you're quickly done. But if you want to do
> things right, you should still go through all the commits to actually update
> the versions and add the new links and new files.
> If you don't, and the temptations is IMO rather high, then you'd create
> another monumental task like the one I went through.
> And that REALLY did not make me happy or considered fun. At all.
>
> As I see it, the primary problem is the lack of people actively contributing
> to the Debian kernel team's work. In general.
Interesting read, and combined with my notes to Didier - is this something where I and some in my team could maybe help?
If you think it would be worthwhile, does it make sense to setup a training session to walk me through the steps so I can attempt a first pass, and then review where the problems are and which bits are useful/not-useful.
If the review/link updating/etc was all done - would it make it easier for a DM to look at it and go "yeah, that's solid" and release?
>
> The amount of open *kernel* bugs is usually around 800. And that's mostly due
> to periodic mass closing of bugs. I have been triaging (kernel) bugs in the
> last say 2 years, but it was mostly me and Salvatore doing that work.
> I asked for assistance but I didn't see anyone step up. And I plan to shift
> (part of) my time/focus to other things and my worry is that that means that
> bug reports get completely unanswered. Like before.
> And having 800+ kernel bugs open isn't helping motivation to spend time on
> non-free stuff like firmware.
>
> Most of the time triaging (kernel) bugs isn't that hard . It usually boils
> down to finding when the issue surfaced (last working version vs first non-
> working version), possibly/ideally bisected to a specific commit and then
> finding the appropriate upstream maintainers and let the bug reporter and the
> upstream maintainer take it from there.
> It's just time consuming.
>
> FTR: Offering to help: great.
> Creating NEW bugs with vague/non-actionable descriptions basically asking for
> new upstream versions ... of which there are already several open in the BTS:
> not so great. It just makes the mountain (of work) higher.
>
My apologies - I really wasn't trying to add to the problem. Not my intent at all. On all other distro's that I work with a bug is usually needed for tracking any work done (and I did reach out by email first :))
My aim with the bugs is to highlight where support is missing for new Intel Meteorlake/AMD Hawkpoint platforms for this years platforms, it wasn't supposed to be vague. I was hesitant to raise the bug, as it is so broad - but it's a valid issue. I do have users on our forum trying to run Debian and are not able to.
I can't highlight all patches needed for Meteorlake/Hawkpoint support - that would be huge. I can highlight that a 6.8 kernel is needed; and which FW packages are needed.
As an aside, it would apply for all system vendors too...if that makes any difference.
I'd really like to figure out how to do something constructive to help with new platforms and getting Debian being friendly to them. I 100% appreciate that time is precious and limited (I do suffer from that too). If feedback from the community is that this isn't of interest then let me know and I shall retreat back under my rock.
Thanks
Mark
Reply to: