Re: [External] Re: ThinkPad laptops preinstalled Linux
Hi Mark!
On 2020/06/03 15:39, Mark Pearson wrote:
>> I guess it should be possible, but the biggest challenge is probably to
>> get the right contact to talk with. Do we have such a contact ?
>>
> I started replying to this thread late last night and after a few iterations I gave up and went to bed 😊 Here goes attempt 5....
First of all, thanks for the great e-mail and making the time to reach
out. By way of introduction, I'm the current Debian Project Leader
(since a bit more than a month now) and I am happy to be available to
help speed things up, feel free to reach out to me personally at any
time if you're frustrated with getting something done in Debian.
> Short answer - you have the right contact. I'm the linux technical lead at Lenovo for the PC team and I'd *love* to improve the Debian experience on Lenovo platforms. It's something I have actively been trying to work on for the last 9 months (with limited success). I'm really happy to see this question and the responses - I would really like to have this conversation with the wider community as to what Lenovo and Debian can do to work better together. Debian is an important distro with a lot of users and an amazing community. I was hoping to be at Debconf 2020 and use that as an opportunity to actually get to meet Debian devs as frankly email has been working poorly and maintainers are just very busy people. Sadly I think Covid has likely stomped on that plan.
Talking at DC20 would've been great, but we can also talk via email and
video calls in the meantime!
> So and let me start the ball rolling by highlighting some good and bad:
>
> The good - Lenovo are expanding what they offer on Linux (we had another big announcement yesterday about doing full config on our workstations with Ubuntu and RHEL). We're asking HW vendors to have Linux support upstream including firmware on LVFS which I think is important. It's not perfect yet but it's getting better and better. We're starting to contribute to open source projects.
> I'm an open source user and a lot of what I see from Lenovo internally is positive and makes me very happy to advocate for what they are doing. Sometimes our process is a bit slow but there is a commitment to doing Linux and doing it right which I'm personally excited by. The Fedora collaboration has worked really well - I think for both sides (obviously talk to your Fedora colleagues for their perspective). They have been a really positive community to work with and it's been productive showing that Lenovo can collaborate with an OS community in the right way.
> Internally at Lenovo there is still a lot of learning about Linux and how it works - and that is happening. Linux is catching on - I have more customer engagements and our Linux sales have been increasing and we haven't turned our websales on yet (coming soon 😉)
>
> The bad - I love Debian but I don't think as a distro you are ready or capable for being pre-loaded on our platforms. I can explain why and I'm very happy to be corrected but more importantly if it can be solved then we can for sure explore the next steps - that would make me really happy!
I 100% agree with you, in my first talk as DPL, this is even a topic
that I've covered when talking with the Brazillian Debian community:
https://peertube.debian.social/videos/watch/cdfe5b24-e3ad-4422-a6f2-15ca4fffd895?start=43m52s
(I talk about this for about 5 minutes, and it was almost midnight and I
was up since 5am so sorry for maybe being a bit incoherent at times)
Anyway, let me summarize what I said on the topic in that video with
some additional notes.
Firstly, I think the typical Debian developer might not care about
Debian being available on an OEM laptop per sé. First thing that most of
us would probably do is to re-install it from official installation media :)
But there's probably a vast amount of Debian users out there that would
want a Debian on their ThinkPad pre-configured. And we'd be delighted
from the Debian side if we could help make that happen.
So in the video I posted some of the initial thoughts that I think we
need to sort out in Debian before we can even start thinking about this.
I'm glad you reached out because you'll be able to give us much better
insight for sure, but at least I have some video evidence to prove that
we put at least *some* thought in to it :)
1. Support
I'm not sure what you did in the case with Fedora, but my guess is you
have some agreement with how to handle support calls for Fedora on the
system. Debian isn't backed by any one company, and in the immediate
future our usual community support channels might be as good as it gets.
I suppose if users know this, understand this, and know what they're
getting, then this becomes less of an issue.
2. OEM Mode installer
Currently we don't have an OEM mode install mode in Debian (that is,
skip questions like timezone, user, password, etc and ask that on first
boot instead).
I believe that there's been some work to make a debian-installer module
for this, but it's been shelved since.
Either way, this isn't a difficult problem to solve on a technical
level, and if we need it to get Debian available on laptops then I think
it would happen fast.
3. Rescue partition
Laptop manufacturers usually don't ship with physical media anymore.
Instead, the laptops have a rescue partition on them for
re-installing/resetting the machine.
As far as I know both installers we currently use in Debian are fine
from installing from a rescue partition, we just need a nice way to set
that up when initially performing an oem style setup from our
installation media. (again, not a huge technical problem, but probably a
bit more work than #2).
...and then there's hardware specific stuff...
> Debian moves somewhat slowly. The general approach seems to me to be to wait for updates to trickle down from the upstream community and I completely understand that approach - I'm sure it is important to maintaining stability *but* it means that Debian is usually broken on our platforms for the first year they are available because that whole process takes a long time. By the time Debian works it's too late for a pre-loaded offering. As an aside RHEL has the same problem but they put a lot of effort into backporting fixes so that their customers get the fixes - and that infrastructure doesn't seem available on Debian to my limited knowledge.
We have processes for both stable release updates and backporting too.
Where they fall short, I'm sure we can look into it further.
> As an important example - the X1 Carbon 7 (which is a popular machine) still doesn't work well with any version of Debian (including experimental or testing) as the audio is broken. Debian users have to jump through a few hoops to get it to work. I've let the maintainer know a number of times what is involved to fix that but it's obviously not a priority (as a heads up - Debian on most Lenovo 2020 platforms is going to suck because of this too). I'm not meaning to point fingers - but just explain why it feels as if Debian and the latest hardware is an awkward fit.
I think what we need to figure out from the Debian side is, whether it's
possible to integrate these kind of 'hot fixes' using our usual
procedures and processes, or whether we might need a lenovo-specific
image and even perhaps its own package archive. We'd of course like to
avoid that, but just from me personally, I think it's worth exploring
what it would take to make it work. If you're willing to share what
worked with RHEL, then maybe we can adapt some of that to our processes too.
> From my point of view what I've been trying to do is to get more involved so I can contribute/backport fixes directly. I get good insight into what issues impact our platforms and when fixes land upstream. It seems the best way to make contribute and make a difference. Unfortunately I've still got a *lot* of learning to do and it's a really slow process because the loop between offering a fix, getting it reviewed to find out what you did wrong and contributing the update is crazy slow (for example I have kernel MR 240 open for four weeks for an OLED brightness issue that a lot of users think is important). My expectation is that as I make fewer mistakes and earn some trust that will help - but until that point (which I'm guessing will take years 😊) I have limited handles that I can pull on to make things happen.
You're right that the trust factor is important in Debian. It helps when
there's a Debian Developer or two who can guide you through some of the
murkier aspects of the project. Again, if I can help you with any of
that, I'd be happy to. I don't think it needs to take years though :)
> My *impression* is there is limited desire to accelerate fixes for Lenovo platforms - I suspect mostly because people are just plenty busy with the things they care about instead and I understand that.
I think we generally care about the hardware that we have, and that our
employers typically buy and deploy. I think "other people's hardware
problems" will always be less exciting than your own. Usually when more
of the latest hardware starts hitting us personally we tend to care
more. I think part of this is just natural.
Maybe since I have your attention I can share some of my thoughts too
that might be directly relevant to the above.
We have many ThinkPad users in Debian, a very large percentage of Debian
Developers have owned at least one ThinkPad (and some multiple ones over
the years, and some multiple ones even at the same time!)).
The work that these Debian Developers do typically go towards helping
those Lenovo laptops work better with Debian. I understand and feel your
frustrations getting those latest features to work in Debian, but Debian
people have been doing that for decades now, which is great, I'm not
complaining. I like that Debian has run so well on my ThinkPad laptops
for all these years. But, in recent years I've also started to think
that it's kind of horrible and ridiculous that Debian Developers pay
full retail price for their laptops considering how much work they put
in making Debian (and often other Linux systems) run better on it. I've
been considering reaching out to both Lenovo and Dell (and other
manufacturers that are popular or important within the project) to see
if we can get much better pricing for our members, whether they're just
closer to cost or even a little subsidized. Maybe you could shed some
light on this topic too :)
On another note, we sometimes use older laptops not because we're
necessarily stuck with them, but also because we don't want to throw
away/discard perfectly good machines just for the sake of having a new
one, while we could use our older machines and help do a bit to protect
the environment at the same time. My x250 is 5 years old and it's still
adequate for me, and while I often have the itch to buy a new laptop (I
very nearly did buy a friend's 7th gen X1 last week, which he bought
sadly just before he lost his job), but decided to hang on for a Ryzen
based version instead. Sorry, I digress :)
So, we have some people (like me) who use 5 year old (and even older)
laptops and are fine with that, but we also have some developers who are
using really obsolete machines and from the Debian side, I'd like to
make it easier for them to upgrade if we can. Debian can already sponsor
some hardware for developers in need, but personally my vision is that
we can work with a hardware manufacturer to get an already
good/subsidised price that we could perhaps help reduce a bit further as
a benefit for our developers. Over the last weekend we had our first
ever online minidebconf, and in these times being stuck on a core 2 duo
is tough :)
Anyway, I probably overshare but you've put in a lot of thought into
your mail and I thought I'd reply properly too.
> A shout out for Hector Martinez who has been helping me a whole bunch. Without his help I likely would have given up and wouldn't even be reading the threads on this forum. If there are more people like Hector (particularly in kernel, audio and graphics) let me know!
As I become more aware of your requirements I can help find more Hectors
too :)
> This email took me a long time to write - I'm *very* aware that I'm new to this and don't want to cause offence. Please take all of the above with the recognition that my viewpoint into Debian is still limited and if I've said anything dumb/wrong/offensive let me know so I can learn what I'm missing.
No offense taken whatsoever, Debian Developers are often the biggest
critics of Debian. Not because they want to be mean, but because they
care and want to see things get fixed. Welcome to Debian and I hope that
you'll join us more formally (and perhaps even vice-versa?) in the future!
-Jonathan
--
⢀⣴⠾⠻⢶⣦⠀ Jonathan Carter (highvoltage) <jcc>
⣾⠁⢠⠒⠀⣿⡁ https://wiki.debian.org/highvoltage
⢿⡄⠘⠷⠚⠋ https://debian.org | https://jonathancarter.org
⠈⠳⣄⠀⠀⠀⠀ Debian, the universal operating system.
Reply to: