Re: Uncoordinated upload of the rustified librsvg
On Wed, Nov 07, 2018 at 08:47:53PM +0100, John Paul Adrian Glaubitz wrote:
> On 11/7/18 8:07 PM, Josh Triplett wrote:
> >> Well, I wouldn't bet on that. I know that a lot of people have the
> >> feeling that rewriting everything in Rust will solve all problems
> >> in software we have nowadays but that's not the case. Rewriting large
> >> projects is associated with a high cost and not many companies are
> >> willing to pay for that. Also, there have already been several
> >> vulnerabilities in Rust and Cargo as well, so the safety is not
> >> really an argument.
> >
> > I really don't feel the need to recreate extensive language arguments
> > here. I think it safe to say that Rust's small handful of documented
> > issues in the standard library pales in comparison to the history of
> > whole classes of bugs in C programs. But the point of this thread is not
> > advocacy, it's simple observation.
>
> I don't think the majority of bugs exist because of C language issues.
Which is not what I said. I said that the small handful of security bugs
in the Rust standard library (none of which had any significant impact)
pale in comparison to the number of security bugs due to C.
And again, this is *not* the thread for language advocacy; there is no
value in having a Rust vs C argument here. This line of argument will
not change whether upstreams adopt Rust, and Debian (and its ports) will
have to handle that. Can we please drop this branch of replies?
> Do you have any data to undermine that argument?
(I think you mean "support that argument"; "undermining" the argument
would be attempting to remove support for the argument.)
There's plenty of such data readily available, not least of which,
summaries of the most common kinds of CVEs; I'm not going to reiterate
it here.
> > I'm not suggesting the world will get rewritten in Rust overnight. It
> > seems a rather safe bet, however, that a non-zero number of additional
> > Rust libraries and binaries will show up in the core ecosystem.
>
> Sure. But also not 95%.
Which is also not what I said. Please drop the hyperbole.
> >>> Running old versions of a library is not a viable long-term strategy.
> >>> Attempting to find alternatives written in C is not a viable long-term
> >>> strategy either; that's running in the wrong direction. Ultimately, the
> >>> new version will need uploading to Debian, and an architecture that
> >>> wants to run a full desktop, or for that matter a server or embedded
> >>> environment, will need to have LLVM support and Rust support.
> >>
> >> I know that. That's why I also criticized the upstream developer,
> >> of librsvg, who happens to be a colleague of mine at SUSE, who was responsible
> >> for that change.
> >
> > For attempting to improve beyond C? Hardly a criticism.
>
> How many CVEs were there on average in librsvg per year that were a result
> of the language?
Note that by saying "improve beyond C", I'm not just referring to
security vulnerabilities.
> Again, any data on that beyond assumption? I am seeing
> 8 CVEs in 6 years. I haven't looked into the details yet whether they were
> a result of the C language or whether they were critical at all:
>
> > https://www.cvedetails.com/vulnerability-list/vendor_id-283/product_id-23082/Gnome-Librsvg.html
>From a *quick* glance, several of them; out of bounds reads, NULL
pointer dereferences...
> >> Will be interesting to see what will happen in the future
> >> when the rustified version of librsvg will try move into the enterprise
> >> distributions.
> >
> > Seems far less likely to encounter issues, given that enterprise
> > distributions target mainstream architectures only.
>
> That's not how enterprise distributions work at all. The main goal is to
> not update at all if it's not necessary. It's not related to the architecture.
I'm well aware of how enterprise distributions work. My point is that
the "regressions" you keep talking about don't apply to a distribution
that doesn't *have* architectures less mainstream than x86 and ARM.
> >>> I think it's reasonable for the *first* such library being uploaded to
> >>> wait a little while to coordinate, which it did.
> >>
> >> It didn't even wait for Rust to stabilize on the architectures it was
> >> recently bootstrapped for. There was no guarantee the Rust compiler will
> >> work on arm32 or mips32 in the foreseeable future.
> >
> > Define "stabilize". And in particular, how were people to know this from
> > https://lists.debian.org/debian-devel-announce/2018/11/msg00000.html ?
>
> Given the fact that you're Rust upstream, I think you should be aware of it.
> If I know about these issues as an irregular Rust upstream contributor with
> just around 20 patches better than you, you're not really making a compelling
> argument.
I am already well aware that architecture-specific issues exist. I'm
also not the person working on fixing them, nor is fixing them something
I have any particular mandate to work on, nor am I (for instance) a MIPS
expert. How are *other* people to know this? An announcement went out
that Rust works on 14 Debian architectures, including every release
architecture. Where is the call for help in maintenance? Where is the
post on the upstream forums asking for assistance? What's visible looks
like a declaration of success, not a call for help.
That's *why* I sent my original post in this thread. Let's try to
arrange help for this.
> >> Given the fact that Rust upstream is always introducing a significant number
> >> of changes with each release, there is quite a chance of regressions of
> >> the compiler on these architectures.
> >
> > This does not relate. The language has active development, like any
> > package that isn't dead upstream. What makes it any *more* likely to
> > have regressions?
>
> The release of completely new upstream versions every six weeks. Compare
> that to OpenJDK, gcc or golang. None of these languages is introducing
> such huge changes on a regular basis.
>
> Are you denying the fact that there isn't a difference between and other
> languages in this regard?
No, I'm holding it up as a feature, like other evergreen software that
gets regular updates and emphasizes ensuring that those updates *don't
break*.
> I have worked with the Rust upstream sources
> well enough to know these issues. You have a regression in Rust 1.25 and
> you will have a very hard time trying to bisect the issues simply because
> you cannot even build 1.25 using a 1.19 compiler because the language
> has changed too much in the mean time.
Why do you need to build 1.25 with a 1.19 compiler in order to bisect?
> I know this is more a problem
> with the compiler source itself than with other packages but I think
> it still shows the current volatility of the language very well.
Development is not volatility.
> > What makes it likely to have regressions is a lack of direct support
> > for such architectures upstream. As a random example: where are the bots
> > that run testsuites on other architectures for PRs?
>
> Well, I know that other languages don't have this issue. You cannot blame
> the lack of these bots on me. I have done as much as I can to help
> Rust upstream beyond their main target architectures. But there is only
> so much energy that I can invest without being paid for that work. I
> have definitely invested a lot of time and energy trying to improve
> Rust. Luckily I have very fast hardware available so that building
> Rust doesn't take too long.
I am *not* blaming this on you, personally. I would hope that support
for these Debian ports depends on more than just you, personally. It
soudns like there's a resource problem. So, again, where is the call for
help? I'll happily go amplify it! I'd be happy to help drive attention
towards the problem, and to work with Rust upstream to provide
accommodations for such bots or other testing infrastructure, and
whatever else might help prevent such issues in the future.
> >>> I don't, however, think it's reasonable to wait indefinitely.
> >>
> >> No one was saying that. But I think it's more reasonable to wait for
> >> the Rust compiler to stabilize
> >
> > Rust is stable. Thank you for your contributions helping it work on more
> > architectures, but "does not have first-tier support for every
> > architecture ever" is not a component of "stabilize".
>
> We're talking in the context of Debian. What is not a tier I architecture
> in Rust is a tier I architecture in Debian. That's the difference. It's
> fine for Rust upstream to set these standards, but it's obvious that these
> standards are in conflict with Debian's standards.
And changing that would require doing work.
> >> There is still no Rust-stable branch in sight which is
> >> most certainly a requirement for Rust to be part of enterprise distributions.
> >
> > This has certainly been discussed upstream, but in general, it's not
> > obvious what this would gain over simply taking any stable release of
> > Rust and packaging it.
>
> Well, if the language continues to add features in new upstream versions,
> there are certainly downstream users interested in using these features
> which will eventually mean that adding package X to SLES will result in
> someone having to backport a newer version of Rust which is not what
> the maintenance teams like to see.
And *there* is where the problem comes in. Enterprise distributions
don't just want to not change, they want to not change except that they
want to add new software. That's not going to work. And trying to make
it work is why people pay enterprise distribution vendors. Again, you
only need to update to new Rust if you update to new upstream software
written in Rust. If you want a distribution that doesn't update to new
upstream versions, don't update to new upstream versions. If you want to
update to new upstream versions, update Rust to a new upstream version.
> >> I know the QA processes associated for SLES to update packages in a release
> >> version and I could imagine that it's not anything less involved for
> >> RHEL or other enterprise distributions. It seems that Rust upstream has
> >> not had any of the enterprise and long-term support distributions in
> >> mind yet. They seem to assume that distributions can just always use the
> >> latest upstream versions.
> >
> > No, we assume that distributions can package Rust alongside Rust
> > software and that the packaged software will work with the packaged
> > Rust. There's no need to use "the latest upstream version"; you only
> > need to update to a new upstream version of Rust if you update to a new
> > upstream version of software written in Rust.
>
> See above. As long as you keep on adding language features the moment
> you release new versions of the compiler, there will be downstream
> users using these features.
I should hope so; they're added because people want to use them.
That changes to C happen on the order of decades does not mean every
other language should emulate that pace.
> >>> If even more coordination had taken place than what already did,
> >>> what would have been the expected outcome?
> >>
> >> A Rust compiler that doesn't regress every six weeks, maybe?
> >
> > It's not reasonable to block the introduction of software written in
> > Rust on some developer (not yet identified) taking the time to
> > contribute the necessary infrastructure upstream to continually test
> > multiple additional uncommon architectures. And that's what would be
> > necessary.
>
> Again, in the context of Debian, this poses a problem. If you as upstream
> say that you don't care if Rust breaks on some of the Rust tier II architectures
> that are release architectures in Debian, there will be a conflict.
I am emphatically *not* saying that. I personally *do* care. But someone
has to do the work.
> Either Rust accepts more architectures as tier I
(Note that "tier 2 but with tests actively run" is another possibility.)
Doing this will require someone who works on the architecture to put in
significant work upstream, just as they do in Debian. There's not a lack
of willingness.
> or you convince Debian to
> make anything but arm64 and x86_64 a release architectures.
I'll refrain from making comments here that would likely be taken
poorly.
> There is a
> clear conflict of interest here which is, again, why I think it's a bad
> idea to use Rust code in a core component like librsvg.
>From the perspective of Debian's less common ports, perhaps. But here's
the fundamental problem, which no amount of discussion in a mailing list
thread is going to solve:
Debian doesn't get to offload the work of supporting less mainstream
architectures to the rest of the ecosystem. Less mainstream
architectures with less developer resources don't get to say "wait up,
slow down".
> >>> Speaking with an upstream Rust hat on in addition to a
> >>> Debian hat: what could Rust do to make life easier for porters?
> >>
> >> Please provide an actual stable version of the Rust compiler that
> >> is supported in the long term and can be shipped by enterprise
> >> distributions.
> >
> > There's a stable version of the compiler every six weeks. Pick one and
> > ship it.
> >
> > If, instead of "stable", you mean "supported on other architectures",
> > that's going to require upstream infrastructure to *test* those
> > architectures on a regular basis.
>
> With stable I mean that I can assume that Rust 1.X compiles the same code
> as Rust 1.Y.
Which you can in fact assume; Rust 1.X will compile the same code as
Rust 1.Y for any X > Y, that's the *fundamental* stability property of
Rust.
> >>> And what could Debian's *considerable* expertise in porting do to make that more
> >>> sustainable upstream? (As an example, it might help if upstream Rust
> >>> folks had access to machines for more architectures, though that's a
> >>> side issue for having an LLVM port in the first place.)
> >>
> >> Debian Ports has worked closely with QEMU upstream to help make significant
> >> improvements to that emulator. So, in most cases, Rust developers can just
> >> use QEMU for the first porting efforts. But there are also porterboxes available
> >> from gcc to which we from Debian Ports also have provided hardware, for example:
> >
> > I'm more suggesting that if people want to see an architecture better
> > supported, it needs to end up in at least tier 2 on
> > https://forge.rust-lang.org/platform-support.html .
>
> Yes, I'm aware of the tier model. I just think upstream should keep more
> invasive changes for a 2.x version and not rewrite half of the compiler
> every six weeks.
Again, please drop the hyperbole.
There is unlikely to *ever* be a Rust 2.x, because there's no intention
to drop backward compatibility with existing Rust code.
Reply to: