Re: Bits from DPL
Hi!
On Sun, Jan 05, 2025 at 10:02:06AM +0100, Jean-Pierre Giraud wrote:
> Le dimanche 05 janvier 2025 à 06:37 +0100, Joost van Baal-Ilić a écrit :
> > I found
> > https://salsa.debian.org/publicity-team/bits/-/pipelines/792035 about a
> > job processing a commit on the Dec 2024 bits post, but not a (failed)
> > job for
> > typesetting the Jan 2025 bits post. Did I miss something or does the
> > ci/cd need
> > some more tweaking to get that rolling?
That is on me actually. We had a workaround to only fail and stop the
pipeline if markdownlint-2024 gives warnings (obviously it just checked
content/2024/). Well, 2025 came and I didn't change the jobs so it
didn't fail.
> > If I push a commit on content/2025/dpl-bits/bits-from-the-dpl-
> > january.md , will
> > a ci/cd job get kicked off, which will process the markdown?
On every commit pushed to our repo (on any branch) the pipeline will be
triggered (provided you changed any markdown file or the ci file).
> Currently, ci/cd scans all .md files in the /content directory (841
> files) and not just file(s) from the last commit. This is why I found it
> urgent to correct all markdown failures. But apparently the process does
> not prevent building files after the faulty files.
This is going to be corrected in [1], if there is a warning on
markdownlint it will stop and fail the pipeline now. I cannot express my
gratitude for these fixes you did. I was supposed to do them, but I have
had a pretty busy end of 2024, sorry for leaving all the work to you.
> The problem for me is that the process takes time (more than 5 minutes).
So, about this, the problem is more on salsa than scanning the 841
files. In general, it takes 15 seconds to install markdownlint and run
it over all files, but salsa is taking a lot of time to find a runner
and prepare the environment to run it. Salsa admins are aware of the
problem, but it's not an easy fix :-(
> It could be interesting if only the files concerned by the last commit
> are parsed.
I think trying to go this way would be too much of a headache and code
when the problem in my opinion is elsewhere (salsa strugling to keep up
right now).
> If the contributorn who produces a commit with markdown
> failures does not fix their mistakes, the job fails, and someone has to
> clean up periodically...
>
> I think it would be helpful
> 1) if everyone corrected their files when they cause failures (or ask
> help to learn how to do that)
Yes, that would be the perfect workflow!
> 2) if everyone had a tool on their computer to parse and fix files before
> pushing them.
I have added a make check rule that does exactly what the markdownlint
job does. So if make check passes it's assured to work on the CI. There
are instructions on the Readme on how to install markdownlint from the
Debian archive.
> This would save everyone time by also avoiding building a docker image on
> each commit. Perhaps a once-a-day scan of /content would be sufficient
> then.
It actually runs on the tip of the branch, so if you do 5 commits and
then push, it will trigger only 1 pipeline on the head of the master
branch. One can also skip ci doing git push -o ci.skip. Also, I don't
think our repos are that active to justify it.
Hopefuly I was able to address your concerns, jipege :-)
Cheers,
Charles
[1] https://salsa.debian.org/publicity-team/bits/-/merge_requests/101
Reply to: