[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Summary of the Debian web team BoF at DC17

[ Please note the cross-post and Reply-To ]

Hi folks,

As promised, here's a quick summary of what was discussed at the web
team BoF session in Montréal.

Thanks to the awesome efforts of our video team, the session is
already online [1]. I've taken a copy of the Gobby notes too,
alongside my small set of slides for the session. [2]

We had a conversation about some of the issues facing the web team.

Initial Agenda
  1. Design work
  2. Migration from CVS
  3. Content

Design Work

People continue to appear on debian-www@lists.debian.org either complaining or
offering re-designs. There are thousands of pages, it's not just a
case of fixing the front page or simply tweaking CSS. If we're going
to make changes, we need continued maintenance, not just short-term
contributors. We have multiple people working on design already, but
we're not following through and making many changes. Why? How can we
make a difference? 

We should be looking at mockups for designs - it's difficult to
evaluate things without that.

CVS continues to be a barrier (more later). It's very difficult to do
"radical" things, and this will put people off. webwml may also be
putting people off. 

If we look into new designs / VCS / backends, it's *imperative* that
we don't lose our translations too - they're critically important.

We should also discuss what part of our website is targeted for which
kind of visitors?

Migration from CVS

CVS is extremely difficult to do radical things (like renaming or
deleting files!) There are differing opinions about how painful CVS
is, but maybe some people are just too used to it and its limitations?
Steve used to be a CVS expert, but recently just working on making
modifications to a small section of the website was effectively
stalled due to the pain of using CVS.

We believe it may be dissuading others from helping, and we should do
something about that. We know that some people are already running
their own CVS<->git conversions and/or gateways right now. Osamu Aoki
has scripts to help with this.

Migrating from CVS to git is not a small task. The models are very

Conversion to git does make for a very large repository, so even small
changes need a full clone. Maybe we can use a git web services
(github/gitlab/similar) to help with this and make it easier for
people to make small changes. We don't want to put off potential

ACTION: Steve is volunteering his own time to make a cvs->git
migration happen this year. He'll need some help to understand more of
the website setup, workflow etc. to make that happen.

Does history matter? Is leaving history in CVS acceptable? Other
options? After some discussion, it became clear that history
preservation has been very useful at times. We came to a clear general
consensus: keeping history is important and we should do that. Moving
to git is, if anything, going to make using history even easier and
therefore more likely. Archaeology in the website history is more
common than most people realise!

There will be more discussion needed around this area in the future,
so we'll get periodic meetings to manage this process. We're not
trying to do the whole thing in one big chunk. Expect to do multiple
conversions as a learning process, with progress visible in the coming
months. Help welcome, but it's also understood that we're struggling
for person power to make this change and that's why we've not done
this yet...

We're also going to be asking for help from non-coding people during
the migration process, to help with verifying things as we go.


How do people currently work on the website source?

 * For some people, check out pages, edit, build locally, commit
 * For other (larjona!), checkout, edit, commit, wait to see if it
   broke things

 * Translators: very different workflow that depends on how CVS works
   (commits are always incremental). There are scripts to help with
   this work, e.g. to mark if translated pages are out of date. This
   does not appear to be well documented anywhere, and this is a
   problem. We'll need to provide new docs for a new workflow, of

There are pages describing how to work on the website, but lots of
people are not finding them / reading them. That's not helping.

Iain Learmonth apparently already started working on VCS-agnostic
interfaces for the website scripts some time ago. Hopefully that
should help us!

Build process

The current process for building the website is very slow. What can we
do to change that?

It's been painful to do changes quickly, e.g. for changing version and
announcing things on release days. There's an "update-parts" script
on the main web server which can help with this. This has been
improved lately:

 * not always pushing to mirrors straight away
 * not descending subdirectories
 * able to change just the top page

"make" is slow for huge trees of subdirectories, and "cvs update" is
very slow too. Can we improve the way that we deploy the website?

It would be lovely to have a staging area for the website build that
we can use. For example, on a release day we could have all the
website work done and ready to ship early. Then when we're done,
simply flip a switch (using a sym-link or similar?) so that everything
is released at once. If we can't make the process *directly* faster,
at least hide the slowness.

We currently have ~8G of content on the website, and this is part of
the reason it's slow!


Is there too much content? There is lots of duplicate information
spread around the website, meaning a lot of it is out of date and

The security section is enormous, and basically static. It sounds like
an ideal set of data to split out.

We want to try and break things up into smaller compartments which can
be owned by teams. non-CVS would make it easier to experiment with
removing content and being able to get it back.

In terms of broken link checking, we apparently already use linklint
(http://www.linklint.org/). This probably needs checking, though - a
human should also check that linked external content continues to be
appropriate. There's a mail alias already existing to see errors from
the website build process. Steve is going to sign up for that!

Suggestion to just auto-switch some broken links to point them to
archive.org maybe? With a more powerful VCS, could we maybe track who
added a link so we can ask them to update it as/when things

[1] http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/debian-web-team-bof.vp8.webm
[2] https://www.einval.com/~steve/talks/Debconf17-webteam-BoF/

Steve McIntyre, Cambridge, UK.                                steve@einval.com
"Because heaters aren't purple!" -- Catherine Pitt

Attachment: signature.asc
Description: PGP signature

Reply to: