Re: Index for the /doc directory under ftp.debian.org

[barbier@linuxfr.org (Denis Barbier)]

On Fri, May 10, 2002 at 02:49:24PM +0200, Javier Fernández-Sanguino Peña wrote:
> On Fri, May 10, 2002 at 11:34:06AM +0200, Denis Barbier wrote:
> > 
> > I can only connect between 8:00PM-11:00PM GMT.  Will do tonight.
> > But I am not sure having useful comments to give, my feeling is that
> > we only need to gather documents and write nice indices, and think
> > about automation for future use.  
> 
> Ok. I will try to be there tonight. IMHO we just need a master index
> (which is mostly automated) and automate a way to publish DDP documents
> differently than currently in the web (since those directories IMHO are 
> too cluttered sp?  to provide Content Negotiation).

I will let jfs summarize what was discussed and what has to be done.
Anyway here is some kind of improvement for the *.txt files,

Currently these files are generated by converting English web pages to
text files.  The 2 attached scripts are a draft to generate HTML
translated pages. (they can afterwards be converted to text files)
Javier, if you want to try them, create a directory in which you put these
2 scripts.  Edit copyfiles and set cvsdir to your wemwml root.
Then run
  ./copyfiles
  make
  ./fixhref
Remaining problems:
  * What to do with broken links, i.e. those pointing to not included
    Debian web pages?  We could either remove <a> tags or point to
    www.debian.org
  * What to do with missing translated files?  As there are few pages,
    we could only ship languages which have translated all these pages.
  * Patch our copy of wml::debian::template to define a local layout
    (navbar,...)
  * And of course write an index.xx.html,

Let me know if you think it is useful.  I will then ask translators
to work on missing files, and will continue to work on these scripts.

Denis

#! /bin/sh

cvsdir=../webwml

function copyFiles () {
        [ -d $1 ] || mkdir -p $1
        cp $cvsdir/$1/.wmlrc $1/
        sed -e 's/\$(WMLBASE)/./g' $cvsdir/$1/Make.lang > $1/Makefile
        rm -rf $1/Pics
        mkdir -p $1/Pics
        cp $cvsdir/$1/Pics/*                    \
           $1/Pics/
        sed -e 's/devel\/website\/tc.data/tc.data/g' $cvsdir/$1/contact.wml > $1/contact.wml
        cp $cvsdir/$1/donations.wml             \
           $cvsdir/$1/license.wml               \
           $cvsdir/$1/social_contract.wml       \
           $cvsdir/$1/support.wml               \
           $1/
        [ -d $1/intro ] || mkdir -p $1/intro
        cp $cvsdir/$1/intro/free.wml            \
           $cvsdir/$1/intro/organization.wml    \
           $cvsdir/$1/intro/why_debian.wml      \
           $1/intro/
        [ -d $1/Bugs ] || mkdir -p $1/Bugs
        if grep -i 'href="\./".*</a>' $cvsdir/$1/Bugs/otherpages.inc >/dev/null 2>&1; then
                sed -e '/[hH][rR][eE][fF]="\.\/"/d' $cvsdir/$1/Bugs/otherpages.inc > $1/Bugs/otherpages.inc
        else
                sed -e '/[hH][rR][eE][fF]="\.\/"/,/<\/[aA]>/d' $cvsdir/$1/Bugs/otherpages.inc > $1/Bugs/otherpages.inc
        fi
        cp $cvsdir/$1/Bugs/Access.wml           \
           $cvsdir/$1/Bugs/Developer.wml        \
           $cvsdir/$1/Bugs/Reporting.wml        \
           $cvsdir/$1/Bugs/server-control.wml   \
           $cvsdir/$1/Bugs/server-refcard.wml   \
           $cvsdir/$1/Bugs/server-request.wml   \
           $1/Bugs/
        cp $cvsdir/$1/devel/constitution.wml $1/
        #  Fix links
        for f in $1/Bugs/*
        do
                sed -e 's/Bugs\///g' $f > `echo $f | sed -e 's/Bugs\///g'`
        done
        rm -rf $1/Bugs/
        for f in $1/intro/*
        do
                sed -e 's/intro\///g' $f > `echo $f | sed -e 's/intro\///g'`
        done
        rm -rf $1/intro/
}

langs='catalan danish german english spanish finnish french croatian hungarian italian japanese korean norwegian swedish chinese'

for l in $langs
do
    copyFiles $l
done

cp $cvsdir/.wmlrc ./

#  Data files
cp $cvsdir/english/Bugs/footer.inc           english/
cp $cvsdir/english/devel/website/tc.data     english/
sed -e 's/intro\///g' $cvsdir/english/intro/organization.data > english/organization.data
cp $cvsdir/english/intro/organization.tags   english/
rm -rf english/template >/dev/null 2>&1
mkdir english/template
cp -a $cvsdir/english/template/debian english/template/
#  Specific stuff
rm -rf chinese/bin >/dev/null 2>&1
cp -a $cvsdir/chinese/bin chinese/

#   This script is invoked in english/ directory
cat <<EOT > touch_translations.pl
#! /usr/bin/perl
exit(0);
EOT
chmod a+x touch_translations.pl

#   Fix top-level Makefiles
sed -e '/^LANGUAGES[ 	]:=/,/^$/{:a; /\\$/N; s/\\\n//; ta}' $cvsdir/Makefile |\
    sed -e "s/^\\(LANGUAGES[ 	]:=\\).*/\\1 $langs/"\ > Makefile

sed -e 's/\$(WMLBASE)/./g' $cvsdir/Makefile.common > Makefile.common

#! /bin/sh

langs="catalan danish german english spanish finnish french croatian hungarian italian japanese korean norwegian swedish chinese"
wmlfiles=`echo english/*.wml | sed -e 's/\.wml//g' -e 's/english\///g' -e 's/ /|/g'`
for d in $langs
do
    for f in $d/*.html
    do
        l=`echo $f | sed -e 's/\.html$//' -e 's/.*\.//'`
        perl -pi -e "\
             s{(<a\\s+href=\"[^:\"]*/)\"}{\$1index\"}gi;
             s{(<a\\s+href=\"(?:[^:\"\#]+/)?[^:\"\#.]+)([\"\#])}{\$1.$l.html\$2}gi;
             s{(<a\\s+href=\")[^:\"\#/]+/($wmlfiles)\\.}{\$1\$2.}goi;
           " $f
    done
done