This is by far the largest patch in the series. I've put it first because it significantly simplifies a number of following patches. I have edited the attached patch somewhat to remove the most boring changes, while still keeping it clear (hopefully) what the changes are. The main purpose of the patch is to significantly simplify the maintenance of the lists of language tasks in debian-cd for releases. Because the lists are now generated automatically there is also less chance of copy/paste or similar errors (of which we have seen examples in the past!). The patch includes a README.tasksel file which explains how the new code works. Please read. Functional changes introduced with this patch ============================================= There are basically two changes: 1) order in which language tasks are grouped 2) inclusion of language tasks for "secondary" desktop tasks 1) order in which language tasks are grouped -------------------------------------------- As can be seen in the patch in the old situation language tasks were grouped by *language*, so you'd have [1]: dutch dutch-desktop dutch-gnome-desktop french french-desktop french-gnome-desktop german german-desktop german-gnome-desktop In the new situation they will be grouped by *type* (plain language tasks first; general desktop tasks next; desktop environment tasks last): dutch french german dutch-desktop french-desktop german-desktop dutch-gnome-desktop french-gnome-desktop german-gnome-desktop The impact of this is probably less than you may think: - it only makes a difference for the GNOME and KDE CD1, and then only for installations done without use of other CDs or network mirror - it only makes a difference for "Key" packages, not "regular" packages However, languages early in the alphabet may get a bit less "complete" support than they previously had. IMO the new order is a lot more fair as it puts all languages on a more equal footing, but the order can be changed back if there are solid reasons to do so. [1] Languages used for examples were taken completely at random... not. 2) inclusion of language tasks for "secondary" desktop tasks ------------------------------------------------------------ By secondary desktop tasks I mean kde-desktop and lxde-desktop for the full CD set. These _are_ listed in the standard task list, but only after the language tasks. Key packages from these tasks are only included after the regular packages for other tasks. Until now language tasks for these desktops were ignored. With the new code they _do_ get included, but again at the very end (but before packages are added from the popcon list).
Split out list of languages supported by D-I and tasksel from task.list files
update-tasks is changed to expand the task.list files to again
include the language tasks. The difference between primary tasks
and extra tasks is now indicated in the task.list file. Packages
for extra tasks are not included in the "essential" lists, but
only in the "full" lists.
diff --git a/CONF.sh b/CONF.sh
index 5011e14..c28512c 100644
--- a/CONF.sh
+++ b/CONF.sh
@@ -185,6 +185,10 @@ export DISKTYPE=CD
#export DISKTYPE=CUSTOM
#export CUSTOMSIZE=XXXX
+# List of languages for which language tasks from tasksel should be
+# included. See tasks/README.tasksel for further info.
+export TASK_LANGLIST=tasksel_d-i.languages
+
# We don't want certain packages to take up space on CD1...
#export EXCLUDE1=exclude
# ...but they are okay for other CDs (UNEXCLUDEx == may be included
diff --git a/Makefile b/Makefile
index 316659e..64565ec 100755
--- a/Makefile
+++ b/Makefile
@@ -118,12 +118,16 @@ $(BDIR):
$(ADIR):
$(Q)mkdir -p $(ADIR)
$(TASKDIR):
+ $(Q)echo "Updating task files..."
$(Q)mkdir -p $(TASKDIR)
$(Q)cp -r $(BASEDIR)/tasks/$(CODENAME)/* $(TASKDIR)
$(Q)set -e; cd $(TASKDIR); \
$(BASEDIR)/tools/update_tasks; \
$(BASEDIR)/tools/generate_di_list; \
$(BASEDIR)/tools/generate_di+k_list
+ $(Q)echo "- task.languages: using 'tasks/$(DI_CODENAME)/$(TASK_LANGLIST)'"
+ $(Q)cp $(BASEDIR)/tasks/$(DI_CODENAME)/$(TASK_LANGLIST) \
+ $(TASKDIR)/task.languages
$(BDIR)/DATE:
$(Q)date '+%Y%m%d' > $(BDIR)/DATE
diff --git a/debian/CONF.sh b/debian/CONF.sh
index 5011e14..c28512c 100644
--- a/debian/CONF.sh
+++ b/debian/CONF.sh
@@ -185,6 +185,10 @@ export DISKTYPE=CD
#export DISKTYPE=CUSTOM
#export CUSTOMSIZE=XXXX
+# List of languages for which language tasks from tasksel should be
+# included. See tasks/README.tasksel for further info.
+export TASK_LANGLIST=tasksel_d-i.languages
+
# We don't want certain packages to take up space on CD1...
#export EXCLUDE1=exclude
# ...but they are okay for other CDs (UNEXCLUDEx == may be included
diff --git a/tasks/README.tasksel b/tasks/README.tasksel
new file mode 100644
index 0000000..7c17e5f
--- /dev/null
+++ b/tasks/README.tasksel
@@ -0,0 +1,137 @@
+This README explains how packages are added to full CDs and DVDs based
+on tasks defined in tasksel.
+
+Note that this document describes the default configuration for debian-cd.
+
+Overview
+========
+One of the main factors deciding in what order packages are added,
+especially for the first DVD and earlier CDs, is the definition of which
+tasksel tasks should be used.
+
+The list of packages to be added is generated dynamically at build time.
+The procedure is somewhat complex and involves a number of (intermediate)
+files.
+
+The general sequence in which packages are added for "complete" images is:
+1) Packages required by Debian Installer
+2) Packages required to install the base system (debootstrap)
+3) Any packages with priority "standard" or higher
+4) Packages from tasksel tasks
+ a) Packages defined as "key" packages for primary tasksel tasks
+ b) Packages defined as "key" packages for language tasks (in the order:
+ '<language>', '<language>-desktop', '<language>-<desktop>-desktop');
+ only language desktop tasks that correspond to a primary task are
+ included
+ c) Regular packages from primary tasksel tasks
+ d) Regular packages from language tasks (as in 4)
+ e) Any packages from secondary tasksel tasks
+ f) Any packages from language tasks corresponding to secondary tasksel
+ tasks
+5) Other packages in the order of their popularity (popcon score)
+
+A tasksel task is defined as "secondary" by adding a '-' after its name
+in a task.list file; other tasksel tasks are "primary".
+
+Language tasks are added grouped per type, not per language. So with three
+languages (A, B, C) and 'desktop' and 'gnome-desktop' as tasks you'll get:
+A
+B
+C
+A-desktop
+B-desktop
+C-desktop
+A-gnome-desktop
+B-gnome-desktop
+C-gnome-desktop
+
+Note that <language>-desktop tasks are always sorted before any
+<language>-<desktop>-desktop tasks.
+
+Files involved
+==============
+Static task files (can be found in tasks/codename):
+- Debian or Debian-<desktop>:
+ top level task file; defines order of other task files to be used,
+ including task-essential[-<desktop>] and task-full[-<desktop>]
+- tasksel_d-i.languages:
+ list of languages used to add language tasks at build time (see detailed
+ explanation below)
+- task.list or task.list.<desktop>:
+ tasksel tasks to be included
+
+Dynamically generated task files created at build time (can be found in
+the subdirectory tasks under the working directory for the build):
+- task.languages
+ copy of the actual language list to be used to add language tasks
+ (see "Language list used at build time" below)
+- task.list[.<desktop>].essential:
+ contains primary tasks from task.list[.<desktop>] followed by
+ corresponding language tasks
+- task.list[.<desktop>].full:
+ copy of task.list[.<desktop>].essential; followed by secondary tasks
+ from tasks from task.list[.<desktop>] and corresponding language tasks
+- task-essential[-<desktop>]:
+ list of key packages generated using task.list[.<desktop>].essential;
+ corresponds to 3a/3b from overview above
+- task-full[-<desktop>]:
+ list of packages generated using task.list[.<desktop>].full; corresponds
+ to 3c-3f from overview above
+
+The generation of these files is done early in a build by the script
+tools/update_tasks.
+
+The file <codename>/tasksel_d-i.languages
+=========================================
+This file contains the superset of all languages that have language tasks
+in the <codename> version of tasksel. The list should contain the language
+names as used in tasksel.
+Any languages that are not supported in the <codename> version of Debian
+Installer should be commented out. Reason is that it makes no sense to
+include them on early CDs as they will never be used during installations
+and thus the space can be better used for other packages.
+
+The following procedure can be used to update the language list:
+- delete the current list of languages (leaving leading comments) from
+ <codename>/tasksel_d-i.languages
+- get the current source for tasksel and do:
+ $ cd tasks
+ $ grep -l "Test-lang" * | sed "s/-desktop//" | sort -u
+- add the resulting list to <codename>/tasksel_d-i.languages
+- check what languages are supported in the version of localechooser for
+ <codename> by checking the file 'languagelist' in its source
+- comment out any languages not supported by localechooser (either not
+ present or commented out in 'languagelist')
+- check the diff between the old and new version and double check any
+ changes
+
+Language list used at build time
+================================
+The language list that is used to add languages tasks is a copy of
+tasks/$DI_CODENAME/$TASKSEL_LANGLIST. By default TASKSEL_LANGLIST is set
+to 'tasksel_d-i.languages' (see CONF.sh).
+
+Note that the file is taken based on DI_CODENAME and not CODENAME! Reason
+is that if those differ DI_CODENAME will almost always be the "newer"
+release and it is better to have a list that reflects the languages
+supported by the version of Debian Installer that is used, especially
+since any languages not supported by the CODENAME version of tasksel will
+be filtered out automatically by the tools/update_tasks script.
+
+Using a custom language list
+----------------------------
+In some cases it may be desirable to only include a subset of the language
+tasks available in tasksel. For example if you're building images for a
+derived distribution targeted at a specific language or set of languages
+and would prefer to not waste space on language-specific packages that
+would never be used but instead include as many "regular" package as
+possible.
+
+In that case you should create a custom language list file. We'd suggest
+to create it in one tasks/<codename> directory and add symlinks in other
+tasks/<codename> directories.
+Then change TASKSEL_LANGLIST in CONF.sh to use your custom list.
+
+Another option is to set 'TASKSEL_LANGLIST=empty' and add the desired
+language tasks (note: the full task names, not just the language names!)
+to all relevant task.list* files.
diff --git a/tasks/lenny/task.list b/tasks/lenny/task.list
index 7188dd2..b93dbe8 100644
--- a/tasks/lenny/task.list
+++ b/tasks/lenny/task.list
@@ -9,175 +9,6 @@ database-server
dns-server
file-server
-# Language tasks
-# Commented out means the language is currently not supported in D-I
-arabic
-arabic-desktop
-arabic-gnome-desktop
-basque-desktop
-basque-gnome-desktop
[... similar tasks for loads of other languages removed ...]
-#xhosa-desktop
-#xhosa-gnome-desktop
-
# Alternative desktop environments
-kde-desktop
-xfce-desktop
+kde-desktop -
+xfce-desktop -
diff --git a/tasks/lenny/task.list.kde b/tasks/lenny/task.list.kde
index 3a45596..863d49f 100644
--- a/tasks/lenny/task.list.kde
+++ b/tasks/lenny/task.list.kde
@@ -1,173 +1,3 @@
-# Main tasks
kde-desktop
desktop
laptop
-
-# Language tasks
-# Commented out means the language is currently not supported in D-I
-arabic
-arabic-desktop
-arabic-kde-desktop
-basque-desktop
-basque-kde-desktop
[... similar tasks for loads of other languages removed ...]
-#xhosa-desktop
-#xhosa-kde-desktop
diff --git a/tasks/lenny/task.list.xfce b/tasks/lenny/task.list.xfce
index 0c2e350..8c875c2 100644
--- a/tasks/lenny/task.list.xfce
+++ b/tasks/lenny/task.list.xfce
@@ -1,173 +1,3 @@
-# Main tasks
xfce-desktop
desktop
laptop
-
-# Language tasks
-# Commented out means the language is currently not supported in D-I
-arabic
-arabic-desktop
-arabic-xfce-desktop
-basque-desktop
-basque-xfce-desktop
[... similar tasks for loads of other languages removed ...]
-#xhosa-desktop
-#xhosa-xfce-desktop
diff --git a/tasks/lenny/tasksel_d-i.languages b/tasks/lenny/tasksel_d-i.languages
new file mode 100644
index 0000000..6ff2a07
--- /dev/null
+++ b/tasks/lenny/tasksel_d-i.languages
@@ -0,0 +1,52 @@
+# See tasks/README.tasksel for information about contents and usage
+# of this file.
+
+# Languages that are commented are not supported in debian-installer
+arabic
+belarusian
+bengali
+bosnian
+brazilian-portuguese
+bulgarian
+catalan
+chinese-s
+chinese-t
+croatian
+cyrillic
+czech
+danish
+dutch
+finnish
+french
+galician
+german
+greek
+hebrew
+hindi
+hungarian
+#icelandic
+#irish
+italian
+japanese
+khmer
+korean
+lithuanian
+macedonian
+norwegian
+persian
+polish
+portuguese
+punjabi
+romanian
+russian
+#serbian
+slovak
+slovenian
+spanish
+swedish
+tagalog
+tamil
+thai
+turkish
+ukrainian
+#welsh
diff --git a/tools/update_tasks b/tools/update_tasks
index b440362..96896aa 100755
--- a/tools/update_tasks
+++ b/tools/update_tasks
@@ -18,6 +18,46 @@ fi
TDIR=$BDIR/update_tasks
mkdir -p $TDIR
+# Sort primary and extra tasks and add language tasks for both
+# Extra tasks are indicated by a "-" in the task.list file
+# When adding language tasks, 'desktop' is sorted before '*-desktop'
+expand_task_list () {
+ tasklist=$1
+ langlist=$2
+
+ task_essential="$(grep -Ev "^(#.*)?(.*-)?[[:space:]]*$" $tasklist)"
+ task_extra="$(grep -Ev "^(#.*)?[[:space:]]*$" $tasklist | \
+ grep -E "(.*-)[[:space:]]*$" | sed "s/[[:space:]]-.*$//")"
+
+ echo "# Main tasks" >$tasklist.essential
+ echo "$task_essential" >>$tasklist.essential
+
+ echo >>$tasklist.essential
+ echo "# Main language tasks" >>$tasklist.essential
+ for task in "" $(echo "$task_essential" | grep "^desktop" || true) \
+ $(echo "$task_essential" | grep -- "-desktop" || true); do
+ for language in $(cat $langlist); do
+ echo $language${task:+-$task}
+ done
+ done >>$tasklist.essential
+
+ cp $tasklist.essential $tasklist.full
+
+ if [ "$task_extra" ]; then
+ echo >>$tasklist.full
+ echo "# Extra tasks" >>$tasklist.full
+ echo "$task_extra" >>$tasklist.full
+
+ echo >>$tasklist.full
+ echo "# Extra language tasks" >>$tasklist.full
+ for task in $(echo "$task_extra" | grep "^desktop" || true) \
+ $(echo "$task_extra" | grep -- "-desktop" || true); do
+ for language in $(cat $langlist); do
+ echo $language${task:+-$task}
+ done
+ done >>$tasklist.full
+ fi
+}
update_full_list () {
file=$1
@@ -65,16 +105,11 @@ update_full_list () {
update_essential_list () {
file=$1
tasklist=$2
- desktoptasks=$3
- tasksel=$4
+ tasksel=$3
(grep -Ev "^(#.*)?[[:space:]]*$" $tasklist ;
echo DONE ;
- cat $tasksel/usr/share/tasksel/debian-tasks.desc) | mawk -v DT=$desktoptasks '
+ cat $tasksel/usr/share/tasksel/debian-tasks.desc) | mawk '
-
- BEGIN {
- split(DT, d_tasks, ",")
- }
/DONE/ {
in_tasks = 1
@@ -105,16 +140,7 @@ update_essential_list () {
if (in_tasks) {
for (taskname in tasklist) {
if (taskname == cur_task) {
- if ((cur_task != "gnome-desktop") &&
- (cur_task != "kde-desktop") &&
- (cur_task != "xfce-desktop")) {
- in_key = 1
- }
- for (dt_num in d_tasks) {
- if (cur_task == d_tasks[dt_num]) {
- in_key = 1
- }
- }
+ in_key = 1
}
}
}
@@ -134,19 +160,28 @@ TASKSEL_DEB=$MIRROR/`mawk '
dpkg -x $TASKSEL_DEB $TDIR/tasksel
-update_essential_list task-essential task.list gnome-desktop \
- $TDIR/tasksel
-
-update_essential_list task-essential-kde task.list.kde kde-desktop \
- $TDIR/tasksel
-
-update_essential_list task-essential-xfce task.list.xfce xfce-desktop \
- $TDIR/tasksel
-
-update_full_list task-full task.list $TMP_PKG
-
-update_full_list task-full-kde task.list.kde $TMP_PKG
-
-update_full_list task-full-xfce task.list.xfce $TMP_PKG
+[ -e task.languages ] || exit 1
+grep -Ev "^(#.*)?[[:space:]]*$" task.languages > $TDIR/languages
+
+for variant in "" kde xfce; do
+ if [ ! -e task.list${variant:+.$variant} ]; then
+ echo "Warning: task.list${variant:+.$variant} does not exist; skipping"
+ continue
+ fi
+
+ expand_task_list \
+ task.list${variant:+.$variant} \
+ $TDIR/languages
+
+ update_essential_list \
+ task-essential${variant:+-$variant} \
+ task.list${variant:+.$variant}.essential \
+ $TDIR/tasksel
+
+ update_full_list \
+ task-full${variant:+-$variant} \
+ task.list${variant:+.$variant}.full \
+ $TMP_PKG
+done
rm -rf $TDIR
diff --git a/tasks/etch/task.list b/tasks/etch/task.list
index 7188dd2..b93dbe8 100644
--- a/tasks/etch/task.list
+++ b/tasks/etch/task.list
@@ -9,175 +9,6 @@ database-server
[FJP: changes are the same as for tasks/lenny/task.list - see below]
diff --git a/tasks/etch/task.list.kde b/tasks/etch/task.list.kde
index 3a45596..863d49f 100644
--- a/tasks/etch/task.list.kde
+++ b/tasks/etch/task.list.kde
@@ -1,173 +1,3 @@
[FJP: changes are the same as for tasks/lenny/task.list.kde - see above]
diff --git a/tasks/etch/task.list.xfce b/tasks/etch/task.list.xfce
index 0c2e350..8c875c2 100644
--- a/tasks/etch/task.list.xfce
+++ b/tasks/etch/task.list.xfce
[FJP: changes are the same as for tasks/lenny/task.list.xfce - see above]
diff --git a/tasks/etch/tasksel_d-i.languages b/tasks/etch/tasksel_d-i.languages
new file mode 100644
index 0000000..6ff2a07
--- /dev/null
+++ b/tasks/etch/tasksel_d-i.languages
@@ -0,0 +1,52 @@
[FJP: changes are the same as for tasks/lenny/tasksel_d-i.languages - see above]
Attachment:
signature.asc
Description: This is a digitally signed message part.