I thought I would give this topic a better heading, in case it's missed by interested parties. I didn't understand all of Neil's comments with regards to including more scrubbing in grip itself, but i did want to answer the call to provide scripts that are being used. I attach mine. If it fails to get to the list as an attachment I will insert in a next message. I don't think I am doing anything you guys havent already thought of. Looking at the first part of my script, I essentially do the following: if rm -rf usr/share/doc \ && rm -rf usr/share/man \ && rm -rf usr/share/doc-base \ && rm -rf var/cache/apt \ && rm -rf var/cache/man \ && rm -rf var/cache/debconf \ && rm -rf var/lib/apt/lists \ && rm -rf var/lib/dpkg/ \ && rm -rf var/log/installer/cdebconf \ && $SUDO rm -rf var/log/news/* and remove perl: rm -rf usr/share/perl Also, after my first (NFS based) boot, I apt-get the purgelocale package and run that to clean out all language locales except English. Then I remove purgelocale itself again. The above lines blow away about 50% of a standard debian minimal install, bringing it in the 100MB range I think (uncompressed). The next part of the script removes some stuff that I found causes problems (such as udev and things that shouldn't be inherited across machines) and then I start installing our own RFS changes ( serial boot console, network setup, rc.local, passwd files etc) that can be ignored. Finally I create the jffs2 images (one containing the summary file). I think the issue of how far one needs to go with minimalizing obviously depends on the size of the flash you intent to use. Given the market trend, I limited my efforts to producing <64MB filesystems and called it a day. With a 256MB flash at ~$12 or so, I have plenty of room to send out future flash upgrades that can be installed side by side. In end, whether I install packages that have /usr/man removed per package or whether I blow away /usr/man at the end doesnt matter. With regards to board (vendor) specific changes, I would hesitate to spend too much time in incorporating such a mechanism in Emdebian. Its a game where you are always catching up to whats in the market place and at some point everyone is going to have to write some scripts to modify an RFS anyway. I also noticed some calls for ready-to-run kernels within this project: I would hesitate there too. We use an at91sam9260 (and G20). I am sure no generic ARM kernel would even boot. Even 9260 specific kernels probably wouldnt boot because all these chip mulitplex their hardware on I/O lines and depending on whose board you buy, you have different hardware available. I got myself in a bad muddle once when a serial port I/O pin got programmed to be used as an SPI interface in a newer kernel release and a dataflash driver was loaded when there was none. For completeness, and in case anyone in interested, I put a copy of my own notes on how to create a Debian RFS at the bottom here. It only applies of you start on Debian based i86 and are able to boot & complete the debootstrap process on your actual target board, with the RFS NFS mounted. I realize this isnt really appropiate for this mailing list. ==================================================================== RootFS creation instructions (ARM, 9260 processor) -------------------------------------------------------------------------------- my working folder is /arm/debootstrap/nfs #1st stage: mkdir /arm/debootstrap/nfs cd /arm/debootstrap/nfs debootstrap --foreign --arch armel --verbose lenny $PWD # make sure we can log in ! cp /etc/passwd etc/passwd cp /etc/shadow etc/shadow cp /etc/group etc/group echo "proc /proc proc defaults 0 0" > etc/fstab echo "192.168.0.3:/arm/debootstrap/nfs / nfs defaults 0 1" >> etc/fstab #Make sure the root account uses /bin/bash as the shell, since tcsh will not be on the bare Debian install ! #Must create dev/console and dev null or dont get a prompt: cd /arm/debootstrap/nfs/dev mknod -m 660 console c 5 1 mknod -m 660 null c 1 3 mknod ttyS0 c 4 64 mknod ttyS1 c 4 65 #Now boot the kernel with: (at u-boot) setenv bootargs console=ttyS0 root=/dev/nfs rw ip=dhcp nfsroot=192.168.0.3:/arm/debootstrap/nfs init=/bin/bash bootd #kernel now boots to bash prompt.... #Next commands are on the ARM board: mount -n -o remount,rw / mount /proc # 2nd stage of debootstrap, within the guest system, to finalise the installation: cd / export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ./debootstrap/debootstrap --second-stage (if you get lots of errors installing certain packages do dpkg --configure -a to see which ones. One of them was debconf but a dpkg-reconfigure debconf solved that) Note: passwd and shadow files must not contain blank lines at the end of the files. and add (on Debian i86 host, with an editor) T0:23:respawn:/sbin/getty -L ttyS0 115200 vt100 to /etc/inittab. #reboot without the init=/bin/bash statement in the u-boot variable #if the deboostrap 2nd staged failed, you have manually install everything in here, since it didnt #get done cd /var/cache/apt/archives/ dpkg -i -R $PWD #manually correct the apt/source.list - it seems hosed. Do this on Debian i86 with an editor deb http://ftp.us.debian.org/debian/ lenny main deb-src http://ftp.us.debian.org/debian/ lenny main deb http://security.debian.org/ lenny/updates main deb-src http://security.debian.org/ lenny/updates main #and do a apt-get update on the board apt-get update #Wrap up with apt-get install dropbear file tcsh hostname ifupdown nano mount The result RFS is what I use as the input to the (hopefully) attached script. Cheers, Gertjan |
#!/bin/bash
# do NOT run this as root !
# 19.03.2006
# Gertjan Hofman
# mostly geared to Debian SID now (unstable)
# July 2009: added sumtool to add JFFS2 block info to device and speed up boot times
case $# in
3) echo ; echo Creating jffs2 image from $1. Expanding to rootfs;;
*) echo "Usage: add_rfs_mod <name of folder containing rfs> <hostname> <eabi/oabi>"
exit 1 ;;
esac
# set up variables:
# =================== START OF DEFINITIONS ====================
SUDO="sudo"
# where the input root filesystem is:
ROOT_IN=$1
# where the output root filesystem is:
ROOT=${PWD}/rootfs
# store present working dir
STARTPWD=${PWD}
# where our files are:
POST_INSTALL_COMMON=$PWD/post_install_common
if [ $3 = "oabi" ] ; then
POST_INSTALL_BINARY=$PWD/post_install_oabi
elif [ $3 = "eabi" ] ; then
POST_INSTALL_BINARY=$PWD/post_install_eabi
else
echo "Illegal ABI $3"
exit 1
fi
# location of OpenEmbedded created binaries (needed for mkfs)
JFFS2_BIN=/usr/sbin/
# =================== END OF DEFINITIONS ====================
echo -n "Removing old rootfile system..............................."
if $SUDO rm -rf ${ROOT}
then
echo "OK"
else
echo "Can not copy the file system to ${ROOT}"
exit
fi
echo -n "Copying rootfile system...................................."
if $SUDO cp -a ${ROOT_IN} ${ROOT}
then
echo "OK"
else
echo "Can not copy the file system to ${ROOT}"
exit
fi
# change the owner to me:
$SUDO chown -R ${USER}:users ${ROOT}
# ok - going into folder:
cd ${ROOT}
#$SUDO tar -xf ../$1
#before we remove apt, lets get a listing of installed packages
echo -n "Storing list of APT packages..............................."
if [ -x /usr/bin/dpkg ] && dpkg --get-selections --admindir=${ROOT}/var/lib/dpkg > ${ROOT}/etc/apt/packages.list
then
echo "OK"
else
echo "FAILED"
exit
fi
# remove docs
echo -n "Removing doc/man pages....................................."
if rm -rf usr/share/doc \
&& rm -rf usr/share/man \
&& rm -rf usr/share/doc-base \
&& rm -rf var/cache/apt \
&& rm -rf var/cache/man \
&& rm -rf var/cache/debconf \
&& rm -rf var/lib/apt/lists \
&& rm -rf var/lib/dpkg/ \
&& rm -rf var/log/installer/cdebconf \
&& $SUDO rm -rf var/log/news/*
then
echo "OK"
else
echo "Failed"
fi
# remove Perl
echo -n "Removing Perl (!).................. ......................."
if rm -rf usr/share/perl
then
echo "OK"
else
echo "Failed"
fi
# save some locales (which one are we using ?)
# note: this is now done by using apt-get install localepurge on the host
# and running localepurge
echo -n "Purging Locales............................................"
if rm -rf usr/share/locale/fr \
&& rm -rf usr/share/locale/de \
&& rm -rf usr/share/locale/ru \
&& rm -rf usr/share/locale/pl \
&& rm -rf usr/share/locale/sv \
&& rm -rf usr/share/locale/es \
&& rm -rf usr/share/locale/ca \
&& rm -rf usr/share/locale/ja \
&& rm -rf usr/share/locale/tr \
&& rm -rf usr/share/locale/vi \
&& rm -rf usr/share/locale/cs \
&& rm -rf usr/share/locale/nl \
&& rm -rf usr/share/locale/hu
then
echo "OK"
else
echo "Failed"
fi
echo -n "Removing mystery .nfs files ..............................."
if find $ROOT -name ".nfs*" -print | xargs rm -f
then
echo "OK"
else
echo "Failed"
fi
echo -n "Empty sys and proc........................................."
# empty sys and proc
#rm -rf sys/*
rm -rf proc/*
echo "OK"
echo -n "Removing udev ............................................."
# remove dev udev remnants
if rm -fr dev/.udev
then
echo "OK"
else
echo "Failed"
fi
echo -n "Removing dnsmasq start up link in rcX.d...................."
# remove dev udev remnants
if rm -fr etc/rc2.d/S15dnsmasq
then
echo "OK"
else
echo "Failed"
fi
echo -n "Removing/creating lib modules............................."
# remove kernel mods, we have none:
if rm -rf lib/modules/* && \
mkdir -p lib/modules && \
cp -a ${POST_INSTALL_BINARY}/lib/modules/* lib/modules/
then
echo "OK"
else
echo "Failed"
fi
echo -n "Removing timezone info ...................................."
# remove timezone info we dont need:
du -a usr/share/zoneinfo/ | grep -v America | awk '{print $2}' | xargs rm -rf
echo "OK"
#entropy is low, so random blocks, but urandom never does...
#echo "replacing dev/random with urandom"
#rm dev/random
#ln -s urandom dev/random
# re-instate some folder structures for apt in case we ever use apt-get
mkdir -p var/cache/apt
mkdir -p var/lib/apt/lists/partial
mkdir -p var/cache/apt/archives/partial
echo -n "Creating xauth dunmmy......................................"
# create a dummy xauth file -login errors ?
mkdir -p usr/X11R6/bin
cp ${POST_INSTALL_COMMON}/usr/X11R6/bin/xauth usr/X11R6/bin
echo "OK"
echo -n "Updating etc files........................................."
cp -a ${POST_INSTALL_COMMON}/etc/* etc/
echo "OK"
echo -n "Updating Webserver files..................................."
cp -a ${POST_INSTALL_COMMON}/usr/local/apache usr/local/
echo "OK"
echo -n "Updating usr/edaq/ (*should be in usr-local*..............."
cp -a ${POST_INSTALL_COMMON}/usr/edaq usr/
echo "OK"
echo -n "Updating usr/bin/ (*should be in usr-local*................"
cp -a ${POST_INSTALL_COMMON}/usr/bin/* usr/bin/
echo "OK"
echo -n "Creating mount points......................................"
# create mount points
test -d pc_home || mkdir pc_home
test -d arm || mkdir arm
test -d permanent || mkdir permanent
test -d mnt/flash01 || mkdir mnt/flash01
test -d mnt/flash02 || mkdir mnt/flash02
test -d mnt/flash03 || mkdir mnt/flash03
test -d mnt/flash04 || mkdir mnt/flash04
test -d evolution_info || mkdir evolution_info
echo "OK"
echo -n "Creating additional dev files.............................."
test -a dev/mtdblock0 || $SUDO mknod dev/mtdblock0 b 31 0
test -a dev/mtdblock1 || $SUDO mknod dev/mtdblock1 b 31 1
test -a dev/mtdblock2 || $SUDO mknod dev/mtdblock2 b 31 2
test -a dev/mtdblock3 || $SUDO mknod dev/mtdblock3 b 31 3
test -a dev/mtdblock4 || $SUDO mknod dev/mtdblock4 b 31 4
# the major=misc=10, the minor is from ./include/linux/miscdevice.h
test -a dev/watchdog || $SUDO mknod dev/watchdog c 10 130
test -a dev/mtd0 || $SUDO mknod dev/mtd0 c 90 0
test -a dev/mtd1 || $SUDO mknod dev/mtd1 c 90 2
test -a dev/mtd2 || $SUDO mknod dev/mtd2 c 90 4
test -a dev/mtd3 || $SUDO mknod dev/mtd3 c 90 6
test -a dev/mtd4 || $SUDO mknod dev/mtd4 c 90 8
test -a dev/ttyS0 || $SUDO mknod dev/ttyS0 c 4 64
test -a dev/ttyS1 || $SUDO mknod dev/ttyS1 c 4 65
test -a dev/ttyS2 || $SUDO mknod dev/ttyS2 c 4 66
test -a dev/ttyS3 || $SUDO mknod dev/ttyS3 c 4 67
echo "OK"
echo -n "Copying EDAQ program files................................."
if cp -a ${POST_INSTALL_COMMON}/usr/local/edaq usr/local/
then
echo "OK"
else
echo "Failed"
fi
echo -n "Creating usr/local/bin and populating......................"
if \
mkdir -p usr/local/bin && \
cp ${POST_INSTALL_COMMON}/usr/local/bin/* usr/local/bin/
then
echo "OK"
else
echo "Failed"
fi
# note: in future, eliminate usr/bin and put everything in usr/local
echo -n "Creating arch dep usr/{local}/bin and populating.........."
if \
mkdir -p usr/local/bin && \
cp ${POST_INSTALL_BINARY}/usr/local/bin/* usr/local/bin/ && \
cp ${POST_INSTALL_BINARY}/usr/bin/* usr/bin/
then
echo "OK"
else
echo "Failed"
fi
echo -n "Creating binary usr/local/lib and populating..............."
if \
mkdir -p usr/local/lib && \
cp -a ${POST_INSTALL_BINARY}/usr/local/lib/* usr/local/lib
then
echo "OK"
else
echo "Failed"
fi
echo -n "Creating script usr/local/lib and populating..............."
if \
cp -a ${POST_INSTALL_COMMON}/usr/local/lib/* usr/local/lib
then
echo "OK"
else
echo "Failed"
fi
#3/2/2008: decided to have /usr/local run off devel host.
echo -n "Note: Moving Usr_Local to symlink.........................."
if \
mv usr/local usr/local.LOCAL && \
ln -s /arm/edaq_usr_local usr/local
then
echo "OK"
else
echo "Failed"
fi
echo -n "Setting hostname .........................................."
if rm -f etc/hostname && echo $2 > etc/hostname
then
echo "OK"
else
echo "Failed"
fi
# link tcl lib - not sure why this is missing:
#echo -n "Creating TCL dynlib symlink................................"
#if cd usr/lib && \
# ln -s libtcl8.5.so.0 libtcl8.5.so && \
# cd - >>/dev/null
#then
# echo "OK"
#else
# echo "Failed"
#fi
#if cd usr/lib && ln -s libtcl8.5.so.0 libtcl8.5.so && cd - > /dev/null
#then
# echo "OK"
#else
# echo "Failed"
#fi
echo -n "Adding LD_LIBRARY_PATH=/usr/local/lib ....................."
if echo "export LD_LIBRARY_PATH=/usr/local/lib" >> etc/profile &&
echo "setenv LD_LIRARTY_PATH /usr/local/lib" >> etc/csh.cshrc
then
echo "OK"
else
echo "Failed"
fi
echo -n "Copying root user files .................................."
if cp -a ${POST_INSTALL_COMMON}/root/.ssh root/
then
echo "OK"
else
echo "Failed"
fi
echo -n "Clean up SVN files ........................................"
if find ${ROOT} -name .svn | xargs rm -rf
then
echo "OK"
else
echo "Failed"
fi
# does the version number file exist ?
if ! [ -r "${STARTPWD}/rfs_version_number" ] ; then
echo "Note: RFS version file not there, creating"
echo $(date) > ${STARTPWD}/rfs_version_number
echo 1000 >> ${STARTPWD}/rfs_version_number
fi
# open the file. First line is a date, the second line is the version number. I am
# not sure why the \n does get into the version_info variable.
# the $NF gets the last field
version_info=$(cat ${STARTPWD}/rfs_version_number)
version_num=$(echo $version_info | awk '{printf $NF}')
# increment the version.
let version_num=version_num+1
# write back to local SVN'ed file:
echo $(date) > ${STARTPWD}/rfs_version_number
echo $version_num >> ${STARTPWD}/rfs_version_number
# write it to the rootfile system
echo $(date) > etc/rfs_version_number
echo $version_num >> etc/rfs_version_number
#=================== done =========================
# return to up directory.
cd ..
# finally, create RFS image
OUTNAME=`date +%H-%M-%S-%F`
OUTNAME=`date +%Y-%m-%d-%H-%M`
rm -f $OUTNAME.jffs2
echo Creating filesystem
# changed from -x lzo in OE to -x zlib in Ubuntu
# -x - remove compressor
# -root build RFS from root=
# --faketime change all file times to zero
# --pad to end of erase block
# -n no clean markers to every erase block
$SUDO ${JFFS2_BIN}/mkfs.jffs2 \
-x rtime \
--root=${ROOT} \
--faketime \
--output=$OUTNAME.jffs2 \
--pad \
--little-endian \
--eraseblock=0x20000 \
-n \
--squash-uids
# --compression-mode=none
$SUDO ${JFFS2_BIN}/sumtool --eraseblock=0x20000 -n -p -i $OUTNAME.jffs2 -o ${OUTNAME}_sum.jffs2
echo Created $OUTNAME.jffs2 filesystem and sum version ${OUTNAME}_sum.jffs2