Re: slow list of package names in low-memory install
Joey Hess wrote:
> John Reiser wrote:
>
>>While using the Debian installer in low-memory mode to put a Debian system
>>onto a NSLU2 (ARM processor with 32MB RAM), I noticed that the installer
>>seemed slow. Running "ps" repeatedly in a concurrent session, I often
>>saw about 80 processes per second, and many command lines of the form:
>> [ <package1> = <package2> ]
>>which tests if two package names are the same. I'm guessing that this is
>>to construct a set represented as a list, such as a list of required
>>packages with no duplicates. Doing this sequentially takes quadratic time.
>>If '[' is built in to the shell, then it can be fast enough anyway.
>
>
> It would help if you told us what stage of the installer this was
> happening in or what the parent process was..
The busybox 'ps' does not display parent process ID. I re-ran the installation,
with a shell in a concurrent session, taking notes. Here is an edited transcript
of multiple 'ps' runs, with some comments about installation progress.
Blank lines frequently indicate separate runs of ps, with same-as-previous
content elided.
Also note that the text-cell graphics of the low memory mode installer
suffer from "no update on return." The last status message persists until
it is replaced, so the last action of a loop or nested control often
looks like it takes much longer than it actually does, because the first
following graphical update might not occur soon. This is another thing
that should be fixed.
=====
--| Installing the base system |--
~ # ps
PID Uid VmSize Stat Command
1 root 452 S init
2 root SWN [ksoftirqd/0]
3 root SW< [events/0]
4 root SW< [khelper]
5 root SW< [kthread]
15 root SW< [kblockd/0]
49 root SW [pdflush]
50 root SW [pdflush]
52 root SW< [aio/0]
51 root SW [kswapd0]
569 root SW< [kseriod]
604 root SW [mtdblockd]
693 root 368 S < udevd --daemon
2502 root 384 S /sbin/syslogd -m 0 -O /var/log/syslog -S
2505 root 368 S /sbin/klogd -c 2
2914 root DW [IxOsal 1]
2918 root SW< [ixp400_eth]
2919 root SW< [ixp400_eth]
2923 root 496 S /bin/sh /sbin/debian-installer
2942 root 1376 S debconf -o d-i /usr/bin/main-menu
2943 root 544 S /usr/bin/main-menu
3171 root SW< [khubd]
7003 root SW [ixp400 eth0]
7024 root 448 S udpkg --configure --force-configure network-console
7025 root 488 S /bin/sh /var/lib/dpkg/info/network-console.postinst configure
7054 root 792 S /usr/sbin/sshd
7075 root 1652 S sshd: installer@pts/0
7078 root 492 S /bin/sh /sbin/debian-installer /bin/network-console-menu
7092 root 1812 S debconf -o d-i /bin/network-console-menu
7093 root 568 S main-menu
7428 root 1652 S sshd: installer@pts/1
7526 root 492 S /bin/sh /sbin/debian-installer /bin/network-console-menu
7556 root 1476 S debconf -o d-i /bin/network-console-menu
7557 root 452 S udpkg --configure --force-configure di-utils-shell
7637 root 484 S /bin/sh -e /var/lib/dpkg/info/di-utils-shell.postinst configure
7758 root 472 S /bin/sh -e /bin/debconf-disconnect /bin/sh
7759 root 520 S /bin/sh
9251 root SW< [scsi_eh_0]
9254 root SW< [usb-storage]
20212 root SW [kjournald]
24079 root 580 S udpkg --configure --force-configure base-installer
24080 root 640 S /bin/sh /var/lib/dpkg/info/base-installer.postinst configure
24209 root 388 S log-output -t debootstrap run-debootstrap --components=main --debian-installer --resolve-deps etch /target h
24210 root 536 S run-debootstrap --components=main --debian-installer --resolve-deps etch /target http://debian.osuosl.org/de
24211 root 668 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
24314 root 348 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
24316 root 400 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
24426 root 228 R [ contrib/binary-mipsel/Packages.gz = main/binary-arm/Packages ]
24427 root 532 R ps
~ #
Installing the base system; Validating zlib1g... [This is obviously a status message
that persists long after it should vanish.]
~ # ps
PID Uid VmSize Stat Command
1 root 452 S init
2 root SWN [ksoftirqd/0]
3 root SW< [events/0]
4 root SW< [khelper]
5 root SW< [kthread]
15 root SW< [kblockd/0]
49 root SW [pdflush]
50 root SW [pdflush]
52 root SW< [aio/0]
51 root SW [kswapd0]
569 root SW< [kseriod]
604 root SW [mtdblockd]
693 root 364 S < udevd --daemon
2502 root 384 S /sbin/syslogd -m 0 -O /var/log/syslog -S
2505 root 364 S /sbin/klogd -c 2
2914 root DW [IxOsal 1]
2918 root SW< [ixp400_eth]
2919 root SW< [ixp400_eth]
2923 root 496 S /bin/sh /sbin/debian-installer
2942 root 1376 S debconf -o d-i /usr/bin/main-menu
2943 root 544 S /usr/bin/main-menu
3171 root SW< [khubd]
7003 root SW [ixp400 eth0]
7024 root 448 S udpkg --configure --force-configure network-console
7025 root 488 S /bin/sh /var/lib/dpkg/info/network-console.postinst configure
7054 root 712 S /usr/sbin/sshd
7075 root 1652 S sshd: installer@pts/0
7078 root 492 S /bin/sh /sbin/debian-installer /bin/network-console-menu
7092 root 1836 S debconf -o d-i /bin/network-console-menu
7093 root 568 S main-menu
7428 root 1532 S sshd: installer@pts/1
7526 root 424 S /bin/sh /sbin/debian-installer /bin/network-console-menu
7556 root 888 S debconf -o d-i /bin/network-console-menu
7557 root 360 S udpkg --configure --force-configure di-utils-shell
7637 root 412 S /bin/sh -e /var/lib/dpkg/info/di-utils-shell.postinst configure
7758 root 412 S /bin/sh -e /bin/debconf-disconnect /bin/sh
7759 root 504 S /bin/sh
9251 root SW< [scsi_eh_0]
9254 root SW< [usb-storage]
20212 root SW [kjournald]
24079 root 580 S udpkg --configure --force-configure base-installer
24080 root 640 S /bin/sh /var/lib/dpkg/info/base-installer.postinst configure
24209 root 388 S log-output -t debootstrap run-debootstrap --components=main --debian-installer --resolve-deps etch /target h
24210 root 708 S run-debootstrap --components=main --debian-installer --resolve-deps etch /target http://debian.osuosl.org/de
24211 root 676 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
30985 root 416 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
30986 root 296 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
30987 root 328 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
32469 root 144 R [ base-passwd = gcc-4.1-base ]
4440 root 188 R [ util-linux = mount ]
5666 root 204 R [ sysv-rc = tcpd ]
[Then: Extracting <pkg>...]
[Then: Installing core packages...]
20212 root SW [kjournald]
24079 root 580 S udpkg --configure --force-configure base-installer
24080 root 640 S /bin/sh /var/lib/dpkg/info/base-installer.postinst configure
24209 root 496 S log-output -t debootstrap run-debootstrap --components=main --debian-installer --resolve-deps etch /target h
24210 root 752 S run-debootstrap --components=main --debian-installer --resolve-deps etch /target http://debian.osuosl.org/de
24211 root 676 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
7757 root 404 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
7761 root 344 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
7762 root 280 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
7763 root 328 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
7806 root 112 R [ dpkg = e2fslibs ]
7808 root 236 R echo
7809 root 532 R ps
~ #
[Then: Unpacking required packages...] @ 34%
20212 root SW [kjournald]
24079 root 580 S udpkg --configure --force-configure base-installer
24080 root 640 S /bin/sh /var/lib/dpkg/info/base-installer.postinst configure
24209 root 496 S log-output -t debootstrap run-debootstrap --components=main --debian-installer --resolve-deps etch /target h
24210 root 752 S run-debootstrap --components=main --debian-installer --resolve-deps etch /target http://debian.osuosl.org/de
24211 root 676 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10011 root 404 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10013 root 360 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10014 root 412 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10015 root 300 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10016 root 328 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10894 root 80 R [ libattr1 = bsdmainutils ]
10898 root 228 R echo
14385 root 52 R [ libslang2 = klogd ]
15410 root 76 R [ dpkg = libconsole ]
16378 root 84 R [ e2fsprogs = libgnutls13 ]
16382 root 168 R echo
17112 root 108 R [ libattr1 = libncurses5 ]
17118 root 236 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
17637 root 12 R [ ncurses-bin = libpam-modules ]
17641 root 232 R echo
18410 root 140 R [ dpkg = libsepol1 ]
18418 root 228 R echo
19527 root 108 R [ libslang2 = libtext-charwidth-perl ]
~ #
24079 root 580 S udpkg --configure --force-configure base-installer
24080 root 640 S /bin/sh /var/lib/dpkg/info/base-installer.postinst configure
24209 root 496 S log-output -t debootstrap run-debootstrap --components=main --debian-installer --resolve-deps etch /target h
24210 root 752 S run-debootstrap --components=main --debian-installer --resolve-deps etch /target http://debian.osuosl.org/de
24211 root 676 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10011 root 404 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10013 root 360 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10014 root 412 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10015 root 300 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
10016 root 328 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
22788 root 192 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
22789 root 164 R [ base-passwd = procps ]
24266 root 276 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
24269 root 532 R ps
24272 root 132 R [ hostname = wget ]
[Then: Unpacking <pkg>...]
[Then: Unpacking the base system...] @ 48%
20212 root SW [kjournald]
24079 root 580 S udpkg --configure --force-configure base-installer
24080 root 640 S /bin/sh /var/lib/dpkg/info/base-installer.postinst configure
24209 root 496 S log-output -t debootstrap run-debootstrap --components=main --debian-installer --resolve-deps etch /target h
24210 root 840 S run-debootstrap --components=main --debian-installer --resolve-deps etch /target http://debian.osuosl.org/de
24211 root 676 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
2540 root 404 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
2542 root 360 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
2543 root 412 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
2544 root 300 S /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
2545 root 328 R /bin/sh -e /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps etch /target http://deb
3427 root 108 R echo
3430 root 136 R [ libncursesw5 = bsdmainutils ]
4521 root 228 R [ nano = diff ]
5811 root 196 R [ man-db = grep ]
9100 root 108 R [ bsdmainutils = libldap2 ]
10361 root 228 R [ man-db = libpam0g ]
11713 root 104 R [ whiptail = libstdc++6 ]
12223 root 96 R [ iputils-ping = libtext-wrapi18n-perl ]
12826 root 220 R [ iputils-ping = logrotate ]
14132 root 188 R [ adduser = ncurses-base ]
14966 root 108 R [ tcpd = nvi ]
15688 root 92 R [ wget = sed ]
16110 root 128 R [ libldap2 = tar ]
16533 root 44 R [ wget = traceroute ]
17055 root 236 R [ libconsole = zlib1g ]
[Then: Unpacking <pkg>...] @ 50%
=====
--
Reply to: