Re: portable mirror?
Hello,
I didn't know about apt-move or apt-mirror, so I developped a solution to
this problem myself, let me share it :-)
The idea is very simple. If you do an "apt-get update" in your home machine
(the one with slow connection) then you can ask apt to give you the list of
the packages that would be upgraded. Well, I decided to write a small python
script that read the list of packages to be upgraded and generate small
scripts to download them in another machine with a good internet connection.
Finally copy the downloaded files to the original machine and finish the
"apt-get upgrade". My script can download the packages to multiple disks too,
but you need disks that are big enough to fit the biggest deb (that may be
10MB or bigger), like a zip.
Well, it helped me a lot before I got a good connection. Take a look and see
if it is worthy. Precise instructions can be found on the top of the script
attached.
Paulo
PS: You will net wget installed.
PS2: This was my first python script, so don't look for good programming
style.
PS3: Any problem, please contact me.
--
Paulo José da Silva e Silva
Professor Assistente do Dep. de Ciência da Computação
(Assistant Professor of the Computer Science Dept.)
Universidade de São Paulo - Brazil
e-mail: rsilva@ime.usp.br Web: http://www.ime.usp.br/~rsilva
Teoria é o que não entendemos o (Theory is something we don't)
suficiente para chamar de prática. (understand well enough to call practice)
#!/usr/bin/env python
#
# Simple script to create scripts to download a debian upgrade in zip disks
# or any "big enough" disks.
#
# Author: Paulo Silva (rsilva@ime.usp.br).
#
# This script should be useful if you want to upgrade a Debian box which
# has a slow connection to the internet, but you have access to a faster one.
# The idea is to use the slow connection to update your package list and then
# use the fast connection to download the deb files for the upgrade. The
# files should be transported to the machine with slow connection with a
# zip disk, or any big enough media. You'll need python installed (of course).
#
# The use is the following:
# 1) Use the slow connection to update the packages list from the net. You
# do this by changing your /etc/apt/source.list file to point to a
# newer Debian version (like potato right now) and doing an
#
# apt-get update
#
# from the prompt.
#
# 2) Generate the scripts that will download the file from the machine with
# fast connection. To achieve this you must do:
# a) Generate a list of the files to download:
# Just type:
#
# apt-get -qq --print-uris dist-upgrade > uris_file
#
# b) Move the uris file to the apt-download-offline directory and type:
#
# python apt-download-offline.py uris_file disk_size[bkm]
#
# For example: 96 megabytes zip disks and using wget you should type:
#
# python apt-download-offline.py uris_file 96m
#
#
# 3) Move the download scripts (apt-download-disk?.py) and run them
# in _different directories_ in the fast connected machine.
# You should make them executable or type something like:
#
# python apt-download-disk?.py
#
# The files will be downloaded in the current directory.
#
# 4) Take the downloaded deb files with the zip disks, copy them to the
# /var/cache/apt/archives directory in the slow connected machine and
# type
#
# apt-get dist-upgrade
#
# The upgrade should take place and no downloads needed.
#
# Enjoy!
#
# LIMITATIONS:
# The disks must be big enough to fit the biggest deb file
# that must be downloaded. zip should do.
# The number of disks used is not optimal. I believe that it is possible
# to prove that with my solution you may use up to 50% more disks than
# the optimal set. This is good enough for me :-)
import sys, string, re, commands
# Open a file for reading, aborting on exeptions.
def openfile(name):
try:
handle = open(name, 'r')
except:
print 'Problems to open file %s!' % name
sys.exit(1)
return handle
# Deal with command line.
def getarguments(argv):
if len(argv) != 3:
print 'Use: apt-download-offiline.py <uris_file> <disk_size[bkm]>'
sys.exit(1)
# Opens input file.
input = openfile(argv[1])
# Gets disksize.
# Verifies if disk size is in the correct format.
disksize = re.match('[0-9]+[bkm]?$', argv[2])
if not disksize:
print 'Wrong disksize format!'
sys.exit(1)
disksize = disksize.group(0)
# Sets the disksize taking into accound the correct multiplier.
mults = {'b': 1, 'k': 1024, 'm': 1024*1024}
if disksize[-1] in mults.keys():
disksize = int(disksize[:-1])*mults[disksize[-1]]
else:
disksize = int(disksize)
# Return parameters.
return input, disksize
# Save the script to download a disk.
def printdisk(disk, disknumber, size):
print 'Disk %d size: %d kbytes' % (disknumber, size/1024)
try:
output = open('apt-download-disk%d.py' % disknumber, 'w')
except:
print 'Problems to create file apt-download-disk%d.py!' % disknumber
sys.exit(1)
# Write the head of the python script.
output.write("""#!/usr/bin/env python
# Disk %d
# Automatically generated by apt-download-offline.py
#
from string import *
from commands import *
import os
""" % disknumber)
output.write("""
# Function to download a file.
def download(url, file, md5sum):
while 1:
os.system("wget -c %s" % url)
original = split(url, '/')[-1]
os.rename(original, file)
trymd5 = getoutput("md5sum %s" % file)
trymd5 = split(trymd5)[0]
if trymd5 == md5sum:
break
else:
os.remove(file)
# Download all the required files.
""")
for deb in disk:
output.write("download(%s, '%s\', '%s')\n" % (deb[1], deb[2], deb[3]))
# Read URI file
def readuri(input, disksize):
urilist = []
totalsize = 0
for line in input.readlines():
uri, package, size, md5 = string.split(line)
size = int(size)
# Asserts that there isn't any package too big to fit a disk.
if (size > disksize):
print 'There is a package bigger than the disk!\n'
sys.exit(1)
totalsize = totalsize + size
urilist.append([size, uri, package, md5])
print 'Total download size: %d kbytes' % (totalsize/1024)
return urilist
###############################################################################
#
# Main script. Creates the script to download disks.
#
# Gets command line arguments.
input, disksize = getarguments(sys.argv)
# Reads the uri file.
urilist = readuri(input, disksize)
urilist.sort()
# Starts first disk.
disknumber = 1
disk = []
slackness = disksize
# Scans the list to create the disks.
while urilist:
# Scans for a package that is small enough to fit the disk.
for i in urilist:
if i[0] <= slackness: break
# Appends the package if it fits the current disk.
if i[0] <= slackness:
disk.append(i)
slackness = slackness - i[0]
urilist.remove(i)
# Starts a new disk if the package doesn't fit.
else:
# prints the script for this disk.
printdisk(disk, disknumber, disksize-slackness)
# Start a new disk with actual package (note that we already know
# that there is no package to big to fit a disk).
disknumber = disknumber + 1
slackness = disksize
disk = []
# Prints the script for the last disk.
printdisk(disk, disknumber, disksize - slackness)
Reply to: