Re: Speeding Up APT - Project Idea

[Ishan Jayawardena <udeshike@gmail.com>]

Hi Michael, Hi all,

As you may remember, I talked with you about integrating debdelta into
apt-get, on the IRC. I found it interesting and therfore, hope to come
up with a project proposal out of it.

As a start, I started looking into details of debdelta and apt's code
to identify what pieces of it will be affected by the integration. Do
you have any suggestions about this? What classes/functions of apt are
more important? What is needed to be added/modified? What are the
possible challenges, requirements and expected results? Is there any
ongoing development related to this? I would also like to know what
are the areas that I have to pay attention and study, in particular.
It would be great if you can share your expertise with me regarding
this.

Thank you.

On 2/5/11, Ishan Jayawardena <udeshike@gmail.com> wrote:
> Hi David,
>
> I noticed that this idea has been around in the discussions sine some
> time [1]. This feature(and other related features) has been requested
> by many users from time to time, and, if implemented, that can be one
> of major improvements of APT and serve many requirements of different
> types of users.
>
>> Splitting up downloading and installing into smaller chunks on the
>> other hand should need 'only' changes in APT and the current download
>> system is designed to download packages in the required order, so
>> you 'just' need to find the right position where you can start a
>> parallel install process…
>
> Currently, when apt queues the archives for downloading, does it order
> them in the same order that it feeds them into dpkg (i.e. is it same
> as the actual installation order)? Or does it use some other ordering?
>
>>> etc.. Also they had mentioned about the official adoption of debdeltas
>>> for Debian.
>>
>> That is another interesting thing. The problem is currently that a
>> download
>> method doesn't get to know which version of the package is currently
>> installed (if any) which is needed as far as i understand it to repack
>> a package to apply the debdiff to… but yeah, nothing which would be
>> impossible to solve i guess.
>
> Yes. If I have not mistaken, Ubuntu has already started implementing
> this. As I learned from the dpkg community, if it is already
> implemented, most of the code is in place already for Debian, and the
> remaining work includes,
>
>    " * get Debian ftpmasters to accept the idea
>      * work with ftpmasters to implement the debdelta generation for
>        the main archive
>      * work with the mirrors team to get the delta files mirrored on
>        all main mirrors
>      * work with the apt maintainers to have apt support debdeltas,
>        when available, on by default "
>
>>
>>> Currently I am learning the internals of APT and looking
>>
>> I think the best way to get to know your way around is to tackle some
>> bugs - the good thing is that you have a lot to choose from¹ - which also
>> helps to prove to the gsoc admins that you know what you are talking
>> about
>> which increases your changes to be accepted into the program.
>> (just as a general hint)
>>
>> ¹ the first time i consider this a good thing… ;)
>
> Thanks for the tips :). I first read the idea on last year's ideas
> list page and when I tried to get started with a little bug, I saw
> this[1]. So I thought of working on it since I thought it was talking
> about almost the same idea.
>
>>
>>> would like to know your ideas and suggestions about these projects and
>>> other new project ideas that you would have related to APT and package
>>> management.
>>
>> Good question! Zero-bug-count would be cool²… ;)
>> More seriously, i thought already in my proposal last year about a way
>> to exclude packages from being included in the binary cache which could
>> be used e.g. on servers to exclude gnome and kde packages from
>> even be parsed into the caches which speeds up the dependency
>> checking which can be quiet noticeable on slower devices.
>>
>> Can't think about other gsoc-fitting projects now, but its pretty late in
>> my timezone now, so maybe after the weekend…
>>
>> ² just said to fix the balance in the universe again after ¹
>>
>>
>> Best regards
>>
>> David Kalnischkies
>>
>> P.S.: I noticed that you were cc'ed to all mails on dpkg list, so i cc'ed
>> you here, too, but the normal case is to not do this if not requested
>> by the sender -- see code of conduct of debian lists.
>>
> It is completely OK.
>
> Thank you for your reply.
>
> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=30505
>
> --
> Regards,
> Ishan Jayawardena.
>


-- 
Regards,
Ishan Jayawardena.