Re: New Method Spec
On Sat, 3 Oct 1998, Jason Gunthorpe wrote:
Sorry for the long include.
> 4. If-Modified-Since (IMS) checking for index files
Not just for index files.
> source
> Refers to an item in source list. More specifically it is the
> broken down item, that is each source maps to exactly one index
> file. Archive sources map to Package files and Source Code
> sources map to Source files.
>
> source file
> Refers to one of the files making up the source code of a
> package. In debian it is one of .diff.gz, .dsc. or .tar.gz.
I sense that some might confuse this and source. Maybe change the name of
source to location?
>
> URI
> Universal Resource Identifier (URI) is a super-set of the
> familiar URL syntax used by web browsers. It consists of an
> access specification followed by a specific location in that
> access space. The form is <access>:<location>. Network addresses
> are given with the form
> <access>://[<user>:<pas>>@]hostname[:port]/<location>. Some
This s/b:
<access>:/[/][<user>[:<pass>]@]hostname[:port][/<location>]
(at least that is how my uri parser handles it)
Not specifing the password in /etc/apt/sources.list, and, with the new
interface, would not be a prob, since the methods can now prompt for info.
> examples:
> file:/var/mirrors/debian/
> ftp://ftp.debian.org/debian
> ftp://jgg:MooCow@localhost:21/debian
> nfs://bigred/var/mirrors/debian
> rsync://debian.midco.net/debian
> cdrom:Debian 2.0r1 Disk 1/
hmm, rsync and cdrom. Cool. But no :/ in the cdrom uri?
> method
> There is a one to one mapping of URI access specifiers to
> methods. A method is a program that knows how to handle a URI
> access type and operates according to the specifications in this
> file.
Maybe allow in the global configuration the capability to replace a default
method handler with a custom one, maybe usefull for testing.
> All methods operate as a sub process of a main controlling parent. 3
> FD's are opened for use by the method allowing two way communication
> and emergency error reporting. The FD's corrispond to the well known
correspond.
> unix FD's, stdin, stdout and stderr.
Maybe as a way of testing for which way it is being used, the method should
check the status of stdin, and see if it is open/active.
> The basic startup sequence depends on how the method is invoked. If
> any command line arguments are passed then the method should start in
> automatic mode. This facility is provided soley to make the methods
> easier to test and perhaps use outside of APT. Upon startup the method
> will print out a header describing its capabilities and requirements.
> After that it either begins processing the command line arugments and
> exits when done or waits for commands to be fed to it.
Does the capabilities and requirements get printed out even in cmdline mode?
That would seem to lead to an incompatibility with the old api.
> Throught operation of the method communication is done via http style
> plain text. Specifically RFC-822 (like the Package file) fields are
> used to describe items and a numeric-like header is used to indicate
> what is happening. Each of these distinct communication messages
> should be sent quickly and without pause.
So, does quoting need to be done on chars sent between the parent and the
method?
>
> In some instances APT may pre-invoke a method to allow things like
> file URI's to determine how many files are available locally.
>
>
> 2.2. Message Overview
> ----------------------
>
> The first line of each message is called the message header. The first
> 3 digits (called the Status Code) have the usual meaning found in the
> http protocol. 1xx is informational, 2xx is successfull and 4xx is
> failure. The 6xx series is used to specify things sent to the method.
> After the status code is an informational string provided for visual
> debugging.
>
> * 100 Capabilities - Method capabilities
>
> * 101 Log - General Logging
>
> * 102 Status - Inter-URI status reporting (login progress)
>
> * 200 URI Start - URI is starting aquire
>
> * 201 URI Done - URI is finished aquire
>
> * 400 URI Failure - URI has failed to aquire
>
> * 401 General Failure - Method did not like something sent to it
>
> * 402 Authorization Required - Method requires authorization to
> access the URI. Authorization is User/Pass
I hope this isn't hardcoded to JUST be user/pass. FTP has user[/pass]
[/account]. I also hope that the methods have the capability to turn of
echoing of some of the prompting.
>
> * 403 Media Failure - Method requires a media change
* 404 Unknown Message
> * 600 URI Aquire - Request a URI be aquired
>
> * 601 Configuration - Sends the configuration space
>
> * 602 Authorization Credentials - Response to the 402 message
>
> * 603 Media Changed - Response to the 403 message
>
> * 605 Shutdown - Exit
>
> Only the 6xx series of status codes is sent TO the method. Furthermore
> the method may not emit status codes in the 6xx range. The Codes 402
> and 403 require that the method continue reading all other 6xx codes
> until the proper 602/603 code is recieved. This means the method must
received
> be capable of handling an unlimited number of 600 messages.
>
> The flow of messages starts with the method sending out a *100
> Capabilities* and APT sending out a *601 Configuration*. After that
> APT begins sending *600 URI Aquire* and the method sends out *200 URI
> Start*, *201 URI Done* or *400 URI Failure*. No syncronization is
> performed, it is expected that APT will send *600 URI Aquire* messages
> at -any- time and that the method should queue the messages. This
> allows methods like http to pipeline requests to the remote server. It
> should be noted however that APT will buffer messages so it is not
> neccessary for the method to be constantly ready to recieve them.
>
>
> 2.3. Header Fields
> -------------------
>
> The following is a short index of the header fields that are supported
>
> URI
> URI being described by the message
>
> Filename
> Location in the filesystem
>
> Last-Modified
> A time stamp in RFC1123 notation for use by IMS checks
>
> Size
> Size of the file in bytes
>
> Resume-Point
> Location that transfer was started
>
> MD5-Hash
> Computed MD5 hash for the file
>
> Message
> String indicating some displayable message
>
> Media
> String indicating the media name required
>
> Site
> String indicating the site authorization is required for
>
> User
> Username for authorization
>
> Password
> Password for authorization
>
> Config-Item
> A string of the form <item>=<value> derived from the APT
> configuration space. These may include method specific values and
> general values not related to the method. It is up to the method
> to filter out the ones it wants.
>
> Single-Instance
> Requires that only one instance of the method be run This is a
> yes/no value.
>
> Pre-Scan
> Method can detect if archives are already available. This is a
> yes/no value.
>
> Version
> Version string for the method
>
> This is a list of which headers each status code can use
>
> 100 Capabilities
> Displays the capabilities of the method. Fields: Version,
> Single-Interface, Pre-Scan
Single-Instance?
Also, possibly the field Config-Item. This would allow the method to tell APT
what it's defaults are, and to even say that it has no configurable parts, and
so APT would not need to send a *601 Configuration* message.
>
> 101 Log
> A log message may be printed to the screen if debugging is
> enabled. This is only for debugging the method. Fields: Message
>
> 102 Status
> Message gives a progress indication for the method. It can be
> used to show pre-transfer status for internet type methods.
> Fields: Message
>
> 200 URI Start
> Indicates the URI is starting to be transfered. The URI is
> specified along with stats about the file itself. Fields: URI,
> Size, Last-Modified, Resume-Point
>
> 201 URI Done
> Indicates that a URI has completed being transfered. It is
> possible to specify a *201 URI Done* without a *URI Start* which
> would mean no data was transfered but the file is now available.
> A Filename field is specified when the URI is directly available
> in the local pathname space. APT will either directly use that
> file or copy it into another location. Fields: URI, Size,
> Last-Modified, Filename, MD5-Hash
In the old api, the file: method would redirect apt to the correct location.
The internet methods would d/l to a certain directory, and not need to notify
apt of the location. Is this not so now? Do the internet methods need to
notify apt of where the downloaded file is located?
>
> 400 URI Failure
> Indicates a fatal URI failure. The URI is not retrievable from
> this source. As with *201 URI Done* *200 URI Start* is not
> required to preceed this message Fields: URI, Message
>
> 401 General Failure
> Indicates that some unspecific failure has occured and the method
> is unable to continue. The method should terminate after sending
> this message. It is intended to check for invalid configuration
> options or other severe conditions. Fields: Message
>
> 402 Authorization Required
> After sending this message the method will expect APT to send a
> *602 Authorization Credentials* message with the required
> information. It is possible for a method to send this multiple
> times. Fields: Site
As above, don't limit it to just a username/password pair. There may be
more/less required. Maybe a username is needed, but the remote end doesn't
require a password. Maybe the username was specified in sources.list, and the
method only needs a password. Maybe more info is required. I suggest the
following change:
402.A Input Needed, with echo
402.B Input Needed, without echo
The method requires some form of input that is needed to send to
the remote end. This could be done at login time, or at each
individual URI. After sending this message the method will expect
APT to send a *602 Authorization Credentials* message with the
required information. It is possible for a method to send this
multiple times. Fields: URI, Site, Config-Item
As a possible extension, the method could pre-scan for all URI's that might
need prompting, and issue all Input Needed msgs early. APT might not
nescessarily send the responses back in the same order, and there might be
multiple prompts for a single URI. If a single URI has more than one Input
Needed msg outstanding, then the use of the field Config-Item is suggested, so
as to keep them all straight.
> 403 Media Failure
> A method that deals with multiple media requires that a new media
> be inserted. The Media field contains the name of the media to be
> inserted. Fields: Media
This could possibly be rolled into the 402 message above, with the use of
a Config-Item header.
> 600 URI Aquire
> APT is requesting that a new URI be added to the aquire list.
> Last-Modified has the time stamp of the currently cache file if
> applicable. Filename is the name of the file that the aquired URI
> should be written to. Fields: URI, Filename Last-Modified
>
> 601 Configuration
> APT is sending the configuration space to the method. A series of
> Config-Item fields will be part of this message, each containing
> an entry from the configuration space. Fields: Config-Item.
>
> 602 Authorization Credentials
> This is sent in response to a *402 Authorization Required*
> message. It contains the entered username and password. Fields:
> Site, User, Password
See above(402.A, 402.B)
602 Input Response
This is sent in response to a *402.A Input Needed, with echo* or a
*402.B Input Needed, without echo*. If the 402.? msg that this is
responding to had a Config-Item field, then that is also sent
unaltered. Fields: URI, Message, Config-Item
> 603 Media Changed
> This is sent in response to a *403 Media Failure* message. It
> indicates that the user has changed media and it is safe to
> proceed. Fields: Media
This could possibly be rolled into *602 Input Response*, with the use of the
a Config-Item header.
>
> 605 Shutdown
> APT sends this to signal the shutdown of the method. The method
> should terminate immidiately. Fields: None
>
I cut out stuff that wasn't relevant to my reply, but left enough in to cut
down on confusion.
Adam
/me goes and hides now.
Reply to: