[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: partial snapshot mirror amd64/bullseye/bookworm


Le 3/2/21 à 10:58 AM, Lucas Nussbaum a écrit :
On 01/03/21 at 22:41 +0000, Paul Wise wrote:
On Mon, Mar 1, 2021 at 5:25 PM Holger Levsen wrote:

For https://trends.debian.net/, I have a local mirror of snapshot.d.o
(with sources only, and only for specific versions). The code used to
create it is available in https://salsa.debian.org/lucas/dhistory/-/blob/master/dhistory

Specifically, it:
- queries the snapshot DB to identify the files and hashes for each
   source package
- fetches and analyses Sources files to identify (source,version) of
   interest, and thus hashes to transfer
- transfers those hashes from snapshot.d.o to my own machine using rsync

The query used for the first step is:
   psql -At service=snapshot-guest -c "select row_to_json(t) from
   (select srcpkg.name as source_name, srcpkg.version as source_version,
    file.name as file_name, file.hash as file_hash, file.size as file_size,
    node_with_ts.first_run as file_first_run, node_with_ts.last_run as file_last_run
   from srcpkg
   inner join file_srcpkg_mapping on srcpkg.srcpkg_id = file_srcpkg_mapping.srcpkg_id
   inner join file on file.hash = file_srcpkg_mapping.hash
   inner join node_with_ts on node_with_ts.node_id = file.node_id
   inner join archive on node_with_ts.archive_id = archive.archive_id
   where archive.name = 'debian') t"

That's the query that would have to be adapted for binary packages and
for a specific date range.

Following discussion on #debian-reproducible, would this be feasible from the snapshot team to allow something like a daily export of the database on a public exposed server? Not necessary the whole, but only restricted suites and arches. Having such public content like UDD, would allow us to have access to files and hashes information. In this case, that would ease to create a mirror script of snapshot.d.o.

Best regards,

Attachment: OpenPGP_signature
Description: OpenPGP digital signature

Reply to: