[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Debian snapshot.debian.org replica on the Amazon cloud?




-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
Hi all,

I have requested if AWS can support with this service. I am hoping to have an answer in the next week or so. However there is one question I wanted to pose to the collective who are managing this service about the data store (not the database - that's no problem).

AWS has several storage options, amongst which one is S3 (live online storage - http://aws.amazon.com/s3) and one of which is cold storage/archive by name of Glacier (http://aws.amazon.com/glacier). Glacier is significantly more cost effective, with access to 'restoring' archives from Glacier taking a 3-5 hour wait. S3 and Glacier are integrated by way of "S3-Archive", whereby files (objects) in S3 can be transparently migrated to Glacier storage in the background after some period. When one of these archived files is required, a client may call a restore on this file (object), and within 3-5 hours (as above) a copy of the object is retrieved and pushed back into live S3 for a number of days (using the S3 Reduced Redundancy service).


We would need to create a front end interface that lets people request the 'restore' of an archived file (and a notification when its available) in a  controlled manner. When you restore a file, its 'revived' back into S3 storage where it can be accessed - for a number of days before being purged from S3 (it still remains in the archive cold storage until deleted permanently).

To give some more colour to this scenario - fresh files could be left in S3 for 365 days, and then archived. When we restore an object, we could do so for a month.

I would appreciate feedback from the group if is this is useful.

Sincerely,

  James

On 27/09/2013 5:32 PM, Paul Wise wrote:
> Hi James,
>
> I'm wondering if Amazon would be interested in donating hosting for a
> replica of the data behind snapshot.debian.org. It is around 17TB last I
> checked and constantly growing as people upload more packages. There is
> also a postgres database with metadata, I think that is about 60-70GB.
> Currently there is only one functional replica of the data.
>



- --
/Mobile:/ +61 422 166 708, /Email:/ james_AT_rcpt.to
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
 
iQIcBAEBAgAGBQJSWPfXAAoJEK7IKHSdhcU8/MYP/j/zzI8xOTCNSQEtVLXtZDm1
Mk/xGSexDBuEIrDU8gpPgCNUUP89yjxKVwR8Uq90mM6wvUfcANNpphpNmI3J95dQ
+WoRF1nkEYqqR5liefCv6IJVlSu7dczpeDgpB86quBeKUXXj0FSqHHHH7iIXj194
4la4q7lcNyxD87t1fFyZ1Wb4IY9H78gMB1YnBzWE9MEfbDF+vtddJbykSr+SR+HM
yfuzuTXtu4UidcIVVtdAplpGm+inobN9alR5B52YAfsPbLbbrNeFFhyZtGv9QyhJ
D0rJR7NhzEfk3uJeDZLjHiQNqld2qaVmCH7usld1tCeirCJzHzUHOx53f9plodd0
njZjqTLlDnPuXbhkM4TqOsPpok89efLNY9oo3LI/O+flfMnHB0XlTC0Ar7AGDdmj
GltvTnftwESFyMs0F+xxbXYEP7L8Ep7X2+HC4F2jMxIsCBy+shF2EkabmZbSmtpy
CpxIQrtmbYF/VwHRDrvgIcYAHaXlFhQxTtNnCy4NZzxEpJ6LCxtniKWRYJDTJCqT
wlomDO0JgfSH0+UMpq39vVSExV6eg7BXzEdky5aZGFnFuegm247QHqbhno50sDgt
WKfuTvYFShSIgMItoj/Tlntbyq62OTZ857DT0aKhq2wwMPJLDBatCh9+UTFzNXeK
m65Ij0vX0sinyGjJ29Js
=BF5x
-----END PGP SIGNATURE-----


Reply to: