Re: Amazon AWS credits for Debian QA work; help needed!
On 01/11/11 at 18:56 +0900, Charles Plessy wrote:
> Le Mon, Oct 31, 2011 at 10:11:28AM +0100, Lucas Nussbaum a écrit :
> >
> > On 23/10/11 at 14:29 +0200, Lucas Nussbaum wrote:
> > > At this point, I'm fairly confident that we will get sponsored by Amazon
> > > to be able to run those tests on the AWS Cloud. I don't have the exact
> > > amount yet, but it's likely that we get enough credits to run ~60 full
> > > archive rebuilds. So it might not be limited to what I'm currently
> > > doing, but could be extended to other tests (installation testing,
> > > execution of code analysis tools, rebuilds in special environments,
> > > etc).
> >
> > I can now confirm that we were awarded $10,000 from Amazon, to use on
> > AWS. (Many thanks to everyone who made it possible!)
>
> Hello Lucas,
>
> this is amzing. I would like to join my thanks to you and everybody who worked
> on this award.
>
> I have been working on using Debian-Installer to prepare super-clean machine
> images for Amazon-compatible clouds, but unfortunately with not so much
> success. See the following bugs to see details about where I block (the
> partitionning).
>
> http://bugs.debian.org/637784
> https://bugs.launchpad.net/ubuntu/+source/partman-base/+bug/862903
>
> I welcome your or others comments about my approach.
It seems that you are trying to partition the EBS volume, while the EBS
volume is supposed to be used as a partition already (at least that's
what I saw in the various AMI I tried).
> I hope that with this or as co-maintainer of the euca2ools package I can help
> you on the archive rebuilds project. Or with other contributions ? But for
> the moment my experience with cloud computing is limited to read D-I partman
> logs on micro-instances…
I don't know yet what will be our exact requirements, but I'll let you
know, sure.
> By the way, I am very curious about the calculations you made to estimate 60
> full rebuilds to 10,000 dollars. I would be very interested to read it if you
> can share it.
I rebuilt 20 packages on each instance type to determine the most
cost-effective instance type, and the time ratio between a rebuild on a
Grid'5000 cluster and a rebuild on that EC2 instance (result: m1.large
instance type is the more cost-effective to rebuild packages ; ratio =
1.35. m2.large instance were faster, but the speed increase was lower
than the cost increase).
With that, I could estimate the cost of a full archive rebuild on
m1.large instances ($159). $159*60 = $9540, which is what I asked for.
The "60 rebuilds per year" number is based on the fact that I did 21
rebuilds last year, and that I had to decline several time when people
asked me for custom rebuilds. I expect the demand to increase if it's
very easy to run custom rebuilds.
Of course, that's all back-of-the-envelope calculations, but that's all
I was asked for.
It's likely that we can reduce the cost of an archive rebuild. For
example, we could use many micro instances for small packages (those
instances are considerably slower and can't build all packages, but are
more cost-effective). We could also use "spot instances" instead of
regular instances.
> Have a nice day, and thanks again to you, “everybody who made it possible”, and
> also to Grid'5000 for the previous rebuilds, because you just gave an
> interesting estimate of how much we would have had to pay to do rebuilds
> without their support.
Thanks :)
Lucas
Reply to: