moving of dinstall's run time on master
This has turned into a heated topic, which was not my intention. I never
meant to inflame anyone. If I stepped on anyones toes, then I apologize for
doing so.
Now, for some clarifications.
My initial email was from the point of service provider. Novare provides the
service of bandwidth to the debian project. We also host several different
machines for debian. Master, murphy(lists), faure, albert(both alphas). New
to the list, and not yet online, are a sparc, and a new box donated by va.
Novare is charged based on how much actual data is transfered per month. We
have a flat rate, upto a certain point. After which, we get charged in
increments. We have not come close to this limit, so we have not experienced
any additional charges. Anyways, if we did, we would just pass them on to our
customers, so this point is mostly moot.
Our upstream provider, however, is charged based on how much bandwidth is
used based on a percentage of the available. Having the mirror pulse happen
during peak net hours(3pm to 7pm, local time), can possibly hurt our upstream.
Now, you could say that we shouldn't care about our upstream, but, if we can
help them out, by having the mirror pulse moved, then I don't see any harm
done.
Jason Gunthorpe, head debian admin, has setup mrtg on all major debian
servers. This includes master. To see the stats, go to
http://master.debian.org/mrtg/. Note, that the numbers have been going down
over the past year. This could probably be attributed to the fact that the
admin team has policed people to not do personal mirrors off master. Having a
mirror pulse contact the tier-1 mirrors, which then contact master, and
pull-in the days updates, has helped the people who keep personal mirrors
spread their load around.
The stats at the address above, also include traffic that is generated by
novare itself. This traffic does not go out to our isp, so it is not included
in the numbers that they tabulate. Part of this novare traffic, includes a
local full mirror of debian, which is used by the alphas(and the soon to be
online sparc), and also by the rest of novare. Master and murphy are not
physically located at novare anymore. They reside at a colocation facility,
which is about 15 miles or so away.
I am a developer(see email above), and I also work at novare, which hosts
debian equipment. So, I have a unique perspective on lots of sides of this
issue. I know some of what master actually does, and what it is capable of
doing. I also know about the donor's perspective. This can sometimes lead me
in an awkward position. Ean Schuessler, who is co-owner of debian, and my
boss, has very often asked for my opinion on internal debian affairs, seeing
as how I have a little over a year of actual 'debian work' over him. He,
however, has been affiliated with debian, as an outsider, of sorts, for much
longer, longer than I even knew debian existed.
I will quote parts of my email, and add comments to them, so as to try and
explain, and to further clarify, what I meant.
My original email will be prepended with '> ', and my restatement will be
surrounded by '-'. Any addition comments will follow the restatement.
----
> The current time that dinstall runs, 1:53pm CST(local), 7:53pm UTC, makes
> the mirror pulse happen around 3:30pm(9:30) to 5:30(11:30). This is during
> peak network hours. This is wrong, and this has to be changed.
-
Dinstall currently runs at 1:53pm(7:53pm). Dinstall usually gets done
anywhere from 3pm(9pm) to 5pm(11pm), with a mirror pulse following immediately
after. This mirror pulse contacts all tier-1 mirrors, ie, the mirrors that
update directly from master. This is currently 9, split into 2 groups,
offset by 10 minutes. 3 of those are non-us mirrors(txs jason for the info).
The average update, for a full mirror, is around 50megs a day.
Peak hours locally are from 3pm(9pm), until normally about 7pm(1am). This
corresponds to when kids get out of school, and get home, and go online, and
also when parents get out of work, and do the same thing.
Having the mass-update happen during peak, means the update might take longer,
as it has to fight with all the other accounts our isp has. Our isp also is
charged differently for its bandwidth. They are charged, not on the amount of
data they send over a month, but by how much bandwidth they use over a much
smaller period. If they have a spike, during the day, they could see their
fees go up. They, of course, would not like this. They have noticed the
large spike coming from us(novare), and have inquired as to why it happens
during the middle of the day.
-
> I want to move the time that dinstall runs, and I was thinking about making
> it run at midnite(12:00am, 6am UTC). This would put the mirror pulse at
> 1:30am (7:30) - 3:30(9:30).
-
We(novare) would like the time that dinstall runs to be changed.
Midnite(6am), is a nice round number. This puts the mirror pulse at about
1am(7am) to 3am(9am).
-
Not much more can be said here.
> This email is not a discussion about whether to move dinstall's run time.
> It will happen no matter what. This email is to discuss possible
> alternatives.
I am not going to restate this, because I find it a little hard to do so. I
will just comment on it.
I am not aware of any technical problems with moving dinstall's time. I was
under the impression that the current time was picked a little 'out of the
blue.' So, I expected no opposition in having it moved. I was looking for
discussions has to when a better time might be, as long as it wasn't during
local peak network hours.
> Did some initial discussion on irc(irc.debian.org(also irc.openprojects.net)
> #debian). Why can't we have dinstall run several times a day, and only have
> the mirror pulse run once? Can dinstall's parts be run separately like
> this?
Again, I don't see much need for restatement.
This was thrown in, kind of as an after thought. It doesn't quite fit in with
the tone of the rest of the email, and should probably go on a
'debian-dinstall@lists.debian.org' setup(joey?).
Next, is a short followup to the original email, explaining something I left
out.
----
> Sorry, forgot to say this won't be happening today, nor tomorrow. I am
> shooting for the end of next week. Say, saturday, apr 3.
This was to allay any fears people might have as to me doing something
unexpectedly. The actual time that dinstall would have its time changed is
negotiable. I wanted to make it clear that it would not happen suddenly.
----
Ok. Done. Phew. Too much typing(I need to talk to Ean and enter into some
finger excercise classes).
I hope I have not left any loose ends, and I hope that this alleviates some
fears, that I have seen on this topic.
Adam, who was only trying to do his job
Reply to: