Moray Allan <moray@sermisy.org> writes: > On 2013-05-07 22:38, Gunnar Wolf wrote: >> tiago_: h01ger, we from team were kind of accused of not being clear >> about money allocation for travel sponsorship last year, so >> I've been thinking that the team should take care of >> procedures, but rating should be done by (semi)random people, >> in short. > > From subsequent discussion, I think we mostly agree: we should look at > doing something along the lines of what Phil has proposed in the past, > i.e. getting a more diverse group (not just debconf-team or our direct > friends) to vote on this. > > I don't think that having truly random (with equal probability for each > person) selection is the best idea, as I fear this could reduce to > finding the people who are most popular out of those who are most > visible -- many Debian people don't have a good idea of who is involved > beyond their own area of interest. (This danger would be especially > present if we, as has been mentioned as an idea, only asked each person > to vote for a few people who they think should be sponsored, rather than > asking them to rate almost everyone.) > > Similarly if we ask too many people to vote, I think they won't take > the task as such a serious responsibility, and won't perform it as well, > as if it's the team is kept fairly compact -- it's not rational to spend > a lot of time looking up information about people if your own rating > won't count for much. I think that's probably true if you were asking them to rate everyone, and would probably give results that were reliable at the extremes, and fuzzy in the middle. Such a process would effectively pick out the easy cases and not help with the hard decision of what the order in the middle of the pack should be, and where to draw the line in that pack between sponsored and non-sponsored. How about giving people a short list of people to rate, on a (largely) random basis, and then combining the results of these ratings? Something like the slashdot moderation system, say. If we know roughly what proportion of funded to unfunded folk we're aiming at, I'd think that rather than simply asking people to put applicants in an order we could instead say to them something like: We would like you to assume that you are responsible for allocation of $10k in travel sponsorship. Here are applicants asking for a total of $15k [or whatever] in sponsorship. Please allocate the sponsorship as you see fit, but treating the task as though the money were you're own. Do not feel that you must allocate all the funds if you think that the candidates are too weak to deserve sponsorship. choosing the numbers so that each judge only sees about 5 applications, say. Perhaps we could also allow people to indicate a difference between thinking that someone should be left unfunded and having no opinion either way. Given that each judge would only get to see a short list of applications, I'd think they should see full details of the applications (rather than anonomised versions) but only after having agreed to keep the detains private. We could do this in multiple rounds, focusing in on comparisons between people where the decision is still unclear. We could perhaps give people who get a lot of don't knows an opportunity to provide more information, but that's probably unfair on those that get roundly rejected and so don't get a second chance. I've no idea how one combines the results from such a collection of partial orderings, but I presume that methods exist. I think if we make it clear that there is no obligation to perform the selection, if you don't like the idea of an NDA, or just don't have time, we should be left with people that have sufficient motivation for the task. Cheers, Phil. -- |)| Philip Hands [+44 (0)20 8530 9560] http://www.hands.com/ |-| HANDS.COM Ltd. http://www.uk.debian.org/ |(| 10 Onslow Gardens, South Woodford, London E18 1NE ENGLAND
Attachment:
pgpZOcjBeq8nl.pgp
Description: PGP signature