[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

[Debconf-discuss] Talk selection for Debconf 7

On 8 Jun 2006, Alexander Schmehl outgrape:

> Dear Manoj,
> [ written during a long "discussion" in #debian-women, which made me
> angry; rewritten, deleted and again rewritten the next morning; This
> mail is intended as constructive critique, not as personal attack
> ]
> * Manoj Srivastava <srivasta@acm.org> [060607 21:49]:
>> An interesting idea came up during debconf; now that we have
>> software for people to vote on talks, and indicate which talks they
>> would be interested in attending.  With such a mechanism in place,
>> I think we can replace the academic committee, except perhaps to
>> fill a few slots left open for the organizers to give to less
>> popular but desrving talks a chance.
> I think you miss the "big picture behind that".  The committees job
> is not only to select "good" from "bad" talks; is also to ensure a
> well balanced conference programm.

        While a fine goal, it is a difficult, subjective one. And the
 execution depends on experience, and judgement, and, as such,
 leaving it to any one who volunteers may not be the best way of
 selecting people who make such a decision.  This is too much of a
 burden to impose on the few who do stand up to be counted, and
 volunteer their time.

> And as usual done on community projects, it's done in a doocratic
> way: Whoever feels capable and willing do the job, decissions are
> made after public discussion.  That's it.

        I am afraid that this might not be adequate criteria for peer
 reviews and sophisticated juggling that determines what "balanced"
 means. This is a huge, complex, task, and is quite quintessentially a
 subjective decision; the more input that is considered, the better it
 represents the views of the people attending the conference, more on
 why it is important to do so below.  I think expecting people who are
 volunteering to be able to shoulder this burden is unreasonable.

> If you - or someone else - thinks we did a bad job, feel free to
> join the team for the next conference.

        Err, I think the entire process of having a "team" which makes
 such a judgement is the wrong mechanism, so just perpetuating the old
 way of doing business is not good enough. Allow me to elaborate.

        The selection of a set of talks is perhaps the most critical
 single decision for a technical conference; the quality of the
 conference, and the benefits to the community, stem from this single

        Let us go to the basic rationale for having talks and why hte
 project, and the sponsors, should care (the benefit of a number of
 people meeting for rapid discussions, for creating and refreshing
 personal relationships, and the innovations and ideas stemming from
 off the cuff shooting the breeze sessions are beyond the scope of
 this analysis).

        Talks are essentially information transfer. To an individual
 developer, talks help them in doing things for Debian.   It is either
 instructive, teaches  them new things, it is in an area they are
 interested in, and perhaps gives them an impetus to do something
 new, something that perhaps had been on the back burner before they
 heard the talk.  Or it explains critical infrastructure for the
 project, or critical processes, that help future contributors to the
 infrastructure, or future assistants (for example, looking up build
 log failures and fixing rc bugs). 

        In this case, no one knows better than the attendee what kind
 of talks would help them improve debian; so collectively, voting for
 talks gets us stuff that motivates/energizes more people; this is
 something a small group of people is less likely to do well.

        So how do we go about implementing this selection, and why not
 select a cathedral select few that have the authority to decide and
 exercise the judgement about what goes in the conference?

        One of the interesting new processes about how to make
 decisions based on incomplete data, and which involves subjective
 judgement of the information that is available, is the market
 mechanism.  The collective judgement of the market, in which each
 investor is trying to maximize their own return on investment,
 outperforms the judgement of experts in the field most of the time.
 As long as the "investors" are involved in the field (investing, or
 security experts, or any domain involvement at all), and have some
 incentive, the collective judgement is often better.

        The people attending the conference are not uninformed -- we
 are spending a considerable effort to attend the conference, and we
 all are vested in improving Debian.  We all have domain expertise in
 what it takes to work on some part of Debian. This part of the
 criteria fits.

        We all have a personal incentive to not have our time wasted
 by talks that one is uninterested in, and  select talks that,
 individually, would help us in our tasks for Debian; so the market
 incentive exists. All attendees would be trying to maximize the
 benefit of the conference for themselves (return on their investment
 of time and effort).  This criteria fits as well.

        We can open up talk selection to the attendees, and improve on
 the results, since collectively the group has a better judgement than
 _any_ group of experts (not that we have a process of selecting for
 expertise in the academic ctte -- it is mostly self selecting, small,
 group, the common criteria being that one stands up to be counted,
 and putting on their shoulder such a requirement is not good).

        The concrete proposal is this:  We already have software
 in place that allows registered attendees to vote on proposed
 talks. Every individual is allocated, say, $1000 in funny money (or,
 1000 points, if you will), to allocate as they please between the
 talks proposed.  The amount of funny money (or points) that a talk
 garners would determine a rough ranking.  Timing issues can be
 discussed (as in, deadline for abstracts, deadline for selection of
 talks, deadline for full paper submission, etc).

        The benefit of this process is that it is transparent, far
 more democratic than open membership in a small group, and leverages
 the distributed domain knowledge about which talk would serve the
 attendees the best.

        We can also reserve a few slots for the organizers to
 "balance" the talk selection, and to allow for a few talks that are
 deserving, but failed to meet the group selection thresholds.

        There are a few potential lacunae in this approach: People may
 vote for "popular" speakers.  I am not sure that this is really an
 issue: firstly, most people would want to minimize the number of
 "boring" talks, so there is a self correcting incentive in place.
 This is similar to some people having "favourite" stocks in the
 market analogy -- because they work for the company, or Daddy worked
 for so-and-so -- but overall, the market still outperforms experts.
 Secondly, this is cathedral think: that a select few people, by
 volunteering, have better judgement than the unwashed masses about
 what is good for them.  If the attendees collectively want to hear
 Joey Hess speak, why should a small group thwart the wishes of the

        Secondly, it was noted that sponsorships are preferentially
 awarded to speakers,  and people may be able to sway enough others to
 vote for their talk in order to attend the conference.  Again, I feel
 this is a low likely-hood corner case; I certainly am not going to
 vote for a talk just so someone can bilk Debian of money. I strongly
 suspect most of us would not, so this is not something I think we
 really need to guard against.  I would rather not presume that the
 people voting are going to defraud the conference.

        So, this is my proposal, and rationale, for improving the
 selection of talks presented at debconf 7, and how to make the
 process more transparent, go from the cathedral to the bazaar model,
 involve more diverse viewpoints, and, by the nature of the problem
 being solved, come up with more optimal results, though of course
 that last part is hard to quantify.

        Most people would want to skip the next part of this email;
 but I do feel I have to respond to the personal attacks, in as
 non-inflammatory  manner that I can.

> But - now a bit more personal note - let me give you a small tip:
> IMHO this was a bad start, before you start to critisize our
> practise you should at least listen how things are done and learn
> why they are done this way.

  A) I was there. I saw the talk selection, and read the abstracts,
     including the talks thatwere rejected.
  B) This is not rocket science.  Submission of papers to peer
     reviewed journals and professional conferences is not an obscure,
     arcane process (unless you have never participated in one)

        The proof of the pudding is in the eating of it. No one
 expects a small team of volunteers to perform like the vast numbers
 of reviewers that professional journals utilize; no small group has
 the breadth of expertise required for this task.  Expecting such from
 a small group of volunteers is unsustainable. Nobody, but nobody,
 does that; journal editors farm out that task to domain experts in
 the field for each paper submitted to them.

> <irony> You know, just because we are younger than you, doesn't mean
> we do the thinks on a gut level.  We have reasons to do things, the
> way we do. </irony> Which leads to the next point: Don't play the "i
> have more experience, because I'm older, so shut up" card.  It
> doesn't work; even worse: It might even be seen as arogant by the
> people you try to "dicsuss" with.

        My age has nothing to do with it.  Neither does
 yours. You really do not want to go there.  Get the chip off your
 shoulder, stop obsessing about  your age, and look at the substance
 of what is being proposed.

> During last nights chat, you barly gave me the chance to explain the
> reasons of our doings, after you asked for them!  So I rather think
> you are not interested in our position, but mostly on discussing /
> trolling.  Sorry, but I have other things to do than to discuss
> stuff that way.

        I am interested in improving the future conferences, and the
 the benefit to the project.  The talk selection, and the importance
 given to streaming video over hacking and collaborative development
 are the two major flaws I, and others, felt existed at this
 conference.  I think we can do better, unless we take this all
 personally and refuse to hear any proposals for improvement, and the
 accompanying critiques of what could have been done better.

"Big Brother is hallucinating."  Elizabeth D Zwicky
(zwicky@cis.ohio-state.edu), title of a comp.risks article
Manoj Srivastava   <srivasta@acm.org>  <http://www.datasync.com/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C

Reply to: