[IGFmaglist] Question about evaluation mechanism - "blank" scores

[Paul Wilson]

It is inevitable that if some MAG members need to abstain from voting on some proposals, then not all proposals will be scored by all MAG members.

But considering the huge number of MAG members, it really does not matter if some proposals are rated by 35 members while others are rating by 36 (say).  With those numbers, every proposal will get a fair hearing.

But it really is essential that by abstaining, a MAG member is NOT giving a 0 score to a proposal, because that represents a huge disadvantage to that proposal.  Rather, as I suggested before, each proposal should receive a final score based on the average of MAG members who DID contribute a score (whether 35, 36 or whatever).

I'd suggest further than MAG members should be able to abstain for any reason they choose.  If I am personally not qualified to rate a proposal, then surely it is better that I voluntarily abstain, than making a guess or a bad judgement on such a proposal?  

In any case, the final score of proposals really must be based on scores contributed, EXCLUDING abstentions.

Thanks

Paul.




On 06/05/2014, at 12:56 AM, Susan Chalmers <susan at susanchalmers.com> wrote:

> Greetings all,
> 
> Paul, yes, thank you for raising this question. I agree in principle that non-scoring (whether out of conflict of interest or, as Marilyn suggested, incompleteness) shouldn't affect the score. I'm wondering, however, whether the Secretariat has a methodology for this? 
> 
> Chengetai, would you mind explaining how that will be handled? Judy raises a good point about the practicalities of this option.
> 
> The question of whether MAG members should as a rule be precluded from voting for their own panels is an issue that perhaps we could explore during the evaluation of this year's process.
> 
> Warm regards,
> Susan
> 
> 
> On Mon, May 5, 2014 at 10:00 AM, Angelic del Castilho <angelic40 at gmail.com> wrote:
> I agree with that Carolina.
> 
> Angelic Alihusain-del Castilho
> 
> Angelic Caroline Alihusain-del Castilho
> POB 3036
> Wanica
> Suriname
> 
> 
> On Mon, May 5, 2014 at 9:23 AM, Carolina Aguerre <carolina at lactld.org> wrote:
> Hi Paul
> I also tend to agree that a blank score should not be  considered a zero  and thus should not affect the overall average score of that session.
> Best
> Carolina
> 
> 
> 
> -------- Original message --------
> From: Paul Wilson <pwilson at apnic.net> 
> Date: 04/05/2014 23:45 (GMT-05:00) 
> To: "igfmaglist at intgovforum.org IGF" <igfmaglist at intgovforum.org> 
> Subject: [IGFmaglist] Question about evaluation mechanism - "blank" scores 
> 
> 
> Hi Chengetai and all,
> 
> can I confirm that if a MAG member chooses not (or omits) to enter a score for any given proposal, that this non-score will have no bearing on the final rating.
> 
> i am opting not to enter a score for several proposals in which I may have a conflict of interest (by organisational association) but I want to be sure that in this case, the proposal score will be calculated as an average of the scores of other MAG members, and that my decision will not effect that score.
> 
> thanks,
> 
> Paul.
> 
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> 
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> 
> 
> 
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> 
> 
> 
> 
> -- 
> Susan Chalmers
> Consultant, Internet Policy
> 
> +1 269 324 4101
> www.susanchalmers.com
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org