[IGFmaglist] IGF 2017 MAG WS Selection Process - a proposal

Mon Jun 12 15:56:09 EDT 2017

Thank you Elizabeth,

comments below.

> On Jun 12, 2017, at 3:47 AM, THOMAS-RAYNAUD Elizabeth <elizabeth.thomas-raynaud at iccwbo.org> wrote:
> 
> Dear Chair,
> 
> Regarding the proposal for how to select the final roster of workshops, I appreciate the opportunity to consider it ahead of the meeting.
> 
> I took some time before replying because I wanted to canvas other views and check my initial assumptions and reactions against those of others so with that feedback, I'm writing to share some concerns about point 2 as proposed. 
> 
> Past practice has been that those above the cut-off have not been re-reviewed by the collective as the care and attention to evaluation had already been taken by the group. Yet, while some have been uncomfortable with overturning any of the evaluation there has been agreement to use a certain number of the designated slots (this year proposed from 72-80). In this way, only a small number of those evaluated might be overruled in favour of those we felt could fill the gaps/imbalances especially as per the goals we've set for ourselves (increasing participation of govts, private sector, developing country diversity/inclusion etc.). On that basis lesser-rated items would be supported by  MAG mentor(s )to improve the session to better meet the required criteria and the MAG would support their inclusion.

I was not intending that we would re-review the top 72, but only do what we have done in past years - that is, with the benefit of the statistical analysis, look at those top 72 as a whole, in order to see if there are any imbalances the MAG should take a closer look at.   If there are adjustments to be made, I would expect in most cases it would mean pulling a few additional workshops in - to reach the goal of 80.

> Re-reviewing those top 72 collectively seems from past experience with those lower than the cut off that we risk consuming a lot of time, and without enough clarity on what the value-add would be. Rather let us focus our time and attention on the rebalancing effort as proposed by the wild card exercise and perhaps complemented in the new session format work. 

Yes, this is what was intended. Apologies if it was not clear enough.

> Regarding point 3, while the variation may identify a stronger difference of view on a proposal, let's use the wild card to first and foremost serve the goal of filling the gaps with the best potential content rather than pre-selecting the options by those that had the widest differences in views. If the first goal is best addressed while meeting the second goal - all the better. Perhaps that was assumed in what was written but I flag it as it was not explicit and we may miss the first goal if we start with the second.

My intent was to use the so-called wildcard process (which could include WS identified in your individual reviews OR taken from the standard deviation list) to address any imbalances or fill the gaps.   So, no intent to walk through the standard deviation list one-by one.

Having said all this, it feels as though we are conflating the wildcard process (which was initially put forth to identify a very small number of proposals felt worthy of additional review during the course of your evaluations) with a process that is intended to address any imbalances present in the “top WS” selected (post MAG review), and these may be better understood/addressed if dealt with separately.

Thank you Elizabeth, hope this helps and let’s see where the full MAG is on this tomorrow.

Best,
Lynn

> Look forward to seeing everyone later this morning.
> Cheers, Elizabeth
> 
> 
> 
> Elizabeth THOMAS-RAYNAUD
> International Chamber of Commerce (ICC)
> Project Director, ICC Business Action to Support the Information Society (BASIS) 
> Senior Policy Executive, Digital Economy
> 
> -----Original Message-----
> From: Igfmaglist [mailto:igfmaglist-bounces at intgovforum.org] On Behalf Of Lea Kaspar
> Sent: 12 June 2017 00:10
> To: Lynn St.Amour
> Cc: IGF Maglist
> Subject: Re: [IGFmaglist] IGF 2017 MAG WS Selection Process - a proposal
> 
> Dear Lynn,
> 
> Thank you for outlining the process to finalise the workshop selection process.
> 
> Two questions to make sure I understand the next steps (apologies if these were raised on last week's call; haven't had a chance to see the summary yet):
> 
> 1. Are there any workshops that will go through automatically (e.g. the top 50)? I seem to remember this being the case last year. I thought that was a good approach that validated the scores given by MAG members who evaluated the proposals in the first round. 
> 
> 2. Could you give a bit more information about step 2 outlined below? You note that all MAG members will now evaluate the top 72 workshops. Does this mean we are meant to score the top 72 workshops again? I'm not sure I understand what exactly we will be evaluating, how the result of this evaluation will be captured, and how the result will influence the final outcome.
> 
> Unfortunately, I won't be able to join you in Geneva this week, but intend to participate remotely. Having clarity about the process will help ensure those of us not in the room can meaningfully contribute.
> 
> Many thanks.
> 
> Warmly,
> Lea
> 
>> On 11 Jun 2017, at 21:23, Kenta Mochizuki <kemochiz at yahoo-corp.jp> wrote:
>> 
>> Dear Chair, 
>> 
>> Thank you so much for your proposal while very busy. 
>> 
>> I personally find your proposal acceptable, and hope our WS selection process will go efficiently and smoothly, while taking into account as many opinions and suggestions as possible.
>> 
>> Thank you again and look forward to meeting you tomorrow.
>> 
>> Sincerely,
>> 
>> Kenta
>> 
>> -----Original Message-----
>> From: Igfmaglist [mailto:igfmaglist-bounces at intgovforum.org] On Behalf Of Lynn St.Amour
>> Sent: Monday, June 12, 2017 4:35 AM
>> To: IGF Maglist <Igfmaglist at intgovforum.org>
>> Subject: [IGFmaglist] IGF 2017 MAG WS Selection Process - a proposal
>> 
>> Dear MAG members,
>> 
>> As discussed during the last MAG calls/MAG list and building on the past 3 - 4 years practices, I would like to propose the following methodology for our WS selection process:
>> 
>> 1 -  MAG precedes on the assumption that the number of Workshop slots to be allocated by the MAG will be 80.
>>   • The first 72 proposals have been highlighted based on the MAG workshop evaluations and following the new process.
>> 
>> 2 - The MAG evaluates those (72) highlighted against the profile of all WS proposals submitted, paying attention to any imbalances or over- representation against our diversity criteria (topic (tags), regional, SH, developed vs. developing, gender, 1st time proposers vs. returning proposers, session types, etc.), and identifies any general reflections or concerns.
>> 
>> 3 - Using the “wildcard” process: each MAG member (if they so choose) can identify one additional WS to be further evaluated by the full MAG due to extenuating circumstances or in response to any perceived imbalances.  As part of this step, MAG members should review the list of WS proposals with the highest standard deviations to see if there are any proposals that should be included in this wildcard process.
>> 
>> 4 - Secretariat can capture Mergers supported by the full MAG.  And, each Merger candidate should have a minimum of 1 (2?)  MAG member(s) to lead the merging discussion as they do not come together easily.
>> 
>> 5 - The entire MAG will participate in this process (no breakout groups), and we will work to find consensus on the chosen Workshops.  Consensus will be judged based on the level of support and the absence of any strong reason based objections.
>> 
>> I hope you find this proposal acceptable.  If something concerns you, please suggest an improvement.
>> 
>> Best regards, and look forward to seeing everyone soon.
>> 
>> Lynn
>> 
>> 
>> 
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
>> _______________________________________________
>> Igfmaglist mailing list
>> Igfmaglist at intgovforum.org
>> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> 
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org
> _______________________________________________
> Igfmaglist mailing list
> Igfmaglist at intgovforum.org
> http://intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org