[IGFmaglist] May evaluation procedure (Was Online registration for May Open Consultations and MAG meeting)

Mon May 5 13:35:12 EDT 2014

Dear GAC members

Thank you very much for your thoughts about the Paris meeting. 

May I suggest that we try to avoid parallel sessions during the meeting but to try do all our tasks at the plenary. Chengetai will send out a draft Agenda in coming hours.

We would start the meeting with the general discussion about the IGF meeting structure, plenary sessions, lessons from NetMundial and how the IGF should respond to the challenges identified in Sao Paolo. This would be for the first day of the meeting.

At the lunch time UNESCO would present its work on the ethics of the information society and would pick MAG members brain on that work.

Decisions on the structure of the meeting, including the number of workshops, will help us to address the workshop selection task. We will devote two days for that debate and decisions. I would rely on the initial evaluation made by you and also your suggestion on possible mergers of proposed workshops. We will organize our debate around identified subjects/issues. Agenda will reflect this proposal.

As per question raised by Paul about abstention from the rating- I agree that some exceptions could be made and those should be not taken into account during the mathematical calculations of the evaluation results. But abstentions should not be abused and should be seen as exception rather than a rule.

We will have  a chance to exchange on the modalities of the Paris meeting during the conference call on Wednesday, 7 May. Hope to meet you then.

Thank you very much for your dedication to the IGF

JK

Sent from Surface

From: Fatima Cambronero
Sent: ‎Saturday‎, ‎May‎ ‎3‎, ‎2014 ‎7‎:‎29‎ ‎PM
To: Carolina Aguerre
Cc: igfmaglist at intgovforum.org

Dear Susan,

Thanks for refreshing us the discussions and agreements that we had in February meeting.

I tend to agree with your words: 

"Based upon these guidelines I do not see any reason for MAG to break into groups, especially since we decided against assigning subsets of proposals to evaluate, to subgroups within the MAG. In other words, it's expected that all MAG members commit to grading all 200-odd workshop proposals. Also, as a matter of logistics, plenary would be much easier for remote participants".

I was participating in my first MAG meeting last year when we had break-out sessions to assess the proposals. It was very complicated for me to follow the discussions remotely. I really got a lot of help from Raul, Mathew, Farza, among others, who were present at the meeting who allowed me to make some contributions.

I hope all MAG members could discuss all the proposals all together. I think this will facilitate the participation of everyone, physical and remote attendees. In my opinion this will also prevent some groups to be stricter than others in the evaluations of proposals.

I also support Carolina's comments of having 2 groups of volunteers: one to do a follow-up / mentoring for mergers and another group to mentor proposals that are on the threshold of acceptance.

Susan, I'm sorry you cannot attend the Paris meeting. I also offer my collaboration to take notes of the process. 

Best Regards,

Fatima

2014-05-02 12:45 GMT-03:00 Carolina Aguerre <carolina at lactld.org>:

Hi All

Susan, you will be very much missed but count on me for notes and catching up.
With respect to your questions and those raised by Subi, Marylin etc I agree that clearer mechanisms for operationalising mergers should be ideal. We tend to devote a great amount of time to those workshops that fall out of the accepted / rejected category and clearer criteria would be very helpful since 3 days might look like a long meeting, but we really must put our heads together around several key issues since September is around the corner.  

Some categories which could be used as basis for these mergers are: 
- relevance of the proposed topic (relevance could be measured against the themes that have been proposed by the MAG for this year and "extraordinary" events such as NTIA announcement and NETmundial for example); 
- representativeness of the proposal following Bill's suggestion below.

I would support the idea of having 2 groups of volunteers, one to do a follow-up / mentoring for mergers and another group to mentor proposals that are on the threshold of acceptance (which should not exceed the number of 5-8 proposals, I think we're already very stretched as a group with the limited time we have this year).

In particular believe there are other topics besides workshops that we should definitely have to dive into thoroughly: a) main session structuration - the input sent by Vlada on 17 April around main sessions could be a good starting point and considering there are innovations this year - such as the BPF which is already making progress. We had also discussed at the last remote MAG meeting to include roundtable sessions too; 
b) the input provided by Netmundial and what we might begin to accomplish in the time we have for the 9th IGF.

Regards
Carolina

On 02/05/2014 11:27 a.m., William Drake wrote:

In previous years we had Excel spreadsheets. Workshop proposals were grouped so one could easily see how many there were per theme and think about the overall program balance.  One also could see how many there were per specific topic under each theme, which made it easier to comparatively evaluate and to think about possible mergers; and how many proposals there were from any given organizer.  This would have been a nice complement to the web form. 

Bill

On May 2, 2014, at 2:49 PM, Marilyn Cade <marilynscade at hotmail.com> wrote:

i want to agree that we do   need some 'guidance' for those whom we ask to merge. Last year, a few folks got sent to merge, and found 'nobody home' at the workshops they were directed to. 

If merging is a standard operating practice, then we have to make sure that it is actually feasible and that the 'receiving workshop group understands what the expectations are that they are open to merging. 

M

Sent from my iPad

On May 2, 2014, at 12:04 PM, "Subi Chaturvedi" <subichaturvedi at gmail.com> wrote:

Dear Janis,Susan and all, 

Many thanks for your very valuable contributions Susan. Also thank you for addressing the concerns that I had raised on the last call about the new evaluation guidelines and the familiarity that all of us must have with them as MAG members-old and new. Your email was most helpful. And we have made fantastic progress from last years. It is indeed most unfortunate that you won't be able to join us in person Susan. I'd be happy to assist the process by taking down detailed notes so that we can continue to improve the evaluation process and for institutional memory and share them with you and the general MAG list.

I am however most interested in how we will will respond to the following issues raised by you before we go into the May meeting :

"
I believe that development of the Procedure should be guided by what the MAG has agreed upon and adopted. (I have attached that document to this email - there was interest expressed in seeing the document during the last call). In relevant part, during Stage 3: 
MAG members look at the results to ensure an overall balance of the themes/topics covered;
MAG members discuss 5-10 proposals just below the threshold of space availability*** to determine if improvements can be made to overcome proposal deficiencies;
Proposers contacted by May 26th and asked to submit revised proposal to address deficiencies by June 16th. If the proposer responds the expectation is they will get a workshop slot;"

Before we are ready to publish the final list of accepted proposals on 30th June and even before we get into the May meetings, we as MAG must have clarity on these issues, so as to avoid any arbitrariness in the process and be completely transparent about the decision making process. 

Janis if you could please address these questions it would be most helpful:

1. Will we have breakout groups this time as well, according to themes?

2. Are we hoping to achieve a thematic balance of workshops or will we follow a straight ratings based approach depending upon the numerical scores achieved by different workshops?

Both these parameters significantly impact the final outcome of the successful workshop proposals and the resultant agenda.

Personally my suggestion would be to do this as MAG collectively, without breakout groups, though we compromise on ease and working convenience but we minimize ambiguity and subjectivity. This is further reinforced by the fact that remote participation if often difficult and unavailable in breakout rooms. This would also allow the entire community present in the room to participate, without having to choose which thematic group they can attend, since it is a parallel process. There was some concern about these issues in the larger community last year. 

3. The MAG members had also discussed a mentoring role for the workshop proposals? Is that limited to the 5-10 proposals, falling just below the threshold level and their review/retrieval? or do we see ourselves taking responsibility for some proposals each  and following through with consistent engagement which translates into capacity building and improvements in the overall workshop quality? Also this can't be a forced engagement and should be on a voluntary basis for both parties involved.

4. What is our stand on mergers? 

 How will these new guidelines be operationalised?

Since these were excellent steps forward which came out in the form of revised guidelines after the MAG meeting in Feb, we shouldn't loose them in translation.

Looking forward to hearing more  on the above.

warmest

Subi

----

Subi Chaturvedi
Member MAG, UN-IGF (Media & Civil Society) 

Member MAG, India-IGF (Civil Society)

Convenor WG-India IGF 

Co-Chair, Netmundial, Brazil (Civil Society)

Assistant Prof. Journalism & Comm,

Lady Shri Ram College for Women (LSR), 

Delhi University, India

Twitter:@subichaturvedi

Founder & Hon. Managing Trustee,

Media For Change

Founder, Chief Mentor & Editor

The Saltlist

www.thesaltlist.org

Independent Documentary Filmmaker, Photographer,

Curator, Media Critic & Scholar

PhD. Scholar,

Indian Institute of Technology (IIT-D), New Delhi

On 2 May 2014 14:37, <karklinsj at gmail.com> wrote:

Dear Susan,

Thank you very much for your suggestions.

It is unfortunate that you will not be able to join us in Paris. You presence would be extremely useful. Thank you for the commitment to contribute to the debates remotely.

As far as I understand we will have streaming from room IX (plenary). It may not be available from the break-out rooms. Chengetai will check with UNESCO hosts.

Hope we ill discuss our Paris meeting modalities during the next conference call on 7 May

Best regards

JK

Sent from Surface

From: susan at susanchalmers.com
Sent: Thursday, May 1, 2014 1:41 AM
To: igfmaglist at intgovforum.org

Dear Bill, all,

Bill, thank you for raising these questions, and you are not the only MAG member without funding to come to Paris -- I will be joining you as a fellow remote participant.

Below some thoughts on the May meeting procedure:

This is of course up to our Chair, but I'd suggest that the procedure for the May evaluation meeting ("Procedure") should be developed, agreed upon, and understood before the meeting takes place.
In February the MAG adopted a 3-stage process. Stage 1 - Initial Screening* / Stage 2 - Evaluation process to be completed by MAG members by May 2nd** / Stage 3 - Discussion and Finalisation of Programme.
I believe that development of the Procedure should be guided by what the MAG has agreed upon and adopted. (I have attached that document to this email - there was interest expressed in seeing the document during the last call). In relevant part, during Stage 3: 
MAG members look at the results to ensure an overall balance of the themes/topics covered;
MAG members discuss 5-10 proposals just below the threshold of space availability*** to determine if improvements can be made to overcome proposal deficiencies;
Proposers contacted by May 26th and asked to submit revised proposal to address deficiencies by June 16th. If the proposer responds the expectation is they will get a workshop slot;
Final schedule published by June 30th.
Based upon these guidelines I do not see any reason for MAG to break into groups, especially since we decided against assigning subsets of proposals to evaluate, to subgroups within the MAG. In other words, it's expected that all MAG members commit to grading all 200-odd workshop proposals. Also, as a matter of logistics, plenary would be much easier for remote participants.
The score for each proposal is tabulated by the Secretariat. Per what was agreed in February, proposals are ranked in order to scale to space availability, which means that we need from the Secretariat a blank template for the IGF programme, with a predetermined number of slots. The "5-10 proposals just below the threshold of space availability" are then discussed by the MAG in concert. 
Perhaps then it would be most appropriate to take an overall look at the programme and determine subject matter balance.

As part of the team that kicked off the workshop evaluation process, I had hoped to be present in May to observe how we all worked through it together, to take notes on the strengths and weaknesses of the process, so that the MAG could re-examine and improve the process for next year. Unfortunately I won't be able to, so might I ask if all MAG members wouldn't mind jotting down a few notes as they go through the process? I'd be willing to work with some others in digesting these and suggesting improvements following the IGF in September.

I'd really like to hear everyone's thoughts on the above, and would love to work with others on implementation of what we agreed upon. What have I missed? 

I believe that if we work together we can improve by leaps and bounds the process that so many found disappointing and frustrating last year!

Warm regards,

Susan

* Re Initial Screening - Chengetai, were any proposals rejected during the initial screening?

** Re Grading - Has everyone graded their proposals? :-)

*** Re Space Availability - May we have an update on the schedule template?

On Tue, Apr 29, 2014 at 1:37 PM, William Drake <wjdrake at gmail.com> wrote:

Hi 

There was some discussion on the last call about the Paris meeting breaking down into groups to consider proposals according to themes.  I would have some concerns about that procedure if it were to lead to outcomes like last year’s, in which some groups were very strict and didn’t approve workshops that didn’t surpass the numerical cut-off point, while other groups were more permissive and for various reasons approved workshops that hadn’t made the numerical cut-off.  There was also some lack of clarity as to whether we were trying to adjust to achieve some measure of comparability across themes, e.g. since some themes attracted @ 50-60 proposals while others got @ 10, did we need to ‘prune’ more in the former case, etc.  

So I’m wondering what the thinking is regarding break out sessions by themes, e.g. dis/advantages relative to doing everything together in ‘plenary.’  

If this approach is to be followed, it’d also be good to know how remote participation would be accommodated.  I don’t imagine I am the only MAG member without funding to come to Paris….Actually, it might be worth knowing, when making decisions on how to organize the work, how many of us will in fact be remote?

Best,

Bill

On Apr 28, 2014, at 2:33 PM, Chengetai Masango <CMASANGO at unog.ch> wrote:

Dear All, 

Online registration for the May Open consultations and MAG meeting is now open until the 15 of May.

http://www.intgovforum.org/cms/registrations-for-open-consultations-and-mag-meeting-paris

I will send out the draft agenda soon.

Best regards,

Chengetai 

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

***********************************************
William J. Drake
International Fellow & Lecturer
  Media Change & Innovation Division, IPMZ
  University of Zurich, Switzerland
Chair, Noncommercial Users Constituency, 
  ICANN, www.ncuc.org
william.drake at uzh.ch (direct), wjdrake at gmail.com (lists),
  www.williamdrake.org
*********************************************** 

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

-- 

Susan Chalmers 
Consultant, Internet Policy

+1 269 324 4101

www.susanchalmers.com

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

<Mail Attachment.txt>_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

***********************************************
William J. Drake
International Fellow & Lecturer
  Media Change & Innovation Division, IPMZ
  University of Zurich, Switzerland
Chair, Noncommercial Users Constituency, 
  ICANN, www.ncuc.org
william.drake at uzh.ch (direct), wjdrake at gmail.com (lists),
  www.williamdrake.org
*********************************************** 

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

-- 

Este mensaje no contiene virus ni malware porque la protección de avast! Antivirus está activa. 

_______________________________________________
Igfmaglist mailing list
Igfmaglist at intgovforum.org
http://mail.intgovforum.org/mailman/listinfo/igfmaglist_intgovforum.org

-- 

Fatima Cambronero
Abogada-Argentina

Phone: +54 9351 5282 668
Twitter: @facambronero
Skype: fatima.cambronero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20140505/80aa387f/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: firma Caro (2).bmp
Type: image/bmp
Size: 92454 bytes
Desc: firma Caro (2).bmp
URL: <http://intgovforum.org/pipermail/igfmaglist_intgovforum.org/attachments/20140505/80aa387f/attachment.bmp>