COST

International Early Stage Researcher Training School on

Applying Expert Judgement Methodologies to Real Problems

Madrid, Sunday 12^th April to Wednesday 15^th April, 2015

Organisers:

Simon French, Dpt. Statistics, University of Warwick, UK

David Rios Insua, Mathematical Sciences Institute, Nat. Res. Council, Spain

Roger Cooke, Resources for the Future and University of Strathclyde

Jesus Rios, IBM Research, Yorktown Heights

Eva Chen, University of Pennsylvannia

 

School website

http://www.icmat.es/congresos/2015/AEJMRP/

Introduction

This Early Stage Researcher Training[1] School will provide an introduction to the theory and practice of applying expert judgement methodologies in decision and risk analyses.  The School will last an evening and two and a half full days during which the attendees will be introduced to the background theory, including relevant behavioural studies.  Much of the training will focus around practical exercises.  In the first of these the attendees will act as experts themselves experiencing the training that they will need to provide in real studies to enable experts to articulate their judgements probabilistically.  In the second study, they will work as analysts in a simulated problem to elicit, analyse and combine expert judgements into a form which could be taken forward to a full risk or decision analysis.  The overall aim of the Training School is to introduce participants to the range of skills that practical application of expert judgement requires. However, some overview of current research fronts will be provided so that doctoral students may set their personal research against a wider backdrop of expert judgement research.

Content of the Doctoral School

The Doctoral School would offer the following lectures and activities:

• Introductory session setting structured quantitative expert judgement within the context of quantitative risk assessment and decision making.
• Lectures including demonstrations and break-out sessions on:

1.         The Expert Judgement Problem: its history and context, including scientific and political background
2.         Cooke’s Classical Model
3.         Bayesian Approaches
4.         The Behavioural Background: System 1 and System 2 Thinking, Heuristics and Biases
5.         The process of expert judgement studies
6.         Using software to conduct and interpret expert judgement studies
7.         Incorporation of expert judgement data into risk and decision analyses
8.         The political processes of drawing expert judgement studies into societal decision making
9.         Design, reporting and peer review of expert judgement studies
10.         Research fronts in expert judgement theory.

Note that there is no intention that these topics would be treated to equal depth.  The major focus would be on the process and techniques of conducting expert judgement studies.

• Case studies of applications of expert judgement in domains such as:

1.         Volcanic risk assessment
2.         Nuclear Safety
3.         Food safety
4.         Aviation safety
5.         Assessment of Environmental Impacts
6.         Energy planning.

• Exercises using Expert Judgement software, specifically Excalibur.
• An Experiential Exercise in which each participant gives his or her judgements on a range of unknowns that will become resolved through the period of the School.  During this exercise each participant will experience firstly how it feels to give judgements of uncertainty unaided, then training in the process of giving such judgements, a structured elicitation and finally calibration of their judgements against the resolutions of the uncertainty.
• An extensive group exercise based around a hypothetical but realistically complex scenario in which the students would experience the process of developing an expert judgement protocol, identifying calibration questions, elicitation of expert judgements, producing probabilities capable of being fed into a risk or decision analysis.
• A feature of the School is that each day, coffee will be taken in small discussion groups with assigned mentors.  The aims of these sessions are to reflect on the course and its application to their personal research topics and applications and the ways that the understandings and skills being developed in the School may be introduced in these.  It is expected that these discussions will be carried on informally at other times during the School.

N.B. Prior to the course, the students will be expected to have completed an introductory session on the Action’s Moodle training site.  This will be roughly equivalent to a further 3 hours of training and will be supported through a forum on the site with interaction from one or more of the tutors on the course. 

Support

During their time in Madrid, they will be supported with other training material on the same site. The students will be provided with electronic copies of notes, slides, case studies, and other material.  Copies will also be available on the Moodle site. Note that students will be expected to bring their own laptops, capable of running Windows software, either directly or through emulation. The discussion forum on the Action’s Moodle site will continue to run in the weeks after it to continue discussion and share experiences in applying the ideas back in their research institutes and universities. 

All content of the course will be archived in the Moodle site.

Length of Course

We foresee the course participants meeting on the Sunday evening, to introduce themselves at an informal dinner in central Madrid. The remaining two and a half full days will be spent at ICMAT-CSIC, a short bus or train journey from central Madrid.  The course will finish immediately before the Action’s conference and workshop planned from mid-day on the Wednesday until early afternoon on Friday. Since we envisage the School running immediately before an IS1304 Workshop/Conference, we would encourage/expect the students to stay on for that activity.

Related Files

• ESR Schedule - 09.04.15 (24.0 KB)