COST

For more updates and info, visit our blog.

Most elicitation methods are focused on making assessments of the uncertainty of single quantities.  However, risk models usually have more than one uncertain input, and we must consider how to represent uncertainty about multiple quantities jointly. The simplest option is to assume independence, but this is often a faulty assumption, especially for tail events. Nevertheless, exposing faulty assumptions is not always enough to convince decision makers that dependence has to be taken into account (and elicited from experts in absence of data); so one of the questions we need to answer first is "why do we care?". We need to formulate and communicate the answer to this question in such a way that decision makers and problem owners are convinced of the importance of modelling dependence.

Neglecting dependence in beliefs when in fact such dependence is present, is maybe the most serious error. Mathematical operations combining many variables can amplify the effects of dependence, even 'negligible' dependence becomes important when it extends to many variables. Figure 1 serves as an illustration. The first graph shows the sum of 100 standard normal variables when these are independent (red), and (blue) when each pair of variables has a trivial product moment correlation of about 0.05. We would need over 1000 samples to distinguish this correlation from zero^[1].

The picture above speaks for itself. Assuming independence in this case gives us a sense of false certainty.

Dependence modelling in general is an active area of research, and methods for dependence elicitation are still very much under development. A few questions that emerge naturally when one starts thinking about structured dependence elicitation methods are:

1. What should we ask experts about, when we want to represent dependence?
2. How should we formulate the questions for experts in an optimal way?
3. How can we evaluate experts'  performance as dependence assessors?

Other questions and research themes will definitely emerge along the way, but for the moment we concentrate on the 3 above. Click on each of the questions for short descriptions of the related challenges.

[1] The empirical correlation’s limiting distribution is normal with mean ρ and variance (1-ρ²)²/N, where ρ is the bivariate correlation. A sample correlation of 0.01 would be significant against the hypothesis that  ρ=0  if 1.65/N^1/2 < 0.01, or N > 27,225.