COST

In order to encourage the practical use of different Expert Judgement methods, we want to offer a selection of various software solutions, which is to be extended continuously.

In the table below is listed tools to support the expert elicitation process. The list is not comprehensive but includes software which seems to be the best known tools for expert elicitation. The focus in most of the listed tools is on parameter value estimation.  All tools are freely available from the web except Elicitator which is still in the development phase. In addition to the tools mentioned in the table there are several different kinds of tools to be utilised in different phases of an expert elicitation process. A quite comprehensive list of these tools can be found in the report by Jeroen Devilee and Anne Knol from the National Institute for Public Health and the Environment of the Netherlands. The report can be found here.

EXCALIBUR (EXpert CALIBration): Excalibur or http://www.lighttwist.net/wp/excalibur

A software package for structured expert judgement elicitation using Cooke’s Classical Model.

ElicitN: http://www.downloadcollection.com/elicitn.htm

ElicitN aids elicitation of expert knowledge of complex systems of count data, focusing on a case study of elicitation of species richness estimates for coral reefs. The software uses newly developed elicitation procedures to elicit probability distributions of counts in a structured and ordered protocol.

R. Fisher, R. A. O’Leary, S. Low-Choy, K. Mengersen, M. J. Caley (2011). A software tool for elicitation of expert knowledge about species richness or similar counts. Environmental Modelling & Software. Vol 30, pp. 1-14.

SHELF (The SHeffield ELicitation Framework): http://www.tonyohagan.co.uk/shelf/

The SHELF is a package of documents, templates and software to carry out elicitation of probability distributions for uncertain quantities from a group of experts. All materials in the SHELF package are made freely available, but they are covered by copyright.

Elicitator: More information about the status of the software is available by e-mail: Elicitator@qut.edu.au

Elicitator may generally be applied to a wide range of regression problems, such as habitat modelling, environmental niche modelling, species distributing mapping and detectability or other ecological responses to environmental change.

James, A., Low Chou, S. & Mengersen, K.L. (2010). Elicitator : an expert elicitation tool for regression in ecology. Environmental Modelling and Software, 25(1), pp. 129-145.

MATCH Uncertainty Elicitation Tool: http://optics.eee.nottingham.ac.uk/match/uncertainty.php#

The MATCH Tool is based on the SHELF elicitation package. The Tool provides a web-based interface for the SHELF R code, which is more user-friendly, offers a little more flexibility for the elicitation methods, and includes a new feature for conducting elicitation remotely.

David E. Morris, Jeremy E. Oakley, John A. Crowe, A web-based tool for eliciting probability distributions from experts. Environmental Modelling & Software, Volume 52, February 2014, pp. 1-4.

UncertWeb - The Elicitator: http://elicitator.uncertweb.org/

The UncertWeb - Elicitator is a web-based tool that allows a pooled estimate of uncertainty of a particular variable of interest to be obtained from expert beliefs. Variables can be continuous or categorical. Uncertainty of continuous variables is defined by minimum, lower quartile, median, upper quartile and maximum values. Categorical variables are assessed by beans and buckets method where user estimates frequencies of separate categories. Web-based system allows remote use in time appropriate to experts.

G. Heuvelink, L. Bastin, D. Cornford, J. P. Gosling, P. Truong, M. Williams. (2011). Expert elicitation of input and model uncertainties. Deliverable 3.1. THEME FP7-ICT-2009-4, ICT for Environmental Services and Climate Change Adaptation (http://ec.europa.eu/information_society/apps/projects/logos/8/248488/080/deliverables/001_UncertWebD31ARES20111005733.pdf)

Variogram elicitation: http://www.variogramelicitation.org

A web-based tool to facilitate the elicitation of variogram estimates from multiple experts.

P. N. Truong, G. B.M. Heuvelink, J. P.l Gosling. (2013). Web-based tool for expert elicitation of the variogram. Computers & Geosciencies. Vol 51. pp. 390-399.

Unicorn: http://www.lighttwist.net/wp/unicorn-download

The name of the package stands for ”UNcertainty analysis wIth CORrelatioNs” and its main focus is dependence modelling for high dimensional distributions. Random variables can be coupled using dependence trees or vines. An extended formula parser is available for defining the model output, as are a number of post-processing options such as report generation, graphical interpretations, conditioning and sensitivity analysis.