You are here: Home » Archive » Phase 1 » Project » Description of work » WP9/JRA3 - ESM Evaluation: developing an Infrastructure

WP9/JRA3 - ESM Evaluation: developing an Infrastructure



The objective of JRA3 is to create an interdisciplinary infrastructure that will facilitate the scientific evaluation of complex ESMs. This infrastructure aims to standardize, harmonize and simplify - where possible - the tools and methodologies used in existing or past model intercomparison projects to evaluate the basic quality of an ESM model. This infrastructure will be a public archive of easy-to-use toolkits. It will include software to prepare required model diagnostics as well as links to pre-assembled most recent state-of-the-art observational datasets. This infrastructure development will add to the IS-ENES portal and evaluation component that can be used as a ‘one-stop-shop’ for modellers who want to evaluate the performance of their ESM.

 The sequence of tasks that will lead to the fulfilment of the JRA3 objective is as follows:

  1. Identify synergies among the different established model intercomparisons in the field (CFMIP, AEROCOM, PMIP, CCMVal, C4MIP). Investigate the common problems involved in documenting model quality and model progress. Evaluate recent efforts made to create compatible model and observational output, and to judge model performance through model intercomparisons. Develop a joint implementation plan to improve the functioning of these intercomparisons. Organize workshops to examine the implementation plan in collaboration with international groups outside the IS-ENES consortium such as PCMDI, NASA, and NCAR.
  2. Provide support structures to ESM modelling teams to implement the appropriate model diagnostics and formatting tools (including CMOR, instrument simulators, variable meaning and unit standards), so that standard evaluation tests can be easily applied. Develop interfaces to bridge the format gaps between model and observational output.
  3. Provide catalogue and access services for the evaluation toolkits together with the corresponding observational datasets. Define standards for documentation of a toolkit. Implement these documentation standards for a set of previously used evaluation procedures in the above mentioned intercomparisons.
  4. Perform a pilot study of the application of the evaluation toolkits to one ESM.


See the full description in the DoW



The WP partners prepared and distributed questionnaires to all the major European ESM groups with the objective to document the model properties that the groups evaluated against observational data, the observational datasets that were used for the evaluation of the different parameters, the methods that were used, and the problems that were encountered. Based on the received questionnaires, the partners began the construction of an Evaluation Portal that aims to facilitate the evaluation process of the ESM groups.

 See also the IS-ENES-WP9 meeting held in June 2011 in Paris



D9.1 Report on evaluation datasets, methods and problems from a survey of European ESM groups
© Copyright ENES Portal 2011



IS-ENES3 has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 824084