2. Principle of use for the base of tests

The purpose of the base is to compare two versions of the code for various configurations and options, in order to detect, for example, developments that can have an unexpected impact in other parts of the code. In general, the base aims at:

• precisely evaluating the differences introduced by a development version, compared to the official version it’s based on.
• consolidating this development version by testing it in a large range of SURFEX configurations.

We ask that each developer runs the base of tests on its own branch, during or at the end of its development time, as he prefers. Before he can propose its development to enter a new official version, he has to provide a text document that:

• sums up the results of the run of the base of tests: list of the test cases that give results different in the two versions.
• explains and justifies the origin of these differences for each concerned test case.

=> an example and framework for such a document is proposed with this article:

Once the branch of the developer is validated through the base of tests, it can be included in the building process of the next official version. Before to begin this building, a discussion between contributers will allow to determine an order of inclusions:

• a first developer enters its modifications, then uses again the base of tests to validate the resulting version.
• a second developer enters its modifications, above these of previous one. He uses the base of tests to compare his version with the version at previous step (this containing only the first contribution).
• and so on.

At the end of the process, the SURFEX team runs the base of tests for a last time, between the final built version and the original official version (on which all developers worked during the building of the new version). The SURFEX team uses the documents containing information about changes in results, provided by developers, to summarize the scientific input of the new version to be published.