Experiences on the subjective evaluation of ALADIN/LACE model 00 UTC

Márta Sallai Buránszki and Szilvia Jenki (HMS)

Historical overview:

• The idea to perform subjective evaluation of the ALADIN model was originally coming from Ales Poredos (HMIS). It was recommended to start the operational evaluation in February, 1997. At the ALADIN workshop in Budapest (16-18 June, 1997) the first experiences were presented and rediscussed. The first evaluation period was 10 February 1997 - 10 May 1997. (Sallai, Wantuch: First experiences with the subjective evaluation system of ALADIN at HMS, Proceedings of the third RC LACE/ALADIN workshop Budapest).
• After the workshop the evaluation was continued without changing the method.The results of the evaluation for the period 1 June 1997 - 31 December 1997 was presented in the ALADIN Newsletter (number 10).
• 1 February, 1998 : modifications regarding the type of the weather situations and ranking of the different parameters (+/- sign).
• The first results of the new scheme was presented at the 5th ALADIN workshop in Prague (June, 1999).

Motivation:

• Subjective evaluation should aim towards better understanding of operational performance of ALADIN model.
• The goal of the subjective evaluation: to give some additional information about the performance of the models on top of the already existing objective verification scores. The results should serve as input for forecasters as well as for the model developers.

The application of the method in Hungary:

• The evaluated forecast ranges: 06-18 UTC and 18-30 UTC
• Regions : Western and Eastern Hungary (only the plane regions are studied)
• Weather types :
  - C-cyclone
  - A-anticyclone
  - Ce-Eastern part of cyclone
  - Cw-Western part of cyclone
  - Ae-Eastern part of anticyclone
  - Aw-Western part of anticyclone
  - Wp-weak pressure gradient
  - Z-zonal flow
• Extreme events:
  - N-nothing special
  - W-strong wind episode
  - HP-heavy precipitation
  - CU-strong convection event
  - CP-cold air pad
• The evaluated parameters:
  - 10 m wind
  - cloudiness
  - maximum temperature
  - minimum temperature
  - precipitation distribution
  - precipitation quantity

Guidance for the evaluation:

(As it was proposed after the round table discussion of the ALADIN workshop in Budapest)

• Total cloud cover:
  - distribution:
  3 = cores of observed and forecasted max. cloud zone <50 km apart
  2= 50 km < cores < 100 km
  1= cores of observed and forecasted max. cloud zone >100km apart
  - quantity :

  obs. \ for.	0	1-2	3-6	7-8
  0	3	+2	+1	+1
  1-2	-2	3	+2	+1
  3-6	-1	-2	3	+2
  7-8	-1	-1	-2	3

  
  
• Wind:
  3= direction +/- 30 degrees, speed +/- 2 m/s
  2= direction +/- 30-50 degrees, speed +/- 3 m/s
  1= worse
  (if for. > obs. then sign of mark should be +, otherwise -)
• Temperature:
  3= the difference +/- 2 degrees
  2= the difference +/- 2-4 degrees
  1= worse
  (if for. > obs. then sign of mark should be +, otherwise -)
• Precipitation:
  - distribution:
  3= cores of observed and forecasted precip. zone <50 km apart
  2= 50 km < cores < 100 km
  1= cores of observed and forecasted precip. zone>100 km apart
  - quantity:

  obs. \ for.	0	<2 mm	2-10 mm	>10 mm
  0	3	+2	+1	+1
  <2 mm	-2	3	+2	+1
  2-10 mm	-1	-2	3	+2
  >10 mm	-1	-1	-2	3

The results of the evaluation

• The evaluation period: 1 February 1998 - 31 December 1998
• The number of the correctly evaluated cases: 1271
• In the evaluated period the type Ce (Eastern part of cyclone) occured in the most cases (279 occasions), and the second most common situation was the anticyclonic (260 cases). The rarest case was the zonal flow.
• Extreme situations occured in 32 % of examined cases. Among these cases the strong wind episodes and the strong convection events were the most frequent.

  Figure 1.   The frequency of the occurrence of the different synoptic and extreme situations

  
  

•  The results without separating the weather types and the extreme situations

  The precipitation distribution is the best ranked parameter in this evaluation period. The forecast of the temperature, chiefly the maximum temperature is the less successful.

  Parameters	wind	cloud	max. temp.	min. temp.	precip. distr.	Precip. Quan.
  Evaluation	2.67	2.66	2.50	2.55	2.72	2.60

  Table 1.   The rate of successful forecast of the different parameters

  
  
•  When the weather situations and extreme situations are separately studied the results are as follows:

  Wind : The ranks are the best in the case of anticyclonic and weak pressure gradient weather situations, or when it was cold air pad in the Carpathian Basin. The prediction is the worst in strong wind periods, it is significantly underestimated, like in the last year. The rate of the overestimations is far bigger than the rate of the underestimations.

  The rate of the successful forecasts

  The rate of the overestimated/underestimated cases (%): over 67 % / under 33% Figure 2.   Subjective verification of wind forecasts in the case of the different synoptic and extreme situations

•   Cloudiness

  When the cloud coverage is the predicted parameter, the rate of successful forecasts is the best in cyclonic and in zonal flow situations. Studying the extreme situations it can be seen, that the rank is the best in strong wind episode, and chiefly in heavy precipitation situation. It is the worst significantly in cold air pad weather situation. The model overestimates the cloudiness in many cases.

  The rate of the successful forecasts The rate of the overestimated/underestimated cases (%) Figure 3.   Subjective verification of cloudiness forecasts in the case of the different synoptic and extreme situations

  
  
•  Maximum temperature

  The forecast of the maximum temperature is the worst in zonal flow situation (when there is a rapid temperature change at high levels). The forecast is not always successful in the case of anticyclone or Western part of anticyclone. In autumn and in winter in anticyclonic weather situation cold air pad very often occurs, it may cause the wrong prediction. Last year the ALADIN model predicted much more low degrees in the most cases. In this evaluation period, in 1998 the difference is not so big between the rate of the overestimation and underestimation.

  The rate of the successful forecasts

  The rate of the overestimated/underestimated cases (%) Figure 4.   Subjective verification of maximum temperature forecast in the case of the different synoptic and extreme situations

•  Minimum temperature

  Studying the minimum temperature forecast we can draw similar conclusion like in the case of maximum temperature. The worst ranks were given in zonal flow situation and when it was cold air pad. In 1998 the rate of underestimation also was bigger than the rate of overestimation, but not so much like in 1997.

  The rate of the successful forecast

  The rate of the overestimated/underestimated cases (%) Figure 5.   Subjective verification of minimum temperature forecast in the case of the different synoptic and extreme situations

•   Precipitation distribution

  Apart from the anticyclonic situation the most successful forecasts are in Ae (Eastern part of anticyclone) and in cyclonic situation. The rate of successful forecasts is the worst in zonal flow or in weak pressure gradient situation. As the strong convection event connects with the weak pressure gradient weather situation in many times, studying the extreme situations it can be seen, that the ranks are the worst in Cu (strong convection event) weather situation.

  The rate of the successful forecasts Figure 6.   Subjective verification of precipitation distribution forecast in the case of the different synoptic and extreme situations

•   Precipitation quantity

  The predicted amounts of precipitation are not good neither in cyclonic, nor in weak pressure gradient situation. When there is strong convection event, the forecast of the model is not successful, at all. The rate of the overestimation/underestimation shows, that the precipitation forecast is overestimated in many cases.

  The rate of the successful forecasts

  The rate of the overestimated/underestimated cases (%) Figure 7.   Subjective verification of precipitation quantity forecasts in the case of the different synoptic and extreme situations

  Studying the regions separately it can be seen, that the forecast of almost every element is better for the Western part of Hungary, but the difference is not big.

  region	wind	Cloud	max.temp	Min. temp	precip. dist	precip. q.
  East	2.71	2.67	2.50	2.53	2.71	2.59
  West	2.62	2.67	2.51	2.56	2.73	2.61

  Table 2.   The results studying separately the different regions and forecast ranges

Summary

• The prediction of the temperature, mostly of the maximum temperature is the weak point of the ALADIN model. The model often underestimates, but in cold air pad situation usually overestimates the values.
• The precipitation distribution is the best ranked parameter in this evaluated period, too.
• The precipitation quantity is regularly overestimated. Due to the convective processes chiefly in summer period the forecast of the amounts of precipitation is not successful, especially in the case of weak pressure gradient situation.
• The ALADIN overestimates the cloudiness in the most cases, like is the last year.
• The wind prediction is the worst in strong wind periods. In this case the wind is significantly underestimated.

Home