High resolution dropsonde data
|
High resolution dropsonde data |
Data processing
---------------
1 Format conversion procedures
1.1 LEAR36, NOAA-GIV and NOAA-P3 datasets conversion to JOSS CLASS format
---------------------------------------------------------------------
1.1.1 These soundings include the correction applied by NCAR/SSSF to
correct for the incorrect usage of the internal radiosonde
temperature rather than the ambient temperature to correct the
reported relative humidity values. For further information on
this correction please contact Hal Cole (cole@ucar.edu).
1.1.2 These files arrived at JOSS in ascii format and were converted to
a high resolution JOSS CLASS format (section 5.0) using JOSS
developed software. JOSS implemented Quality Control procedures
(described below) in an attempt to remove any noise contained in
the raw 0.5 s ascii files. These procedures are in addition to
typical JOSS Quality Control procedures detailed in section 6.0.
The high resolution class files are of varying resolution but
consist of the raw 0.5 s data minus the 'BAD' data points. These
files then were linearly interpolated to 2 s resolution JOSS CLASS
files. The pressures and times in the original ascii files were
increasing. JOSS reversed the pressures and times in the high
resolution CLASS files and in the final 2 s resolution interpolated
CLASS files.
1.1.3 For files containing a launch time, JOSS processing entailed using
the data point whose time was the closest to the launch time plus
20 seconds in an attempt to prevent recording data while the
dropsonde was still in the launch bay and/or during the
dropsonde's environmental acclimation. In addition, JOSS
processing required that the launch pressure be less than 600 mb
and the launch dry bulb temperature be less than 30 deg C.
Furthermore, JOSS processing checked the initial data points near
the launch point to ensure that only the most suitable data points
were used. Picking a bad launch point would cause the unnecessary
removal of many good points due to the rate of change checks
(section 4.4). Only data points whose times occurred after the
launch time were written to the high resolution class file. Files
not containing launch times or containing many bad data points
needed special processing in order to 'force' the JOSS processing
software to chose quality launch points.
1.1.4 The raw 0.5 s vertical resolution time, temperature, relative
humidity, altitude, pressure, dew point, latitude, longitude, wind
direction and wind speed were kept without change. However, JOSS
implemented rate of change quality control checks which determined
whether a data point was written to the high resolution class file.
These quality control checks included checks for excessive lapse
rates, pressure changes, and ascension rates between successive points
1.1.5 The lapse rate was calculated for each data point based on
the altitude and dry bulb temperature at two levels, the current
0.5 s level and the previous 0.5 s level. If the absolute value of
the calculated lapse rate exceeded 75 deg C/km, then the data point
was not written to the high resolution class file. If the current
data point contained a missing altitude, but not a missing
dry bulb temperature, a -10 m/s ascension rate was assumed and
based on this assumption, a lapse rate was then calculated. If
the absolute value of this assumed lapse rate exceeded 75 deg C/km
then the current data point was not written to the high resolution
class file.
1.1.6 If a data point's pressure and altitude were missing, that point was
not kept. This explains why the number of soundings in the 2 s data
set is lower than in the 0.5 sec data set (NOAA-GIV and NOAA-P3).
Some soundings had no pressure or altitude information within them.
Furthermore, if the absolute value of the pressure change between
the current time and the previous time was greater than or equal to
5 mb/sec, than that data point was not written to the high resolution
class file.
1.1.7 The ascension rate was calculated for each data point (excluding
the surface, where it was given a missing value) based on the
altitude and time at two levels, the current 0.5 s level and the
previous 0.5 s level. If the data point for which the ascension
rate was being calculated had missing time or altitude, the
ascension rate was not calculated at that level and was flagged as
missing. If the data point prior to that for which the ascension
rate was being calculated had a missing time or altitude, then the
data point two data points prior to the present one was used, and so
on until one with non-missing time and altitude was found. If there
are no non-missing previous data points the ascension rate is set
to missing. Raw 0.5 s data was not written to the high resolution
class file if the ascension rate was less than or equal to -40 m/s
or greater than or equal to 10 m/s.
1.1.8 The U and V wind components were calculated based on the wind speed
and direction at the 0.5 s level.
1.1.9 Due to CLASS format constraints, Dew Points less than -99.9 Deg C
were given a value of -99.9 Deg C and were flagged as being
questionable.
1.1.10 A 2 s vertical resolution file was created using existing 2 second
interval data points from the high resolution CLASS file beginning
with 0 s. Pressure, dry bulb temperature, relative humidity, U
and V wind components, latitude and longitude, were linearly
interpolated if no original 2 second interval data point existed
within the high resolution class file. Dew point temperature, total
wind speed and wind direction, ascension rate and altitude were
calculated. The altitude was calculated by assuming that the final
data point within the sounding was the surface and since the
surface was the ocean surface, the altitude at this point was
assumed to be 0 m, as long as the pressure at this point was
>= 960 mb. If the pressure was < 960 mb the altitudes were not
recalculated. If the altitudes were not recalculated the altitudes
within the files are those reported by the dropsonde. For those
that were recalculated from the bottom-up using a version of the
hypsometric equation (Holton 1979, Hess 1959, Herzegh 1988). The
ascension rate recalculation followed the procedure detailed in
section 4.7 minus the rate of change condition and for 2 s vertical
resolution data points. The altitude recalculation utilized the
hypsometric equation and used the virtual temperature if the
relative humidity was not missing. Otherwise the dry bulb
temperature was employed.
To determine whether a file had recalculated altitudes, just see
if the first data point within the file (the "surface") has a
zero altitude, if so, the altitudes WERE recalculated. If not,
the altitudes WERE NOT recalculated.
Missing 2 s variables were interpolated using the closest high
resolution points surrounding the variable to be interpolated.
The distance between these points determined the Quality Control
flags written in the CLASS file's QC fields detailed below in
section 5.2. If the data points used in the interpolation were
less than 20 seconds apart, the interpolated datum was flagged
"UNCHECKED", between 20 s and 40 s, the datum was flagged
"QUESTIONABLE"; for points greater than 40 s apart , the
data were not interpolated and missing values were inserted into
the file (NOAA-GIV and LEAR36), or interpolated and the interpolated
point was flagged "BAD" (NOAA-P3). Thus it is important to examine
the flags within this dataset as data may have been interpolated
over large intervals.
1.2 UK-C130 dataset conversion to JOSS CLASS format
---------------------------------------------------
1.2.1 These files arrived in FRONTS87 format and were converted to
a high resolution JOSS CLASS format. JOSS "reversed" the dropsonde
data so that the pressure decreases from the beginning of the file
to the end as in upsondes.
1.2.2 The raw 1.5 s vertical resolution temperature, relative humidity,
pressure, dew point, latitude, longitude, wind direction and wind
speed were kept without change.
1.2.3 The UKMO set the altitude of last observed data point to zero
regardless of whether it reached the surface or not. One of the
flags within the FRONTS87 format was used by UKMO to signify
whether the dropsonde reached the surface. JOSS used this flag so
that when the dropsonde was considered to have reached the surface,
we kept the altitudes as is. However, if the dropsonde was not
considered to have reached the surface we set all altitudes within
that sounding to missing. The altitudes were set to missing in 106
of the soundings.
1.2.4 The ascension rate was calculated for each data point (excluding
the surface, where it was given a missing value) based on the
altitude and time at two levels, the current level and the previous
level. If the data point for which the ascension rate was being
calculated had missing time or altitude, the ascension rate was not
calculated at that level and was flagged as missing. If the data
point prior to that for which the ascension rate was being
calculated had a missing time or altitude, then the data point two
data points prior to the present one was used, and so on until one
with non-missing time and altitude was found. If there are no
non-missing previous data points the ascension rate is set to
missing.
1.2.5 The U and V wind components were calculated based on the wind speed
and direction at the level.
1.2.6 Due to CLASS format constraints, Dew Points less than -99.9 Deg C
were given a value of -99.9 Deg C and were flagged as being
questionable.
1.3 US-C1307 and US-C1308 datasets conversion to JOSS CLASS format
--------------------------------------------------------------
1.3.1 The 10 sec resolution pressure, temperature, relative humidity, and
wind speed were kept as is.
1.3.2 Wind speeds were converted from knots to m/s.
1.3.3 The U and V wind components were calculated based on the wind speed
and direction at the 10 s level.
1.3.4 Dew points were calculated via the equations from Bolton (1980).
Due to CLASS format constraints, Dew Points less than -99.9 Deg C
were given a value of -99.9 Deg C and were flagged as being
questionable.
1.3.5 At this point, the automated and visual quality control processes
were conducted (see section 4.2).
1.3.6 Altitudes were calculated by assuming that the final point reached
by the radiosonde was the surface, and since the surface was the
ocean surface, the altitude at this point was assumed to be 0, as
long as the pressure at this point was >= 960 mb. If the pressure
was < 960 mb the altitudes were not recalculated. Also, the
altitudes were not recalculated for data points that had any of the
pressure, temperature, or relative humidity values flagged as bad
by either of the quality control processes. Such bad data points
were not used at all in any altitude calculations. The calculation
was done from the surface 0 point up using a version of the
hypsometric equation (Holton 1979, Hess 1959, Herzegh 1988).
1.3.7 The ascension rate was calculated for each data point (excluding
the surface, where it was given a missing value) based on the
altitude and time at two levels, the current 10 s level and the
previous 10 s level. If the data point for which the ascension rate
was being calculated had missing time or altitude, the ascension
rate was not calculated at that level and was flagged as missing.
If the data point prior to that for which the ascension rate was
being calculated had a missing time or altitude, then the data point
two data points prior to the present one was used, and so on until
one with non-missing time and altitude was found. If there are no
non-missing previous data points the ascension rate is set to missing.
2 Quality Control Procedures
2.1 LEAR36, NOAA-GIV, NOAA-P3, USC1307 and US-C1308, UK-C130 datasets:
JOSS Quality Control Procedures
----------------------------------------------------------------------
These datasets underwent the JOSS QC process which consisted of
internal consistency checks and visual quality control. The
internal consistency checks included gross limit checks on all
parameters and vertical consistency checks on temperature,
pressure, and ascension rate. These checks are different from the
checks described in section 4.1. in that the 4.1 checks were point to
point checks between adjacent times and were only vertical consistency
checks while the Automatic Quality Control software employed six
second averaging in its checking procedure and included gross limit
checks. In addition, the Auto QC used a different set of parameters
during its vertical consistency checks. More importantly, the Auto QC
DID NOT REMOVE data points. It only changed QC flags. JOSS then
visually examined each sounding.
1 Automated Quality Control Procedures
1.1 Gross Limit Checks
These checks were conducted on each sounding and data were
automatically flagged as appropriate. Only the data point
under examination was flagged. JOSS conducted the
following gross limit checks on the NOAA P-3 FASTEX sounding
dataset, the Gulfstream FASTEX sounding dataset, the Lear
FASTEX sounding dataset and the USAF C-130 FASTEX sounding
dataset. In the table P = pressure,
T = temperature, RH = relative humidity, U = U wind
component, V = V wind component, B = bad, and
Q = questionable.
----------------------------------------------------------------
Parameter(s) Flag
Parameter Gross Limit Check Flagged Applied
----------------------------------------------------------------
Pressure < 0 mb or > 1050 mb P B
Altitude < 0 m or > 40000 m P, T, RH Q
Temperature < -80C or > 30C T Q
Dew Point < -99.9C or > 25C RH Q
> Temperature T, RH Q
Relative Humidity < 0% or > 100% RH B
Wind Speed < 0 m/s or > 100 m/s U, V Q
> 150 m/s U, V B
U Wind Component < 0 m/s or > 100 m/s U Q
> 150 m/s U B
V Wind Component < 0 m/s or > 100 m/s V Q
> 150 m/s V B
Wind Direction < 0 deg or > 360 deg U, V B
Ascent Rate < -30 m/s or > 10 m/s P, T, RH Q
----------------------------------------------------------------
1.2 Vertical Consistency Checks
These checks were conducted on each sounding and data were
automatically flagged as appropriate. The Auto QC software
employed six-second averaging for these checks. These checks
were started at the lowest level of the sounding and
compared the end points of a six second interval. In the case
of checks ensuring that the values increased/decreased
as expected, only the data points under examination were
flagged. However, for the other checks, all four data
points used in the examination were flagged.
All items within the table are as previously defined.
----------------------------------------------------------------
Vertical Consistency Parameter(s) Flag
Parameter Check Flagged Applied
----------------------------------------------------------------
Time increasing/equal None None
Altitude increasing/equal P, T, RH Q
Pressure decreasing/equal P, T, RH Q
> 3 mb/s or < -3 mb/s P, T, RH Q
> 4 mb/s or < -4 mb/s P, T, RH B
Temperature < -15 C/km P, T, RH Q
< -30 C/km P, T, RH B
from surface to 800 mb:
> 25 C/km (not applied
at p < 275mb) P, T, RH Q
> 40 C/km (not applied
at p < 275mb) P, T, RH B
for pressures < 800 mb:
> 5 C/km (not applied
at p < 275mb) P, T, RH Q
> 30 C/km (not applied
at p < 275mb) P, T, RH B
Ascent Rate change of > 5 m/s
or < -5 m/s P Q
change of > 9 m/s
or < -9 m/s P B
----------------------------------------------------------------
2 Visual Quality Control Procedures
Each sounding was then visually examined for problems that
were not able to be captured via the automated checks
described in item .1 above. These problems typically
included oddities in the dew point and wind profiles. These
two parameters can be highly variable, and hence, the
automated checking is more difficult. The visual checking
procedure has two main purposes: First, as a check on the
results provided by the automatic checks, and second, as a
more stringent check on the more variable parameters.
2.2 UK-C130 dataset: FASTEX Archive Quality Control Procedures
--------------------------------------------------------------
This dataset was validated by the University of Reading before its
sending to the FASTEX Central archive.
UCAR/JOSS applied its quality control procedures and the quality flags
are updated.
3 Additional quality control processing information or dataset remarks:
Aircraft
--------
LEAR36 Dataset Remarks (from OFPS checkings - May 1998)
-----------------------------------------------------
.1 Relative Humidity Issues
UCAR/JOSS has conducted intercomparisons of the relative humidity
measurements of the various dropsonde systems and the FASTEX
ships. For information on these intercomparisons please contact
Jim Moore (moore@ucar.edu).
Some other problems with specific dropsondes include, relative
humidities having oscillations between very moist and very dry
over entire soundings, single data points that are either VERY
DRY or VERY MOIST relative to data points directly above and
below, as well as sometimes erratic values.
.2 Wind Problems
There are several files with significant problems with the winds.
This includes problems such as drastic speed and directional
changes, wind speeds that are too high, and winds that are erratic
in their speed and/or direction.
About 10 of the drops had little or no wind data.
.3 Launch Location Problems
Several soundings did not have lat/lon information available at
the time of release of the sonde. For the header items:
Launch Location (lon,lat,alt):
we used the first lat/lon position available from the data. The
first lat/lon position for these soundings was typically ~30 s
after release. The soundings affected had the following nominal
launch times:
13 January 1997
14:31:11.0 (position from release)
14 February 1997
11:26:41.0 (position from 26 s after release)
11:51:14.0 (position from 6 s after release)
12:14:27.0 (position from 8 s after release)
12:37:39.0 (position from 16 s after release)
13:02:38.0 (position from 48 s after release)
13:39:38.0 (position from 10 s after release)
14:10:06.0 (position from 20 s after release)
14:18:25.0 (position from 16 s after release)
14:23:10.0 (position from release)
14:44:29.0 (position from 20 s after release)
14:58:36.0 (position from 12 s after release)
15:32:13.0 (position from 8 s after release)
16 February 1997
14:34:47.0 (position from 18 s after release)
17 February 1997
23:16:22.0 (position from 10 s after release)
23:38:06.0 (position interpolated from previous and later release)
18 February 1997
00:15:09.0 (position from 6 s after release)
00:31:01.0 (position from 16 s after release)
.4 Dropsondes Not Reaching the Surface
As noted above in section 4.10 when soundings did not appear to
reach the surface, JOSS did not recalculate altitudes. This
occurred in about 6 of the dropsondes for the Lear. Most of
those that did not reach the surface were from the 14-17 February
1997 flights. The others were spread evenly
over the other flights.
NOAA-GIV Dataset Remarks (from OFPS checkings - May 1998)
-----------------------------------------------------
.1 Relative Humidity Issues
UCAR/JOSS has conducted intercomparisons of the relative humidity
measurements of the various dropsonde systems and the FASTEX
ships. For information on these intercomparisons please contact
Jim Moore (moore@ucar.edu).
Some other problems with specific dropsondes include, relative
humidities having oscillations between very moist and very dry
over entire soundings, single data points that are either VERY
DRY or VERY MOIST relative to data points directly above and
below, as well as sometimes erratic values.
.2 Wind Problems
There are several files with significant problems with the winds.
This includes problems such as drastic speed and directional
changes, wind speeds that are too high, and winds that are erratic
in their speed and/or direction.
About 35 of the drops had little or no wind data.
.3 Launch Location Problems
Several soundings did not have lat/lon information available at
the time of release of the sonde. For the header items:
Launch Location (lon,lat,alt):
we used the first lat/lon position available from the data. The
first lat/lon position for these soundings was typically ~30 s
after release. The soundings affected had the following nominal
launch times:
22 January 1997
11:28:55.0 (position from 94 sec after release)
11:34:26.0 (position and altitude from 10 sec after release)
11:38:49.0 (position and altitude from 28 sec after release)
27 February 1997
11:50:48.0 (position from time of release)
12:09:41.0 (position from time of release)
12:28:16.0 (position from 26 sec after release)
12:52:28.0 (position from 10 sec after release)
.4 Dropsondes Not Reaching the Surface
As noted above in section 4.10 when soundings did not appear to
reach the surface, JOSS did not recalculate altitudes. This
occurred in about 43 of the dropsondes for the G-IV. Most of
those that did not reach the surface were from the 29 January
and 1 February 1997 flights. The others were spread evenly
over the other flights.
NOAA-P3 Dataset Remarks (from OFPS checkings - May 1998)
-----------------------------------------------------
.1 Relative Humidity Issues
UCAR/JOSS has conducted intercomparisons of the relative humidity
measurements of the various dropsonde systems and the FASTEX
ships. For information on these intercomparisons please contact
Jim Moore (moore@ucar.edu).
.2 Wind Problems
There are some files with significant problems with the winds.
This includes problems such as drastic speed and directional
changes, wind speeds that are too high, and winds that are erratic
in their speed and/or direction.
UK-C130 Dataset Remarks (from OFPS checkings - May 1998)
-----------------------------------------------------
.1 Missing Winds
103 of the dropsondes had no wind information.
.2 Altitudes
As mentioned above, when the dropsonde did not reach the surface the
UKMO still set the final data point altitude to 0.0 m and recalculated
the altitudes in the remainder of the sounding. For soundings that
did not reach the surface (as per the flag set by UKMO) JOSS set all
altitudes to missing as the altitudes prior to the recalculation by
UKMO were unrecoverable.
.3 The 12 January flight
There are no winds available for the dropsondes from this flight.
Many of the dropsondes from this flight had reported relative
humidities > 100.0%.
.4 Relative humidities in general - UCAR/JOSS has conducted
intercomparisons of the relative humidity measurements of the
various dropsonde systems and the FASTEX ships. For information
on these intercomparisons please contact Jim Moore (moore@ucar.edu).
US-C1307 Dataset Remarks (from OFPS checkings - November 1997)
US-C1308 -----------------------------------------------------
The dew point/relative humidity and temperature values
just after release were prior to the radiosonde
acclimating to the environment. Many dropsondes were too
dry and warm in the first 30 s after release.
About 20 of the soundings had significant problems with
the winds.
Top of page
Updated: 2 February 1999