Drifter data processing

The data were first converted from binary to ASCII format (as raw
counts) and then converted to physical units.  Optical data were
calibrated using the calibration factors provided by Satlantic,
Inc.  Missing, saturated, and anomalous data were replaced with
NaNs.   Repeated and out-of-order data were removed.  The data
were then despiked as follows:

Despiking Method

The despiking method used was based on a combination of
statistical and subjective criteria.

The input parameters and typical values used (in brackets) were
as follows:

   * numav : number of data points to average in a running
     average (30)

   * n : least number of standard deviations from the mean that
     is acceptable (2)

   * minstd : minimum standard deviation used (value varies)

   * initav : estimated mean for the good data points in the
     first set of 30 data points (value varies)

The steps taken were as follows:

  1. A running average and standard deviation is calculated in
     groups of 30 (numav) data points.

  2. If the standard deviation is less than a guessed lower
     estimate for the standard deviation (minstd), the guessed
     estimate (minstd) is used instead (this prevents stds of
     zero).

  3. For the first 30 points, if any of the points deviate from a
     guess for the initial mean (initav) by more than n minimum
     standard deviations (n*minstd), they are removed.

  4. After the initial set of 30 points, the despiking method is
     as follows: Each data point is compared with the mean and
     standard deviation of the previous 30 points. If it differs
     from the mean by more than n standard deviations, it is
     removed.

  5. After the initial despiking has been made, a second
     despiking is performed on data where any remaining spikes
     are obvious (such as latitude, longitude, and occasionally
     SST data). This second type of despiking is simply based on
     visual estimation of the maximum deviation that should occur
     between data points.



Optical despiking
Before applying the above method to radiance data, each set of
radiance measurements was first divided by Ed490 to remove
diurnal variations. Poor data points were then determined by
recording the positions of spikes present in these ratios. The
corresponding data points were removed from the radiance data.
Poor data points in the Ed490 data set were assumed to correspond
to the poor data points present in the ratio of Lu412 to Ed490.
Note that this method also removes some good data points, since
some of the data points removed from the radiance data may have
been caused by poor Ed490 values, and vice versa.

Time
The decimal day was calculated from the day of year and datatime.
The datatime is the satellite GMT time minus the data age.

Location and Sea Surface Temperature
Because of storage constraints during the data retrieval from the
optical drifters, several of the optical drifter parameters were
recorded alternately. Hence, note that measurements of location
(latitude and longitude) do not occur simultaneously with
measurements of sea-surface temperature (SST) in the optical
drifter data sets.



[back] back to the data page

Webpage by Jasmine S. Bartlett of Oregon State University
