Overview
==========

Acoustic Doppler Current Profilers (ADCPs) use the Doppler shift of an
outgoing frequency along a beam to return measured velocity along that
beam.  ADCPs typically have 4 beams, allowing measurement of velocity
in 3 dimensions, in bins of varying distance from the transducer head.
In this application, the ADCP is mounted on a CTD rosette frame and
lowered along with the rest of the package, hence called Lowered ADCP
("LADCP").  The goal is to determine water velocities to the full
depth of the ocean for each deployment.  Ocean velocities are very
small (few cm/s), especially in the deep water. The package is
typically lowered or raised at 1m/s for much of the water column,
creating a challenging data processing situation.

Lowered ADCP data requires ancillary information to improve the
processed data product.  Over generations, the processing has improved
and these ancillary components are systematically collected along with
the LADCP data.  In older datasets, they may be missing entirely.  The
directory structure and naming convention for the submitted LADCP
datasets (WOCE+CLIVAR) attempt to rationalize 20 years of data
acquisition and processing by a collection of scientists who were
inventing the field of LADCP acquisition and processing, so there is
some variety in the organization and naming of the orginal datasets.
Evolution of the instrumentation and processing techniques have
continued.

Each data submission is at the "cruise" level, matching the CCHDO
cruise legs and CTD station+cast nomenclature.

Each data submission contains three directories:
  - docs: cruise metadata, renaming (if applicable)
        notes or reports that could be found
  - raw-level0: original data required to reprocess (all that could be found)
  - processed-level1: original files produced (if ascii)
                       or netCDF conversion (if matlab)


raw-level0 directory: The raw data can be considered to be "level 0",
i.e. original data from an instrument.  The goal of the level-0 data
is that someone would be able to take modern processing techniques and
apply them to older datasets.


processed-level1: The processed data can be considered to be "level
1", i.e. algorithms combining data from ancillary measurements along
with the LADCP data, transformations and editing, and other algorithms
are applied along with some quality control, to create a profile of
ocean velocity.  These data products are the "state of the art at the
time"; they have not been reprocessed with modern methods.  Note that
different processing methods contain different tools to estimate
errors--these are contained in the datasets and are accessible to
users.  Users should be cautioned: LADCP velocities are likely to be
most reliable in regions with strong scattering and strong currents.
In regions with low scattering and weak currents, for example the
center of any ocean gyre and below the main thermocline, LADCP
velocities are likely to be suspect.

level-2-products: There is no level-2 submission at present.  Level-2
data would be robust enough that there are no particluar caveats
about using any of it, and errors would be quantified and documented.



Details:
===========

* directory = "docs"
=====================

- cruise_meta.txt: computer-readable file # PI, WOCE Expocode, cruisename, etc
- *_ladcpmeta.txt: table of instrument settings (where data exist)
- stacast_*rename: notes about file renaming
      "downlooker"      (downward-looking instrument: renaming files)
      "uplooker"        (upward-looking instrument: renaming files)
      "ctd_timeseries"  (rationalization of ctd timeseries names)
      "inverse"         (acquisition did not follow CTD names;
                         processed using 'inverse method')
      "shear"           (acquisition *did* follow CTD names;
                         processed using 'shear' method)
      "gps_timeseries"  (rationalization of gps timeseries names)
      "depth_soundspeed"(rationalization of depth-soundspeed names)
- anything else  : notes, reports, or journals about the cruise


* directory = "raw-level0"
===================================

Over time, processing algorithms have improved and ancillary data
have been added.


instrument                                   what is gained
---------------                          -----------------------
Lowered ADCP (+internal heading)               (shear profile)
+ position (start and end)                     barotropic component
+ CTD time series                              accurate depth and speed of sound
+ shipboard ADCP                               constraint (surface velocities)
+ position timeseries (not just start/end)     inversion processing method
+ accelerometer                                vertical velocity


Differences in data acquisition strategy and naming convention
over the years means that the original LADCP data file might not match
the official CCHDO CTD station and cast.  Files have been renamed to
match CTD station and cast, and records have been kept to connect the
original file name to the submitted file name.

Sometimes an instrument fails and another (sometimes a different
frequency) is used for the duration of the cruise.  In newer cruises
there might be an upward-looking and a downward-looking instrument for
a single cast.  The organization of the raw data reflects this and
tries to unify and describe the data across the 25 years and 53 WOCE
and CLIVAR cruises.


 directories:
================
* directory = docs
 -------------------
 
    cruise_meta.txt  # PI, WOCE Expocode, cruisename, etc
    stacast_*rename  # notes about file renaming
    [any digital journal or notes kept at sea]

* directory = raw-level0
----------------------------

sta = station number
cast = cast number

- ctd_timeseries
    ctd_timeseries_00056_001.txt

                       ctd_timeseries_%05d_%03d.txt % (station, cast)  # no gps
    ctd_timeseries_00036_001_gps.txt ctd_timeseries_%05d_%03d_gps.txt % (station, cast)  # with gps

- depth_soundspeed
    depth_soundspeed_00019_002.txt   depth_soundspeed_%05d_%03d.txt % (station, cast)  # if it exists

- gps_timeseries
    gps_timeseries_00019_002.txt     gps_timeseries_%05d_%03d.txt % (station, cast)  # if it exists

- ladcp_raw/
    [downlooker,uplooker]
         BB150, WH150, WH300

           example files:
             ladcp_raw/downlooker/WH150/DLWH15000073_001.PD0
             ladcp_raw/downlooker/WH150/DLWH15000073_001.PD0
             ladcp_raw/uplooker/WH300/ULWH30000073_001.PD0
             ladcp_raw/downlooker/BB150/DLBB15000073_001.PD0
             ladcp_raw/downlooker/WH150/DLWH15000073_001.PD0

             ladcp_raw/uplooker/WH300/ULWH30000073_001.PD0
             ladcp_raw/uplooker/WH300/ULWH30000073_001.PD0

-  sadcp_nc # shipboard ADCP at the station
           wh300, nb150, os150bb, os150nb,  # directory named by instrument
           os75bb, os75nb, os38bb, os38nb]       A%-5d%03d.nc (station_cast)


For the complete processed shipboard ADCP data, go to JASADCP


* directory = processed-level1
=================================

cruise
    docs
        cruise_meta.txt  # PI, WOCE Expocode, cruisename, etc
        stacast_*rename  # notes about file renaming
        [any digital journal or notes kept at sea]
    proc
       inverse_Visbeck
           [I00123.002.nc]                  I%05d_%03d.nc  % (station, cast)
       inverse_Visbeck_Thurnherr
           [I00123.002.nc]                  I%05d_%03d.nc  % (station, cast)
       shear_Firing
           [S00123.002.nc]                  S%05d_%03d.nc  % (station, cast)


example:

2008_I6S/proc/inverse_Visbeck/I00045_001.nc
2008_I6S/proc/shear_Firing/S00045_001.nc

2008_I6S/data/ladcp_raw/downlooker/BB150/DLBB15000045_001.PD0
2008_I6S/data/ctd_timeseries/ctd_timeseries_00045_001_gps.txt
2008_I6S/data/sadcp_nc/nb150/A00045_001.nc
2008_I6S/data/gps_timeseries/gps_timeseries_00045_001.txt
2008_I6S/data/depth_soundspeed/depth_soundspeed_00045_001.txt


============================================
References
----------


Two primary processing mechanisms have been used: 'shear' and 'inverse'.
References are :

 - shear

Fischer, J., and M. Visbeck. 1993.
“Deep Velocity Profiling with Self-Contained ADCPs.”
Journal of Atmospheric and Oceanic Technology 10 (5): 764–73.
https://doi.org/10.1175/1520-0426(1993)010<0764:DVPWSC>2.0.CO;2.


 - inverse

Visbeck, Martin. 2002.
“Deep Velocity Profiling Using Lowered Acoustic Doppler Current Profilers: Bottom Track and Inverse Solutions.”
Journal of Atmospheric and Oceanic Technology 19 (5): 794–807.
https://doi.org/10.1175/1520-0426(2002)019<0794:DVPULA>2.0.CO;2.

Other resources:

JASADCP:Primary USA archive for processed shipboard ADCP data
(Joint Archive for Shipbord ADCP)
http://ilikai.soest.hawaii.edu/sadcp/

USA archive for WOCE, CLIVAR, and GOSHIP hydrographic measurements
https://cchdo.ucsd.edu

============================================
File formats:

suffix                   file format
-------                 ---------------
.txt                      Ascii text
.rst                      Ascii text ("Restructured Text markdown language)

.pdf                      Portable Document Format (PDF)
.s                        text: Linux bash commands (ascii)

.nc                       Network Common Data Format (NetCDF)  

.PD0     (Note 1)         Instrument manufacturer format, see below
.NB0     (Note 2)         Instrument manufacturer format, see below


** Note 1 - PD0 data format for workhorse and broadband instruments
   - WorkHorse_Commands_and_Output_Data_Format_Mar16_957-6156-00.pdf

** Note 2 -  - PD0 data format for narrowband instruments
  - NB_VM_Tech_Manual_67119_-_Data_Format.pdf




