NCEI accepts open file formats maintained by a standards organization rather than proprietary or product-specific options. Self-describing formats like netCDF and HDF include valuable metadata that support data compatibility and long-term information preservation. PDF or PDF for archiving (PDF/A) are preferred for archived documents.
Above all, choose a format that works well with the type of data you’re submitting. Additionally, consider the format’s long-term (decades into the future) compatibility with data access software and infrastructure. For example, tabular data in comma-separated UTF-8 or ASCII files are much more likely to be compatible with future operating systems than proprietary formats (e.g., Lotus 1-2-3).
Before choosing a data format, review the National Archives and Records Administration (NARA) transfer formats list. Refer to the Library of Congress Sustainability of Digital Formats page when considering formats outside of the NARA list. All submissions are subject to review, and may need to undergo changes to format, filename, packaging, and other criteria to meet system or preservation requirements before being approved.
| Generic Data Types | Preferred Format(s) | Comment(s) |
|---|---|---|
| Tabular Data |
|
ASCII or UTF/UTF8/UNICODE encoding; See additional NetCDF guidance for many kinds of observation data (i.e, profiles, time series, trajectories, etc.) |
| Geospatial Information Systems (GIS) |
|
|
| Text |
|
Text documents must meet NOAA accessibility guidelines for compliance with Section 508 |
| Metadata | ISO 191xxx (series of international Geographic Metadata) (.xml) | The preferred representation of metadata in these standard formats is eXtensible Markup Language (xml) |
| Moving Images |
|
Please contact ncei.info@noaa.gov before submitting video data. Be sure to assess potential copyright and privacy issues before archiving. |
| Still Images | TIFF | |
| Others |
|
File Naming Conventions
The names of files submitted to NCEI must:
- Be clear and easy to understand
- Exclude special characters (such as punctuation and symbols)
- Use appropriate file format extensions
There may be other file naming and packaging rules depending on the type of data you submit. Contact NCEI for additional help, or review the NCEI metadata creation guide.
Supporting Documentation
An archive requires sufficient information to read, understand and characterize data holdings in accordance with documentation standards. This documentation contextualizes data, allowing current and future users to:
- Read the encoded format including descriptions of the data variables/content
- Understand data lineage pertaining to how it was created
- Characterize data quality to assess its usability
- Identify and distinguish data
- Trust the data integrity
Common supporting documents include user guides, format specifications, algorithm theoretical basis documents, and read-me files. Other forms of documentation may be appropriate for archiving to support data stewardship. NCEI may request additional documentation in order to meet these requirements.
Data providers are responsible for ensuring that documents comply with Federal Section 508 accessibility requirements. The NOAA Central Library Section 508 compliance page has many resources, such as how to create 508 compliant documents and accessibility testing tools.
Collection Metadata
NCEI requires standardized metadata for every collection or dataset to support discoverability, understandability and interoperability. This documentation includes a standard description, instructions for accessing and citing the data, and other pertinent details. Collection and metadata creation is supported through the various data submission methods.