The systems and instruments that collect data about Earth don’t sleep. As those observing tools do their job, NCEI grows an ever-expanding archive of information about Earth, its environment, and space weather.
The amount of data that NCEI ingests and securely stores is growing, and that growth is expected to accelerate in the future. As of May 2022, NCEI stored about 44 petabytes of data, which includes a secure copy. That is the equivalent size of almost 150 years of high-definition video. By 2030, we expect to have 400 petabytes of data. Currently, NCEI manages thousands of datasets in the archive and more than 400 products.
Data volumes are increasing due to new and more complex data systems like uncrewed systems (UxS), more international satellite data, and the archiving of weather and climate forecast models. Our data come from many sources: weather stations, radar, UxS, acoustics monitoring, ‘Omics, satellite systems, and other unique collections from field measurements, tree rings, ice cores, and instruments on animals.
Additionally, a new NCEI effort called Synchronized NCEI Archive Projects Program, aptly referred to as SNAPP, has accelerated the pace of ingesting data at NCEI. SNAPP improves our ability to track data through the archiving process and identify areas of improvement to decrease the time it takes to archive data. In Fiscal Year 2021, SNAPP completed 66 archive projects while receiving 45 additional archive requests, exceeding expectations.
Our data dissemination capabilities are growing in the cloud through collaboration with the NOAA Open Data Dissemination (NODD) program, formerly known as the Big Data Program. NODD works with Amazon Web Services, Google Cloud Platform, and Microsoft Azure to make NOAA data available in the cloud. The full list of datasets is available on the NODD website with direct links to the Cloud Service Provider landing pages. For example, over 50 million depth measurements of the ocean floor are available through this collaboration. Our datasets are also searchable through Google Dataset Search.
Users of our data benefit from more robust data delivery systems, rapid availability, and a rising awareness of NOAA resources. One goal of accessibility is to continue to fine-tune searches and delivery methods to match the needs of users and streamline the discovery process.
Our data are used in a variety of fields and economic sectors, primarily: professional, scientific, and technical services; education; public administration; legal; construction; manufacturing; finance; utilities; and agriculture.
NCEI’s Data: Nuts and Bolts
Our archive includes data, metadata, and products from a broad range of time periods, scientific disciplines, and geographic locations. NCEI environmental data spans a spectrum of archive mechanisms, naming conventions, file formats, and governance strategies. NCEI develops software, APIs, visualization methods, and other services to enhance data access, discovery, and interoperability.
Our digital data are stored on computer servers and tape drives. But NCEI stewards more than digital data. Physical materials dating back to the 1800s also fall within the archival duties of NCEI. Our temperature-controlled facility stores nearly 40,000 boxes of documents, more than 140,000 reels of microfilm, and about 650,000 pieces of microfiche.
