Release 2024
Release Tag: RELEASE-2024
Release Date: January 29, 2024
Contents: JSON manifest
neonUtilities: use v2.4.1 or higher
NEON’s fourth data Release, RELEASE-2024, includes 161 data products annotated by 129 DOIs.
- RELEASE-2024 includes Instrumented Systems (IS) data through June 2023, and Observational Systems (OS) and Airborne Observation Platform (AOP) data through December 2022.
- Data products that are exclusively hosted by an external repository are excluded. These are:
- Phenocam data products: Phenology images (DP1.00033.001), Land-water interface images (DP1.20002.001), and Snow depth and understory phenology images (DP1.00042.001);
- AERONET product Spectral sun photometer - calibrated sky radiances (DP1.00043.001)
- Gap-filled products hosted by AmeriFlux, including Temporally interpolated soil temperature (DP2.00006.001), Temporally interpolated shortwave and longwave radiation (net radiometer) (DP2.00020.001), and Temporally interpolated triple aspirated tower temperature (DP2.00023.001).
- Data products that are exclusively downloadable via another data product are associated with the DOI of the downloadable data product.
- Some data products or subsets of data products (certain date ranges or field sites) have been excluded. Data may be available in excluded products and time periods; these data are provisional, and subject to change. Please refer to the exclusion table below.
- AOP data products will be exclusively available in this release until December 2024. The DOIs for the AOP data products in previous releases have been tombstoned.
Data Exclusions
| Data Product ID | Data Product Name | What is excluded for reasons other than general lag time | Reason for exclusion |
|---|---|---|---|
| DP*.300**.001 | All AOP Data Products *for STEI 2020-08: All AOP Data Products except for DP3.30015.001, DP3.30024.001, DP3.30025.001 |
CLBJ 2018-04, MLBS 2021-06, MOAB 2019-08 and 2021-04, PUUM 2019-01, REDB 2019-05, TOOL 2022-08, UKFS 2019-05, UNDE 2019-06, STEI 2020-08* | Dense and mis-classified atmospheric noise are in the discrete lidar point cloud data, resulting in orthorectification artifacts in spectrometer and camera data products. In most cases, a small portion of each site is affected by this issue, and the large majority of the site is still accurately classified (lidar) and orthorectified (spectrometer and camera). |
| DP1.10020.001 | Ground beetle sequences DNA barcode | 2022 | Lab analyses pending for 2022 samples |
| DP1.10022.001 | Ground beetles sampled from pitfall traps | 2022 | Lab analyses pending for 2022 samples |
| DP1.10041.001 | Mosquito-borne pathogen status | 2022-11 and 2022-12 | Lab analyses pending for samples from late 2022 |
| DP1.10064.002 | Rodent pathogen status, tick-borne | 2022-11 and 2022-12 | Lab analyses pending for samples from late 2022 |
| DP1.10081.001 | Soil microbe community composition | 2019, 2020, 2021, and 2022 | New data processing pipeline pending |
| DP1.10092.001 | Tick-borne pathogen status | 2022-11 and 2022-12 | Lab analyses pending for samples from late 2022 |
| DP1.10107.001 | Soil microbe metagenome sequences | 2020, 2021, and 2022 | Lab analyses pending |
| DP1.10108.001 | Soil microbe marker gene sequences | 2020, 2021, and 2022 | Lab analyses pending |
| DP1.20048.001 | Discharge field collection | 2022-10, -11, and -12 | Discharge-related products are released by water year. |
| DP1.20066.001 | Aquatic plant bryophyte macroalgae clip harvest | 2022 | Lab analyses pending for 2022 samples |
| DP1.20072.001 | Aquatic plant, bryophyte, lichen, and macroalgae point counts in wadeable streams | 2022 | Lab analyses pending for 2022 samples |
| DP1.20086.001 | Benthic microbe community composition | 2019 through 2022 | New data processing pipeline pending |
| DP1.20093.001 | Chemical properties of surface water | BARC, SUGG, and FLNT: 2020, 2021, and 2022 | Filtration problem in Domain 03 raised data quality concerns; resolution in progress. |
| DP1.20105.001 | Fish sequences DNA barcode | 2022 | Data updates pending |
| DP1.20107.001 | Fish electrofishing, gill netting, and fyke netting counts | 2022 | Data updates pending |
| DP1.20120.001 | Macroinvertebrate collection | 2022 | Lab analyses pending for 2022 samples |
| DP1.20126.001 | Macroinvertebrate metabarcoding | 2022 | Lab analyses pending for 2022 samples |
| DP1.20141.001 | Surface water microbe community composition | 2019 through 2022 | New data processing pipeline pending |
| DP1.20163.001 | Periphyton, seston, and phytoplankton chemical properties | 2020, 2021, and 2022 | Lab analyses pending and optimization of taxonomy workflow in progress. |
| DP1.20166.001 | Periphyton, seston, and phytoplankton collection | 2020, 2021, and 2022 | Lab analyses pending and optimization of taxonomy workflow in progress. |
| DP1.20206.001 | Stable isotopes in surface water | BARC, SUGG, and FLNT: 2021 and 2022 | Filtration problem in Domain 03 raised data quality concerns; resolution in progress. |
| DP1.20219.001 | Zooplankton collection | 2022 | Lab analyses pending for 2022 samples |
| DP1.20221.001 | Zooplankton metabarcoding | 2022 | Lab analyses pending for 2022 samples |
| DP1.20267.001 | Gauge height | 2022-10, -11, and -12 | Discharge-related products are released by water year. |
| DP1.20277.001 | Benthic microbe group abundances | 2022 | Lab analyses pending for 2022 samples |
| DP1.20278.001 | Surface water microbe group abundances | 2022 | Lab analyses pending for 2022 samples |
| DP1.20279.001 | Benthic microbe metagenome sequences | 2020, 2021, and 2022 | Lab analyses pending |
| DP1.20280.001 | Benthic microbe marker gene sequences | 2020, 2021, and 2022 | Lab analyses pending |
| DP1.20281.001 | Surface water microbe metagenome sequences | 2020, 2021, and 2022 | Lab analyses pending |
| DP1.20282.001 | Surface water microbe marker gene sequences | 2020, 2021, and 2022 | Lab analyses pending |
| DP1.30001.001 | LiDAR slant range waveform | BLAN 2022-05, WREF 2022-07, SRER 2022-08 | Withheld for further QA |
| DP1.30003.001 | Discrete return LiDAR point cloud | MOAB 2022-04 | Classified point clouds are missing |
| DP1.30024.001 | Elevation - LiDAR | CLBJ 2017-05 | Geolocation error in portion of Digital Terrain Model (DTM) |
| DP3.30010.001 | High-resolution orthorectified camera imagery mosaic | HOPB 2022-08, UNDE 2022-06 | Camera data are missing |
| DP3.30015.001 | Ecosystem Structure | STEI 2019-06 | Publication issue |
| DP4.00130.001 | Continuous discharge | REDB 2021-10 to 2022-12; all sites 2022-10 to 2023--06 | REDB - Poor data in water year 2022 are present that will be corrected before next release. All sites - Discharge-related products are released following water year. |
| DP4.00133.001 | Stage-discharge rating curves | 2022-10, -11, and -12 | Discharge-related products are released by water year. |
| DP4.00200.001 | Bundled data products - eddy covariance | BARR 2017-05 to 2017-08 BART 2016-12 to 2017-01 BONA 2017-10 CLBJ 2017-09 to 2017-10 CPER 2017-11 DELA 2017-02 DSNY 2017-02 JORN 2017-03 to 2017-08 LAJA 2017-08 to 2018-04 NOGP 2017-09 ORNL 2017-03 to 2017-06 SERC 2017-03 to 2017-05 TEAK 2018-06 UNDE 2017-02 WREF 2017-09 to 2018-02 YELL 2017-08 |
Site start dates have been consistently defined in this release. The excluded site-months are prior to a site passing initial operation review or prior to installation of the 3D sonic anemometer. |
Changes between RELEASE-2023 and RELEASE-2024
- For all OS data products, data quality improvement and error correction are continuous and ongoing processes. Routine data entry errors (such as transcription errors, duplicate records, or adjusted spatial coordinates) have been corrected where possible. Due to the large volume of data (>20 million data records), these corrections are not described individually. The table below describes significant changes to data products between releases.
- Isolated sites and time periods for most IS products were reprocessed to incorporate corrections to sensor installation periods, geolocation data, and manual quality flags.
- For all AOP data products, data quality improvement is an ongoing process. The bulk of the changes in the 2024 Release involved updating the Ecosystem Structure (DP3.30015.001) product, using an improved implementation of the pit-free Canopy Height Model algorithm. In addition, L3 lidar data products for two site-years were corrected to remove erroneous noise and properly classify mis-classified noise in the discrete lidar point cloud data. This mis-classified noise resulted in artifacts in the derived lidar data products, as well as orthorectification errors in the spectrometer and camera data products. A number of sites were excluded from RELEASE-2024 to provide time to correct for this mis-classified lidar noise issue identified in other sites. Finally, minor corrections were made to various AOP data products, sites, and years including fixing spatial referencing, re-naming files, removing duplicated files, and adding missing QA reports in order to improve consistency and follow AOP's standard naming and data organization conventions. Due to the large volume of data, the more minor corrections are not described individually, but the table entries below summarize significant changes to data products between releases. For more information on the AOP 2024 Release Updates, please refer to AOP Data Availability Notification – Release 2024.
| Data Product ID | Data Product Name | Summary of changes between RELEASE-2023 and RELEASE-2024 |
|---|---|---|
| DP1.00002.001 | Single aspirated air temperature | Data were reprocessed with an adjustment to the quality control algorithm that checks for sufficient air flow through the aspirated shield (flow test). The adjusted algorithm continues to raise the final quality flag when the data fail the flow test, but impacted data are no longer removed prior to averaging. |
| DP1.00003.001 | Triple aspirated air temperature | Data were reprocessed with an adjustment to the quality control algorithm that checks for sufficient air flow through the aspirated shield (flow test). The adjusted algorithm continues to raise the final quality flag when the data fail the flow test, but impacted data are no longer removed prior to averaging. |
| DP1.00006.001 | Primary precipitation | Data were reprocessed with an interim algorithm to better show spurious trace precipitation, reduce bias in long-term sums, and improve quality flagging. |
| DP1.00013.001 | Wet deposition chemical analysis | Uncertainty fields were removed from lab data as they were inaccurately documenting the method detection limit. A "sampling impractical" workflow was established in July 2023. |
| DP1.00014.001 | Shortwave radiation (direct and diffuse pyranometer) | The minimum range threshold for direct radiation was reduced from 0 W/m^2 to -5 W/m^2 to accommodate uncertainty in the measurement. Data were reprocessed. |
| DP1.00066.001 | Photosynthetically active radiation (quantum line) | Data were reprocessed using NEON’s new instrument processing pipeline. Computation of skewness and kurtosis statistics were updated to reflect the formulation presented in the product's Algorithm Theoretical Basis Document. |
| DP1.00095.001 | Soil CO2 concentration | Data from NEON site PUUM are now available and included in RELEASE-2024. |
| DP1.10026.001 | Plant foliar traits | For 2021 chlorophyll samples, all records were flagged to communicate improper analytical lab storage at -20C instead of ultracold temps. In addition, the relativeAccuracyScale field set to 'not available' because the lab was not using an appropriate check standard. For 2020 and 2021 elements data, the 'soil contamination' part of the flags associated with Fe or Mn values were removed. In the long-term summary table for carbon and nitrogen analyses, uncertainty values contain updated metrics that better conform with NEON definitions of uncertainty terms. |
| DP1.10033.001 | Litterfall and fine woody debris production and chemistry | Added new table, ltr_vegetationCover, to enable subplot, plot, and site level scaling of litter production. In the long-term summary table for carbon and nitrogen analyses, uncertainty values contain updated metrics that better conform with NEON definitions of uncertainty terms. |
| DP1.10043.001 | Mosquitoes sampled from CO2 traps | The dryIceStatus column from the mos_trapping table has been changed to CO2Status to reflect the use of CO2 cylinders to bait mosquito traps at numerous sites. A value of "present" means that the CO2 bait was still available at the time of trap collection, and "absent" means it ran out. "Flow obstructed" indicates that the dry ice vent hole was plugged or the tubing for the CO2 release was clogged. |
| DP1.10047.001 | Soil physical and chemical properties, distributed initial characterization | In the lab summary table, a column formerly called 'analyteAccuracy' was renamed to 'analyteAbsoluteError' to better conform with NEON definitions of uncertainty terms. |
| DP1.10055.001 | Plant phenology observations | Added new table, phe_identificationHistory, to track changes to taxonID in the phe_perindividual table. |
| DP1.10058.001 | Plant presence and percent cover | The subplot naming convention for Base plots was updated across all data products with observations and samples that originate in Base plots. The new format consists of the identity of the plot point in the southwest corner of the subplot, the scale or size of the subplot, and, for those subplots smaller than 100 m2, the corner of the 100 m2 subplot in which the smaller subplot is located. For more details, see the associated Data Notification: NEON Terrestrial Observation System (TOS) Base plot subplot renaming. |
| DP1.10067.001 | Root biomass and chemistry, periodic | In the long-term summary table for carbon and nitrogen analyses, uncertainty values contain updated metrics that better conform with NEON definitions of uncertainty terms. In addition, values for ‘samplingImpractical’ are now recorded at the level of the core rather than the plot. The ‘rootSamplingPossible’ field has also been removed, and records that formerly had a ‘rootSamplingPossible’ value of “N” now have a ‘samplingImpractical’ value of “obstruction” to more clearly and simply indicate that sampling was attempted but was not possible. In addition, if ‘samplingImpractical’ is not either “OK” or “obstruction”, edits were made to remove ‘clipCellNumber’, ‘clipID’, ‘sampleID’, and ‘sampleFate’ data that are not relevant when no physical sample was collected. |
| DP1.10086.001 | Soil physical and chemical properties, periodic | Several soil samples were flagged as being collected from resampled X, Y locations. Some instances of oven start dates or pH processed dates were removed if they were before the sample collect dates (this was a data entry error). Fall 2021 soil samples from ORNL were flagged to indicate missing pH data. In the long-term summary table for carbon and nitrogen analyses, uncertainty values contain updated metrics that better conform with NEON definitions of uncertainty terms. |
| DP1.10111.001 | Site management and event reporting | All TOS plot values in the 'locationID' field were updated to include the three letter sampling code to work with `geoNEON` functions. |
| DP1.20048.001 | Discharge field collection | Stage and discharge values are now all published in the dsc_fieldData table regardless of measurement type (flowmeter vs. ADCP; measurement type is published in dsc_fieldData:dischargeBoutTypeID). The dsc_fieldDataADCP table is still published and contains the survey metadata and secondary variables. All discharge values are now published in liters per second in the dsc_fieldData:finalDischarge field. Both dsc_fieldData:totalDischarge and dsc_fieldData:totalDischargeCalcQF are no longer published. The staff gauge named location is now published in this data product for each bout in dsc_fieldData:staffGaugeID. Temporary hydrologic conditions at the staff gauge and discharge transect locations are now published in dsc_fieldData (field names: gaugeTempHydroCond, gaugeTempHydroCondLoc, dscTempHydroCond, dscTempHydroCondLoc). For previously released records, data in these fields were migrated from the Gauge height (DP1.20267.001) data product and populated in this data product. The gag_fieldData table is no longer published in this data product. All numeric fields now have consistency in units (all now metric) and fields that report the unit for a measurement or provide a quality flag for inconsistent units are no longer published. All previously released records have been edited to fit the new data product structure. |
| DP1.20063.001 | Aquatic plant bryophyte chemical properties | alg_clipHarvest.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. |
| DP1.20066.001 | Aquatic plant bryophyte macroalgae clip harvest | alg_clipHarvest.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. D01 state-level taxa obfuscation: Prior to the 2024 data release, publication of species identifications were obfuscated to a higher taxonomic rank when the taxon was found to be listed as threatened, endangered, or sensitive by any state within the domain. Obfuscating state-listed taxa across an entire domain has created challenges for data users studying biodiversity. |
| DP1.20072.001 | Aquatic plant, bryophyte, lichen, and macroalgae point counts in wadeable streams | apc_pointTransect.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. D01 state-level taxa obfuscation: Prior to the 2024 data release, publication of species identifications were obfuscated to a higher taxonomic rank when the taxon was found to be listed as threatened, endangered, or sensitive by any state within the domain. Obfuscating state-listed taxa across an entire domain has created challenges for data users studying biodiversity. Reduced sampling: Until 2022, D11 BLUE collected transect data at 0.5 m intervals. Due to the stream width, data collection in one day is difficult especially during times of extreme heat. |
| DP1.20086.001 | Benthic microbe community composition | amb_fieldParent.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. |
| DP1.20120.001 | Macroinvertebrate collection | inv_fieldData.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. D01 state-level taxa obfuscation: Prior to the 2024 data release, publication of species identifications were obfuscated to a higher taxonomic rank when the taxon was found to be listed as threatened, endangered, or sensitive by any state within the domain. Obfuscating state-listed taxa across an entire domain has created challenges for data users studying biodiversity. |
| DP1.20126.001 | Macroinvertebrate metabarcoding | inv_fieldData.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window' |
| DP1.20163.001 | Periphyton, seston, and phytoplankton chemical properties | alg_fieldData.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. |
| DP1.20166.001 | Periphyton, seston, and phytoplankton collection | alg_fieldData.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. |
| DP1.20190.001 | Reaeration field and lab collection | The sensor data table rea_conductivityFieldData was moved to large file storage, and is now downloaded as a separate file for each sampling event. |
| DP1.20193.001 | Salt-based stream discharge | The sensor data table rea_conductivityFieldData was moved to large file storage, and is now downloaded as a separate file for each sampling event. |
| DP1.20219.001 | Zooplankton collection | zoo_fieldData.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. |
| DP1.20221.001 | Zooplankton metabarcoding | zoo_fieldData.biophysicalCriteria options have been updated to 'OK - within bout window', 'conditions not met: sampled after fish', and 'conditions not met: outside bout window'. |
| DP1.20288.001 | Water quality | Chlorophyll data streams were corrected due to the RFU and mg/L units being transposed. A complete list of sites and dates with incorrect data in RELEASE-2023 can be found in the issue log. |
| DP3.30015.001 | Ecosystem structure | This data product was updated for all sites and years, using an improved implementation of the pit-free Canopy Height Model. Please see AOP Data Availability Notification – Release 2024 for more details. |
| DP3.30015.001, DP3.30024.001, DP3.30025.001 | Ecosystem structure, Elevation - LiDAR, Slope and aspect - LiDAR | Re-processed to remove and re-classify mis-classified noise in the discrete lidar point clouds for CHEQ 2020-08 (delivered under STEI 2020-08) and CLBJ 2018-04. These mis-classified points previously resulted in artifacts in the derived L3 lidar raster products, which are now corrected. |
| DP4.00001.001 | Summary weather statistics | Data were reprocessed to incorporate the algorithm changes in primary precipitation (DP1.00006.001). |
| DP4.00200.001 | Bundled data products - eddy covariance | The turbulent CO2 concentration (DP1.00034.001) and CO2 flux (DP4.00067.001) field calibration was improved by evaluating variance and removing erroneous gas cylinders from regression and reevaluating to maximize data quality and availability. Flux footprint characteristics (DP4.00201.001) statistics table now includes the stability parameter (z/L) and the Obukhov length (L).The Atmospheric CO2 and d13C variables under the Atmospheric CO2 isotopes data product (DP1.00036.001) are now corrected using the NEONIso package with basic files including the Bowling et al., 2003 correction and expanded files including both Bowling et al., 2003 correction and a linear regression correction (Fiorella et al., 2021). Fiorella RP, Good SP, Allen ST, Guo JS, Still CJ, Noone DC, Anderegg WR, Florian CR, Luo H, Pingintha-Durden N, Bowen GJ (2021). “Calibration Strategies for Detecting Macroscale Patterns in NEON Atmospheric Carbon Isotope Observations.” Journal of Geophysical Research - Biogeosciences. https://doi.org/10.1029/2020JG005862. |