Surface and groundwater elevation, conductivity, and temperature algorithm updates
September 23, 2024
The following data products have been reprocessed using NEON’s new instrument processing pipeline: Elevation of groundwater (DP1.20100.001), Specific conductivity in groundwater (DP1.20015.001), Temperature of groundwater (DP1.20217.001), and Elevation of surface water (DP1.20016.001). All provisional data have been updated, and data for all time and all sites will be updated in RELEASE-2025 (expected to be issued in late January 2025). Two data products have also been newly published with this update: Temperature (digital thermistor) of surface water (DP1.20054.001) and Specific conductivity of surface water (DP1.20008.001).
The main differences between previous and newly published data include:
Elevation of surface water (DP1.20217.001) now includes a 1-minute table with instantaneous water column height and nonsystematic uncertainty in the expanded download package. This 1-minute feeds into NEON level 4 Stage-discharge rating curves (DP4.00133.001) and Continuous Discharge (DP4.00130.001).
Sensor logged data has been incorporated to gap fill time periods where data streaming failed resulting in significantly increased data availability.
Additional flagging has been added, including:
zeroPressure flag - applied to all data products when the sensor raw pressure value is below 0 kPa, indicating a dry sensor.
missingTemp flag - applied to Specific conductivity of surface water (DP1.20008.001) when the sensor temperature data is unavailable for conversion from actual conductivity.
logData flag - applied to all data products when log files have been used to gap fill missing data. Note that raw values should not differ between streamed and logged data, but log data have a higher temporal uncertainty than streamed data.
logDataDate flag - applied to all data products when log data have corrupted dates. These data have been assigned a date-time following the readout time of the proceeding data point. Data with a logDateError flag of 1 may have a high temporal uncertainty.
Slight differences in the α and β quality flags and quality metrics (expanded download package) used to determine the final quality flags. See details in the NEON Algorithm Theoretical Basis Documents (ATBD) for surface water (NEON.DOC.001198) and groundwater (NEON.DOC.001328).
Quality metrics indicating the use of a valid sensor calibration (“validCalPassQM”, “validCalFailQM”, and “validCalNAQM” columns) are now populated in the expanded download package. See the variables file included in the download package for full descriptions of these metrics. Note that these are informational quality metrics and do not impact the computation of the final quality flag.
The expanded uncertainty for Elevation of surface water (DP1.20217.001) and Elevation of groundwater (DP1.20100.001) has been updated to account for the elevation uncertainty of the sensor in relation to other sensors at that NEON site. This does not account for real-world uncertainty, which equals the uncertainty of the GPS coordinates used to transform the sensor elevation to meters above sea level. Real-world uncertainty values can be downloaded using the geoNEON package in the NEONScience/NEON-geolocation Github repository.
The formulation for expanded uncertainty for Specific conductivity in groundwater (DP1.20015.001) has been updated to fix an error in the previous algorithm implementation.
All code used to generate the data products is open-source and available in the NEONScience/NEON-IS-data-processing Github repository. For questions or more information please Contact Us and select the appropriate product number in the data product dropdown list.
