Optimizing the Observational Sampling Designs
The initial science designs implemented for the NEON Terrestrial Observation System (TOS) and Aquatic Observation System (AOS) data products were developed in collaboration with Technical Working Groups (TWGs) composed of community experts. The science design specifications were based on subject matter expertise informed by analyses of published datasets and NEON prototype data. Now that NEON has been fully operational for several years, there is an opportunity to use available data to statistically evaluate the current implementation of TOS and AOS science designs and adjust accordingly.
Statistical analyses to guide science design optimization must:
- Describe if/where the data collected to date support reduction of TOS and AOS sampling effort without compromising NEON's ability to meet design requirements. For example, these analyses should help NEON staff identify target sites for a reduction in sampling effort, spatial scales where replication is not necessary, temporally redundant sampling, etc. A minimum of three years of data per product and NEON site is typically required to implement these analyses.
- Evaluate effects of design modifications on uncertainty and ability to detect year-to-year changes for key response variables (e.g., assess how reduced spatial and/or temporal sampling effort would change the detectable effect size for a response variable of interest).
The analyses of NEON data products enable assessment of how well the implementation of the science designs meet observatory goals. Moreover, results of these analyses allow the NEON TOS and AOS teams to effectively prioritize sampling efforts when required by funding or logistical constraints. Many such analyses and resulting sampling modifications have been implemented (as detailed in data product user guides and issue logs), with additional analyses expected throughout NEON's operations.
Overview of Optimization Process:
Identify the key scientific questions for the dataset of interest
The first step in the evaluation of a NEON data product is to identify specific questions about the dataset that the NEON data user community would be likely to ask. For example, do small mammal communities differ between National Land Cover Database (NLCD) classes within a NEON site? Or, at which NEON sites do measures of aboveground herbaceous biomass significantly change from one year to the next? Does ground beetle activity vary more with season or across years? Questions and analytical approaches used in these optimization analyses are developed in consultation with NEON TWGs.
Design and implement the statistical analysis, using NEON data collected to date
After determining the response variable(s) of interest and the types of ecological effects to target in an assessment of the data, statistical analyses are developed to (1) assess how well data collected under the current science design enables answering the ecological question(s), and (2) determine how altering the science design affects the ability to generate the same answer (e.g., if there is an 80% probability of detecting a 20% change in aboveground herbaceous biomass from one year to the next, how would that statistical power change if spatial replication is reduced from 8 plots to 6?).
Based on the above approach, the key questions in the optimization analyses typically are:
- Where is the variability in the data? For example, what proportion of the variance at each site exists at the year, bout, plot, and subplot scales? Understanding the components of variance provides a critical component of the actionable directions for modifying sampling size by providing answers to questions such as:
- At what scales does the sample design capture the most variation?
- How do variance components differ among sites?
- Is it possible to detect meaningful interannual variation in key response variables?
- Is it possible to reduce replication and produce parameter estimates within +/- 10% of those achieved with full replication according to the initial design greater than 90% of the time?
Statistical analyses that address such questions provide data-based guidance on where sampling efforts should be prioritized, and where opportunities for optimization might exist. For example, sites with greater variation among years might require a prioritization of temporal sampling. Alternatively, there would be a need to preserve spatial replication at sites with smaller temporal variance components and larger variance associated with spatial components of the design (e.g., across habitat types).
Seek expert and program approval for any resulting recommendations
If the data analyses support changes in the science design, either across the Observatory or at a subset of sites, the NEON science staff provide the analysis results and recommendations to the appropriate TWG for review and discussion. If the TWG expresses strong, majority support for the recommendations, the proposal is then taken through NEON's internal change management process for review and approval prior to implementation.