Comfort in Consistency: The Critical Job of NEON's Calibration and Validation Laboratory

By Sandra Chung

Deep inside the technical facility at NEON’s headquarters, test design engineer Guillermo Oviedo walks into a room with solid black walls. He stands next to a waist-high table surrounded by black curtains, types something into a nearby computer and clicks a mouse. A brilliant cone of light shoots toward the ceiling from the center of the table. The source of the light is a sausage-shaped bulb, selected by the National Institute of Standards and Technology (NIST) from among hundreds like it. NIST chose the bulb for how closely its 1,000-watt glare resembles real sunlight at wavelengths that plants use for photosynthesis. The bulb takes seven and a half minutes to ramp up to full power. A prompt pops up on the computer telling Oviedo to insert the sensor, a palm-size black metal cylinder. It fits perfectly into a mount with a small, round opening pointing toward the dazzling light. NEON scientists will use more than 1,000 sensors just like this one to measure photosynthesis active radiation (PAR), or the amount of daylight energy available for plants to capture and use to make sugars. Oviedo’s role is to make sure that all 1,028 sensors are measuring what they are supposed to measure, and doing it with consistent accuracy. NEON scientists will use more than 11,000 sensors of 44 different types to make dozens of different kinds of measurements, from soil moisture to water temperature to wind speed. Oviedo and his 10 colleagues are responsible for calibrating and validating all of NEON’s sensors as well as 1,300 gas cylinders each year. “With the amount of data we’re collecting, we’re going to have to prove that our data are very accurate,” says Laura Leyba-Newton, NEON’s calibration, validation, and audit laboratory manager. “And people are going to ask us to show them how we’re doing it.”

Accuracy Matters
Scientists and engineers place a premium on accurate measurement, and it’s often not a simple task to verify that a measuring device is accurate enough to trust. For example, your car speedometer may say you’re cruising along at the speed limit when police radar records you traveling 15 mph faster. You might go to traffic court and ask the police officer to show proof of his radar gun training, to demonstrate the radar calibration procedure and to provide evidence that the radar had been properly calibrated not long before it was pointed at your car. If the gun wasn’t properly tested against a respected standard, you argue, there’s no way of knowing how well it was working at the moment it was used to collect the key piece of evidence supporting your speeding ticket. In most cases it takes strict training, equipment and documentation for police to verify the accuracy of a single measurement made with a single radar gun. Similarly, NEON has to vouch for millions of measurements made with thousands of sensors over the next 30 years. Each NEON sensor must be calibrated and validated each year to ensure that it is collecting data as consistently and accurately as possible. For a project as massive as NEON, calibration and validation pose an enormous technical challenge. NEON opted to design its own calibration and validation facility from the ground up to reduce the cost and variability of sensor calibration. Many research projects outsource their calibrations to specialized facilities. But NEON’s many sensors will be spread out across the country, and the cost of sending them to other calibration facilities would be “outrageous,” says Leyba-Newton. Moreover, not every facility uses the same methods and equipment. It’s as if a chain manufacturing company had different technicians manufacture one link apiece using a variety of tools and techniques. The links are bound to come out a little different, and the chain company can’t guarantee the reliability of the whole chain in the same way it could if all the links were made the same way by a single person or machine. Just as a manufacturing company employs quality control experts to ensure that its products perform reliably, NEON employs calibration and validation experts to control the quality of its main product: data. The huge body of data collected by NEON sensors could be used to generate unprecedented scientific analyses that inform key decisions about how to manage natural resources. “If our info is not valid, then people may make bad decisions,” says Oviedo. “We have to be accurate. If people can’t trust our numbers, they won’t use our results.”

Quality Through Consistency
At NEON, all sensors of the same type will be calibrated and validated using the same process at the same facility. An eleven-person team, including engineers, scientists and technicians, is designing standard calibration and validation processes to meet requirements from the scientists developing NEON’s data collection protocols. “Everything we do has to be accepted by the scientific community,” says Leyba-Newton. The engineers will need, on average, a full year to validate their designs by collecting enough test data that support their efficacy. Oviedo, for example, is currently testing and refining the PAR sensor calibration process that he designed for NEON. The more daylight hits a PAR sensor, the higher the voltage across the sensor wires. The brightest sunlight that hits the Earth at sea level generates only 11 thousandths of a volt — not even enough to make a single brain cell fire. The sensor calibration process has to block out perturbations that can easily wreck such a sensitive measurement. Dark drapes and walls in the all-black room absorb excess light and reduce reflections that might interfere with PAR sensor tests. The tabletop also sits on air-cushioned legs to insulate it from any incidental vibrations. “It’s good up to a 3.7 earthquake,” Oviedo says. Accuracy is crucial, but so is speed. “Because of the quantity of data that we have, our processes have to be very efficient,” Leyba-Newton says. Automation not only saves time, it reduces the involvement of humans in the process. Humans are not as precise or as predictable as machines can be, so every step in the process that involves a human touch adds a little more unwanted variability to the results. Oviedo and his colleagues are designing, building and testing machines, or “fixtures,” that automate the calibration process as much as possible. One NEON-designed calibration fixture can handle 60 of NEON’s 4,000 PRT temperature probes at a time, with a robotic arm that transfers the sensors from one temperature-controlled water bath to another. Computer software controls the fixtures and, in some cases, instructs a human technician on how, where, and when to place sensors or flip a switch. Several volunteers from other NEON departments recently visited the calibration lab to test-drive some of the fixtures. NEON engineer Aaron Joos coached the volunteers through his procedure for calibrating humidity sensors. Joos’ procedure integrates a computerized tracking system already in use in other NEON engineering departments. The system automates the onerous task of documenting the locations and calibration statuses of thousands of individual sensors. Instead of recording everything in a paper log or typing it into a computer, the volunteers scanned barcode tags on the sensors and equipment to register their identities in the software that controls the humidity sensor calibration. In addition to the barcode tags that identify them to the control system, the sensors also sport microchips that can carry an amount of information equivalent to a small text file, Joos says. Chips on all the sensors will eventually store information about who installed the sensor, who calibrated it, when, and where, down to a particular slot at a particular fixture, Leyba-Newton says. Moreover, if a technician’s training for particular calibration procedure isn’t up-to-date, the system will prohibit the technician from performing calibrations. All this is just to make sure the sensors are working properly in the laboratory. Once the sensors go out into the field, many other factors begin to affect the quality of the data they collect. For example, a PAR sensor that’s incorrectly installed or tipped over by birds won’t measure daylight accurately. An upcoming facility in Longmont will house outdoor calibration fixtures for instruments that need to measure natural light and wind. In addition, NEON’s calibration, validation, and audit department includes auditors who will periodically review a random selection of NEON’s data and biological samples to make sure the companies that analyze them are doing so correctly and consistently. The close attention to detail and constant focus on improvement and quality in this line of work isn’t for everyone, Oviedo says. But NEON’s calibration, validation and audit team seem to be well-suited to the job in terms of both skill and enthusiasm. Most of the team members have experience in industrial-scale, or bulk, calibration, Leyba-Newton says. And many seem to relish the many-faceted challenge of creating calibration processes and fixtures that are as effective and immaculate as possible. “This job is fun because I have to do the mechanical design, the electrical design, software, process control,” Oviedo says. “It is very complete. It’s allowing me to grow in areas that I typically wouldn’t.” When asked if she and her colleagues enjoy designing the laboratory from the ground up, Leyba-Newton’s face brightens and she answers right away. “Absolutely. If you couldn’t create something from scratch, it would be boring.”