The EFI-NEON Forecasting Challenge in the Classroom

Dozens of individuals and teams have participated in the Ecological Forecasting Initiative Research Coordination Network (EFI-RCN)’s NEON Ecological Forecasting Challenge, which challenges people to create ecological forecasts using data from the NEON program. If you have not had a chance to participate yet, don’t worry: this initiative is ongoing. Educators across the country are using the Challenge with their undergraduate and graduate students. Some of them have shared their stories and strategies below.

Laying the Foundations for Tomorrow’s Ecological Forecasters

The EFI-NEON Forecasting Challenge was established in 2020 via a grant from the National Science Foundation (NSF). Dr. Quinn Thomas, an associate professor and faculty fellow at Virginia Tech in the Departments of Forest Resources and Environmental Conservation and Biological Sciences, is the principal investigator for the grant. He explains, “We wanted to create a research coordination network around ecological forecasting with a primary goal of setting the rules and developing the cyberinfrastructure for others who want to use NEON data for forecasting. That way, instead of having to start from scratch to figure out what data to forecast or how to get data in the format you need, you can just focus on the cool way you want to build your forecast.”

EFI-RCN and the NEON program have put together a host of resources and instructions for would-be forecasters and educators who wish to use the Challenge in the classroom. While the initial Challenge required participants to submit forecasts at set times, the Challenge is now ongoing. That means teams can submit their forecasts at any time—and even submit forecasts daily. EFI Community Manager Jody Peters of the University of Notre Dame says, “The idea is that we would like the challenge to be something that can continually run in the background. We’ll have a couple of big pushes each year where we encourage people to submit, but the infrastructure is there so you can use it in a class at any time of the year.”

Forecast teams can submit forecasts for different themes at any time throughout the calendar year. Round 2 will continue through the end of 2022, and Round 3 will start in early 2023. The current grant will keep the challenge running through 2024.

The infrastructure that supports the rolling Challenge is distributed and cloud-based, with a variety of open-source and community-available software tools to streamline common processes such as downloading data and uploading forecasts. The cyberinfrastructure has evolved over the last year to better meet the needs of the forecasting community. Thomas explains, “Figuring out all these little pieces of software that are needed to reduce barriers to entry have been some of the lessons learned. We’re continuing to build more and more of these tools to support the community and make it easier to participate.”

Lessons from the Classroom

“What we found is that one of the most exciting areas for the forecasting challenge was use in the classroom,” says Thomas. He used the Challenge in an Ecological Modeling and Forecasting course at Virginia Tech open to both undergrads and grad students. He had five different groups (with 3-4 students each) in his class, each tackling a different Challenge theme.

A number of educators have used the Challenge as a culminating project or real-world experience in their courses. Dr. Brett Melbourne, an associate professor of Ecology at the University of Colorado – Boulder, used the Challenge in an interdisciplinary quantitative biology graduate program. Three of his students banded together to create a novel forecasting model for the Phenology Challenge. He says that the EFI-NEON Challenge is an ideal real-world research project for students. “What makes it ideal is that it is very typical, in the sense that we were using real data, and a lot of the effort went into data cleaning. I think it’s good to have students tackle a real-world project where the data can be messy.” While he did not believe that their forecast performed particularly well, for Melbourne, that was not the point. He says his students developed important problem-solving, teamwork and creativity skills in addition to data acumen.

Drs. Kim Novick and Mallory Barnes co-taught a graduate-level readings course focused on land-atmosphere interactions at Indiana University – Bloomington. The EFI-NEON Challenge provided much-needed hands-on application to complement the readings. Their students focused on the Soil Moisture Forecasting Challenge. Two groups participated, one using a process-based model and one using a machine learning model. Novick says, “We wanted them to think more critically about the drivers of soil moisture evaporative transfer and also learn more about forecasting.” The machine learning group ultimately submitted a forecast which they felt performed fairly well.

Thomas, Melbourne, and Novick shared some tips and advice for educators interested in using the Challenge in the classroom:

• Start simple. Thomas says, “It’s OK to use simple models…and it’s OK if they’re not very good. You can always build in more complexity later, and you can always get better through time.” While some students may think they need to have a novel or complex model in order to participate, it may be better for new forecasters to experiment with some simple, tried-and-true models and open-source code.
• Figure out your objective. Is your primary objective for students to learn data science skills? To understand the various approaches to forecasting? Or to test theories they have about ecological drivers? Whether your focus is primarily on the data science or the ecology, past participants recommend figuring out your specific objectives for your students and guiding them accordingly.
• Don’t worry about results. All three stressed that the point of the Challenge was learning, not performing better than all the other forecasts submitted. Students participating in the Challenge are developing important data skills and learning about different approaches to forecasting, even if their forecast does not end up performing well. Melbourne says, “About 50-75% of their time was spent in data wrangling and learning about the Challenge. Ultimately, they only submitted a couple of forecasts, so I would be surprised if they did very well. But the point wasn’t to compete against the other teams. It was to develop teamwork and apply their skills in a real-world situation.”
• Make connections. Have questions? Reach out—people at both EFI and the NEON program are happy to help (NEON staff are eager to be contacted with questions about NEON data). Thomas says, “You can reach out to me or others who have done this in the past. We’re very happy to answer questions and provide best practices.” Novick would like to develop more collaborations with students and educators at other universities for the next Challenge. “I want to help them make those connections early on.”
• Don’t be afraid to dive in! “Don’t feel like you have to have a perfect understanding of the Challenge in order for your students to get something out of it,” says Novick. “It’s a learning process; students and their professors can learn together.”

Educators wishing to bring the Challenge to their own classrooms will find a wealth of resources from both NEON and EFI-RCN. And these resources are growing all the time. “We’re collecting best practices and trying to put them out there,” says Thomas. The cyberinfrastructure and library of open-source tools will continue to grow as well. If funding can be secured, the program managers would love to see the Challenge continue for the life of the NEON program.

Thomas envisions a growing role for the Challenge in education. “It’s authentic; it’s not the professor just making up some assignment. Students like the idea that it’s bigger than them. And for an emerging field like forecasting in ecology, we want to target the next generation, so education is a good place to start."