Using an API Token when Accessing NEON Data with neonUtilities

NEON data can be downloaded from either the NEON Data Portal or the NEON API. When downloading from the Data Portal, you can create a user account. Read about the benefits of an account on the User Account page. You can also use your account to create a token for using the API. Your token is unique to your account, so don't share it.

Using a token is optional! You can download data without a token, and without a user account. Using a token when downloading data via the API, including when using the neonUtilities package, links your downloads to your user account, as well as enabling faster download speeds. For more information about token usage and benefits, see the NEON API documentation page.

For now, in addition to faster downloads, using a token helps NEON to track data downloads. Using anonymized user information, we can then calculate data access statistics, such as which data products are downloaded most frequently, which data products are downloaded in groups by the same users, and how many users in total are downloading data. This information helps NEON to evaluate the growth and reach of the observatory, and to advocate for training activities, workshops, and software development.

Tokens can be used whenever you use the NEON API. In this tutorial, we'll focus on using tokens with the neonUtilities R package.

If you've never downloaded NEON data using the neonUtilities package before, we recommend starting with the Download and Explore tutorial before proceeding with this tutorial.

In the next sections, we'll get an API token from the NEON Data Portal, and then use it in neonUtilities when downloading data.

Get a NEON API Token

The first step is create a NEON user account, if you don't have one. Follow the instructions on the Data Portal User Accounts page. If you do already have an account, go to the NEON Data Portal, sign in, and go to your My Account profile page.

Once you have an account, you can create an API token for yourself. At the bottom of the My Account page, you should see this bar:

Click the 'GET API TOKEN' button. After a moment, you should see this:

Click on the Copy button to copy your API token to the clipboard:

Use API token in neonUtilities

In the next section, we'll walk through saving your token somewhere secure but accessible to your code. But first let's try out using the token the easy way.

First, we need to load the neonUtilities package and set the working directory:

# install neonUtilities - can skip if already installed, but
# API tokens are only enabled in neonUtilities v1.3.4 and higher
# if your version number is lower, re-install
install.packages("neonUtilities")

# load neonUtilities
library(neonUtilities)

# set working directory
wd <- "~/data" # this will depend on your local machine
setwd(wd)

NEON API tokens are very long, so it would be annoying to keep pasting the entire text string into functions. Assign your token an object name:

NEON_TOKEN <- "PASTE YOUR TOKEN HERE"

Now we'll use the loadByProduct() function to download data. Your API token is entered as the optional token input parameter. For this example, we'll download Plant foliar traits (DP1.10026.001).

foliar <- loadByProduct(dpID="DP1.10026.001", site="all", 
                        package="expanded", check.size=F,
                        token=NEON_TOKEN)

You should now have data saved in the foliar object; the API silently used your token. If you've downloaded data without a token before, you may notice this is faster!

This format applies to all neonUtilities functions that involve downloading data or otherwise accessing the API; you can use the token input with all of them. For example, when downloading remote sensing data:

chm <- byTileAOP(dpID="DP3.30015.001", site="WREF", 
                 year=2017, check.size=F,
                 easting=c(571000,578000), 
                 northing=c(5079000,5080000), 
                 savepath=wd,
                 token=NEON_TOKEN)

Token management for open code

Your API token is unique to your account, so don't share it!

If you're writing code that will be shared with colleagues or available publicly, such as in a GitHub repository or supplemental materials of a published paper, you can't include the line of code above where we assigned your token to NEON_TOKEN, since your token is fully visible in the code there. Instead, you'll need to save your token locally on your computer, and pull it into your code without displaying it. There are a few ways to do this, we'll show two options here.

• Option 1: Save the token in a local file, and source() that file at the start of every script. This is fairly simple but requires a line of code in every script.

• Option 2: Add the token to a .Renviron file to create an environment variable that gets loaded when you open R. This is a little harder to set up initially, but once it's done, it's done globally, and it will work in every script you run.

Option 1: Save token in a local file

Open a new, empty R script (.R). Put a single line of code in the script:

NEON_TOKEN <- "PASTE YOUR TOKEN HERE"

Save this file in a logical place on your machine, somewhere that won't be visible publicly. Here, let's call the file neon_token_source.R, and save it to the working directory. Then, at the start of every script where you're going to use the NEON API, you would run this line of code:

source(paste0(wd, "/neon_token_source.R"))

Then you'll be able to use token=NEON_TOKEN when you run neonUtilities functions, and you can share your code without accidentally sharing your token.

Option 2: Save token to the R environment

To create a persistent environment variable, we use a .Renviron file. Before creating a file, check which directory R is using as your home directory:

# For Windows:
Sys.getenv("R_USER")

# For Mac/Linux:
Sys.getenv("HOME")

Check the home directory to see if you already have a .Renviron file, using the file browse pane in RStudio, or using another file browse method with hidden files shown. Files that begin with . are hidden by default, but RStudio recognizes files that begin with .R and displays them.

If you already have a .Renviron file, open it and follow the instructions below to add to it. If you don't have one, create one using File -> New File -> Text File in the RStudio menus.

Add one line to the text file. In this option, there are no quotes around the token value.

NEON_TOKEN=PASTE YOUR TOKEN HERE

Save the file as .Renviron, in the RStudio home directory identified above. Double check the spelling, this will not work if you have a typo. Re-start R to load the environment.

Once your token is assigned to an environment variable, use the function Sys.getenv() to access it. For example, in loadByProduct():

foliar <- loadByProduct(dpID="DP1.10026.001", site="all", 
                        package="expanded", check.size=F,
                        token=Sys.getenv("NEON_TOKEN"))