Evaluation

Last updated on 2025-04-25 | Edit this page

Estimated time: 15 minutes

Overview

Questions

  • How do I use Tools for Exploratory Evaluation in Hydrologic Research (TEEHR) to evaluate my NextGen models in NGIAB?

Objectives

  • Explain how TEEHR interacts with NGIAB
  • Evaluate a NextGen run using TEEHR in NGIAB

Using Tools for Exploratory Evaluation in Hydrologic Research (TEEHR) with NGIAB

A TEEHR evaluation run is executed by default upon execution of guide.sh. A separate runTeehr.sh script is also available in the NGIAB-CloudInfra repository.

What is TEEHR?

TEEHR is a Python-based package enabling analysis of hydrologic model performance (RTI International, 2025). TEEHR also provides tools to fetch the U.S. Geological Survey (USGS) streamflow data and NWM gridded and point-based retrospective and near real-time forecast data. This functionality supports comprehensive model evaluation and uncertainty analysis. NGIAB leverages TEEHR as a tool for hydrological model evaluation (CIROH, 2024). Researchers can explore a comprehensive range of metrics, including error statistics, skill scores, hydrologic signatures, and uncertainty quantification.

Gridded Data vs. Point Data

Gridded data consists of values at regularly spaced 2-dimensional cells that form a grid covering a region on Earth. Point data consists of values on specific 0-dimensional points. The NWM real-time forecast produces point-type stream routing and reservoir variables.

TEEHR folder contents

TEEHR consolidates data from the USGS and NWM, allowing side‐by‐side visual comparisons of observed and simulated over the model run intervals. Figure 1 shows the default comparison of the modeled outlet hydrograph and the corresponding time series from the NWM 3.0. While the default configuration produces this view, the TEEHR user documentation provides additional examples of capabilities for customized plotting functions.

A hydrograph spanning years 2017-2022. The x-axis is labeled "Datetime", and the y-axis is labeled "streamflow_hourly inst [m^3/s]". The blue line represents the NextGen run (labeled "ngen"), and the orange line represents the NWM 3.0 time series (labeled "nwm30_retrospective"). A legend is in the upper-right corner explaining the colors of these lines.
Figure 1: Comparison of the NextGen-based model (labeled “ngen”; blue line) and the NWM time series (labeled “nwm30_retrospective”; orange line) for the same location. The figure is automatically generated by the TEEHR-based analysis that accompanies the guide.sh script included with NGIAB and is named timeseries_plot_streamflow_hourly_inst.html in the teehr folder.

In addition to hydrograph visualization, TEEHR also calculates key performance metrics such as Kling-Gupta Efficiency (KGE), Nash-Sutcliffe Efficiency (NSE), and Relative Bias (RB) to quantify the accuracy of model predictions against observed data (Gupta et al., 2009; Nash and Sutcliffe, 1970). Users do not need to understand all these metrics immediately, and can refer to the references above to learn more. These results are then stored in a standardized output directory. For instance, the metrics.csv file in the TEEHR folder structure contains aggregated statistics for each configuration.

Through this systematic approach, NGIAB and TEEHR together allow hydrologists and stakeholders to inspect, compare, and refine model predictions in an open-source environment.

Your Turn

Go ahead and execute the TEEHR run using guide.sh and explore your TEEHR folder!

guide.sh Tips

  • The default TEEHR image is fine to use.

Key Points

  • Tools for Exploratory Evaluation in Hydrologic Research (TEEHR) is a Python-based backage for hydrologic model evaluation.
  • NGIAB uses TEEHR to assess model performance, comparing predictions against USGS streamflow and NWM data and calculating performance metrics.
  • TEEHR runs automatically with the main guide.sh NGIAB script.