Evaluating NextGen’s Performance in the MARFC Region with NGIAB
The National Weather Service's Middle Atlantic River Forecast Center (MARFC) sees large variations in the performance of the National Water Model 3.0. Through its support for regionalized parameters and models, NOAA-OWP’s Next Generation Water Resources Modeling Framework (NextGen framework) offers a potential solution to address these inconsistencies. As such, this study took advantage of NextGen in a Box (NGIAB) to evaluate the NextGen framework’s performance in the MARFC region.
This study evaluated three operational hydrologic modeling frameworks targetted at the National Water Model (NWM): the Community Hydrologic Prediction System (CHPS), the NextGen framework, and version 3.0 of the National Water Model itself.
- CHPS is the current operational framework used by NOAA's River Forecast Centers. It incorporates the SNOW-17 model for snowmelt and the Sacramento Soil Moisture Accounting (SAC-SMA) model for runoff generation.
- For the early phases of this study, the NextGen framework was used with the default model configuration provided by the NGIAB ecosystem, which combines the Noah-OWP-Modular land surface model and the Conceptual Functional Equivalent (CFE) rainfall runoff model [2].
- After initial runs with the baseline configuration, Noah-OWP-Modular was replaced with SNOW-17 output and simplified Potential Evapotranspiration (PET) values from the MARFC database.
- The models were calibrated using two objective functions: Kling-Gupta Efficiency (KGE) [6][7] and Nash-Sutcliffe Efficiency (NSE) [4][5].
- The National Water Model 3.0 uses the Noah-MP land surface model coupled with the Weather Research and Forecasting Hydrologic model (WRF-Hydro) [2][3] to simulate hydrological processes across CONUS.
The case studies focused on the Westfield and Elkland basins in North-Central Pennsylvania. These basins provide good locations for comparison due to the presence of USGS stream gages and their "flashy" behavior, characterized by rapid and unpredictable rises and falls in streamflow. Additionally, both Westfield and Elkland were sites of catastrophic flooding during Tropical Storm Debby in 2024, which allowed for the models to be evaluated on a recent extreme flood event. Results from Westfield, PA are shown in Figure 1.
Figure 1) Nash-Sutcliffe Efficiency (NSE) Metric for simulations from 2007 to 2020.
The simulations executed for the case study are listed below:
- Uncalibrated CHPS
- Calibrated CHPS
- Uncalibrated NextGen (AORC forcings)
- Uncalibrated NextGen (MARFC forcings)
- Calibrated NextGen (AORC forcings; calibrated on KGE objective function)
- Calibrated NextGen (AORC forcings; calibrated on NSE objective function)
- Calibrated NextGen (MARFC forcings; calibrated on KGE objective function)
- National Water Model 3.0
These simulations were executed on the JetStream2 platform with the support of the CIROH Research Cyberinfrastructure Team.
Figure 2) Percent Bias and NSE of simulations from 2007 to 2020 generated with different forcings - AORC (referred as NextGen) and MARFC Hourly Operational Grids (referred as NextGen FC).
The researchers observed that the calibrated runs for both NextGen and CHPS showed significant improvements over uncalibrated runs in both Westfield and Elkland basins. Among the calibrated runs, CHPS marginally outperformed NextGen. Additionally, both CHPS and NextGen outperformed NWM 3.0 in Westfield basin.
When running the NextGen-based models, substitution of SNOW-17 output and simpler PET for the NOAH-OWP reduced calibration time by 50% and produced similar outputs.
However, the models that performed best during calibration did not necessarily perform best during the Hurricane Debby extreme event. This suggests that including a greater number of extreme events in the calibration period is necessary to improve model robustness.
Figure 3) Hydrograph of Tropical Storm Debby on 08/09/2024. It is important to note that the observed data, denoted by black dots, cuts out halfway through the hydrograph. For this window, peak flow was estimated by the USGS to be just under 20,000 cfs, which is nearly double the second largest event on record.
NGIAB served as a critical enabler in this study, allowing for rapid iteration and in-depth exploration of the NextGen framework’s configuration space. This evaluation of objective functions, forcing datasets, and baseline models would have been significantly more challenging with traditional workflows.
The presentation and output data for this study have been made available on Hydroshare.
References
- National Oceanic and Atmospheric Administration. Community Hydrologic Prediction System (CHPS) Documentation [Internet]. National Oceanic and Atmospheric Administration; [updated 2024 Nov 26; cited 2025 Jul 28]. Available from: https://vlab.noaa.gov/web/chps
- Araki R, Ogden FL, McMillan HK. Testing Soil Moisture Performance Measures in the Conceptual‐Functional Equivalent to the WRF-Hydro National Water Model. JAWRA Journal of the American Water Resources Association. 2025 Feb;61(1):e70002.
- National Oceanic and Atmospheric Administration. The National Water Model [Internet]. National Water Prediction Service; [cited 2025 Jul 29]. Available from: https://water.noaa.gov/about/nwm
- Tijerina‐Kreuzer D, Condon L, FitzGerald K, Dugger A, O’Neill MM, Sampson K, Gochis D, Maxwell R. Continental hydrologic intercomparison project, phase 1: A large‐scale hydrologic model comparison over the continental United States. Water Resources Research. 2021 Jul;57(7):e2020WR028931.
- Moriasi DN, Arnold JG, Van Liew MW, Bingner RL, Harmel RD, Veith TL. Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Transactions of the ASABE. 2007;50(3):885-900.
- Gupta HV, Kling H, Yilmaz KK, Martinez GF. Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. Journal of Hydrology. 2009 Oct 20;377(1-2):80-91.
- Vrugt JA, de Oliveira DY. Confidence intervals of the Kling-Gupta efficiency. Journal of Hydrology. 2022 Sep 1;612:127968.