6 posts tagged with "nextgen"

This month, we are excited to showcase two case studies that utilized our cyberinfrastructure tools and services. These case studies demonstrate how CIROH's cyberinfrastructure is being utilized to support hydrological research and operational advancements.

1. ngen-datastream and NGIAB​

Overview:​

CIROH’s cloud computing resources have allowed for the development of ngen-datastream, which automates the process of collecting and formatting input data for NextGen, orchestrating the NextGen run through NextGen In a Box (NGIAB), and handling outputs. This software allows users to run NextGen in an efficient, relatively painless, and reproducible fashion, increasing community access to the NextGen framework. ngen-datastream is already community accessible (https://github.com/CIROH-UA/ngen-datastream/tree/main) and making an impact on research. A major component of this software is the Amazon Web Services (AWS) cloud-based research datastream (https://github.com/CIROH-UA/ngen-datastream/tree/main/research_datastream). The research datastream is a CONUS-wide recurring NextGen simulation configured by the community. The terraform to build the AWS infrastructure exists in the ngen-datastream repository and current development focuses on CI/CD and enabling community contribution to the research datastream via edits to the NextGen configuration. Ultimately, these tools help distribute access throughout the community to cutting edge hydrologic research, maximizing the pace of progress of research to operations in hydrology.

Contribution to CIROH:​

• Automation: It automates the process of collecting, formatting, and validating input data for NextGen, streamlining model preparation.
• Flexibility: It allows users to provide their own input files to run NextGen.
• Scalable Infrastructure: It utilizes AWS state machine to provide access to high-performance computing (HPC) resources.

Infrastructure Utilized:​

• Elastic Compute Cloud (EC2)
• Simple Storage Service (S3)
• AWS Lamda and Step Functions

2. TEEHR​

• PI : Katie van Wekhoven
• Co-PI : Matt Denno (Development Lead)
• Developer : Sam Lamont

Project Overview:​

The goal of this project is to investigate, design, and build a prototype hydrologic model/forecast evaluation system (TEEHR) that will significantly improve our ability to evaluate continental-scale datasets and will provide a robust and consistent evaluation tool for CIROH and OWP research. Design priorities include easy integration into common research workflows, rapid execution of large-scale evaluations, simplified exploration of performance trends and drivers, inclusion of common and emergent evaluation methods, efficient data structures, open-source and community development, and easy extensibility.

Contribution to CIROH:​

• TEEHR-HUB: It is a JupyterHub environment, running the TEEHR image, with AWS services (EFS and S3) to provide a scalable platform for hydrologic research.
• Data Processing: TEEHR-HUB has successfully processed the AORC (v3.0 retrospective) gridded precipitation data to the MERIT basins, as well as the CONUS 40-year retrospective (v3.0 and USGS).
• Testbed Integration: TEEHR-HUB’s compatibility with various testbeds allows researchers to experiment with different hydrologic models and datasets.
• Evaluation - TEEHR is being used (or is planned for use) by several CIROH research teams to evaluate large scale model results.

Infrastructure Utilized:​

• Elastic Kubernetes Service (EKS) (including supporting AWS services) - Scalable computing resources to host JupyterHub Dask and Spark
• Elastic File System (EFS) - Shared data drive for cached data and shared documents (notebooks, etc.)
• Simple Storage Service (S3) - Bucket storage for large public and private datasets

CIROH Developers Conference 2024

The CIROH team recently participated in the 2nd Annual CIROH Developers Conference (DevCon24), held from May 29th to June 1st,2024. The conference brought together a diverse group of water professionals to exchange knowledge and explore cutting-edge research in the field of hydrological forecasting.

Reflecting CIROH's current research focus, the conference explored topics including hydrological modeling (NextGen), flood inundation mapping, hydroinformatics, social science, and community engagement. Attendees got the opportunity to delve deeper into specific areas through its well-structured training track. This year the tracks were:

• NextGen
• Flood Inundation Mapping (FIM)
• Hydrological Applications of Machine Learning (ML)
• Hydroinformatics
• Cross-cutting

This year, various workshops leveraged cloud technologies. Notably, we provided access to the 2i2c JupyterHub environment, a cloud-based platform for interactive computing, for ten workshops. This facilitated seamless access to powerful computing resources for participants. Additionally, we provided AWS instances to support four workshops.

Presentation Slides:

You can find the presentation slides here. To learn more about CIROH's work or connect with the team, visit our website at CIROH-website.

Conference Website:

Learn More

We've release NGIAB v1.1.0! This release fixes issues:

• #21
• #67
• #44

More info: https://github.com/CIROH-UA/NGIAB-CloudInfra/releases/tag/v1.1.0

Visit NGIAB News

We've introduced a fresh addition within the NGIAB, offering the most up-to-date insights on NGIAB and NextGen monthly updates.

Visit NGIAB Release Notes Page

NextGen Framework Forcings​

A new forcing processor tool has been made public. This tool converts any National Water Model based forcing files into ngen forcing files. This process can be an intensive operation in compute, memory, and IO, so this tool facilitates generating ngen input and ultimately makes running ngen more accessible.

Read more​

Visit Github

Adding posts​

What file name to use?​

DocuHub will extract a YYYY-MM-DD date from many patterns such as YYYY-MM-DD-my-blog-post-title.md or YYYY/MM/DD/my-blog-post-title.md. This enables you to easily group blog posts by year, by month, or to use a flat structure.

Example (with Metadata/Front matter)​

To publish in the blog, create a Markdown file within the blog directory. For example, create a file at /blog/2019-09-05-hello-docuhub.md

e.g.

---
title: Welcome DocuHub
description: This is my first post on DocuHub.
slug: welcome-DocuHub
authors:
  - name: John Doe
    title: Co-creator of Product 1
    url: <Youe github product or external article link>
    image_url: <Author pic url>
  - name: Jane Doe
    title: Co-creator of Product 2
    url: <Youe github product or external article link>
    image_url: <Author pic url>
tags: [hello, docuhub, nextgen]
hide_table_of_contents: false
---

Welcome to this blog. This blog is created with [**DocuHub 2**](https://docs.ciroh.org/).