Community NextGen and NGIAB Updates
- June 2024 Update
update ngen-DataStream
In June, 9 PRs were merged. Ngen-datastream was updated to be compatible with the new hydrofabric data format provided by hfsubset version 2.1.1.
In June, 9 PRs were merged. Ngen-datastream was updated to be compatible with the new hydrofabric data format provided by hfsubset version 2.1.1.
news NGIAB-data-preprocessor
Four PRs were merged in June. We are happy to annonce the new release v1.0.1. This release includes various improvements and bug fixes.
Four PRs were merged in June. We are happy to annonce the new release v1.0.1. This release includes various improvements and bug fixes.
feature NextGen Model Framework (ngen)
Eleven PRs merged in June. Explore more updates here.
Eleven PRs merged in June. Explore more updates here.
news CIROH DevCon24 Conference Slides
Access the slides from the 8 NextGen Track workshops held at CIROH DevCon24. Slides can be found here.
Access the slides from the 8 NextGen Track workshops held at CIROH DevCon24. Slides can be found here.
- May 2024 Update
news NGIAB-CloudInfra
The NextGen In A Box (NGIAB) workshop last week at CIROH DevCon24 was a huge success. New sample dataset is created “AWI-006”. We had 50+ attendees and most of them were successfully able to run NGIAB on their local machine. For those without the admin access, we provided them the AWS instances for the workshop. This workshop introduced a new geospatial visualization tool for visualizing outputs from NGIAB runs. During May, 16 PRs were merged. Slides can be found at:
The NextGen In A Box (NGIAB) workshop last week at CIROH DevCon24 was a huge success. New sample dataset is created “AWI-006”. We had 50+ attendees and most of them were successfully able to run NGIAB on their local machine. For those without the admin access, we provided them the AWS instances for the workshop. This workshop introduced a new geospatial visualization tool for visualizing outputs from NGIAB runs. During May, 16 PRs were merged. Slides can be found at:
update NGIAB-HPCInfra
Run NGIAB on HPC using NGIAB-HPCInfra. This repo is upgraded to use AWI-006 dataset. For more details, please refer README at
Run NGIAB on HPC using NGIAB-HPCInfra. This repo is upgraded to use AWI-006 dataset. For more details, please refer README at
news ngen-DataStream
NextGen Simulation Development Tools workshop was a huge success. It provided attendees with a hands-on experience using the ngen-datastream tool. In May, 21 PRs were merged. Explore the updates. Slides for the DevCon can be found at:
NextGen Simulation Development Tools workshop was a huge success. It provided attendees with a hands-on experience using the ngen-datastream tool. In May, 21 PRs were merged. Explore the updates. Slides for the DevCon can be found at:
news NGIAB-data-preprocessor
The NextGen Simulation Development Tools workshop at CIROH DevCon24 showcased the tool's capabilites of simplifying data preparation for next-gen simulations.
The NextGen Simulation Development Tools workshop at CIROH DevCon24 showcased the tool's capabilites of simplifying data preparation for next-gen simulations.
- April 2024 Update
news NGIAB-data-preprocessor
Introducing a tool for NGIAB data preparation. A workshop is planned for CIROH DevCon24 under NextGen Simulation Development Tools.
Introducing a tool for NGIAB data preparation. A workshop is planned for CIROH DevCon24 under NextGen Simulation Development Tools.
news NGIAB-CloudInfra
In April, we are gearing up to present the NextGen In A Box (NGIAB) workshop at CIROH DevCon24
In April, we are gearing up to present the NextGen In A Box (NGIAB) workshop at CIROH DevCon24
news ngen-DataStream
In April, we are getting ready to present NextGen DataStream at CIROH DevCon24. A workshop is planned under NextGen Simulation Development Tools
In April, we are getting ready to present NextGen DataStream at CIROH DevCon24. A workshop is planned under NextGen Simulation Development Tools
feature NextGen Model Framework (ngen)
Thirteen PRs merged in April. Explore the updates.
Thirteen PRs merged in April. Explore the updates.
- March 2024 Update
news Improvement of NGIAB HPCInfra Repositories
Automating build of NGIAB HPCInfra using GitHub Actions
Automating build of NGIAB HPCInfra using GitHub Actions
feature ngen-DataStream
In March, the python internal to the datastream was containerized. This included implementing catchment-specific configuration for PET, CFE, and Noah OWP (https://github.com/CIROH-UA/ngen-datastream/pull/46). The scripts and plotting tools were developed to perform benchmarking and preliminary results have been collected.
In March, the python internal to the datastream was containerized. This included implementing catchment-specific configuration for PET, CFE, and Noah OWP (https://github.com/CIROH-UA/ngen-datastream/pull/46). The scripts and plotting tools were developed to perform benchmarking and preliminary results have been collected.
- Feb 2024 Update
news Improvement of NGIAB CI Pipeline for Pull Requests from Forked Repositories
Three pull requests have been successfully merged. Pull requests submitted using forked repositories are now automatically tested in the CI Pipeline. Upon merging changes to the main repository, another CI pipeline is triggered to retest the build and push the image to DockerHub. Additionally, new AWI_004 sample data is now available for 09 VPU for unit testing.
Three pull requests have been successfully merged. Pull requests submitted using forked repositories are now automatically tested in the CI Pipeline. Upon merging changes to the main repository, another CI pipeline is triggered to retest the build and push the image to DockerHub. Additionally, new AWI_004 sample data is now available for 09 VPU for unit testing.
feature ngen-DataStream
In February, two pull requests were merged. For small runs, you can access ngen-DataStream at: https://github.com/CIROH-UA/ngen-datastream/tree/main/examples
In February, two pull requests were merged. For small runs, you can access ngen-DataStream at: https://github.com/CIROH-UA/ngen-datastream/tree/main/examples
- Jan 2024 Update
news Run NextGen In A Box(NGIAB) with Singularity on HPC without docker support!!!
If you want to use NGIAB on HPC that does not support docker, we have a solution for you. Please follow the steps at this link.
If you want to use NGIAB on HPC that does not support docker, we have a solution for you. Please follow the steps at this link.
news NextGen In A Box (NGIAB)
We have made some improvements to NGIAB: 10 PRs merged. Sample input data updated to use AWI_003(with new data provider names). Boost upgraded to v1.79. Self-hosted Runner can start and stop automatically with GitHub Actions. NGIAB image can run in auto mode. Geopackage support added. Documentation available at CIROH DocuHub
We have made some improvements to NGIAB: 10 PRs merged. Sample input data updated to use AWI_003(with new data provider names). Boost upgraded to v1.79. Self-hosted Runner can start and stop automatically with GitHub Actions. NGIAB image can run in auto mode. Geopackage support added. Documentation available at CIROH DocuHub
feature ngen-DataStream
3 PRs merged in January. Documentation available at: /products/nextgeninaboxDocker/nextgenDatastream
3 PRs merged in January. Documentation available at: /products/nextgeninaboxDocker/nextgenDatastream
- Dec 2023 Update
news NGIAB presentation at AGU!!!
NextGen In A Box: Advancing Collaborative Modeling for Enhanced Water Resource Management presented by Arpita. We had a full house!
NextGen In A Box: Advancing Collaborative Modeling for Enhanced Water Resource Management presented by Arpita. We had a full house!
news NextGen In A Box
NGIAB Updates: Merged CI pipeline changes PR#74 by Benjamin Lee, Added case study map and a plot with output results PR#72 by Shahab Alam, PR#70 by Josh Cu
NGIAB Updates: Merged CI pipeline changes PR#74 by Benjamin Lee, Added case study map and a plot with output results PR#72 by Shahab Alam, PR#70 by Josh Cu
feature NextGen Model Framework
5 PRs merged in December.
5 PRs merged in December.
feature T-route
12 PRs merged in December.
12 PRs merged in December.
feature Hydrofabric
Lynker is proud to support CIROH with a freely accessable resource for geospatial data: https://www.lynker-spatial.com/. Updated transects/cross section runners
Lynker is proud to support CIROH with a freely accessable resource for geospatial data: https://www.lynker-spatial.com/. Updated transects/cross section runners
feature NextGen Framework Forcings
December updates include an (almost) complete stream script that will produce daily ngen outputs. Documentation (readme) included. ngen-datastream/subsetting has been deprecated and hfsubset integrated into the stream. Benchmarking has begun.
December updates include an (almost) complete stream script that will produce daily ngen outputs. Documentation (readme) included. ngen-datastream/subsetting has been deprecated and hfsubset integrated into the stream. Benchmarking has begun.
note Community Support
December is always an AGU Month! San Francisco (the traditional home of AGU) brings us to the season where we reflect on our own work for the year and finally learn what our coworkers have been doing all this time! Checkout the presentation slides from CIROH at below link.
December is always an AGU Month! San Francisco (the traditional home of AGU) brings us to the season where we reflect on our own work for the year and finally learn what our coworkers have been doing all this time! Checkout the presentation slides from CIROH at below link.
- Nov 2023 Update
news NextGen Research Lightning Talk
Presented at CIROH NextGen Research Lightning Talk Webinar
Presented at CIROH NextGen Research Lightning Talk Webinar
feature NextGen Model Framework
14 PRs merged in November. Adding new tests for sqlite and new geopackage-based hydrofabric support.
14 PRs merged in November. Adding new tests for sqlite and new geopackage-based hydrofabric support.
feature T-route
13 PRs merged in November. Added additional json/geojson support
13 PRs merged in November. Added additional json/geojson support
feature Hydrofabric
Lynker is proud to support CIROH with a freely accessable resource for geospatial data: https://www.lynker-spatial.com/. Updated hfsubset to support using the v2.0 hydrofabric by default. Added Cross-section support to the hydrofabric. 13 PRs merged in November. Added additional json/geojson support
Lynker is proud to support CIROH with a freely accessable resource for geospatial data: https://www.lynker-spatial.com/. Updated hfsubset to support using the v2.0 hydrofabric by default. Added Cross-section support to the hydrofabric. 13 PRs merged in November. Added additional json/geojson support
feature NextGen Framework Forcings
14 PRs merged towards the Research Data Stream. Fixed lingering pyarrow issues. Added new support for all segments to be dockerized. Added S3 support with regards to the control flow design, to support one reusable, configurable sequence
14 PRs merged towards the Research Data Stream. Fixed lingering pyarrow issues. Added new support for all segments to be dockerized. Added S3 support with regards to the control flow design, to support one reusable, configurable sequence
note Community Support
Singularity support is coming to AWI for HPC users that are able to run Singularity builds. Work on serialization (passing from one run to the next with model states still in memory or imported from a file) is ongoing, relative temporal configurations of NextGen with consideration to model configuration changes between timesteps, and better general support for framework polymorphism.
Singularity support is coming to AWI for HPC users that are able to run Singularity builds. Work on serialization (passing from one run to the next with model states still in memory or imported from a file) is ongoing, relative temporal configurations of NextGen with consideration to model configuration changes between timesteps, and better general support for framework polymorphism.
- Oct 2023 Update
note NGIAB Updates
15 runs of the whole build process, 7 successful, and 2 pending. Over 550 pulls of the container image, number of 'canonical' runs (uploading their metadata and results) is coming in a future version.
15 runs of the whole build process, 7 successful, and 2 pending. Over 550 pulls of the container image, number of 'canonical' runs (uploading their metadata and results) is coming in a future version.
feature NextGen Model Framework
Updates to SUMMA candidate model, performance improvements for memory usage and spatial domain tooling to decouple the currently used geojson Feature and geometry classes into their own simple features interface. This is an abstract interface, with a coupled Boost.Geometry backend that will be used as the default backend.
Updates to SUMMA candidate model, performance improvements for memory usage and spatial domain tooling to decouple the currently used geojson Feature and geometry classes into their own simple features interface. This is an abstract interface, with a coupled Boost.Geometry backend that will be used as the default backend.
note T-route
22 Pull Requests were worked on in October with 11 merged, and a focus on testing and getting the compiling process to be tested and reliable particularly for downstream NGIAB and HPC usage of t-route
22 Pull Requests were worked on in October with 11 merged, and a focus on testing and getting the compiling process to be tested and reliable particularly for downstream NGIAB and HPC usage of t-route