Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “end-to-end”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Enabling kilometer-scale E3SM land model simulation over North America: A new integrated framework solution

This study introduces a novel framework designed to enhance the performance, scalability, and portability of the kilometer-scale E3SM Land Model (km-ELM) within the E3SM modeling infrastructure. By seamlessly integrating cutting-edge data tools, we address existing challenges such as slow performance, limited scalability, and difficulties in software integration in current data-driven ELM simulation over large geographic areas. Our innovative approach leverages the KiloCraft data toolkit to generate unified inputs for simulations ranging from a single-cite case, to a 72,083-cell regional case to a continental configuration encompassing 21.6 million land grid cells at a 1 km × 1 km resolution. We conduct extensive strong- and weak-scaling experiments on three state-of-the-art supercomputers, utilizing up to 100,800 CPU cores across 2400 compute nodes to evaluate end-to-end metrics including wall-clock time, simulation-years-per-day (SYPD), initialization costs, and I/O throughput. Our results reveal the land (LND) component’s efficient scaling, demonstrating near-ideal weak scaling and strong-scaling parallel efficiencies reaching up to 87% at 50,400 cores. We confirm portability and reproducibility through bitwise-equivalent outputs across different machines using identical inputs over supported machines. Notably, at extreme scales, we identify I/O as a critical bottleneck and that leads to effective solution with the SCORPIO/ADIOS stack. Collectively, these findings validate the deployment of km-ELM at a continental scale with high parallel efficiency and provide essential guidance on configuration, decomposition, and I/O settings for optimized kilometer-scale land simulations in E3SM. This work emphasizes the innovative design and practical solutions that enhance the operational capabilities of km-ELM, focusing on software performance and scalability while leaving detailed scientific evaluations of simulated land processes for future investigations.

E3SM land model (ELM), km-ELM, scalability, perfor↗
DOE OSTI2026

From 2D to 4D: a containerized workflow and browser to explore dynamic chromatin architecture

Background Characterizing the physical organization of the genome is essential for understanding long-range gene regulation, chromatin compartmentalization, and epigenetic accessibility. Hi-C experiments generate two-dimensional (2D) genome-wide contact maps of chromatin interactions by capturing the spatial proximity between genomic loci, which reveal interaction frequencies but lack the spatial resolution needed to interpret the three-dimensional (3D) genome structure(s). Emerging evidence suggests that epigenetic regulation is closely linked to 3D genome architecture, and that structural changes over time (4D) drive key biological processes in development, disease, and environmental response. Thus, integrating 3D structure with functional data is critical for a more complete understanding of genome regulation. Previous work, most notably the 4DHiC chromosome modeling framework, has shown that physical multi-dimensional modeling approaches rooted in polymer physics and molecular dynamics can resolve these structures at biologically meaningful resolutions by integrating temporal Hi-C data with physical constraints to uncover dynamic chromosome reorganization. Thus, molecular dynamics simulations, constrained by Hi-C contact matrices, can resolve fine-scale structural changes and reveal functionally significant transitions in chromatin conformation. Results Herein, we present the 4D Genome Browser Workflow (4DGBWorkflow) and the 4D Genome Browser (4DGB). The algorithm is based on the 4DHiC method, and the containerized tool is an end-to-end workflow that can transform, filter, and view 4D epigenomics and chromatin datasets, allowing non-specialists to apply three-dimensional modeling principles to diverse datasets and experimental conditions. The software executes on a laptop running macOS, Linux or Windows. From input Hi-C files (.hic), the 4DGBWorkflow produces 3D reconstructions of chromosomes, integrates the reconstruction with track data (e.g., epigenetic marks, transcriptome profiles), and provides comparative visualization of the results in a single workflow. Conclusions The 4DGBWorkflow and 4D Genome Browser are open-source tools for comparative analysis and visualization of 4D chromosome datasets, including chromatin architecture and epigenomic signals. Automatic integration of Hi-C data with molecular dynamics democratizes the construction of time resolved 3D genome structures, simplifying complex simulations and data integration schemes.

3D Genome Browser↗
DOE OSTI2026

Data for An End-to-End Pipeline for Succinic Acid Production at an Industrially Relevant Scale Using Issatchenkia orientalis

Microbial production of succinic acid (SA) at an industrially relevant scale has been hindered by high downstream processing costs arising from neutral pH fermentation for over three decades. Here, we metabolically engineer the acid-tolerant yeast Issatchenkia orientalis for SA production, attaining the highest titers in sugar-based media at low pH (pH 3) in fed-batch fermentations, i.e. 109.5 g/L in minimal medium and 104.6 g/L in sugarcane juice medium. We further perform batch fermentation using sugarcane juice medium in a pilot-scale fermenter (300×) and achieve 63.1 g/L of SA, which can be directly crystallized with a yield of 64.0%. Finally, we simulate an end-to-end low-pH SA production pipeline, and techno-economic analysis and life cycle assessment indicate our process is financially viable and can reduce greenhouse gas emissions by 34–90% relative to fossil-based production processes. We expect I. orientalis can serve as a general industrial platform for production of organic acids.

Metabolomics↗
DOE OSTI2025

TRISO Particle Isolation and Graphite Removal Using SRNL Vapor Digestion Technology

Tri-structural isotropic fueled reactors are planned to come online in the next decade, but there is not currently a widely agreed upon disposition pathway for this new fuel stream. For long term used nuclear fuel storage, there would be a significant benefit if the waste volume could be reduced or mitigated. Most of the volume from used TRISO fuel contains graphite. SRNL currently has an active patent which describes one pathway to digest the graphite. One benefit in the utilization of this pathway is that multiple cylindrical pieces can be stacked end-to-end in a reaction tube, then nitric acid and water vapor can be flowed through the tube, and the weight change of the individual cores can provide both a reaction profile and overall oxidant use efficiency. The reaction of nitric acid and graphite passes through a series of intermediate products – NO 2 , NO, and N 2 O – from reaction and decomposition to eventually form N2. The data shows that NO2 grows in with increasing temperature between 500-600°C and then decreases by way of either reaction with graphite or decomposition. Nitric oxide, a reaction and thermal decomposition product of NO 2 , exhibits a consistent decline as a function of temperature, which is consistent with the literature. Similarly, the concentrations of N 2 O and N 2 increase as a function of temperature as their reactions with graphite become more favorable.

Advanced Reactors↗
DOE OSTI2020

An intra-node HPC network architecture with nanosecond-scale photonic switches

We propose a single-stage network architecture for intra-node connectivity that makes use of nanosecond-scale photonic switches. Although buffering at the switch points is of vital importance for complex multi-stage networks, this is not the case for smaller-scale single-stage networks where the end nodes are located only one hop apart. By limiting the buffering to the end points, the proposed architecture manages to minimize the required electro-optic and opto-electronic conversions, leading in this way to both low end-to-end latency and better energy efficiency. Combining these advantages with nanosecond-scale switching times can allow for high-throughput operation even for frequent switch reconfigurations. The performance of the proposed architecture is evaluated via discrete-event simulations for a wide range of synthetic-traffic cases. The simulation results show that high-throughput operation of ≥90% can be achieved even for small message sizes, i.e., 32 KB for all-to-all communication and 2 KB for uniform random traffic, at a data rate of 400 Gb/s and a switch reconfiguration time of ≤72 ns. Moreover, if 100-ns reconfiguration times are achievable as opposed to 150-ns, then for the all-to-all traffic case a 16% and 28% reduction in completion time can be achieved for message sizes of 8 KB and 1 KB, respectively. In a forthcoming era of optically interfaced processors and accelerators, nanosecond-scale photonic switches appear as a highly promising solution for keeping up with the intra-node bandwidth scaling due to their high-bandwidth, low-latency and fast-switching capabilities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

This paper proposes a machine-learning (ML)-aided cognitive approach for effective bandwidth reconfiguration in optically interconnected datacenter/high-performance computing (HPC) systems. The proposed approach relies on a Hyper-X-like architecture augmented with flexible-bandwidth photonic interconnections at large scales using a hierarchical intra/inter-POD photonic switching layout. We first formulate the problem of the connectivity graph and routing scheme optimization as a mixed-integer linear programming model. A two-phase heuristic algorithm and a joint optimization approach are devised to solve the problem with low time complexity. Then, we propose an ML-based end-to-end performance estimator design to assist the network control plane with intelligent decision making for bandwidth reconfiguration. Numerical simulations using traffic distribution profiles extracted from HPC applications traces as well as random traffic matrices verify the accuracy performance of the ML design estimator ( < <#comment/> 9 % <#comment/> error) and demonstrate up to 5 × <#comment/> throughput gain from the proposed approach compared with the baseline Hyper-X network using fixed all-to-all intra/inter-portable data center interconnects.

Chen, Xiaoliang (ORCID:0000000278056237)↗
DOE OSTI2022

Sim2Ls: FAIR simulation workflows and data

Just like the scientific data they generate, simulation workflows for research should be findable, accessible, interoperable, and reusable (FAIR). However, while significant progress has been made towards FAIR data, the majority of science and engineering workflows used in research remain poorly documented and often unavailable, involving ad hoc scripts and manual steps, hindering reproducibility and stifling progress. We introduce Sim2Ls (pronounced simtools) and the Sim2L Python library that allow developers to create and share end-to-end computational workflows with well-defined and verified inputs and outputs. The Sim2L library makes Sim2Ls , their requirements, and their services discoverable, verifies inputs and outputs, and automatically stores results in a globally-accessible simulation cache and results database. This simulation ecosystem is available in nanoHUB, an open platform that also provides publication services for Sim2Ls , a computational environment for developers and users, and the hardware to execute runs and store results at no cost. We exemplify the use of Sim2Ls using two applications and discuss best practices towards FAIR simulation workflows and associated data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

UAS Edge Computing of Energy Infrastructure Damage Assessment

Energy infrastructure assessments are needed within 72 hours of natural disasters, and previous data collection methods have proven too slow. We demonstrate a scalable end-to-end solution using a prototype unmanned aerial system that performs on-the-edge detection, classification (i.e., damaged or undamaged), and geo-location of utility poles. The prototype is suitable for disaster response because it requires no local communication infrastructure and is capable of autonomous missions. Collections before, during, and after Hurricane Ida in 2021 were used to test the system. The system delivered an F1 score of 0.65 operating with a 2.7 s/frame processing speed with the YOLOv5 large model and an F1 score of 0.55 with a 0.48 s/frame with the YOLOv5 small model. Geo-location uncertainty in the bottom half of the frame was ~8 m, mostly driven by error in camera pointing measurement. With additional training data to improve performance and detect additional types of features, a fleet of similar drones could autonomously collect actionable post-disaster data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Stacked filters: learning to filter by structure

We present Stacked Filters, a new probabilistic filter which is fast and robust similar to query-agnostic filters (such as Bloom and Cuckoo filters), and at the same time brings low false positive rates and sizes similar to classifier-based filters (such as Learned Filters). The core idea is that Stacked Filters incorporate workload knowledge about frequently queried non-existing values. Instead of learning, they structurally incorporate that knowledge using hashing and several sequenced filter layers, indexing both data and frequent negatives. Stacked Filters can also gather workload knowledge on-the-fly and adaptively build the filter. We show experimentally that for a given memory budget, Stacked Filters achieve end-to-end query throughput up to 130x better than the best alternative for a workload, either query-agnostic or classifier-based filters, and depending on where data is (SSD or HDD).

Computer Science↗
DOE OSTI2020

Hindsight logging for model training

In modern Machine Learning, model training is an iterative, experimental process that can consume enormous computation resources and developer time. To aid in that process, experienced model developers log and visualize program variables during training runs. Exhaustive logging of all variables is infeasible, so developers are left to choose between slowing down training via extensive conservative logging, or letting training run fast via minimalist optimistic logging that may omit key information. As a compromise, optimistic logging can be accompanied by program checkpoints; this allows developers to add log statements post-hoc, and "replay" desired log statements from checkpoint---a process we refer to as hindsight logging. Unfortunately, hindsight logging raises tricky problems in data management and software engineering. Done poorly, hindsight logging can waste resources and generate technical debt embodied in multiple variants of training code. In this paper, we present methodologies for efficient and effective logging practices for model training, with a focus on techniques for hindsight logging. Our goal is for experienced model developers to learn and adopt these practices. To make this easier, we provide an open-source suite of tools for Fast Low-Overhead Recovery (flor) that embodies our design across three tasks: (i) efficient background logging in Python, (ii) adaptive periodic checkpointing, and (iii) an instrumentation library that codifies hindsight logging for efficient and automatic record-replay of model-training. Model developers can use each flor tool separately as they see fit, or they can use flor in hands-free mode, entrusting it to instrument their code end-to-end for efficient record-replay. Our solutions leverage techniques from physiological transaction logs and recovery in database systems. Evaluations on modern ML benchmarks demonstrate that flor can produce fast checkpointing with small user-specifiable overheads (e.g. 7%), and still provide hindsight log replay times orders of magnitude faster than restarting training from scratch.

Computer Science↗
DOE OSTI2024

Spatialyze: A Geospatial Video Analytics System with Spatial-Aware Optimizations

Videos that are shot using commodity hardware such as phones and surveillance cameras record various metadata such as time and location. We encounter suchgeospatial videoson a daily basis and such videos have been growing in volume significantly. Yet, we do not have data management systems that allow users to interact with such data effectively. In this paper, we describe Spatialyze, a new framework for end-to-end querying of geospatial videos. Spatialyze comes with a domain-specific language where users can construct geospatial video analytic workflows using a 3-step, declarative,build-filter-observeparadigm. Internally, Spatialyze leverages the declarative nature of such workflows, the temporal-spatial metadata stored with videos, and physical behavior of real-world objects to optimize the execution of workflows. Our results using real-world videos and workflows show that Spatialyze can reduce execution time by up to 5.3×, while maintaining up to 97.1% accuracy compared to unoptimized execution.

Computer Science↗
DOE OSTI2023

Utah FORGE 3-2417: Fiber-Optic Geophysical Monitoring of Reservoir Evolution - Workshop Presentation

This is a presentation on the Fiber-Optic Geophysical Monitoring of Reservoir Evolution at the Utah FORGE Milford Site project by Rice University, presented by Prof. Jonathan Ajo-Franklin. The project's objective was to develop an end-to-end fiber-optic sensing approach for EGS to track the initial zone of fracture creation, zones of connected mechanically compliant fractures, and zones of flowing fractures linking the injection/production well pair. This approach would also be used to integrate data into an improved thermo-hydro-mechanical model (THM). This presentation was featured in the Utah FORGE R&D Annual Workshop on September 8, 2023. The workshop provided a valuable opportunity to explore the progress made in each of the 17 Research and Development projects funded under Solicitation 2020-1 which aim to enhance our understanding of the crucial factors influencing the development of Enhanced Geothermal Systems (EGS) reservoirs and resources.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Utah FORGE 3-2417: Fiber-Optic Geophysical Monitoring of Reservoir Evolution - 2024 Annual Workshop Presentation

This is a presentation on the Fiber-Optic Geophysical Monitoring of Reservoir Evolution by Rice University, presented by Jonathan Ajo-Franklin. This video slide presentation discusses the development of an end-to-end fiber-optic sensing approach for EGS to track the (1) initial zone of fracture creation, (2) zones of connected mechanically compliant fractures, (3) zones of flowing fractures, and (4) the integration of the data into an improved THM model. This presentation was featured in the Utah FORGE R&D Annual Workshop on August 13, 2024.

15 GEOTHERMAL ENERGY↗
DOE OSTI2026

2020 Can Do Colorado E-Bike Mini Pilot Program Study

### The Colorado Energy Office conducted a mini pilot program study as part of the Can Do Colorado initiative, providing e-bikes to 13 low-income participants. The program aimed to encourage energy-efficient transportation during the COVID-19 pandemic as transit services were reduced and people were concerned about exposure. The insights garnered from this small-scale pilot study informed the design of a full-scale, 2-year pilot in locations across Colorado. For more information about the mini pilot program, see NLR's [Preliminary Results Report](https://www.nlr.gov/docs/fy21osti/79657.pdf). Micromobility options such as e-bikes offer a solution for improving energy efficiency for short-distance trips, especially in urban areas. Pedal-assist e-bikes use an electric motor and battery to help power the bike. The motor amplifies the power behind each pedal stroke, augmenting the energy you put into the bike. #### Data Collection Agency The Colorado Energy Office conducted the survey. #### Survey Methodology Participants in the program received a Momentum LaFree E+ e-bike (Class 1) and accessories at no cost and manually submitted travel data and feedback for 3 months using the CanBikeCo App. The smartphone app, developed in partnership with NLR, used a customized version of the [NLR OpenPATH platform](https://www.nlr.gov/transportation/openpath.html). #### Survey Records and Data Survey records include a total of 13 participants. This dataset contains 3 months of end-to-end, multimodal travel data manually submitted via smartphone app by 13 low-income essential workers in the greater Denver area. The data includes distance, mode (e.g., e-bike, car, transit), trip purpose, and demographic information.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

2021–2022 Can Do Colorado E-Bike Full-Scale Pilot Program Study

In 2021–2022, the Colorado Energy Office conducted a full-scale pilot program study on e-bike usage as part of the Can Do Colorado initiative, providing e-bikes to low-income participants across the state. A [2020 mini pilot program study](https://www.nlr.gov/transportation/secure-transportation-data/tsdc-2020-can-do-colorado-e-bike-pilot-program.html) informed the full-scale study. Both studies used pedal-assist e-bikes, which feature an electric motor and battery to help power the bike. The motor amplifies the power behind each pedal stroke, augmenting the energy you put into the bike. #### Data Collection Agency The Colorado Energy Office conducted the study in partnership with local organizations in Adams and Broomfield counties (Smart Commute Metro North), Boulder (Community Cycles), Durango (Four Corners Office for Resource Efficiency), Fort Collins (City of Fort Collins), Pueblo (Pueblo County), and Vail (Town of Vail). #### Survey Methodology Program participants received an e-bike and accessories at no cost and manually submitted travel data and feedback via the CanBikeCO smartphone app. Developed in partnership with NLR, the app used a customized version of the open-source [NLR OpenPATH platform](https://www.nlr.gov/transportation/openpath.html). #### Survey Records and Data Survey records include 170 participants. The six datasets contain up to 18 months of partially automated travel diaries, combining sensed and surveyed travel behavior data—patterns of multimodal, end-to-end, individual human mobility—as well as demographic information from participants. The number of e-bike trips and e-bike miles traveled per location are 1,560 and 4,179 for Adams and Broomfield counties; 8,481 and 27,000 for Boulder; 2,815 and 6,307 for Durango; 3,483 and 7,080 for Fort Collins; 4,022 and 14,887 for Pueblo, and 1,206 and 3,3361 for Vail.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

2022 Bull E-Bike Pilot Program Study

In 2022, the City of Durham conducted an e-bike pilot program study to learn more about how electric bikes (e-bikes) could improve the transportation experience in the "Bull City" (a.k.a., Durham, North Carolina). The study used pedal-assist e-bikes, which feature an electric motor and battery to help power the bike. The motor amplifies the power behind each pedal stroke, augmenting the energy you put into the bike. #### Data Collection Agency The City of Durham's Transportation Department conducted the study. #### Survey Methodology Program participants used electric-assist e-bikes for at least 4 weeks between August and November 2022 in exchange for sharing information about their experiences, including tracking their travel via a smartphone app. In addition to the e-bike, participants received maintenance support along with a helmet, bike lock, and other accessories. Data collection was enabled by the [NLR OpenPATH platform](https://www.nlr.gov/transportation/openpath.html). #### Survey Records and Data Survey records include a total of 76 participants. The dataset contains 3 months of partially automated travel diaries, combining sensed and surveyed travel behavior data—patterns of multimodal, end-to-end, individual human mobility—as well as demographic and socioeconomic information from participants. The number of total trips was 6,488, the number of e-bike trips was 2,183, and the number of e-bike miles traveled was 5,450.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

2023 University of Puerto Rico at Mayagüez National Institute for Congestion Reduction Study

# 2023 University of Puerto Rico at Mayagüez National Institute for Congestion Reduction Study The 2023 University of Puerto Rico at Mayagüez National Institute for Congestion Reduction (NCIR) Study provided insight into the travel patterns and associated energy consumption of participants. Study results helped researchers identify opportunities for the development of policies that could incentivize the use of alternative modes of travel such as transit and micromobility. Such travel modes reduce congestion by reducing the miles traveled by privately owned vehicles in urban and rural areas. The [National Institute for Congestion Reduction](https://nicr.usf.edu/) provides multimodal congestion reduction strategies through real-world deployments that leverage advances in technology, big data science, and innovative transportation options to optimize the efficiency and reliability of the transportation system for all users. ## Data Collection Agency The University of Puerto Rico at Mayagüez conducted the study. ## Survey Methodology The study was conducted in Spanish. Data collection was enabled via the open-source [NREL OpenPATH platform](https://www.nrel.gov/transportation/openpath). The resulting dataset consists of partially automated travel diaries—combining sensed and surveyed data reflecting patterns of multimodal, end-to-end, individual human mobility—as well as demographic and socioeconomic information from the 17 participants. ## Survey Records, Data, and Documentation Study records include 17 participants. The total number of trips was 458 and total non-air-miles traveled was approximately 1,469.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗