Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stream processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM↗
DOE OSTI2024

Designing and prototyping extensions to the Message Passing Interface in MPICH

As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. Here, in this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Integrated low-temperature PVC and polyolefin upgrading

Polyolefins and their chlorinated derivatives such as polyvinyl chloride (PVC) are among the most prevalent plastics in global production and waste streams. Traditional waste-to-energy methods such as incineration and pyrolysis, as well as most chemical upcycling methods for PVC utilization, require thorough, high-temperature dechlorination to prevent the release of toxic chlorinated compounds. Here, we present here a strategy for upgrading discarded PVC into chlorine-free fuel range hydrocarbons and hydrogen chloride in a single-stage process catalyzed by chloroaluminate ionic liquids. This approach offsets endothermic dechlorination and carbon-carbon bond cleavage with exothermic alkylation and hydrogen transfer by isobutane or isopentane in a low-temperature tandem process. The light isoalkanes are available from refinery processes and partly from recycling of the product stream. This process is suitable for handling real-world mixed and contaminated PVC and polyolefin waste streams.

Zhang, Wei [East China Normal Univ. (ECNU), Shangh↗
DOE OSTI2026

rustpix

rustpix is a high-performance, open-source Rust library with first-class Python bindings (via PyO3) for processing pixel-detector data in neutron imaging. It targets time-stamping detectors such as Timepix3 (TPX3) at ORNL's Spallation Neutron Source (VENUS beamline), where each detected neutron deposits charge across a cluster of pixels within a very high-rate event stream (96M+ hits/sec). rustpix parses TPX3 event data in parallel using memory-mapped I/O, offers four interchangeable clustering algorithms (ABS adjacency-based search, DBSCAN, graph/union-find connected components, and a parallel grid method), and extracts weighted, super-resolved centroids to produce neutron-event lists. A streaming architecture lets it process files larger than available memory. rustpix is distributed as a pip-installable Python package (with NumPy integration), Rust crates, a command-line tool, and an interactive GUI; it writes HDF5, Apache Arrow, and CSV; and it is designed to extend to TPX4 and other detector types. Released as open-source under the MIT License.

Zhang, Chen [Oak Ridge National Laboratory (ORNL),↗
DOE OSTI2025

Real-time data processing for serial crystallography experiments

We report the use of streaming data interfaces to perform fully online data processing for serial crystallography experiments, without storing intermediate data on disk. The system produces Bragg reflection intensity measurements suitable for scaling and merging, with a latency of less than 1 s per frame. Our system uses the CrystFEL software in combination with the ASAP::O data framework. In a series of user experiments at PETRA III, frames from a 16 megapixel Dectris EIGER2 X detector were searched for peaks, indexed and integrated at the maximum full-frame readout speed of 133 frames per second. The computational resources required depend on various factors, most significantly the fraction of non-blank frames ('hits'). The average single-thread processing time per frame was 242 ms for blank frames and 455 ms for hits, meaning that a single 96-core computing node was sufficient to keep up with the data, with ample headroom for unexpected throughput reductions. Further significant improvements are expected, for example by binning pixel intensities together to reduce the pixel count. We discuss the implications of real-time data processing on the `data deluge' problem from recent and future photon-science experiments, in particular on calibration requirements, computing access patterns and the need for the preservation of raw data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Cell-free bioelectrocatalytic platform for carbon dioxide reduction (Final Technical Report)

The University of Minnesota (UMN) EcoSynBio Team aimed to develop a cell-free, enzyme-based platform for the electro- biocatalytic conversion of CO2 into formate as a platform chemical for further upgrading. This type of bio electrocatalytic process delivers a clean product stream without the need for extensive separation from the electrolyte as in electrochemical synthesis and microbial processes. The reduction reaction is catalyzed by metal-dependent formate dehydrogenases (mFDHs) that are capable of efficient electrocatalytic CO2 reduction without the need of costly co-factors. The development of an efficient, scalable electrobiocatalytic process with high total turnover numbers and viable space time yields, however, was not without its challenges. The UM team has developed a protein-based scaffolding system that facilitates enzyme stabilization and attachment to electrodes along with electron transfer. Yet, although FDHs are highly promising enzymes for cell-free, electrobiochemical CO2 reduction, they are also greatly understudied and especially for applications in electrocatalysis. The UM team used the best described mFDH from Clostridium as its benchmark system and spent significant time and effort in attempting to replicate published data and finally, redesigned a recombinant production system for proper metal co-factor incorporation. The UM team has also identified a small set of new enzyme homologs from extreme microorganisms with superior stabilities that have yielded initial structural data for further engineering. In addition, a new bioelectrocatalytic reactor system has been developed that can be 3D printed and used for enzyme attachment to electrodes. In summary the project has generated critical basic information for the further development of this class of enzymes for the electricity driven reduction of CO2 into formate as platform chemical for upgrading into various other chemicals, including fuels.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Batch Extraction Studies to Evaluate Trace Element Behavior in PUREX Conditions

The multilab Intentional Forensics Venture is working to identify which stable elements (i.e., taggants) at trace concentrations relative to U would persist throughout the nuclear fuel cycle in a voluntary fuel tagging scheme. A taggant would provide the nuclear forensics community with a “barcode” to help identify nuclear materials found outside of regulatory control. A portion of this project was focused on reprocessing effects and determining which, if any, elements would coextract with U(VI) in standard Pu–U reduction extraction (PUREX) conditions. Elements with a propensity to coextract could, in theory, be used as taggants from a PUREX perspective. Although retention is not a performance requirement, the taggant signature would need to partition predictably from the U stream after the PUREX process to maintain forensic utility. This report documents results from several batch extraction studies with numerous trace elements from HNO 3 (1.5–5 M), with and without U(VI), into 30% tri-n-butyl phosphate (TBP) in kerosene. Extraction and back-extraction tests were used to evaluate nearly 60 elements in surrogate conditions for PUREX, and distribution coefficients (i.e., D-values) for most species were <0.1, indicating few species are likely to co-extract with U through PUREX. Additional studies are needed to optimize sample volumes and dilutions to dial in these low D-values. The D-values (D) were determined for several of the more promising elements, including Re and Se. Ultimately, we conclude that only a limited number of the ~ 60 elements investigated are extractable in the U stream of PUREX, based on measured D values, meaning most candidate elemental taggants would likely be lost at this stage of the nuclear fuel cycle, even when considering a range of acid concentrations.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2025

Machine Learning for Automated Weld Quality Monitoring and Control

Resistance Spot Welding (RSW) is a critical process in the automotive industry, valued for its cost-effectiveness, short cycle time, and robustness. However, achieving consistent high-quality joints remains challenging due to the complex interplay of various factors, like materials, processes, and manufacturing uncertainties, etc. Under the collaborative project between Oak Ridge National Laboratory (ORNL) and General Motors (GM), we have developed a robust and expansible machine learning (ML) framework aimed at enhancing quality control in RSW. By harnessing the power of machine learning, we have developed the ability to ensure every aspect of the welding process, from the initial process design stage to the final weld joint quality. The framework operates by analyzing a variety of data streams, including in-line process signals, process parameters, materials, and postprocessed weld joint data. Through this analysis, the models have been trained to detect deviations from optimal quality standards, leveraging their ability to identify signature data patterns and anomalies within in-line signals and construct complex correlations between these signals and weld quality parameters. Meanwhile, the machine learning framework is designed to adapt to a variety of materials, including high strength steels and aluminum alloys, etc. Its flexible architecture facilitates the incorporation of diverse data sources and features, enabling precise modeling and prediction across a broad range of material properties and weld quality variables. The expansible ML frameworks represent a promising transformation in weld quality monitoring and control, empowering industry to achieve high levels of efficiency, consistency, and reliability in manufacturing.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

Machine Learning for Automated Weld Quality Monitoring and Control

Resistance Spot Welding (RSW) is a critical process in the automotive industry, valued for its cost-effectiveness, short cycle time, and robustness. However, achieving consistent high-quality joints remains challenging due to the complex interplay of various factors, like materials, processes, and manufacturing uncertainties, etc. Under the collaborative project between Oak Ridge National Laboratory (ORNL) and General Motors (GM), we have developed a robust and expansible machine learning (ML) framework aimed at enhancing quality control in RSW. By harnessing the power of machine learning, we have developed the ability to ensure every aspect of the welding process, from the initial process design stage to the final weld joint quality. The framework operates by analyzing a variety of data streams, including in-line process signals, process parameters, materials, and postprocessed weld joint data. Through this analysis, the models have been trained to detect deviations from optimal quality standards, leveraging their ability to identify signature data patterns and anomalies within in-line signals and construct complex correlations between these signals and weld quality parameters. Meanwhile, the machine learning framework is designed to adapt to a variety of materials, including high strength steels and aluminum alloys, etc. Its flexible architecture facilitates the incorporation of diverse data sources and features, enabling precise modeling and prediction across a broad range of material properties and weld quality variables. The expansible ML frameworks represent a promising transformation in weld quality monitoring and control, empowering industry to achieve high levels of efficiency, consistency, and reliability in manufacturing.

42 ENGINEERING↗
DOE OSTI2025

CO2 upgrading into bioproducts using a two-step abiotic–biotic system

The valorization of CO2 to chemicals beyond C1-2 products is receiving significant interest; however, the direct electrosynthesis of Cn molecules (n > 4) remains a challenge. Here, we present a two-step abiotic-biotic system for upgrading CO2 into the biopolymer, poly(3-hydroxybutyrate). In the electrolysis system, CO2 is converted into C2 oxygenates using a Cu-Ag tandem electrocatalyst. The electrolysis process generates a liquid stream containing ~ 200 mM acetate in a bio-compatible electrolyte. This electrosynthesized acetate is then fed to a bioreactor, where the substrate is upgraded by Cupriavidus necator to biopolymer with a maximum rate of 32 ± 3.5 mg L-1 h-1. We further demonstrate the purification of the resulting biopolymer into a powder. The high productivity of the abiotic-biotic system demonstrates its feasibility for sustainable chemical manufacturing.

CO2 upgrading↗
DOE OSTI2026

Continental-Scale Controls on Hyporheic Respiration Revealed by Knowledge-Guided Machine Learning

Hyporheic zone sediments regulate organic matter turnover and in-stream respiration, yet controls on sediment respiration remain poorly constrained across heterogeneous river networks, limiting prediction of stream metabolism and carbon processing at continental scales. Here, we integrate observations from ~90 river corridors across the United States in the WHONDRS consortium with a knowledge-guided machine learning (KGML) framework that couples thermodynamic rate theory with machine learning to identify dominant controls on hyporheic respiration. Diagnostic analyses show that organic matter concentration and thermodynamic favorability define an upper bound on respiration potential, whereas biological catalytic capacity and physical accessibility jointly govern realized respiration rates through interaction effects. To represent unmeasurable accessibility constraints, we use the mechanistic model as a scaffold for KGML, allowing machine learning to target residual structure not explained by process theory. This hybrid framework improves predictive skill relative to both the mechanistic model alone and fully data-driven models while preserving interpretability. These results indicate that variability in hyporheic respiration is largely mechanistically structured and demonstrate how integrating process theory with explainable AI enhances predictive performance while enabling scalable synthesis of river corridor observations.

Zheng, Jianqiu↗
DOE OSTI2024

Diaspora: Resilience-enabling services for science from HPC to edge

Scientific applications of interest to DOE must increasingly engage distributed resources (e.g., instruments, remote computers, data stores, edge devices) and deliver more stringent levels of service (e.g., uninterrupted processing of experiment data streams). In such systems, state is distributed and components can fail in many ways, often silently, making application resilience a major concern. Addressing the resilience needs of such applications requires methods for gaining knowledge of resources and applications and for translating that knowledge into action. We are working on addressing these needs in the context of multi-messenger astronomy, where detecting and responding to unusual transient events in multiple cosmic messengers (gravitational wave, electromagnetic, high- energy particles) from different instruments leads to a federated learning problem.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Investigating the impacts of used nuclear fuel direct dissolution on the radiolytic longevity of solvent and butyramide extractants

Removing the nitric acid (HNO3) dissolution step in used nuclear fuel (UNF) reprocessing would reduce the volume of radioactive waste streams generated, thereby, improving process efficiency. A promising strategy for this is the direct dissolution of UNF that has been pretreated by voloxidation into an organic solvent composed of specialized extractants and diluent. However, removal of the aqueous HNO3 phase from the envisioned reprocessing system has the potential to drastically change the suite of radiation-induced processes occurring, and thus, alter the longevity of proposed reagents. Furthermore, the impacts of fission product and transuranic metal ion complexation on the aforementioned radiation-induced processes is poorly understood, and yet can cause significant changes in radiolytic longevity. To bridge these knowledge gaps and support the continued development of direct dissolution strategies, we present an investigation into the impacts of direct dissolution conditions on the gamma radiation-induced degradation of N,N-di-(2-ethylhexyl) butyramide (DEHBA) and N,N-di-(2-ethylhexyl)isobutyramide (DEHiBA) ligands—candidate replacements for tributyl phosphate—in pre-equilibrated n-dodecane solvent in the presence and absence of envisioned loading amounts of uranium.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2025

Conceptual Model Testing Related to SDU 6 Drainwell Observations

From its inception in the early 1950s through the end of the Cold War in the early 1990s, the Savannah River Site (SRS) produced nuclear materials for national defense in five reactors. Additionally, irradiated reactor fuel and target tubes were dissolved in nitric acid to recover plutonium and uranium using the PUREX (Plutonium Uranium Reduction EXtraction) process. Liquid waste from these chemical separations processes was then stored onsite in 51 underground tanks. Eight waste storage tanks have been operationally closed (i.e. cleaned and grouted) and the remaining tanks hold a mixture of liquids, insoluble solids, and precipitated salts (SRMC-LWP-2022-00001), the latter generated by evaporating water from the liquid waste. Waste is currently being retrieved from tanks and separated into 1) high-radioactivity, low-volume, and 2) low-radioactivity, high-volume components, principally through the Salt Waste Processing Facility (SWPF) (SRMC-LWP-2023-00001). The former waste stream is vitrified in the Defense Waste Processing Facility (DWPF), stored onsite, and destined for offsite disposal in a deep geologic repository. The latter stream is mixed with dry cementitious materials in the Saltstone Production Facility (SPF) and the wet slurry placed in onsite Saltstone Disposal Units (SDUs) within the Saltstone Disposal Facility (SDF), where it hardens into a cement waste form termed saltstone. A low-infiltration surface cover system will be placed over the SDF at closure, where SDUs will then be in the subsurface post-closure (SRR-CWDA-2019-00001).

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

Stream Chemistry, Synoptic Surveys, East Fork Poplar Creek Watershed, TN, USA; April 2023 to February 2025

Impacts of developed land cover on stream chemistry can be difficult to discern from natural variability, particularly in carbonate watersheds where weathering of urban infrastructure and lithology generate similar signatures. We evaluated how spatial patterns of stream chemistry varied across perennial and non-perennial tributaries spanning an urban-to-forested gradient in a mid-order, carbonate-dominated watershed. This data package contains a processed and compiled summary of stream chemistry and properties obtained from 12 synoptic surveys of 54 stream sites across the East Fork Poplar Creek watershed located near Oak Ridge, TN, United States. The sites include non-perennial tributaries, perennial tributaries, and the main stem and span forested to urban (highly developed) land cover gradients. The data package includes the processed and flagged chemical data (WaDE_SynopticSummary_FinalChemistry), metadata describing data flagging and analysis (WaDE_SynopticSummary_Metadata), information about each site and its contributing subcatchment (WaDE_SynopticSummary_SiteInformation), and a comparison of instrument and field detection limits used to determine method detection limits for the study (WaDE_SynopticSummary_DetectionLimitComparison). Stream chemistry includes stream parameters measured in situ using multiparameter probes (dissolved oxygen, pH, specific conductance, temperature) and solutes including nutrients (nitrate, ammonium, soluble reactive phosphorus), dissolved organic carbon, dissolved inorganic carbon, major cations (calcium, magnesium, potassium, sodium), major anions (chloride, sulfate), and a broad suite of minor and trace elements.

EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SURFACE ↗
DOE OSTI2026

Glass Formulation Development for Al, Fe, and Na Phosphate HLW

The primary objective of the work described herein was to develop and identify HLW glass compositions and glass forming additive blends that achieve high waste loadings and processing rates for high phosphorus HLW streams while maintaining acceptable glass properties. Another objective was to determine the effect of phosphate form (aluminum, iron, sodium) on feed processing properties, glass production rates, and product quality while vitrifying a high phosphorus HLW stream. This was accomplished through a combination of crucible-scale tests, vertical gradient furnace tests, and confirmation tests on a DM100 melter system.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

Techno-Economic, Feasibility, and Life Cycle Analysis of Renewable Propane: 2025 Update

To clarify the current and future landscape for renewable propane (RP) production, this work evaluates the value proposition of recovering RP from existing and planned hydroprocessed esters and fatty acids (HEFA) biorefineries and surveys emerging technologies under development or deployment. HEFA biorefineries co-produce a propane-rich fuel gas stream, normally used to meet HEFA process heat requirements, from which propane can be recovered and sold to create an additional revenue stream alongside liquid transportation fuels such as renewable diesel (RD) and sustainable aviation fuel (SAF). This report updates and extends a 2022 analysis of RP recovery from HEFA facilities by escalating capital and operating costs to 2024 prices, incorporating recent policy developments (including the Section 45Z Clean Fuel Production Credit), evaluating RP recovery for both RD- and SAF-focused HEFA facilities at two scales (3,000 and 75,000 barrels per day of feedstock), and quantifying the impact of RP recovery on HEFA liquid-fuel carbon intensity (CI) and associated tax credits using the 45ZCF-GREET model. For a 3,000 BPD RD-focused HEFA facility, approximately 3.5 million gallons per year (MGPY) of RP can be recovered; in this base case, the estimated payback period is 18 months based on the total installed cost of the RP recovery equipment and 36 months based on the total capital investment for the entire RP recovery project. The payback period is slightly shorter for the analogous SAF-focused configuration (approximately 4.3 MGPY RP). Sensitivity analysis shows that CAPEX magnitude, RP recovery plant scale, and CI-driven tax credit valuations are the dominant determinants of project viability. RP recovery may increase the CI of HEFA liquid fuels, which can reduce liquid-fuel tax credits (a key revenue stream for the HEFA biorefinery) and lengthen payback periods. However, RP recovery generally remains economically favorable across a wide range of plausible scenarios and market conditions. The report also summarizes emerging pathways that could expand future RP supply.

09 BIOMASS FUELS↗
DOE OSTI2026

High waste loading glass formulation development for High-Mn HLW

One of the primary objectives of the work described herein was to develop and identify HLW glass compositions and glass forming additive blends that achieve high waste loadings and processing rates for high manganese HLW streams while maintaining acceptable glass properties. Another objective was to determine the effect of the form of manganese (MnO, MnO2, MnCO3) on feed processing properties, glass production rates, and product quality while vitrifying a high manganese HLW stream. This was accomplished through a combination of crucible-scale tests, vertical gradient furnace tests, and confirmation tests on a DM100 melter system.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗