Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

It is widely known that neural networks (NNs) are universal approximators of continuous functions. However, a less known but powerful result is that a NN with a single hidden layer can accurately approximate any nonlinear continuous operator. This universal approximation theorem of operators is suggestive of the structure and potential of deep neural networks (DNNs) in learning continuous operators or complex systems from streams of scattered data. Here, in this work, we thus extend this theorem to DNNs. We design a new network with small generalization error, the deep operator network (DeepONet), which consists of a DNN for encoding the discrete input function space (branch net) and another DNN for encoding the domain of the output functions (trunk net). We demonstrate that DeepONet can learn various explicit operators, such as integrals and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations. We study different formulations of the input function space and its effect on the generalization error for 16 different diverse applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A lightweight, user-configurable detector ASIC digital architecture with on-chip data compression for MHz X-ray coherent diffraction imaging

Today, most X-ray pixel detectors used at light sources transmit raw pixel data off the detector ASIC. With the availability of more advanced ASIC technology nodes for scientific application, more digital functionalities from the computing domains (e.g., compression) can be integrated directly into a detector ASIC to increase data velocity. In this paper, we describe a lightweight, user-configurable detector ASIC digital architecture with on-chip compression which can be implemented in 130 nm technologies in a reasonable area on the ASIC periphery. In addition, we present a design to efficiently handle the variable data from the stream of parallel compressors. The architecture includes user-selectable lossy and lossless compression blocks. The impact of lossy compression algorithms is evaluated on simulated and experimental X-ray ptychography datasets. This architecture is a practical approach to increase pixel detector frame rates towards the continuous 1 MHz regime for not only coherent imaging techniques such as ptychography, but also for other diffraction techniques at X-ray light sources.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

[Data Set] East Fork Poplar Creek Sonde Data at Kilometer 16.2 Water Year 2020

This data set reports stream water temperature, specific conductance, pH, dissolved oxygen percent saturation, dissolved oxygen concentration, and turbidity for East Fork Poplar Creek at a location approximately 16.2 kilometers upstream from the mouth of the creek.

temperature, pH, specific conductance, dissolved o↗
DOE OSTI2026

Vulcan-Forge: Architecture and Design of a Multi-Modal Forensic Analysis Plugin for CALDERA

Forge and VULCAN together describe an open-architecture cybersecurity analysis ecosystem that unifies forensic artifact processing, detection engineering, and vulnerability intelligence within integrated platforms. Forge operates as a plugin for MITRE CALDERA, ingesting diverse evidence formats—including EVTX, PCAP/PCAPNG, CSV, JSON, YAML, XML, binaries, and archives—to construct a unified artifact graph enriched with severity scoring, TLP classification, and audit trails. It provides subsystems for artifact parsing, streaming structured-data visualization, NetworkMiner-based packet inspection, PE/.NET binary analysis, and LLM-assisted triage and rule generation, with outputs validated against CCCS-YARA and pySigma schemas. VULCAN complements this by serving as a cybersecurity analyst platform that integrates a Neo4j knowledge graph, Qdrant vector retrieval, SSVC-based triage, and a local LLM to deliver CVE intelligence and forensic analysis through a multi-source ingest pipeline drawing from NVD, CISA KEV, EPSS, MITRE ATT&CK, and CAPEC. Together, they bridge structured threat intelligence with automated forensic analysis and detection workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Systems, methods, and devices for failure detection of one or more energy storage devices

An energy storage device management system can include a management portion for charging/discharging an energy storage device and an ultrasound interrogation portion for passing ultrasound energy through the energy storage device during charge/discharge cycles. A memory stores a stream of capture data instances derived from ultrasound energy exiting the energy storage device and baseline ultrasound data instances corresponding with the energy storage device during normal charging/discharging thereof. A processor can compare each capture data instance with the baseline ultrasound data and detect abnormal operating states of the energy storage device. A warning system can issue a notification when abnormal operating states are detected.

Kowalski, Jeffrey A.↗
DOE OSTI2023

ADCP Upsample

This R script takes reads the csv file of stream-wise velocity data and generates a new time series of with comparable but distinct noise using the Langevin method and library. It simulates a time series measured at 1/10 the sampling rate by calculating a moving average of 10, though other sampling rates can be calculated by adjusting the moving average window. A stationary signal is calculated by taking the difference of the original signal and the moving average, then normalizing by the standard deviation of the moving average. The stationary signal is used to calculate drift and diffusion coefficients. Those coefficients are used to generate a new stationary signal. The new stationary signal is un-normalized and added to the moving average (or new equivalent signal) to create a new up-sampled velocity time-series.

Green, Nathan↗
DOE OSTI2024

Data and scripts associated with a manuscript on a meta-analysis synthesizing stream biogeochemical response to wildfires across space and time (v2)

This data package is associated with the publication “Catchment characteristics modulate the influence of wildfires on nitrate and dissolved organic carbon in lotic systems across space and time: A meta-analysis” submitted to Global Biogeochemical Cycles (Cavaiani et al. 2025). This study uses meta-analytical techniques to evaluate the effect of wildfire on in-stream responses in burned and unburned watersheds. The study aims to provide additional insight into the range of responses and net influences that wildfires have on hydro-biogeochemistry across broad spatial scales, burn extents, and the persistence of water-quality change. This study compiles data and metadata from 18 total publications that includes 1) surface water geochemistry data (dissolved organic carbon; nitrate), 2) climate classifications, 3) year of the wildfire, 4) the time lag between when the fire occurred and when the sampling occurred, and 5) study design of the publication. In total, this meta-analysis draws data that spans 8 climate guilds, 3 biomes, 62 watersheds, and 20 unique wildfires. See Sites_meta_data.csv for citations of the papers used in this meta-analysis. All R scripts and the associated data can also be found on GitHub at This data package was originally published in March 2024. It was updated in April 2025 (v2; new and modified files). See the change history section in the readme for more details. This data package contains five primary folders that include the following: (1) inputs; (2) output for analysis; (3) initial plots; (4) R scripts; and (5) GIS data. The data package also contains a data dictionary (dd) that provides column header definitions and a file-level metadata (flmd) file that describes every file. The “inputs” folder contains a list of all publications identified during the formal web search and an indication of whether each publication was included in the final analysis. Additionally, it includes site-level metadata, catchment characteristics, and GIS data for all publications included in the final analysis. The “Output_for_analysis” folder contains all data frames and figures generated from each R script used for additional data analysis. The “initial_plots” folder includes all exploratory figures that will be included in a supplemental and figures that will be submitted with the manuscript for publication. The “R_scripts” folder contains the scripts that perform all the data manipulations, statistical analyses, and plots. The “gis_data” folder includes shape files for each fire included in this meta-analysis. This data package contains the following file types: csv, pdf, jpeg, cpg, dbf, prj, shp, shp.ea.iso.xml, shp.iso.xml, shx.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

National Energy Water Treatment & Speciation (NEWTS): A Water & Critical Mineral Database and Dashboard

The scarcity of water resources, the need for beneficial water reuse, and the challenges of wastewater treatment are becoming increasingly pressing in economic, social, and environmental domains. Addressing these concerns requires effective treatment strategies to manage wastewater streams and tackle environmental and economic issues. Furthermore, the recovery of critical minerals from the waste streams associated with energy production holds the promise of offsetting treatment costs and securing local sources of valuable minerals. However, relevant data on these waste streams are dispersed and challenging to locate. The process of ingesting such data into modeling software often involves multiple steps, requiring data restructuring to meet software-input requirements. The non-standardized reporting of water data makes data aggregation and reformatting a time-consuming process. Additionally, essential attributes necessary for modeling water treatment and mineral scale formation are frequently missing. Moreover, data gaps vary depending on the region of interest. Consequently, there is a pressing need for high-quality energy-water composition data that can be easily imported into water chemistry modeling software. To address this need, the National Energy Technology Laboratory has created the National Energy Water Treatment and Speciation (NEWTS) Database and Dashboard—a free online tool catering to community leaders and water researchers. NEWTS facilitates a comprehensive understanding of the composition of energy-related wastewater streams in the United States. The datasets provide detailed concentrations and speciation of major and minor aqueous compounds in energy-related wastewater streams, including power plant leachate, acid mine drainage, brackish water, and oil and gas produced water across the United States. Many of the aqueous species are critical minerals (Li, REEs) in high demand to modernize the world’s energy infrastructure. Many of the datasets also contain volumetric flow-rates needed to model the treatment and reuse scenarios in advanced aqueous chemistry software programs. The NEWTS Database and Dashboard offer public access to hitherto challenging-to-access datasets, presented in a standardized format that is tailored for easy input into aqueous chemistry modeling software. By performing the work needed to transform dispersed, disparate data sources into unified, model-ready datasets, NEWTS serves as an essential resource in advancing water treatment research and sustainable water resource management.

produced water management↗
DOE OSTI2025

Developing Fluorescence-Based Sensors to Support Rare Earth Element Separation

Rare earth elements (REEs) are essential to most renewable energy technologies. Unfortunately, as we transition to sustainable energy production, the demand for REEs is rapidly growing well beyond current rates of production. As a result, novel means of efficient, scalable, and easily adaptable methods for processing primary and recycle feedstocks are needed. Development and integration of sensors for highly selective in-line monitoring can support more efficient design and testing of such novel separation processes, as well as more cost-effective deployment of those separation flowsheets. Work here will explore the application of fluorescence spectroscopy, a highly sensitive and selective technique, to quantify multiple lanthanides in complex mixtures including known interferents or quenching agents. Results include identification of the optimal excitation wavelength and the limit of detection of various rare earth elements as well as the performance of data-science-based quantification approaches in streams where “unknowns” are present. Overall, the data science tools in conjunction with optical sensor data were able to quantify analytes in the presence of other lanthanides which can be anticipated in the actual industrial stream. Here we include characterization of lanthanides in a microfluidic device similar to those used in new process development. This study demonstrates the capability of utilizing fluorescence spectroscopy to quantify analytes in a complicated solution matrix, suggesting this is a successful approach for in-line monitoring to optimize the separation efficiency in an industrial stream.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Analysis of DWPF Sludge Batch 6 (Macrobatch 7): Pour Stream Glass Samples

The Defense Waste Processing Facility (DWPF) began processing Sludge Batch 6 (SB6), also referred to as Macrobatch 7 (MB7), in June 2010. SB6 is a blend of the heel of Tank 40 from Sludge Batch 5 (SB5), H-Canyon Np transfers and SB6 that was transferred to Tank 40 from Tank 51.1 SB6 was processed using Frit 418. Sludge is received into the DWPF Chemical Processing Cell (CPC) and is processed through the Sludge Receipt and Adjustment Tank (SRAT) and Slurry Mix Evaporator Tank (SME). The treated sludge slurry is then transferred to the Melter Feed Tank (MFT) and fed to the melter. During processing of each sludge batch, the DWPF is required to take at least one glass sample to meet the objectives of the Glass Product Control Program (GPCP) and to complete the necessary Production Records so that the final glass product may be disposed of at a Federal Repository. The DWPF requested various analyses of radioactive glass samples obtained from the melter pour stream during processing of SB6 as well as reduction/oxidation (REDOX) analysis of MFT samples to determine the impact of Argon bubbling. Sample analysis followed the Task Technical and Quality Assurance Plan (TTQAP) and an Analytical Study Plan (ASP). Four Pour Stream (PS) glass samples and two MFT slurry samples were delivered to the Savannah River National Laboratory (SRNL) from the DWPF. Table 1-1 lists the sample information for each pour stream glass sample. SB6 PS3 (S03472) was selected as the official pour stream sample for SB6 and full analysis was requested. This report details the visual observations of the as-received SB6 PS No.3 glass sample as well as results for the chemical composition, Product Consistency Test (PCT), radionuclide content, noble metals, and glass density. REDOX results will be provided for all four pour stream samples and vitrified samples of MFT-558 and MFT-568A. Where appropriate, data from other pour stream samples will be provided.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

From the Fire: A Deeper Look at the Phoenix Stream

We use 6 yr of data from the Dark Energy Survey to perform a detailed photometric characterization of the Phoenix stellar stream, a 15° long, thin, dynamically cold, low-metallicity stellar system in the Southern Hemisphere. We use natural splines, a nonparametric modeling technique, to simultaneously fit the stream track, width, and linear density. This updated stream model allows us to improve measurements of the heliocentric distance (17.4 ± 0.1 (stat.) ±0.8 (sys.) kpc) and distance gradient (−0.009 ± 0.006 kpc deg$^{−1}$) of Phoenix, which corresponds to a small change of 0.13 ± 0.09 kpc in heliocentric distance along the length of the stream. We measure linear intensity variations on degree scales, as well as deviations in the stream track on ∼2° scales, suggesting that the stream may have been disturbed during its formation and/or evolution. We recover three peaks and one gap in linear intensity along with fluctuations in the stream track. Compared to other thin streams, the Phoenix stream shows more fluctuations and, consequently, the study of Phoenix offers a unique perspective on gravitational perturbations of stellar streams. We discuss possible sources of perturbations to Phoenix, including baryonic structures in the Galaxy and dark matter subhalos.

79 ASTRONOMY AND ASTROPHYSICS↗