Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59

HIDES (Hybrid Intrusion Detection for Energy Systems) [SWR-19-65]

Hybrid Intrusion Detection for Energy Systems (HIDES) is a conglomerate detector that incorporates three major components: (1) Signature-based detection that involves information technology (IT) related anomaly detection rules, (2) Behavior-based detection that utilizes specially crafted rulesets for detecting abnormal behavior regarding the power system's supervisory control and data acquisition (SCADA) communication streams, and (3) Learning-based detection that combines SCADA communication with phasor measurements to identify otherwise non-detectible attack vectors. The software itself interconnects intrusion detection engine (IDE) enabling communications between the generated OT and IT alerts with phasor measurements for the learning-based detection system. The resulting detection system can detect a wide array of IT, OT, and physical attacks in a real-time setting.

Rivera, Joshua↗

Using High Performance Computing to Understand Roles of Labile and Nonlabile U(VI) on Hanford 300 Area Plume Longevity

Evolution of a hexavalent uranium [U(VI)] plume at the Hanford 300 Area bordering the Columbia River is investigated to evaluate the roles of labile and nonlabile forms of U(VI) on the longevity of the plume. A high fidelity, three-dimensional, field-scale, reactive flow and transport model is used to represent the system. Richards equation coupled to multicomponent reactive transport equations are solved for times up to 100 years taking into account rapid fluctuations in the Columbia River stage resulting in pulse releases of U(VI) into the river. The peta-scale computer code PFLOTRAN developed under a DOE SciDAC-2 project is employed in the simulations and executed on ORNL's Cray XT5 supercomputer Jaguar. Labile U(VI) is represented in the model through surface complexation reactions and its nonlabile form through dissolution of metatorbernite used as a surrogate mineral. Initial conditions are constructed corresponding to the U(VI) plume already in place to avoid uncertainties associated with the lack of historical data for the waste stream. The cumulative U(VI) flux into the river is compared for cases of equilibrium and multirate sorption models and for no sorption. The sensitivity of the U(VI) flux into the river on the initial plume configuration is investigated. The presence of nonlabile U(VI) was found to be essential in explaining the longevity of the U(VI) plume and the prolonged high U(VI) concentrations at the site exceeding the EPA MCL for uranium.

99 GENERAL AND MISCELLANEOUS↗

Land Surface Modeling 2.0 for agricultural climate change impact assessments

Focus Area: This white paper addresses DOE AI4ESP focus area #2 by providing a sketch blueprint for a next generation, hybrid AI/process-based global Land Surface Modeling (LSM) framework to improve projections of climate change impacts on the land surface system including agriculture and the hydrological cycle. Science Challenge: Climate change impacts on agriculture are highly uncertain: -50% to +150% global production changes for major grains under high-end climate change, for example. The core challenge in land surface system predictability is the large gap between the scale at which the relevant biological processes act and the scale at which the risks need to be assessed. Conventional empirical and process-based modeling approaches are insufficient. A new multi-scale modeling paradigm which employs AI/ML to learn from new streams of remote sensing data and targeted ’gene-to-global’ simulations is needed.

54 ENVIRONMENTAL SCIENCES↗

Colorado State University Extension Industrial Assessment Center

Since its inception in 1984, the Colorado State University Industrial Assessment Center has performed industrial assessments at more than 720 manufacturing facilities in Colorado, Montana, Nebraska, Nevada, New Mexico, North Dakota, South Dakota, Utah, and Wyoming. From 2017 to 2019 there was a funding gap and the IAC shutdown. In 2020 through DOE extension funding the University relaunched the IAC as an extension center in order to provide assessments to underserved areas. Under this award, the CSU Industrial Assessment Center (IAC) was rebuilt with the help of student employees and the director. The CSU team experienced difficulty as the program was in the process of being restarted right as the 2020 pandemic hit. Nonetheless, the CSU IAC was instrumental in providing energy assessments to manufacturers in Colorado and Wyoming during the period of performance of 09/2019 – 12/2022. The Department of Energy's Industrial Assessment Centers (IACs) provide a valuable service to small and medium-sized manufacturers seeking to optimize their operations. These university-based centers offer no-cost, on-site assessments conducted by engineering faculty and students, analyzing energy consumption, production processes, and waste streams. They utilize advanced data acquisition systems to record operational data, and then use the data to create assessment recommendations. These recommendations form the foundation of the comprehensive energy report. The comprehensive report delivers actionable recommendations for enhancing energy efficiency, reducing waste, and reducing greenhouse gas emissions and improving productivity, often identifying significant cost savings. Furthermore, IACs facilitate access to implementation grants, enabling the businesses to readily adopt these improvements. This program not only strengthens individual businesses but also contributes to national goals of training the next generation of energy experts, as well as increasing industrial competitiveness and reduced environmental impact.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗

Waveform Simulation Framework: User Manual with Tutorials

This manuscript describes the Waveform Simulation Framework (WSF), a Python-based framework that provides a unified, programmable interface for generating synthetic seismograms for applications such as seismic array design, method development, and special event analysis. WSF standardizes how users define sources, receivers, and velocity models while abstracting simulator-specific configuration details, enabling workflows that are largely independent of the underlying numerical engine. The document provides installation guidance and tutorial-driven examples for three WSF simulator wrappers—WSF PyFK, WSF SW4, and WSF SPECFEM2D—illustrating end-to-end workflows from forward waveform simulation to common post-processing tasks (e.g., visualization and backprojection) using consistent data products (e.g., ObsPy Stream objects and SAC files).

97 MATHEMATICS AND COMPUTING↗

Coherent Structures at Ion Scales in Fast Solar Wind: Cluster Observations

We investigate the nature of magnetic turbulent fluctuations, around ion characteristic scales, in a fast solar wind stream, by using Cluster data. Contrarily to slow solar wind, where both Alfvénic (δb{sub ⊥}≫δb{sub ∥}) and compressive (δb{sub ∥}≫δb{sub ⊥}) coherent structures are observed, the turbulent cascade of fast solar wind is dominated by Alfvénic structures, namely, Alfvén vortices, with a small and/or finite compressive part, with the presence also of several current sheets aligned with the local magnetic field. Several examples of vortex chains are also recognized. Although an increase of magnetic compressibility around ion scales is observed also for fast solar wind, no strongly compressive structures are found, meaning that the nature of the slow and fast winds is intrinsically different. Multispacecraft analysis applied to this interval of fast wind indicates that the coherent structures are almost convected by the flow and aligned with the local magnetic field, i.e., their normal is perpendicular to B, which is consistent with a two-dimensional turbulence picture. Understanding intermittency and the related generation of coherent structures could provide a key insight into the nonlinear energy transfer and dissipation processes in magnetized and collisionless plasmas.

79 ASTRONOMY AND ASTROPHYSICS↗

ADARA

The ADARA software is a collection of programs which gather neutron scattering data from multiple detector banks, multiplex the data into a single stream and distribute that stream to analysis, reduction and translation utilities in real-time.

Miller, Ross↗

Spectroscopic and physicochemical measurements for on-line monitoring of used nuclear fuel separation processes

Separation processes for used nuclear fuel are often complicated and challenging due to the high constraints in purity of the products and safeguards of the process streams. In order to achieve a safe, secure and efficient separation process, the liquid streams in the separation process require close monitoring. Due to the high radiation environment, sampling of the materials is difficult. Availability of a detection technique that is remote, non-destructive and can avoid time-delay caused by retrieving samples would be beneficial and could minimize the exposure to personnel and provide material accountancy to avoid diversion (non-proliferation). For example, Ultra Violet (UV), Visible (Vis), Near-Infrared (NIR) and Raman spectroscopy that detect and quantify elements present in used nuclear fuel, e.g. lanthanides, actinides and molecules such as nitrate, can be used. In this work, we have carried out NIR and Raman spectroscopy to study aqueous solutions composed of different concentrations of nitric acid, sodium nitrate, and neodymium at varied temperatures. A chemometric model for online monitoring based on the PLS-Toolbox (MATLAB) software has been developed and validated to provide chemical composition of process streams based on spectroscopic data. In conclusion, both of our NIR and Raman spectra were useful for H{sup +} and NO{sub 3} prediction, and only NIR was helpful for the Nd{sup 3+} prediction.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DUNE Database Development

The DUNE experiment will produce vast amounts of metadata, which describe the data coming from the read-out of the primary DUNE detectors. Various databases will make up the overall DB architecture for this metadata. ProtoDUNE at CERN is the largest existing prototype for DUNE and serves as a testing ground for - among other things - possible database solutions for DUNE. The subset of all metadata that is accessed during offline data reconstruction and analysis is referred to as ‘conditions data’ and it is stored in a dedicated database. As offline data reconstruction and analysis will be deployed on HTC and HPC resources, conditions data is expected to be accessed at very high rates. It is therefore crucial to store it in a granularity that matches the expected access patterns allowing for extensive caching. This requires a good understanding of the sources and use cases of conditions data. This contribution will briefly summarize the database architecture deployed at ProtoDUNE and explain the various sources of conditions data. We will present how the conditions data is retrieved and streamed from the databases and how it is handled to match expected access patterns.

43 PARTICLE ACCELERATORS↗

Sampling Low-Level Waste - Building a Program - 16101

Low level radioactive waste is routinely sampled and analyzed to demonstrate compliance with Title 10 Code of Federal Regulations Part 61. The sample results are frequently used to create scaling factors to relate the activity of hard to detect (HDT) radionuclides to gamma emitting activity if there is a correlation between the key and scaled radionuclide pair. The activity of gamma emitters can be used with these scaling factors to calculate activity of the low level waste generated in the plant. Another method is to use the sample results directly and scale the total activity to the volume or weight of the waste in the container. Either method assumes that the sample results are representative of the total waste in the container. This requires that the waste streams be accurately defined and that the samples contain radionuclides at ratios that are representative of the bulk waste. This is not always the case and deviations between the sample results and the actual radionuclide concentrations in the waste can lead to miss-classification of the waste resulting in regulatory non-compliance, increased waste disposal costs or both. A deliberate sampling methodology that considers the context of the waste generation and the inherent limitations in practical sampling methodologies will enhance the licensee's ability to demonstrate the analytical results from the sample are representative and defendable. A good sampling program starts with the establishment of waste streams that are consistent with both production mechanisms and plant systems. Once waste streams have been established, data quality objectives can be determined and then the methods for obtaining an appropriate sample can be developed along with the identification of sample points and sample schedules. Sometimes it may not be practical or even possible to obtain a truly representative sample in which case the results from the measurement of the sample must be evaluated within the context of the expected values and the factors inhibiting the collection of a truly representative sample. This paper will explore objectives of a sampling program in relation to the regulations and identify methods that can be used to obtain and evaluate sample data that can be defended as 'representative'. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Functional Verification for Endcap Concentrator ASICs in the High-Granularity Calorimeter Upgrade of CMS

The High-Granularity Calorimeter (HGCAL) of CMS will undergo a major upgrade during Long-Shutdown 3. The Endcap Concentrator (ECON) ASICs represent key elements in the readout chain, processing trigger (ECON-T) and data (ECON-D) streams from the HGCROC to the lpGBT. The ECONs will operate in a radiation environment with a High-Energy Hadron (HEH) flux of $3\cdot10^{6} cm^{-2}s^{-1}$. This contribution describes the Universal Verification Methodology (UVM)-based functional verification of the ECON ASICs focusing on the re-use of existing components to manage the complexity of the verification environment.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Grid Operator Analytics and Assessment Tools for Inverter- Based Resources Dominated Grid (GOAAT-IBR) Project Update

This presentation provides an update on the OPTIMA GOAAT project, with emphasis on the cloud-native data platform developed in-house to ingest, manage, and operationalize high-resolution power system data. Since our last NASPI presentation, accessible via OSTI ID #2671437, the project team advanced the design and deployment of a scalable architecture capable of handling both synchronized and non-synchronized streams, including PMU, point-on-wave (POW), COMTRADE, and SCADA data. These materials review the project status, recent progress, and key lessons learned. The core of the presentation examines the architecture and engineering of our cloud-native ingestion and data management platform. We then explain how pipelines were designed to collect, normalize, time-align, store, and serve heterogeneous data at scale. We will discuss design choices such as data models, streaming versus batch ingestion, storage tiers, and interoperability with analytics applications. Practical experiences with cloud-native technologies were shared during the event, including benefits, limitations, and integration challenges in a utility environment, along with methods used to improve performance, reduce latency, and optimize resource usage. The presentation also showcases user interface designs and visualization tools that convert raw measurements and analytics results into intuitive, actionable insights for operators and engineers. During the presentation examples were provided demonstrating how visualization, event views, and summarized analytics enhance situational awareness and support operational decision-making. These use cases illustrate how a well-designed data infrastructure can bridge the gap between high-volume measurements and practical grid operations.

Aminifar, Farrokh↗

A spatial characterization of the Sagittarius dwarf galaxy tidal tails

We measure the spatial density of F turnoff stars in the Sagittarius dwarf tidal stream, from Sloan Digital Sky Survey data, using statistical photometric parallax. We find a set of continuous, consistent parameters that describe the leading Sgr stream's position, direction, and width for 15 stripes in the north Galactic cap, and three stripes in the south Galactic cap. We produce a catalog of stars that has the density characteristics of the dominant leading Sgr tidal stream that can be compared with simulations. We find that the width of the leading (north) tidal tail is consistent with recent triaxial and axisymmetric halo model simulations. The density along the stream is roughly consistent with common disruption models in the north, but possibly not in the south. We explore the possibility that one or more of the dominant Sgr streams has been misidentified, and that one or more of the ''bifurcated'' pieces is the real Sgr tidal tail, but we do not reach definite conclusions. If two dwarf progenitors are assumed, fits to the planes of the dominant and ''bifurcated'' tidal tails favor an association of the Sgr dwarf spheroidal galaxy with the dominant southern stream and the ''bifurcated'' stream in the north. In the north Galactic cap, the best fit Hernquist density profile for the smooth component of the stellar halo is oblate, with a flattening parameter q = 0.53, and a scale length of r {sub 0} = 6.73. The southern data for both the tidal debris and the smooth component of the stellar halo do not match the model fits to the north, although the stellar halo is still overwhelmingly oblate. Finally, we verify that we can reproduce the parameter fits on the asynchronous MilkyWay@home volunteer computing platform.

79 ASTRONOMY AND ASTROPHYSICS↗

The understudied winter: Evidence of how precipitation differences affect stream metabolism in a headwater

Climate change is causing pronounced shifts during winter in the US, including shortening the snow season, reducing snowpack, and altering the timing and volume of snowmelt-related runoff. These changes in winter precipitation patterns affect in-stream freeze-thaw cycles, including ice and snow cover, and can trigger direct and indirect effects on in-stream physical, chemical, and biological processes in ~60% of river basins in the Northern Hemisphere. We used high-resolution, multi-parameter data collected in a headwater stream and its local environment (climate and soil) to determine interannual variability in physical, chemical, and biological signals in a montane stream during the winter of an El Niño and a La Niña year. We observed ~77% greater snow accumulation during the El Niño year, which caused the formation of an ice dam that shifted the system from a primarily lotic to a lentic environment. Water chemistry and stream metabolism parameters varied widely between years. They featured anoxic conditions lasting over a month, with no observable gross primary production (GPP) occurring under the ice and snow cover in the El Niño year. In contrast, dissolved oxygen and GPP remained relatively high during the winter months of the La Niña year. These redox and metabolic changes driven by changes in winter precipitation have significant implications for water chemistry and biological functioning beyond the winter. Our study suggests that as snow accumulation and hydrologic conditions shift during the winter due to climate change, hot-spots and hot-moments for biogeochemical processing may be reduced, with implications for the downstream movement of nutrients and transported materials.

54 ENVIRONMENTAL SCIENCES↗

Time Ordered Astrophysics Scalable Tools

This software package provides tools for astrophysical experiments which record data in the form of individual time streams from discrete detectors. TOAST provides tools from meta-data manipulation and job set up, I/O operation, telescope pointing reconstruction, and map-making. It also provides tools for constructing simulated observations.

Kisner, Theodore↗

Big Data, Big Solutions

Data—lots of data—generated in seconds and piling up on the internet, streaming and stored in countless databases. Big data is important for commerce, society and our nation’s security. Yet the volume, velocity, variety and veracity of data is simply too great for any single analyst to make sense of alone. It requires advanced, data-intensive computing. Simply put, data-intensive computing is the use of sophisticated computers to sort through mounds of information and present analysts with solutions in the form of graphics, scenarios, formulas, new hypotheses and more. This scientific capability is foundational to PNNL’s energy, environment and security missions. Senior Scientist and Division Director Bill Pike and his team are developing analytic tools that are used to solve important national challenges, including cyber systems defense, power grid control systems, intelligence analysis, climate change and scientific exploration.

97 MATHEMATICS AND COMPUTING↗

Using NEXRAD and Rain Gauge Precipitation Data for Hydrologic Calibration of SWAT in a Northeastern Watershed

The value of watershed-scale, hydrologic and water quality models to ecosystem management is increasingly evident as more programs adopt these tools to evaluate the effectiveness of different management scenarios and their impact on the environment. Quality of precipitation data is critical for appropriate application of watershed models. In small watersheds, where no dense rain gauge network is available, modelers are faced with a dilemma to choose between different data sets. In this study, we used the German Branch (GB) watershed (~50 km 2 ), which is included in the USDA Conservation Effects Assessment Project (CEAP), to examine the implications of using surface rain gauge and next-generation radar (NEXRAD) precipitation data sets on the performance of the Soil and Water Assessment Tool (SWAT). The GB watershed is located in the Coastal Plain of Maryland on the eastern shore of Chesapeake Bay. Stream flow estimation results using surface rain gauge data seem to indicate the importance of using rain gauges within the same direction as the storm pattern with respect to the watershed. In the absence of a spatially representative network of rain gauges within the watershed, NEXRAD data produced good estimates of stream flow at the outlet of the watershed. Three NEXRAD datasets, including (1)*non-corrected (NC), (2) bias-corrected (BC), and (3) inverse distance weighted (IDW) corrected NEXRAD data, were produced. Nash-Sutcliffe efficiency coefficients for daily stream flow simulation using these three NEXRAD data ranged from 0.46 to 0.58 during calibration and from 0.68 to 0.76 during validation. Overall, correcting NEXRAD with rain gauge data is promising to produce better hydrologic modeling results. Given the multiple precipitation datasets and corresponding simulations, we explored the combination of the multiple simulations using Bayesian model averaging.

54 ENVIRONMENTAL SCIENCES↗