Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Parallelizing the Unpacking and Clustering of Detector Data for Reconstruction of Charged Particle Tracks on Multi-core CPUs and Many-core GPUs

We present results from parallelizing the unpacking and clustering steps of the raw data from the silicon strip modules for reconstruction of charged particle tracks. Throughput is further improved by concurrently processing multiple events using nested OpenMP parallelism on CPU or CUDA streams on GPU. The new implementation along with earlier work in developing a parallelized and vectorized implementation of the combinatoric Kalman filter algorithm has enabled efficient global reconstruction of the entire event on modern computer architectures. We demonstrate the performance of the new implementation on Intel Xeon and NVIDIA GPU architectures.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Multiscale drivers of extreme southern California flooding: ENSO, MJO, North Pacific jet, and atmospheric rivers

Extreme rainfall and flooding, driven by a powerful atmospheric river (AR) and a persistent Madden-Julian Oscillation (MJO), hit Southern California in February 2024 during the 2023–2024 El Niño, affecting over 10 million people. ARs are key contributors to extreme rainfall and flooding along the U.S. West Coast. Although the AR-MJO link has been documented, its spatio-temporal variability remains a major forecasting and risk-management challenge. Combining precipitation, stream gauge and demographic data, we quantify the physical drivers and population exposure to this extreme event. Leveraging a Lagrangian MJO precipitation tracking algorithm, we unravel the multiscale interactions responsible for the AR’s development. El Niño favored a large, long-lived MJO that interacted with the North Pacific Jet (NPJ) over more than three weeks. The MJO convective outflow modulated the NPJ by inducing negative potential vorticity advection along the tropopause. The ensuing NPJ extension and acceleration induced explosive cyclogenesis, whose AR-driven moisture transport resulted in extreme rainfall.

Atmospheric dynamics↗
DOE OSTI2021

Accounting for uncertainty in complex alluvial aquifer modeling by Bayesian multi-model approach

Alluvial aquifers by nature are complex caused by varied depositional environments. Developing a reliable groundwater model to represent an alluvial aquifer is non-trivial. Also, relying on a single best calibrated model may not be sufficient because of an inadequate choice of model parameter values. To better understand groundwater dynamics and improve model prediction reliability, this study presents a Bayesian multi-model uncertainty quantification (BMMUQ) framework to account for model parameter uncertainty in complex alluvial groundwater modeling. The methodology was applied to the agriculturally intensive Mississippi River alluvial aquifer (MRAA), Northeast Louisiana. An aquifer architecture was first constructed using 7,259 well logs in the MRAA area which covers three fluvial deposits (alluvium, braided-stream terrace, and braided-stream terrace-loess). A 12-layer MODFLOW model was then developed to address the alluvial aquifer complexity and well calibrated through a genetic algorithm. This study quantified model parameter uncertainty in hydraulic conductivity and specific storage of sand facies. Bayesian model averaging (BMA) with the Expectation Maximization (EM) algorithm was adopted to derive posterior model weights and head variances of 50 alternative conceptual groundwater flow models, and thereby obtains BMA ensemble model predictions instead of only relying on the best calibrated conceptual model. Overall, results show that an estimated around 950 million m3 of groundwater storage loss occurs in 2015 with respect to the beginning of 2004, due to high groundwater demand for irrigation in the MRAA area. Explicitly quantifying model uncertainty can produce more reliable groundwater level predictions from BMA ensemble model. The presented groundwater modeling framework improves our understanding of the MRAA and provides a valuable tool to assist agricultural water management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Cylinders and jets in crossflow: Wake formations as a result of varying geometric proximities

The combined flow physics of several canonical flow configurations is experimentally studied. Here, we analyze an array of jets issuing into a crossflow, then immediately navigating past an array of cylinders. This is achieved with a 2 × 3 triangular pattern of jets and symmetric cylinders at three jets to crossflow velocity ratios, enabling near-complete optical access of each jet, with velocities measured by time-resolved particle image velocimetry. Jet trajectories reveal that each configuration adheres to a power-law trend and that greater penetration is achieved by the downstream and confined jets compared to the more conventional upstream one. Recirculation regions of the upstream and downstream jets are nearly independent, with the confined jet encompassing regions of overlap with both. Turbulent statistics reveal the influence of geometric placement and velocity ratio on the time-averaged velocity, anisotropy, and Reynolds stresses incurred by each jet. Galilean decomposition utilizes a supplemental crossflow-only velocity field to delineate the influence of each jet's low- and high-pressure regions on the otherwise uniform stream. Proper orthogonal decomposition suggests that increased jet penetration decreases the number of modes required for truncation in the investigated spanwise plane. Vortex identification algorithms are applied to the reconstructed flow fields, reaffirming that with increasing velocity ratio, the jets generate vortices of their own in similar statistical formations as the cylinders. This investigation provides a foundation to aid future modeling efforts toward characterizing flow physics of importance in designing and passively controlling transverse jets with varying blockage proximities in a crossflow.

Kristo, Paul J. (ORCID:0000000225605175)↗
DOE OSTI2020

Dynamic Power Network State Estimation with Asynchronous Measurements

The operation of distribution networks is becoming increasingly volatile, due to fast variations of renewables and, hence, net-loading conditions. To perform a reliable state estimation under these conditions, this paper considers the case where measurements from meters, phasor measurement units, and distributed energy resources are collected and processed in real time to produce estimates of the state at a fast time scale. Streams of measurements collected in real time and at heterogenous rates render the underlying processing asynchronous, and poses severe strains on workhorse state estimation algorithms. In this work, a real-time state estimation algorithm is proposed, where data are processed on the fly. Starting from a regularized least-squares model, and leveraging appropriate linear models, the proposed scheme boils down to a linear dynamical system where the state is updated based on the previous estimate and on the measurement gathered from a few available sensors. The estimation error is shown to be always bounded under mild condition. Numerical simulations are provided to corroborate the analytical findings.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Trigger-DAQ and Slow Controls Systems in the Mu2e Experiment

The muon campus program at Fermilab includes the Mu2e experiment that will search for a charged-lepton flavor violating processes where a negative muon converts into an electron in the field of an aluminum nucleus, improving by four orders of magnitude the search sensitivity reached so far. Mu2e's Trigger and Data Acquisition System (TDAQ) uses otsdaq as its solution. Developed at Fermilab, otsdaq uses the artdaq DAQ framework and art analysis framework, under the-hood, for event transfer, filtering, and processing. otsdaq is an online DAQ software suite with a focus on flexibility and scalability, while providing a multi-user, web-based, interface accessible through the Chrome or Firefox web browser. The detector Read Out Controller (ROC), from the tracker and calorimeter, stream out zero-suppressed data continuously to the Data Transfer Controller (DTC). Data is then read over the PCIe bus to a software filter algorithm that selects events which are finally combined with the data flux that comes froma Cosmic Ray Ve to System (CRV). A Detector Control System (DCS) for monitoring, controlling, alarming, and archiving has been developed using the Experimental Physics and Industrial Control System (EPICS) Open Source Platform. The DCS System has also been itegrated into otsdaq. The installation of the TDAQ and the DCS systems in the Mu2e building is planned for 2021-2022, and a prototype has been built at Fermilab's Feynman Computing Center. We report here on the developments and achievements of the integration of Mu2e's DCS system into the online otsdaq software.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

A Block-Based Triangle Counting Algorithm on Heterogeneous Environments

Triangle counting is a fundamental building block in graph algorithms. In this article, we propose a block-based triangle counting algorithm to reduce data movement during both sequential and parallel execution. Our block-based formulation makes the algorithm naturally suitable for heterogeneous architectures. The problem of partitioning the adjacency matrix of a graph is well-studied. Our task decomposition goes one step further: it partitions the set of triangles in the graph. By streaming these small tasks to compute resources, we can solve problems that do not fit on a device. We demonstrate the effectiveness of our approach by providing an implementation on a compute node with multiple sockets, cores and GPUs. The current state-of-the-art in triangle enumeration processes the Friendster graph in 2.1 seconds, not including data copy time between CPU and GPU. Using that metric, our approach is 20 percent faster. When copy times are included, our algorithm takes 3.2 seconds. This is 5.6 times faster than the fastest published CPU-only time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Block-Based Triangle Counting Algorithm on Heterogeneous Environments

Triangle counting is a fundamental building block in graph algorithms. In this paper, we propose a block-based triangle counting algorithm to reduce data movement during both sequential and parallel execution. Our block-based formulation makes the algorithm naturally suitable for heterogeneous architectures. The problem of partitioning the adjacency matrix of a graph is well-studied. Our task decomposition goes one step further: it partitions the set of triangles in the graph. By streaming these small tasks to compute resources, we can solve problems that do not fit on a device. We demonstrate the effectiveness of our approach by providing an implementation on a compute node with multiple sockets, cores and GPUs. The current state-of-the-art in triangle enumeration processes the Friendster graph in 2.1 seconds, not including data copy time between CPU and GPU. Using that metric, our approach is 20 percent faster. When copy times are included, our algorithm takes 3.2 seconds. This is 5.6 times faster than the fastest published CPU-only time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Advanced Health Information Technology Analytic Framework and Application to Hazard Detection

Health Information Technology (HIT) aims to improve healthcare outcomes by organizing and analyzing various health-related data. With data accumulating at a staggering rate, the importance of real-time analytics has been increasing dramatically, shifting the focus of informatics from batch processing to streaming analytics. HIT is also facing unprecedented challenges in adapting to this new requirement and leveraging advanced IT technologies. This paper introduces a HIT data and compute platform that supports multi-granularity real-time analytics from heterogeneous data sources. The paper first identifies functional requirements and proposes a framework that satisfies the requirements using state-of-the-art big data technologies including Apache Kafka, Spark Structured Streaming Engine, and Delta Lake. To demonstrate its capability to support data analytics in multiple time granularities analytics, a statistical process control-based hazard detection algorithm has been implemented on top of the framework to detect unexpected hazards from order cancellation data of the Department of US Veterans Affairs (VA) in near real-time.

Kumar, Mohit↗
DOE OSTI2024

Unconventional Quantum Advantages for Computation (U-QuAC)

While quantum computing offers the promise of exponential advantages, limited quantum speedups are known, especially for practical applications. To open new avenues for quantum advantages, we propose Unconventional Quantum Advantages for Computation (U-QuACs), with respect to unconventional resources such as space (number of bits or quantum bits of memory required to solve a problem), accuracy of solution, communication, or energy consumption. We focus on space-efficient quantum algorithms, where we seek to design algorithms that solve a problem using much less space than the total size of the input. A natural setting in which space is critical is the streaming model of computation, where the input data arrives sequentially in pieces that must each be processed individually. Streaming is motivated by a variety of problems including analysis of internet traffic or social networks. We design the first exponential quantum space advantage for a natural streaming problem, which also constitutes the first quantum advantage for approximating a discrete optimization problem, albeit with respect to space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Substructure in the stellar halo near the Sun: II. Characterisation of independent structures

In an accompanying paper, we present a data-driven method for clustering in ‘integrals of motion’ space and apply it to a large sample of nearby halo stars with 6D phase-space information. The algorithm identified a large number of clusters, many of which could tentatively be merged into larger groups. The goal here is to establish the reality of the clusters and groups through a combined study of their stellar populations (average age, metallicity, and chemical and dynamical properties) to gain more insights into the accretion history of the Milky Way. To this end, we developed a procedure that quantifies the similarity of clusters based on the Kolmogorov–Smirnov test using their metallicity distribution functions, and an isochrone fitting method to determine their average age, which is also used to compare the distribution of stars in the colour–absolute magnitude diagram. Also taking into consideration how the clusters are distributed in integrals of motion space allows us to group clusters into substructures and to compare substructures with one another. We find that the 67 clusters identified by our algorithm can be merged into 12 extended substructures and 8 small clusters that remain as such. The large substructures include the previously known Gaia-Enceladus, Helmi streams, Sequoia, and Thamnos 1 and 2. We identify a few over-densities that can be associated with the hot thick disc and host a small metal-poor population. Especially notable is the largest (by number of member stars) substructure in our sample which, although peaking at the metallicity characteristic of the thick disc, has a very well populated metal-poor component, and dynamics intermediate between the hot thick disc and the halo. We also identify additional debris in the region occupied by Sequoia with clearly distinct kinematics, likely remnants of three different accretion events with progenitors of similar masses. Although only a small subset of the stars in our sample have chemical abundance information, we are able to identify different trends of [Mg/Fe] versus [Fe/H] for the various substructures, confirming our dissection of the nearby halo. We find that at least 20% of the halo near the Sun is associated to substructures. When comparing their global properties, we note that those substructures on retrograde orbits are not only more metal-poor on average but are also older. We provide a table summarising the properties of the substructures, as well as a membership list that can be used for follow-up chemical abundance studies for example.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Evolutionary NAS in Light of Model Stability for Accurate Continual Learning

Continual learning, the capability to learn new knowledge from streaming data without forgetting the previous knowledge, is a critical requirement for dynamic learning systems, especially for emerging edge devices such as self-driving cars and drones. However, continual learning is still facing the catastrophic forgetting problem. Previous work illustrate that model performance on continual learning is not only related to the learning algorithms but also strongly dependent on the inherited model, i.e., the model where continual learning starts. The better stability of the inherited model, the less catastrophic forgetting and thus, the inherited model should be elaborately selected. Inspired by this finding, we develop an evolutionary neural architecture search (ENAS) algorithm that emphasizes the Stability of the inherited model, namely ENAS-S. ENAS-S aims to find optimal architectures for accurate continual learning on edge devices. On CIFAR-10 and CIFAR-100, we present that ENAS-S achieves competitive architectures with lower catastrophic forgetting and smaller model size when learning from a data stream, as compared with handcrafted DNNs.

Du, Xiaocong↗
DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗
DOE OSTI2020

Paleo-Megadroughts and Abrupt Climate Changes in the Speleothem Records. Final report

This project is motivated by the speleothem isotope records in Asia, which show regional responses in the hydrologic cycle to different climate forcings. Speleothem isotopic records are typically interpreted in terms of local precipitation variations or monsoon intensity. Our study demonstrates that non-local processes also play an important role. We started this project to understand the regional difference in speleothem isotopic composition between the Last Glacial Maximum (LGM) and the present-day. The record in Southwest China showed greater depletion during the LGM compared to those in East China. Our modeling and analysis showed that speleothems record, in addition, large scale changes in atmospheric circulation and moisture transport and their subsequent impact on precipitation. We developed an algorithm to partition total precipitation according to their formation dynamics, namely into frontal and non-frontal precipitation, and showed that the two have different trends and hence different causal mechanism. We then focused our subsequent attention on circulation impacts on precipitation changes. We applied a machine learning algorithm to detect rainbands in the ERA-Interim reanalysis product, and showed that the seasonal migrations of the rainbands are tied to the seasonal migrations of the jet stream, in particular the northerlies of the jet meanders. These northerlies, in turn, are partly topographic Rossby waves excited as the upstream westerlies impinge on the Tibetan Plateau. The seasonal variations of these upstream westerlies thus contribute to the seasonal movements of the rainbands and regional precipitation changes. Our analysis of the modern precipitation isotope record further confirms the importance of jet stream changes in the isotopic variations and shows that isotope-enriched years have reduced summer seasonality, with less pronounced northward migration of the jet.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

How Dynamic Time Warping Can Assist Conventional Cross-correlation

Waveform cross-correlation is a sensitive phase-matched filtering technique that can detect seismic events for nuclear explosion monitoring. However, there are outstanding challenges with correlation detectors, most notably a direct dependence on the completeness of the waveform template library. To ameliorate these challenges, we investigate how dynamic time warping (DTW) may make waveform correlation more robust. DTW analyzes the differences between two time series and attempts to “warp” one time series relative to another in a recursive manner. We apply DTW to synthetic earthquake and recorded explosion templates to expand the capability of correlation detectors. We explore what conditions (e.g., source, station distance, frequency bands) and/or DTW algorithms generate stronger correlation scores. We show that DTW performs well on noisy signals and can dramatically improve the cross-correlation coefficient between a template and data-stream waveform. We conclude with recommendations on how to utilize DTW in nuclear monitoring detection.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2022

Substructure in the stellar halo near the Sun: I. Data-driven clustering in integrals-of-motion space

Context. Merger debris is expected to populate the stellar haloes of galaxies. In the case of the Milky Way, this debris should be apparent as clumps in a space defined by the orbital integrals of motion of the stars. Aims. Our aim is to develop a data-driven and statistics-based method for finding these clumps in integrals-of-motion space for nearby halo stars and to evaluate their significance robustly. Methods. We used data from Gaia EDR3, extended with radial velocities from ground-based spectroscopic surveys, to construct a sample of halo stars within 2.5 kpc from the Sun. We applied a hierarchical clustering method that makes exhaustive use of the single linkage algorithm in three-dimensional space defined by the commonly used integrals of motion energy E, together with two components of the angular momentum, L z and L ⊥ . To evaluate the statistical significance of the clusters, we compared the density within an ellipsoidal region centred on the cluster to that of random sets with similar global dynamical properties. By selecting the signal at the location of their maximum statistical significance in the hierarchical tree, we extracted a set of significant unique clusters. By describing these clusters with ellipsoids, we estimated the proximity of a star to the cluster centre using the Mahalanobis distance. Additionally, we applied the HDBSCAN clustering algorithm in velocity space to each cluster to extract subgroups representing debris with different orbital phases. Results. Our procedure identifies 67 highly significant clusters (> 3σ), containing 12% of the sources in our halo set, and 232 subgroups or individual streams in velocity space. In total, 13.8% of the stars in our data set can be confidently associated with a significant cluster based on their Mahalanobis distance. Inspection of the hierarchical tree describing our data set reveals a complex web of relations between the significant clusters, suggesting that they can be tentatively grouped into at least six main large structures, many of which can be associated with previously identified halo substructures, and a number of independent substructures. This preliminary conclusion is further explored in a companion paper, in which we also characterise the substructures in terms of their stellar populations. Conclusions. Our method allows us to systematically detect kinematic substructures in the Galactic stellar halo with a data-driven and interpretable algorithm. The list of the clusters and the associated star catalogue are provided in two tables available at the CDS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Reflection and transmission of electromagnetic pulses at a planar dielectric interface -- theory and quantum lattice simulations

There is considerable interest in the application of quantum information science to advance computations in plasma physics. A particular point of curiosity is whether it is possible to take advantage of quantum computers to speed up numerical simulations relative to conventional computers. Many of the topics in fusion plasma physics are classical in nature. In order to implement them on quantum computers it will require couching a classical problem in the language of quantum mechanics. Electromagnetic waves are routinely used in fusion experiments to heat a plasma or to generate currents in the plasma. The propagation of electromagnetic waves is described by Maxwell equations with an appropriate description of the plasma as a dielectric medium. Before advancing to the tensor dielectric of a magnetized plasma, this paper considers electromagnetic wave propagation in a one-dimensional inhomogeneous scalar dielectric. The classic theory of scattering of plane electromagnetic waves at a planar interface, separating two different dielectric media, leads to Fresnel equations for reflection and transmission coefficients. In contrast to plane waves, this paper is on the reflection and transmission of a spatially confined electromagnetic pulse. Following an analytical formulation for the scattering of a Gaussian pulse, it is deduced that the maximum transmission coefficient for a pulse is $\sqrt{n_2/n_1}$ times that for a plane wave; the incident and transmitted pulses propagating in dielectric media with refractive indices $n_1$ and $n_2$, respectively. The analytical theory is complemented by numerical simulations using a quantum lattice algorithm for Maxwell equations. The algorithm, based on the Riemann-Silberstein-Weber representation of the electromagnetic fields and expressed in term of qubits, is an interleaved sequence of entangling operators at each lattice site and unitary streaming operators which transmit information from one site to an adjacent lattice site. Besides substantiating results from the theory for Gaussian pulses, numerical simulations show their validity for non-Gaussian pulses. Apart from their time-asymptotic forms, the simulations display an interplay between the incident, reflected, and transmitted pulses in the vicinity of the transition region between two dielectric media.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Qubit lattice algorithm simulations of the scattering of a bounded two dimensional electromagnetic pulse from an infinite planar dielectric interface

Qubit lattice algorithm (QLA) simulations are performed for a two-dimensional spatially bounded pulse propagating onto a plane interface between two dielectric slabs. QLA is an initial value scheme that consists of a sequence of unitary collision and streaming operators, with appropriate potential operators, that recover Maxwell equations in inhomogeneous dielectric media to the second order in the lattice discreteness. For the case of total internal reflection, there is transient energy transfer into the second medium due to the evanescent fields as the Poynting unit vector of the pulse is rotated from its incident to reflected direction. Because of the finite spatial extent of the pulse, a self-consistent Goos–Hanchen-type displacement along the interface is found without imposing any explicit interface boundary conditions on the fields. For normal incidence, the standard Fresnel coefficients are recovered for appropriately averaged QLA fields. Energy is conserved at all times to seven significant figures.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗