Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Deducing neutron star equation of state parameters directly from telescope spectra with uncertainty-aware machine learning

Neutron stars provide a unique laboratory for studying matter at extreme pressures and densities. While there is no direct way to explore their interior structure, X-rays emitted from these stars can indirectly provide clues to the equation of state (EOS) of the superdense nuclear matter through the inference of the star's mass and radius. However, inference of EOS directly from a star's X-ray spectra is extremely challenging and is complicated by systematic uncertainties. The current state of the art is to use simulation-based likelihoods in a piece-wise method which relies on certain theoretical assumptions and simplifications about the uncertainties. It first infers the star's mass and radius to reduce the dimensionality of the problem, and from those quantities infer the EOS. Here we demonstrate a series of enhancements to the state of the art, in terms of realistic uncertainty quantification and a path towards circumventing the need for theoretical assumptions to infer physical properties with machine learning. We also demonstrate novel inference of the EOS directly from the high-dimensional spectra of observed stars, avoiding the intermediate mass-radius step. Our network is conditioned on the sources of uncertainty of each star, allowing for natural and complete propagation of uncertainties to the EOS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Hindered segmental dynamics in associative protein hydrogels studied by neutron spin-echo spectroscopy

Transient binding between associating macromolecules can cause qualitative changes to chain dynamics, including modes of conformational relaxation and diffusion, through tethering effects imparted by long-range connectivity. Here, in this work, the role of binding on short-time segmental dynamics in associative polymer gels is investigated by neutron spin-echo (NSE) measurements on a class of model artificial coiled-coil proteins with a systematically varied architecture, probing timescales of 0.1–130 ns, and length scales close to the molecular radius of gyration. The results illustrate effects of transient cross-linking on chain dynamics on different timescales, manifested in changes in segmental relaxation behavior with variations in strand length, chain concentration, and sticker distribution (endblock- vs midblock-functionalized). In all gels, a short-time cooperative diffusion mode is seen over all wave vectors, analogous to a semidilute solution, with no transitions seen at any known structural length scale. However, the diffusion coefficients are found to decrease with increasing junction density across all gels, with the strand length and number of stickers per chain in each gel appearing to play a relatively minor role. The slowing of cooperative diffusion with junction density contrasts with classical predictions of a greater restoring force for fluctuation dissipation due to the increased elasticity, suggesting additional effects of the coiled-coil junctions such as an enhancement in local viscosity that slows dynamics. Notably, the relaxation rates for all gels can be rescaled by the interjunction spacing inferred from small-angle neutron scattering, where they collapse onto a master curve suggestive of self-similar dynamics even in networks with different strand lengths and chain architectures. On long timescales (but shorter than the junction exchange time), a slowing of network relaxation is observed, resulting in a nondecaying plateau in the spin-echo amplitude attributed to a freezing of chain dynamics due to tethering. A characteristic length scale corresponding to the extent of dynamic fluctuations is estimated for each gel, which appears to be smaller than the interjunction spacing but similar to the correlation blob size of the overlapping strands. The results indicate an important role of transient binding on molecular-scale dynamics in associative polymer gels, even on timescales shorter than the junction exchange time, in addition to its effects on long-range self-diffusion previously observed.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Improving Subsurface Stress Characterization for Carbon Dioxide Storage Projects

Researchers at New Mexico Tech, Los Alamos National Laboratory (LANL), Sandia National Laboratory (SNL), and other collaborators are developing a methodology to improve characterization of stress in the subsurface by means of a model-based inversion of reservoir engineering data, time lapse seismic measurements, and microseismicity. Historically, various geophysical techniques have been used in efforts to understand the state of stress in the subsurface through direct and indirect imaging of stress sensitive features (faults and fractures) and observations of transient stress related observations (time variant elastic moduli and microseismicity). Standard direct and indirect techniques for seismic fault and fracture imaging suffer from detectability limits due to reliance on high data quality and multiplicity. Variations in effective elastic properties from inversion of high quality seismic (Vertical Seismic Profile (VSP)) data may be used to infer stress near the wellbore through integration with independent experimental characterization of the stress-velocity relationship. Given a sufficiently robust observation network, microseismic emissions may be inverted to characterize focal mechanisms which, together with supporting assumptions and constraints, inform estimates of in-situ stress. While each of these techniques contributes in part to the characterization of stress within limited spatial and temporal domains, no one method provides an unambiguous stress measurement or a predictive capability over a site scale spatial extent. Challenges associated with solution non-uniqueness, measurement ambiguity, and irregular sampling may be greatly minimized through combination of one or more of independent measurements within a common framework in which realistic geological, hydrodynamic, geomechanical, and seismological constituent process models may act as constraints.

02 PETROLEUM↗
DOE OSTI2021

Identifiability and predictability of integer- and fractional-order epidemiological models using physics-informed neural networks

Here we analyze a plurality of epidemiological models through the lens of physics-informed neural networks (PINNs) that enable us to identify time-dependent parameters and data-driven fractional differential operators. In particular, we consider several variations of the classical susceptible-infectious-removed (SIR) model by introducing more compartments and fractional-order and time-delay models. We report the results for the spread of COVID-19 in New York City, Rhode Island and Michigan states and Italy, by simultaneously inferring the unknown parameters and the unobserved dynamics. For integer-order and time-delay models, we fit the available data by identifying time-dependent parameters, which are represented by neural networks. In contrast, for fractional differential models, we fit the data by determining different time-dependent derivative orders for each compartment, which we represent by neural networks. We investigate the structural and practical identifiability of these unknown functions for different datasets, and quantify the uncertainty associated with neural networks and with control measures in forecasting the pandemic.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Simple and sophisticated models of Taylor’s cylinder impact

We present a study based on the simplest possible models for Taylor’s cylinder impact problem, in addition to examina-tion of using convolutional neural networks (CNNs) to map cylinder profiles to strength calibrations. We find that the approximate treatments of Taylor and Hawkyard compare well with hydrodynamic simulations using an equivalent assumption of constant flow stress. The CNN models prove to be well suited to successfully infer parameterizations of the Preston-Tonks-Wallace model of plastic deformation based on the deformed profile of an impacted cylinder.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Scaling Inference Using Triton to Accelerate Particle Physics at the LHC and DUNE

DUNE and the LHC experiments consist of unique and cutting-edge particle detectors that create massive, complex, and rich datasets with billions of events. They require sophisticated algorithms to reconstruct and interpret the data. Modern machine learning algorithms provide a powerful toolset to detect and classify particles, from familiar image processing convolutional neural networks to newer graph neural network architectures. A full reconstruction of these particle collisions requires novel approaches to handle the computing challenge of processing so much raw data. In a series of studies, physicists from Fermilab, CERN, and university groups explored how to accelerate their data processing using the Triton Inference Server.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Electrochemical Random-Access Memory: Progress, Perspectives, and Opportunities

Non-von Neumann computing using neuromorphic systems based on analogue synaptic and neuronal elements has emerged as a potential solution to tackle the growing need for more efficient data processing, but progress toward practical systems has been stymied due to a lack of materials and devices with the appropriate attributes. Recently, solid state electrochemical ion-insertion, also known as electrochemical random access memory (ECRAM) has emerged as a promising approach to realize the needed device characteristics. ECRAM is a three terminal device that operates by tuning electronic conductance in functional materials through solid-state electrochemical redox reactions. This mechanism can be considered as a gate-controlled bulk modulation of dopants and/or phases in the channel. Early work demonstrating that ECRAM can achieve nearly ideal analogue synaptic characteristics has sparked tremendous interest in this approach. More recently, the realization that electrochemical ion insertion can be used to tune the electronic properties of many types of materials including transition metal oxides, layered two-dimensional materials, organic and coordination polymers, and that the changes in conductance can span orders of magnitude has further attracted interest in ECRAM as the basis for analogue synaptic elements for inference accelerators as well as for dynamical devices that can emulate a wide range of neuronal characteristics for implementation in analogue spiking neural networks. At its core, ECRAM shares many fundamental aspects with rechargeable batteries, where ion insertion materials are used extensively for their ability to reversibly store charge and energy. Computing applications, however, present drastically different requirements: systems will require many millions of devices, scaled down to tens of nanometers, all while achieving reliable electronic-state tuning at scaled-up rates and endurances, and with minimal energy dissipation and noise. Further, in this review, we discuss the history, basic concepts, recent progress, as well as the challenges and opportunities for different types of ECRAM, broadly grouped by their primary mobile ionic charge carrier, including Li, protons, and oxygen vacancies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Computational design of highly signalling-active membrane receptors through solvent-mediated allosteric networks

Abstract Protein catalysis and allostery require the atomic-level orchestration and motion of residues and ligand, solvent and protein effector molecules. However, the ability to design protein activity through precise protein–solvent cooperative interactions has not yet been demonstrated. Here we report the design of 14 membrane receptors that catalyse G protein nucleotide exchange through diverse engineered allosteric pathways mediated by cooperative networks of intraprotein, protein–ligand and –solvent molecule interactions. Consistent with predictions, the designed protein activities correlated well with the level of plasticity of the networks at flexible transmembrane helical interfaces. Several designs displayed considerably enhanced thermostability and activity compared with related natural receptors. The most stable and active variant crystallized in an unforeseen signalling-active conformation, in excellent agreement with the design models. The allosteric network topologies of the best designs bear limited similarity to those of natural receptors and reveal an allosteric interaction space larger than previously inferred from natural proteins. The approach should prove useful for engineering proteins with novel complex protein binding, catalytic and signalling activities.

Chemistry↗
DOE OSTI2025

Neural network-based model of galaxy power spectrum: fast full-shape galaxy power spectrum analysis

ABSTRACT We present a neural network-based emulator for the galaxy redshift-space power spectrum that enables several orders of magnitude acceleration in the galaxy clustering parameter inference, while preserving 3$\sigma$ accuracy better than 0.5 per cent up to $k_{\mathrm{max}}$ = 0.25 $\, h\text{Mpc}^{-1}$ within Lambda-cold dark matter ($\Lambda$CDM) and around 0.5 per cent $w_0$–$w_a$CDM. Our surrogate model only emulates the galaxy bias-invariant terms of one-loop perturbation theory predictions, these terms are then combined analytically with galaxy bias terms, counter-terms, and stochastic terms in order to obtain the non-linear redshift-space galaxy power spectrum. This allows us to avoid any galaxy bias prescription in the training of the emulator, which makes it more flexible. Moreover, we include the redshift $z \in [0,1.4]$ in the training which further avoids the need for re-training the emulator. We showcase the performance of the emulator in recovering the cosmological parameters of $\Lambda$CDM by analysing the suite of 25 AbacusSummit simulations that mimic the Dark Energy Spectroscopic Instrument luminous red galaxies at $z=0.5$ and 0.8, together as the emission line galaxies at $z=0.8$. We obtain similar performance in all cases, demonstrating the reliability of the emulator for any galaxy sample at any redshift in $0 \lt z \lt 1.4$. We will make our emulator public at github repository.

Trusov, Svyatoslav (ORCID:0000000224146720)↗
DOE OSTI2023

Semi-supervised graph neural networks for pileup noise removal

Abstract The high instantaneous luminosity of the CERN Large Hadron Collider leads to multiple proton–proton interactions in the same or nearby bunch crossings (pileup). Advanced pileup mitigation algorithms are designed to remove this noise from pileup particles and improve the performance of crucial physics observables. This study implements a semi-supervised graph neural network for particle-level pileup noise removal, by identifying individual particles produced from pileup. The graph neural network is firstly trained on charged particles with known labels, which can be obtained from detector measurements on data or simulation, and then inferred on neutral particles for which such labels are missing. This semi-supervised approach does not depend on the neutral particle pileup label information from simulation, and thus allows us to perform training directly on experimental data. The performance of this approach is found to be consistently better than widely-used domain algorithms and comparable to the fully-supervised training using simulation truth information. The study serves as the first attempt at applying semi-supervised learning techniques to pileup mitigation, and opens up a new direction of fully data-driven machine learning pileup mitigation studies.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Sentinel

Network intrusion detection systems (NIDS) are commonplace in network security but they frequently employ algorithms that are computational demanding requiring hardware and software with significant power requirements. Two examples of such resource-intensive algorithms used for network security are regular expression matching and broader signature pattern matching which are commonly used in deep packet inspection (DPI). Network security algorithms that have large power requirements may be a challenge for low-power internet-of-things (IoT) environments, which generally lack the power resources to implement complex security measures like computationally expensive DPI at the edge. Furthermore, IoT environments incorporating 5G standalone networks have network latency constraints beyond just power that make DPI at the edge even more difficult. Programmable logic is ideally suited for machine learning inference for DPI because of its deep instruction level parallelism and single-cycle memory access. Machine learning approaches for DPI have been explored before using the programmable logic of field programmable gate arrays (FPGA) as a potential solution for NIDS approaches that would be power-suitable for IoT. However, those previous programmable logic NIDS approaches utilize either a supervised or unsupervised learning model. Sentinel utilizes the ensemble of these two machine learning approaches known as a semi-supervised approach which has shown promise in NIDS implementations. Sentinel provides a programmable logic implementation of a semi-supervised approach for DPI which operates at much lower power and latency than a GPU implementation with negligible loss of accuracy due to quantization through a logistic regressor.

Anderson, MatthewW [Idaho National Laboratory (INL↗
DOE OSTI2024

Graph Neural Network-based Tracking as a Service

Recent studies have shown promising results for track finding in dense environments using Graph Neural Network (GNN)-based algorithms. However, GNN-based track finding is computationally slow on CPUs, necessitating the use of coprocessors to accelerate the inference time. Additionally, the large input graph size demands a large device memory for efficient computation, a requirement not met by all computing facilities used for particle physics experiments, particularly those lacking advanced GPUs. Furthermore, deploying the GNN-based track-finding algorithm in a production environment requires the installation of all dependent software packages, exclusively utilized by this algorithm. These computing challenges must be addressed for the successful implementation of GNN-based track-finding algorithm into production settings. In response, we introduce a ``GNN-based tracking as a service'' approach, incorporating a custom backend within the NVIDIA Triton inference server to facilitate GNN-based tracking. This paper presents the performance of this approach using the Perlmutter supercomputer at NERSC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗
DOE OSTI2024

Inferring the Focal Depths of Small Earthquakes in Southern California Using Physics-Based Waveform Features

Determining the depths of small crustal earthquakes is challenging in many regions of the world, because most seismic networks are too sparse to resolve trade-offs between depth and origin time with conventional arrival-time methods. Precise and accurate depth estimation is important, because it can help seismologists discriminate between earthquakes and explosions, which is relevant to monitoring nuclear test ban treaties and producing earthquake catalogs that are uncontaminated by mining blasts. Here, we examine the depth sensitivity of several physics-based waveform features for ~8000 earthquakes in southern California that have well-resolved depths from arrival-time inversion. We focus on small earthquakes (2 < M L < 4) recorded at local distances (<150 km), for which depth estimation is especially challenging. We find that differential magnitudes (M w /M L –M c ) are positively correlated with focal depth, implying that coda wave excitation decreases with focal depth. We analyze a simple proxy for relative frequency content, Φ≡log 10 (M 0 )+3log 10 (f c ), and find that source spectra are preferentially enriched in high frequencies, or “blue-shifted,” as focal depth increases. Here, we also find that two spectral amplitude ratios Rg 0.5–2 Hz/Sg 0.5–8 Hz and Pg/Sg at 3–8 Hz decrease as focal depth increases. Using multilinear regression with these features as predictor variables, we develop models that can explain 11%–59% of the variance in depths within 10 subregions and 25% of the depth variance across southern California as a whole. We suggest that incorporating these features into a machine learning workflow could help resolve focal depths in regions that are poorly instrumented and lack large databases of well-located events. Some of the waveform features we evaluate in this study have previously been used as source discriminants, and our results imply that their effectiveness in discrimination is partially because explosions generally occur at shallower depths than earthquakes.

58 GEOSCIENCES↗
DOE OSTI2022

Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark

We present our development experience and recent results for the MLPerf Tiny Inference Benchmark on field-programmable gate array (FPGA) platforms. We use the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware codesign of optimized neural networks on FPGAs. We present the design and implementation process for the keyword spotting, anomaly detection, and image classification benchmark tasks. The resulting hardware implementations are quantized, configurable, spatial dataflow architectures tailored for speed and efficiency and introduce new generic optimizations and common workflows developed as a part of this work. The full workflow is presented from quantization-aware training to FPGA implementation. The solutions are deployed on system-on-chip (Pynq-Z2) and pure FPGA (Arty A7-100T) platforms. The resulting submissions achieve latencies as low as 20 $\mu$s and energy consumption as low as 30 $\mu$J per inference. We demonstrate how emerging ML benchmarks on heterogeneous hardware platforms can catalyze collaboration and the development of new techniques and more accessible tools.

Borras, Hendrik↗
DOE OSTI2023

Fast uncertainty estimates in deep learning interatomic potentials

Deep learning has emerged as a promising paradigm to give access to highly accurate predictions of molecular and material properties. A common short-coming shared by current approaches, however, is that neural networks only give point estimates of their predictions and do not come with predictive uncertainties associated with these estimates. Existing uncertainty quantification efforts have primarily leveraged the standard deviation of predictions across an ensemble of independently trained neural networks. This incurs a large computational overhead in both training and prediction, resulting in order-of-magnitude more expensive predictions. Here, we propose a method to estimate the predictive uncertainty based on a single neural network without the need for an ensemble. This allows us to obtain uncertainty estimates with virtually no additional computational overhead over standard training and inference. We demonstrate that the quality of the uncertainty estimates matches those obtained from deep ensembles. We further examine the uncertainty estimates of our methods and deep ensembles across the configuration space of our test system and compare the uncertainties to the potential energy surface. Finally, we study the efficacy of the method in an active learning setting and find the results to match an ensemble-based strategy at order-of-magnitude reduced computational cost.

Chemistry↗
DOE OSTI2020

Framework For Spatial Agricultural Crop Yield Prediction Model Development

This framework was developed to provide data preprocessing for spatiotemporal agricultural yield data and remote sensing data for modelling using artificial neural networks (ANNs) to predict subfield crop yield estimates. The software includes methods to train, validate, and test ANN models. It also include methods to infer on new remote sensing data.

Griffel, LloydM.↗