Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Teaching a neural network to attach and detach electrons from molecules

Abstract Interatomic potentials derived with Machine Learning algorithms such as Deep-Neural Networks (DNNs), achieve the accuracy of high-fidelity quantum mechanical (QM) methods in areas traditionally dominated by empirical force fields and allow performing massive simulations. Most DNN potentials were parametrized for neutral molecules or closed-shell ions due to architectural limitations. In this work, we propose an improved machine learning framework for simulating open-shell anions and cations. We introduce the AIMNet-NSE (Neural Spin Equilibration) architecture, which can predict molecular energies for an arbitrary combination of molecular charge and spin multiplicity with errors of about 2–3 kcal/mol and spin-charges with error errors ~0.01e for small and medium-sized organic molecules, compared to the reference QM simulations. The AIMNet-NSE model allows to fully bypass QM calculations and derive the ionization potential, electron affinity, and conceptual Density Functional Theory quantities like electronegativity, hardness, and condensed Fukui functions. We show that these descriptors, along with learned atomic representations, could be used to model chemical reactivity through an example of regioselectivity in electrophilic aromatic substitution reactions.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Motion Dynamics of Motile Microbes in Pore-Networks and its Implications for Reactive Transport Processes

This report outlines new methods to improve simulations of microbial transport and microbially mediated reactions in porous media. A range of experimental, modeling, and machine learning tools are introduced to make these simulations faster, more reliable, and useful for real-world applications. At the microscopic level, the study investigates how different types of bacteria move through confined spaces. A new artificial intelligence tool called DeepTrackStat, is introduced to track motions dynamics as observed in videos of particles migrating through pore networks. This tool is especially helpful for studying fast-moving microbes and requires less computing power than traditional tracking methods. At larger scales, the research looks at how microbes and chemicals interact in zones where surface water and groundwater meet. To connect the small- and large-scale findings, the study presents a neural network model called STAMNet. This tool helps scale up detailed small-scale microbial motion behaviors to predict large-scale environmental changes more efficiently. By combining lab experiments, computer models, and artificial intelligence, the research presented supports smarter environmental decision-making, especially in bioremediation of contaminated groundwater and protection of water quality.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Adaptive Client Selection in Federated Learning: A Network Anomaly Detection Use Case

Federated Learning (FL) has become a ubiquitous approach for training machine learning models on decentralized data, addressing the myriad privacy concerns inherent in traditional centralized methods. However, the efficiency of FL depends on effective client selection and robust privacy preservation mechanisms. Inadequate client selection may lead to suboptimal model performance, while insufficient privacy measures risk exposing sensitive data. This paper proposes a client selection framework for FL that integrates differential privacy and fault tolerance. Our adaptive approach dynamically adjusts the number of selected clients based on model performance and system constraints, ensuring privacy through calibrated noise addition. We evaluate our method on a network anomaly detection use case using the UNSW-NB15 and ROAD datasets. Results show up to a 7% increase in accuracy and a 25% reduction in training time compared to FedL2P. Moreover, we highlight the trade-offs between privacy budgets and model performance, with higher privacy budgets reducing noise and improving accuracy. Our fault tolerance mechanism, while causing a slight performance drop, enhances robustness to client failures. Statistical validation using Mann-Whitney U tests confirms the significance of these improvements (p < 0.05).

Marfo, William [University of Texas at El Paso,Dep↗
DOE OSTI2023

Fission with Exotic Nuclei (Abbreviated Report)

Nuclear fission is a key mechanism involved in the synthesis of heavy elements in the Cosmos and is the primary explanation for the stability of superheavy elements. Nevertheless, our knowledge of fission remains extremely fragmented. Most experiments have been conducted only on a tiny number of stable actinide nuclei and are often incomplete, leading to gaps in our basic understanding of the process. For many radioactive isotopes, basic fission data such as the charge or mass distribution of the fragments is unknown. These gaps cannot always be filled by simulation alone. Common fission models contain too many free parameters and lack predictive power. In contrast, the fundamental theory of fission under development at LLNL is much more predictive, but its current computational cost is too high to be used extensively for data evaluations. A unique window of opportunity to resolve these limitations has recently opened: the U.S. nuclear science community is ramping up major experimental programs at the Facility for Rare Isotope Beams (FRIB, the DOE flagship facility in low-energy nuclear science), and techniques from machine learning have shown great potential to simplify the use of a fundamental, quantum-mechanical theory of fission. This project has two components. On the experimental side, we acquired and deployed at the HIGS facility a new dual Frisch-Grid ionization chamber to measure correlated fragment-mass, kinetic energy, and angular distributions of fission fragments from induced fission. This new device was used to perform measurements of charge, mass and total kinetic energy of fission fragments in the photofission of 238 U and eight gamma-ray beam energies between 6.2 and 13 MeV, which allowed extracting high-precision independent yields for this reaction. The device was also used to perform measurements of the same quantities in the neutron-induced fission of 234 U with monoenergetic beams of energy between 5 and 8 MeV. In parallel, we collaborated with a team at Commissariat à l’énergie atomique et aux énergies alternatives (CEA) to perform a series of measurements of fission yields in inverse kinematics for the two isotopes of 236 U and 240 Pu. The experiment took place at the Grand Accélérateur National d’Ions Lourds in France in June 2023. The deployment of the VAMOS spectrometer with a new array called PISTA allowed determining the excitation energy of the fissioning system within 1 Mega-electronvolts. The second component of the project involved using deep neural networks to build fast and reliable emulators of our current fission models. In an invited paper published in Frontier in Physics, we showed that autoencoders could successfully compress nuclear wavefunctions in nuclear density functional theory. We achieved a dimensionality reduction of the order of two orders of magnitude while keeping the error in the total energy to less than 0.01%. In a second paper submitted to Physical Review Letters in June 2023 with our collaborators at CEA, we showed that variational autoencoders can learn the collective degrees of freedom driving the fission process.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Exploring electron-beam induced modifications of materials with machine-learning assisted high temporal resolution electron microscopy

Directed atomic fabrication using an aberration-corrected scanning transmission electron microscope (STEM) opens new pathways for atomic engineering of functional materials. In this approach, the electron beam is used to actively alter the atomic structure through electron beam induced irradiation processes. One of the impediments that has limited widespread use thus far has been the ability to understand the fundamental mechanisms of atomic transformation pathways at high spatiotemporal resolution. Here, we develop a workflow for obtaining and analyzing high-speed spiral scan STEM data, up to 100 fps, to track the atomic fabrication process during nanopore milling in monolayer MoS 2 . An automated feedback-controlled electron beam positioning system combined with deep convolution neural network (DCNN) was used to decipher fast but low signal-to-noise datasets and classify time-resolved atom positions and nature of their evolving atomic defect configurations. Through this automated decoding, the initial atomic disordering and reordering processes leading to nanopore formation was able to be studied across various timescales. Using these experimental workflows a greater degree of speed and information can be extracted from small datasets without compromising spatial resolution. This approach can be adapted to other 2D materials systems to gain further insights into the defect formation necessary to inform future automated fabrication techniques utilizing the STEM electron beam.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Spatio–Temporal Machine Learning for Regional to Continental Scale Terrestrial Hydrology

Integrated hydrologic models can simulate coupled surface and subsurface processes but are computationally expensive to run at high resolutions over large domains. Here we develop a novel deep learning model to emulate subsurface flows simulated by the integrated ParFlow–CLM model across the contiguous US. We compare convolutional neural networks like ResNet and UNet run autoregressively against our novel architecture called the Forced SpatioTemporal RNN (FSTR). The FSTR model incorporates separate encoding of initial conditions, static parameters, and meteorological forcings, which are fused in a recurrent loop to produce spatiotemporal predictions of groundwater. We evaluate the model architectures on their ability to reproduce 4D pressure heads, water table depths, and surface soil moisture over the contiguous US at 1 km resolution and daily time steps over the course of a full water year. The FSTR model shows superior performance to the baseline models, producing stable simulations that capture both seasonal and event–scale dynamics across a wide array of hydroclimatic regimes. The emulators provide over 1,000× speedup compared to the original physical model, which will enable new capabilities like uncertainty quantification and data assimilation for integrated hydrologic modeling that were not previously possible. Our results demonstrate the promise of using specialized deep learning architectures like FSTR for emulating complex process–based models without sacrificing fidelity.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

A Mitigation Strategy for the Prediction Inconsistency of Neural Phase Pickers

Neural phase pickers—neural networks designed and trained to pick seismic phase arrivals—have proven to be a powerful tool for developing earthquake catalogs. However, these pickers suffer from prediction inconsistency in which the results they produce change, sometimes substantially, even under a small perturbation to the input waveform. This problem has not been addressed by the developers and users of these pickers. In this study, we show how prediction inconsistency can negatively affect the completeness of earthquake catalogs developed using neural phase pickers. Further, we show that simply using a small step size for the sliding window when processing continuous waveform data and aggregating the results significantly mitigates this problem. We also highlight the importance of training datasets for increasing the consistency and other performance metrics.

58 GEOSCIENCES↗
DOE OSTI2022

Reduction of detection limit and quantification uncertainty due to interferent by neural classification with abstention

Many measurements in the physical sciences can be cast as counting experiments, where the number of occurrences of a physical phenomenon informs the prevalence of the phenomenon's source. Often, detection of the physical phenomenon (termed signal) is difficult to distinguish from naturally occurring phenomena (termed background). In this case, the discrimination of signal events from background can be performed using classifiers, and they may range from simple, threshold-based classifiers to sophisticated neural networks. These classifiers are often trained and validated to obtain optimal accuracy, however we show that the optimal accuracy classifier does not generally coincide with a classifier that provides the lowest detection limit, nor the lowest quantification uncertainty. Here, we present a derivation of the detection limit and quantification uncertainty in the classifier-based counting experiment case. We also present a novel abstention mechanism to minimize the detection limit or quantification uncertainty a posteriori. We illustrate the method on two data sets from the physical sciences, discriminating Ar-37 and Ar-39 radioactive decay from non-radioactive events in a gas proportional counter, and discriminating neutrons from gammas in an inorganic scintillator and report results therefrom.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

E(n)-Equivariant cartesian tensor message passing interatomic potential

Machine learning potential (MLP) has been a popular topic in recent years for its capability to replace expensive first-principles calculations in some large systems. Meanwhile, message passing networks have gained significant attention due to their remarkable accuracy, and a wave of message passing networks based on Cartesian coordinates has emerged. However, the information of the node in these models is usually limited to scalars, and vectors. In this work, we propose High-order Tensor message Passing interatomic Potential (HotPP), an E(n) equivariant message passing neural network that extends the node embedding and message to an arbitrary order tensor. By performing some basic equivariant operations, high order tensors can be coupled very simply and thus the model can make direct predictions of high-order tensors such as dipole moments and polarizabilities without any modifications. The tests in several datasets show that HotPP not only achieves high accuracy in predicting target properties, but also successfully performs tasks such as calculating phonon spectra, infrared spectra, and Raman spectra, demonstrating its potential as a tool for future research.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

How Cooperative Engagement Programs Strengthen Sequencing Capabilities for Biosurveillance and Outbreak Response

The threat of emerging and re-emerging infectious diseases continues to be a challenge to public and global health security. Cooperative biological engagement programs act to build partnerships and collaborations between scientists and health professionals to strengthen capabilities in biosurveillance. Biosurveillance is the systematic process of detecting, reporting, and responding to especially dangerous pathogens and pathogens of pandemic potential before they become outbreaks, epidemics, and pandemics. One important tool in biosurveillance is next generation sequencing. Expensive sequencing machines, reagents, and supplies make it difficult for countries to adopt this technology. Cooperative engagement programs help by providing funding for technical assistance to strengthen sequencing capabilities. Through workshops and training, countries are able to learn sequencing and bioinformatics, and implement these tools in their biosurveillance programs. Cooperative programs have an important role in building and sustaining collaborations among institutions and countries. One of the most important pieces in fostering these collaborations is trust. Trust provides the confidence that a successful collaboration will benefit all parties involved. With sequencing, this enables the sharing of pathogen samples and sequences. Obtaining global sequencing data helps to identify unknown etiological agents, track pathogen evolution and infer transmission networks throughout the duration of a pandemic. Having sequencing technology in place for biosurveillance generates the capacity to provide real-time data to understand and respond to pandemics. We highlight the need for these programs to continue to strengthen sequencing in biosurveillance. By working together to strengthen sequencing capabilities, trust can be formed, benefitting global health in the face of biological threats.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Uncertainty quantification of graph convolution neural network models of evolving processes

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Geometry-complete perceptron networks for 3D molecular graphs

Abstract Motivation The field of geometric deep learning has recently had a profound impact on several scientific domains such as protein structure prediction and design, leading to methodological advancements within and outside of the realm of traditional machine learning. Within this spirit, in this work, we introduce GCPNet, a new chirality-aware SE(3)-equivariant graph neural network designed for representation learning of 3D biomolecular graphs. We show that GCPNet, unlike previous representation learning methods for 3D biomolecules, is widely applicable to a variety of invariant or equivariant node-level, edge-level, and graph-level tasks on biomolecular structures while being able to (1) learn important chiral properties of 3D molecules and (2) detect external force fields. Results Across four distinct molecular-geometric tasks, we demonstrate that GCPNet’s predictions (1) for protein–ligand binding affinity achieve a statistically significant correlation of 0.608, more than 5%, greater than current state-of-the-art methods; (2) for protein structure ranking achieve statistically significant target-local and dataset-global correlations of 0.616 and 0.871, respectively; (3) for Newtownian many-body systems modeling achieve a task-averaged mean squared error less than 0.01, more than 15% better than current methods; and (4) for molecular chirality recognition achieve a state-of-the-art prediction accuracy of 98.7%, better than any other machine learning method to date. Availability and implementation The source code, data, and instructions to train new models or reproduce our results are freely available at https://github.com/BioinfoMachineLearning/GCPNet.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml

In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx ZCU102 evaluation board. The latency is reduced to 3 ms per image when increasing the batch size to ten, corresponding to the use case where the autonomous vehicle receives inputs from multiple cameras simultaneously. We show, through aggressive filter reduction and heterogeneous quantization-aware training, and an optimized implementation of convolutional layers, that the power consumption and resource utilization can be significantly reduced while maintaining accuracy on the Cityscapes dataset.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Universal Polarization Transformations: Spatial Programming of Polarization Scattering Matrices Using a Deep Learning‐Designed Diffractive Polarization Transformer

Abstract Controlled synthesis of optical fields having nonuniform polarization distributions presents a challenging task. Here, a universal polarization transformer is demonstrated that can synthesize a large set of arbitrarily‐selected, complex‐valued polarization scattering matrices between the polarization states at different positions within its input and output field‐of‐views (FOVs). This framework comprises 2D arrays of linear polarizers positioned between isotropic diffractive layers, each containing tens of thousands of diffractive features with optimizable transmission coefficients. After its deep learning‐based training, this diffractive polarization transformer can successfully implement N i N o = 10 000 different spatially‐encoded polarization scattering matrices with negligible error, where N i and N o represent the number of pixels in the input and output FOVs, respectively. This universal polarization transformation framework is experimentally validated in the terahertz spectrum by fabricating wire‐grid polarizers and integrating them with 3D‐printed diffractive layers to form a physical polarization transformer. Through this set‐up, an all‐optical polarization permutation operation of spatially‐varying polarization fields is demonstrated, and distinct spatially‐encoded polarization scattering matrices are simultaneously implemented between the input and output FOVs of a compact diffractive processor. This framework opens up new avenues for developing novel devices for universal polarization control and may find applications in, e.g., remote sensing, medical imaging, security, material inspection, and machine vision.

Optical neural networks↗
DOE OSTI2024

Hydrologic connectivity and dynamics of solute transport in a mountain stream: Insights from a long-term tracer test and multiscale transport modeling informed by machine learning

The movement of solutes in a watershed is a complex process with multiple interactions and feedbacks across spatial and temporal scales. Modeling the dynamics of solute transport along diverse hydrologic pathways within watersheds – from hillslopes to stream channels and in and out of the hyporheic zones – is challenging but critically important, as these processes integrate and contribute to the biogeochemical functioning of the river corridor up to the river network scale. Here we use results from a long-term network-scale tracer test at the H.J. Andrews experimental forest in western Cascade Mountains, Oregon, USA to inform a multiscale framework for transport in stream corridors. The framework uses a Lagrangian-based subgrid model to represent the effects of hyporheic exchange flow and advective transport at stream network scales. The spatially and temporally resolved stream discharge needed for the transport model is imputed across the river system by an entity-aware long short-term memory network. Modeled concentrations show good agreements with the observations and exhibit power scaling laws indicative of a very wide range of timescales over which hyporheic exchange flow occurs. Our results demonstrate a data-informed modeling framework that links dynamical processes occurring at small scales to a network context to help understand how changes at reach scale cascade into network-scale effects, providing a useful tool for sustainable river basin management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Machine learning for endoleak detection after endovascular aortic repair

Diagnosis of endoleak following endovascular aortic repair (EVAR) relies on manual review of multi-slice CT angiography (CTA) by physicians which is a tedious and time-consuming process that is susceptible to error. We evaluate the use of a deep neural network for the detection of endoleak on CTA for post-EVAR patients using a novel data efficient training approach. 50 CTAs and 20 CTAs with and without endoleak respectively were identified based on gold standard interpretation by a cardiovascular subspecialty radiologist. The Endoleak Augmentor, a custom designed augmentation method, provided robust training for the machine learning (ML) model. Predicted segmentation maps underwent post-processing to determine the presence of endoleak. The model was tested against 3 blinded general radiologists and 1 blinded subspecialist using a held-out subset (10 positive endoleak CTAs, 10 control CTAs). Model accuracy, precision and recall for endoleak diagnosis were 95%, 90% and 100% relative to reference subspecialist interpretation (AUC = 0.99). Accuracy, precision and recall was 70/70/70% for generalist1, 50/50/90% for generalist2, and 90/83/100% for generalist3. The blinded subspecialist had concordant interpretations for all test cases compared with the reference. In conclusion, our ML-based approach has similar performance for endoleak diagnosis relative to subspecialists and superior performance compared with generalists.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Complex Oxides for Brain–Inspired Computing: A Review

The fields of brain-inspired computing, robotics, and, more broadly, artificial intelligence (AI) seek to implement knowledge gleaned from the natural world into human-designed electronics and machines. In this review, the opportunities presented by complex oxides, a class of electronic ceramic materials whose properties can be elegantly tuned by doping, electron interactions, and a variety of external stimuli near room temperature, are discussed. The review begins with a discussion of natural intelligence at the elementary level in the nervous system, followed by collective intelligence and learning at the animal colony level mediated by social interactions. An important aspect highlighted is the vast spatial and temporal scales involved in learning and memory. The focus then turns to collective phenomena, such as metal-to-insulator transitions (MITs), ferroelectricity, and related examples, to highlight recent demonstrations of artificial neurons, synapses, and circuits and their learning. First-principles theoretical treatments of the electronic structure, and in situ synchrotron spectroscopy of operating devices are then discussed. The implementation of the experimental characteristics into neural networks and algorithm design is then revewed. Finally, outstanding materials challenges that require a microscopic understanding of the physical mechanisms, which will be essential for advancing the frontiers of neuromorphic computing, are highlighted.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Ductile failure and damage localization in Al6061-T6 characterized by in situ X-ray computed tomography and neural network segmentation

Damage initiation and localization in uniaxial tension tests of an Al6061-T6 alloy are evaluated and quantified using in situ X-ray computed tomography and a neural network segmentation strategy. This segmentation strategy offers significant advantages over traditional thresholding in quantifying microstructural and damage evolution metrics. Void nucleation was observed to occur throughout the deformation process, whereas void volume did not grow significantly until later stages of deformation. Localized analysis determined that void nucleation begins in and around the necked region during the early stages of deformation, whereas limited damage nucleation occurs outside of the neck until late stages of deformation. Lastly, inhomogeneities in void and particle spacing in the initial undeformed state of the material are observed to correlate well with the location of damage localization and necking during deformation. These results provide new insights into the ductile failure of Al alloys and how preexisting defects influence damage and rupture.

36 MATERIALS SCIENCE↗