Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Convolutional neural networks for intra-hour solar forecasting based on sky image sequences

Accurate and timely solar forecasts play an increasingly critical role in power systems. Compared to longer forecasting timescales, very short-term solar forecasting has lagged behind in both research and practice. In this paper, we propose deep convolutional neural networks (CNNs) to provide operational intra-hour (10-minute-ahead to 60-minute-ahead) solar forecasts. We develop two CNN structures inspired by a widely-used CNN architecture. The CNNs are tailored to our solar forecasting regression tasks and rely solely on sky image sequences. Case studies based on six years of data (over 150,000 data points) demonstrate that the best CNN model has forecast skill scores of 20%-39% over the naive persistence of cloudiness benchmark, even at these very short timescales. The CNNs also have consistently superior performance when compared to shallow machine learning models with meteorological predictors, where the improvement averages around 7%. The sensitivity analyses show that the sky image length, resolution, and weather conditions have impacts on the deep learning model accuracy. In our intra-hour problem with specific setups, two sky images with a 10-minute 128 x 128 resolution yield the most accurate forecasts. Current limitations, future work, and deployment challenges and solutions are also discussed.

14 SOLAR ENERGY↗
DOE OSTI2020

Deep neural network improves the estimation of polygenic risk scores for breast cancer

Polygenic risk scores (PRS) estimate the genetic risk of an individual for a complex disease based on many genetic variants across the whole genome. Here, we compared a series of computational models for estimation of breast cancer PRS. A deep neural network (DNN) was found to outperform alternative machine learning techniques and established statistical algorithms, including BLUP, BayesA, and LDpred. In the test cohort with 50% prevalence, the Area Under the receiver operating characteristic Curve (AUC) were 67.4% for DNN, 64.2% for BLUP, 64.5% for BayesA, and 62.4% for LDpred. BLUP, BayesA, and LPpred all generated PRS that followed a normal distribution in the case population. However, the PRS generated by DNN in the case population followed a bimodal distribution composed of two normal distributions with distinctly different means. This suggests that DNN was able to separate the case population into a high-genetic-risk case subpopulation with an average PRS significantly higher than the control population and a normal-genetic-risk case subpopulation with an average PRS similar to the control population. This allowed DNN to achieve 18.8% recall at 90% precision in the test cohort with 50% prevalence, which can be extrapolated to 65.4% recall at 20% precision in a general population with 12% prevalence. Interpretation of the DNN model identified salient variants that were assigned insignificant p values by association studies, but were important for DNN prediction. These variants may be associated with the phenotype through nonlinear relationships.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Toward accelerating rare-earth metal extraction using equivariant neural networks

The separation of rare-earth metals, vital for numerous advanced technologies, is hampered by their similar chemical properties, making ligand discovery a significant challenge. Traditional experimental and quantum chemistry approaches for identifying effective ligands are often resource-intensive. We introduce a machine learning protocol based on an equivariant neural network, Allegro, for the rapid and accurate prediction of binding energies in rare-earth complexes. Key to this work is our newly curated dataset of rare-earth metal complexes—made publicly available to foster further research—systematically generated using the Architector program. This dataset distinctively features functionalized derivatives of proven rare-earth-chelating scaffolds, hydroxypyridinone (HOPO), catecholamide (CAM), and their thio-analogues, selected for their established efficacy in binding these elements. Trained on this valuable resource, our Allegro models demonstrate excellent performance, particularly when trained to directly predict DFT-level binding energies, yielding highly accurate results that closely correlate with theoretical calculations on a diverse test set. Furthermore, this strategy exhibited strong out-of-sample generalization, accurately predicting binding energies for an isomeric HOPO-derivative ligand not seen during training. By substantially reducing computational demands, this machine learning framework, alongside the provided dataset, represent powerful tools to accelerate the high-throughput screening and rational design of novel ligands for efficient rare-earth metal separation.

Gupta, Ankur K. [Lawrence Berkeley National Labora↗
DOE OSTI2022

Embedding Learning in Hybrid Quantum-Classical Neural Networks

Quantum embedding learning is an important step in the application of quantum machine learning to classical data. In this paper we propose a quantum few-shot embedding learning paradigm, which learns embeddings useful for training downstream quantum machine learning tasks. Crucially, we identify the circuit bypass problem in hybrid neural networks, where learned classical parameters are optimized to represent the dataset without the quantum kernel. We observe that the few-shot learning embeddings generalize to unseen classes, and suffer less from the circuit bypass problem in terms of better occupation of the parameter space compared with embeddings learned from regression and classification.

Liu, Minzhao↗
DOE OSTI2021

A Bayesian Neural Network Ensemble Approach for Improving Large-Scale Streamflow Predictability

Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising of a hierarchy of models (e.g.,AI driven model/component/parameterization selection). We propose a Bayesian neural network ensemble approach for improving large-scale streamflow predictability and understanding in a changing environment that combines multiple Earth system land model predictions by calculating spatiotemporally varying model weights and biases while accounting for various types of observations at multiple scales with uncertainty.

58 GEOSCIENCES↗
DOE OSTI2025

Artificial neural networks estimate evapotranspiration for Miscanthus × giganteus as effectively as empirical model but with fewer inputs

Estimating actual evapotranspiration (ET) is particularly crucial for addressing how vegetation affects the water balance of ecosystems. ET estimation can be complex with empirical models due to their many parameters and reliance on aridity. In contrast, artificial neural networks (ANNs) could potentially estimate ET with fewer and more common meteorological parameters. In this study, we trained two ANNs, one using a feed-forward approach (FFN) and the other a nonlinear auto-regressive network (NARX), to predict ET and compared them to the commonly used empirical model Granger and Gray (GG). We trained our models on a nine-year eddy covariance (EC) dataset for Miscanthu s × giganteus ( M . × giganteus ) from Illinois (UIEF), then tested them using out-of-sample data from both UIEF and a different location in Iowa (SABR) to compare the accuracy of FFN, NARX, and GG models in estimating daily ET. A combination of air temperature (T a ) and solar radiation (R s ) was chosen as inputs due to the highest R 2 for FFN (R 2 = 0.79, 0.81, and 0.79 for training, testing, and validation, respectively) and only T a for NARX (R 2 = 0.70 for out-of-sample validation). The predictive power of the FFN model was superior to the NARX and GG models at the UIEF site (R 2 = 0.84, 0.70, and 0.83 for out-of-sample validation, respectively). Our analysis showed that ANN approaches are as accurate as empirical approaches for estimating ET but use fewer inputs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Prediction of chronic kidney disease progression using recurrent neural network and electronic health records

Chronic kidney disease (CKD) is a progressive loss in kidney function. Early detection of patients who will progress to late-stage CKD is of paramount importance for patient care. To address this, we develop a pipeline to process longitudinal electronic heath records (EHRs) and construct recurrent neural network (RNN) models to predict CKD progression from stages II/III to stages IV/V. The RNN model generates predictions based on time-series records of patients, including repeated lab tests and other clinical variables. Our investigation reveals that using a single variable, the recorded estimated glomerular filtration rate (eGFR) over time, the RNN model achieves an average area under the receiver operating characteristic curve (AUROC) of 0.957 for predicting future CKD progression. When additional clinical variables, such as demographics, vital information, lab test results, and health behaviors, are incorporated, the average AUROC increases to 0.967. In both scenarios, the standard deviation of the AUROC across cross-validation trials is less than 0.01, indicating a stable and high prediction accuracy. Our analysis results demonstrate the proposed RNN model outperforms existing standard approaches, including static and dynamic Cox proportional hazards models, random forest, and LightGBM. The utilization of the RNN model and the time-series data of previous eGFR measurements underscores its potential as a straightforward and effective tool for assessing the clinical risk of CKD patients concerning their disease progression.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2025

Resolving the Solvation Structure and Transport Properties of Aqueous Zinc Electrolytes from Salt-in-Water to Water-in-Salt Using Neural Network Potential

Zn Cl 2 solutions are promising electrolytes for aqueous zinc-ion batteries. Here, we report a joint computational and experimental study of the structural and dynamic properties of aqueous Zn Cl 2 electrolytes with concentrations ranging from salt-in-water to water-in-salt (WIS). By developing a neural network potential (NNP) model, we perform molecular dynamics (MD) simulations with accuracy but at much larger lengths and longer timescales. The NNP predicted structures are validated by the structure factors measured by X-ray total scattering experiments. The MD trajectories provide a comprehensive and quantitative picture of the Zn 2 + solvation shell structures. Additionally, we find that the O − H covalent bonds in water are strengthened with increasing salt concentration, thus expanding the electrochemical stability window of aqueous electrolytes. In terms of dynamic properties, the calculated and experimentally measured conductivities are in good agreement. Through the analysis of the calculated cation transference number, we propose a three-stage charge carrier transport mechanism with increasing concentration: independent ion transport, strongly correlated ion transport, and small positive charge carrier diffusion through negatively charged polymeric clusters. Our study provides fundamental atomic scale insights into the structure and transport properties of the Zn Cl 2 electrolyte that can aid the optimization and development of WIS electrolytes. Published by the American Physical Society 2025

25 ENERGY STORAGE↗
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Component-Level Inverse Design of Transmon Qubits Using Neural Networks

Designing a superconducting qubit to realize specific Hamiltonian parameters typically requires iterating through a time and compute-intensive forward loop in which the designer chooses a layout geometry, simulates it, extracts circuit parameters such as capacitances, and refines the geometry. We study the inverse version of this task using a neural-network workflow that maps target Hamiltonian parameters directly to component-level layout parameters, which we subsequently demonstrate on a planar transmon layout. During training, we pair the inverse model with a frozen forward surrogate model and evaluate the loss in Hamiltonian space rather than in layout-parameter space. In validation against a conventional EM solver, 97% of generated designs produce usable geometries, and the inverse-plus-surrogate pipeline reaches mean percent errors of 0.73% for qubit frequency and 1.58% for anharmonicity, comparable to or below the fabrication and simulation-to-measurement uncertainty expected for academic-process transmon devices of this type. A single pipeline query takes ~60 ms on CPU, versus ~2 min for a conventional EM capacitance extraction on the same hardware, a speedup of approximately 2,000x. Batching minimizes the AI model inference overhead, reducing the runtime to 3.1 microseconds per sample on CPU and 2.6 microseconds per sample on GPU at a batch size of 2048, resulting in speedups of 3.9 x 10^7 and 4.6 x 10^7, respectively, relative to a single conventional CPU EM extraction. Our results indicate that component-level inverse design usefully extends and complements conventional EM simulation, including for small datasets on the order of 1,000 samples.

Seidel, Olivia [Fermilab; Texas U., Arlington]↗
DOE OSTI2021

Inverse design of two-dimensional materials with invertible neural networks

The ability to readily design novel materials with chosen functional properties on-demand represents a next frontier in materials discovery. However, thoroughly and efficiently sampling the entire design space in a computationally tractable manner remains a highly challenging task. To tackle this problem, we propose an inverse design framework (MatDesINNe) utilizing invertible neural networks which can map both forward and reverse processes between the design space and target property. This approach can be used to generate materials candidates for a designated property, thereby satisfying the highly sought-after goal of inverse design. We then apply this framework to the task of band gap engineering in two-dimensional materials, starting with MoS 2 . Within the design space encompassing six degrees of freedom in applied tensile, compressive and shear strain plus an external electric field, we show the framework can generate novel, high fidelity, and diverse candidates with near-chemical accuracy. We extend this generative capability further to provide insights regarding metal-insulator transition in MoS 2 which are important for memristive neuromorphic applications, among others. This approach is general and can be directly extended to other materials and their corresponding design spaces and target properties.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A Reconfigurable Neural Network ASIC for Detector Front-End Data Compression at the HL-LHC

Despite advances in the programmable logic capabilities of modern trigger systems, a significant bottleneck remains in the amount of data to be transported from the detector to off-detector logic where trigger decisions are made. We demonstrate that a neural network (NN) autoencoder model can be implemented in a radiation-tolerant application-specific integrated circuit (ASIC) to perform lossy data compression alleviating the data transmission problem while preserving critical information of the detector energy profile. For our application, we consider the high-granularity calorimeter from the Compact Muon Solenoid (CMS) experiment at the CERN Large Hadron Collider. The advantage of the machine learning approach is in the flexibility and configurability of the algorithm. By changing the NN weights, a unique data compression algorithm can be deployed for each sensor in different detector regions and changing detector or collider conditions. To meet area, performance, and power constraints, we perform quantization-aware training to create an optimized NN hardware implementation. The design is achieved through the use of high-level synthesis tools and the hls4ml framework and was processed through synthesis and physical layout flows based on a low-power (LP)-CMOS 65-nm technology node. The flow anticipates 200 Mrad of ionizing radiation to select gates and reports a total area of 3.6 mm 2 and consumes 95 mW of power. The simulated energy consumption per inference is 2.4 nJ. Furthermore, this is the first radiation-tolerant on-detector ASIC implementation of an NN that has been designed for particle physics applications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or collective communication to fetch remote data, with optimal performance depending on (i) dataset characteristics, (ii) training scale, and (iii) interconnection network. Empirical analysis shows collective communication excels with larger mini-batch sizes and/or fewer processes, whereas one-sided communication outperforms at larger scales. We propose MDLoader, a hybrid in-memory data loader for distributed graph neural network training. MDLoader features a model-driven performance estimator that dynamically selects between one-sided and collective communication at the beginning of training using Tree of Parzen Estimators (TPE). Evaluations on NERSC Perlmutter and OLCF Summit show MDLoader outperforms single-backend loaders by up to 2.83 × and predicts the suitable communication method with 96.3% (Perlmutter) and 94.3% (Summit) success rate.

Bae, Jonghyun↗
DOE OSTI2020

Accelerated Modeling of Lithium Diffusion in Solid State Electrolytes using Artificial Neural Networks

Abstract Previous efforts to understand structure‐function relationships in high ionic conductivity materials for solid state batteries have predominantly relied on density functional theory (DFT‐) based ab initio molecular dynamics (MD). Such simulations, however, are computationally demanding and cannot be reasonably applied to large systems containing more than a hundred atoms. Here, an artificial neural network (ANN) is trained to accelerate the calculation of high accuracy atomic forces and energies used during such MD simulations. After carefully training a robust ANN for four and five element systems, nearly identical lithium ion diffusivities are obtained for Li 10 GeP 2 S 12 (LGPS) when benchmarking the ANN‐MD results with DFT‐MD. Applying the ANN‐MD approach, the effect of chlorine doping on the lithium diffusivity is calculated in an LGPS‐like structure and it is found that a dopant concentration of 1.3% maximizes ionic conductivity. The optimal concentration balances the competing consequences of effective atomic radii and dielectric constants on lithium diffusion and agrees with the experimental composition. Performing simulations at the resolution necessary to model experimentally relevant and optimal concentrations would be infeasible with traditional DFT‐MD. Systems that require a large number of simulated atoms can be studied more efficiently while maintaining high accuracy with the proposed ANN‐MD framework.

Rao, Karun K.↗
DOE OSTI2022

Predicting path-dependent diffusion barrier spectra in vast compositional space of multi-principal element alloys via convolutional neural networks

The emergent multi-principal element alloys (MPEAs) provide a vast compositional space to search for novel materials for technological advances. How to screen promising compositions from such an ample design space for targeted properties is a grand challenge. In this report, we demonstrate the state-of-the-art deep learning technology—convolutional neural network (CNN)—in predicting path-dependent vacancy migration energy barrier spectra, which are critical to diffusion behavior and many high-temperature properties, in the hyperdimensional composition space of MPEAs. The developed CNN model, fully capturing local chemical features surrounding each vacancy, accurately and efficiently predicts migration energy barrier of MPEAs with different degrees of chemical short-range order and at any unseen compositions. By varying the size of the local region encapsulating vacancy in the CNN model, we reveal that the length scale influencing vacancy migration is surprisingly extensive, up to its six nearest neighboring shells. The efforts of the CNN model make it promising for developing a database of diffusion barriers for various MPEA systems, which would have profound implications for accelerating alloy screening and discovering new compositions with desirable properties.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Modeling the Solvation and Acidity of Carboxylic Acids Using an Ab Initio Deep Neural Network Potential

Formic and acetic acid constitute the simplest of carboxylic acids, yet they exhibit fascinating chemistry in the condensed phase such as proton transfer and dimerization. The go-to method of choice for modeling these rare events have been accurate but expensive ab-initio molecular dynamics simulations. Here, we present a deep neural network potential trained using accurate ab-initio data that can be used in tandem with enhanced-sampling methods to perform an efficient exploration of the free-energy surface of aqueous solutions of weak carboxylic acids. In particular, we show that our model captures proton dissociation and provides a good estimate of the pK a , as well as the dimerization of formic and acetic acid. This provides a suitable starting point for applications in different research areas where computational efficiency coupled with the accuracy of ab-initio methods is required.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗