Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Data-Driven Strategies for Accelerated Materials Design

The ongoing revolution of the natural sciences by the advent of machine learning and artificial intelligence sparked significant interest in the material science community in recent years. The intrinsically high dimensionality of the space of realizable materials makes traditional approaches ineffective for large-scale explorations. Modern data science and machine learning tools developed for increasingly complicated problems are an attractive alternative. An imminent climate catastrophe calls for a clean energy transformation by overhauling current technologies within only several years of possible action available. Tackling this crisis requires the development of new materials at an unprecedented pace and scale. For example, organic photovoltaics have the potential to replace existing silicon-based materials to a large extent and open up new fields of application. In recent years, organic light-emitting diodes have emerged as state-of-the-art technology for digital screens and portable devices and are enabling new applications with flexible displays. Reticular frameworks allow the atom-precise synthesis of nanomaterials and promise to revolutionize the field by the potential to realize multifunctional nanoparticles with applications from gas storage, gas separation, and electrochemical energy storage to nanomedicine. In the recent decade, significant advances in all these fields have been facilitated by the comprehensive application of simulation and machine learning for property prediction, property optimization, and chemical space exploration enabled by considerable advances in computing power and algorithmic efficiency. In this Account, we review the most recent contributions of our group in this thriving field of machine learning for material science. We start with a summary of the most important material classes our group has been involved in, focusing on small molecules as organic electronic materials and crystalline materials. Specifically, we highlight the data-driven approaches we employed to speed up discovery and derive material design strategies. Subsequently, our focus lies on the data-driven methodologies our group has developed and employed, elaborating on high-throughput virtual screening, inverse molecular design, Bayesian optimization, and supervised learning. We discuss the general ideas, their working principles, and their use cases with examples of successful implementations in data-driven material discovery and design efforts. Furthermore, we elaborate on potential pitfalls and remaining challenges of these methods. Finally, we provide a brief outlook for the field as we foresee increasing adaptation and implementation of large scale data-driven approaches in material discovery and design campaigns.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Approaching hydro-equivalent ignition in laser direct-drive via target design optimization using novel statistical modeling

Laser direct-drive offers significant advantages in terms of target simplicity, improved energy coupling, and large fuel masses over indirect drive. However, performance degradations from hydrodynamic and laser-plasma instabilities seeded and driven by the direct illumination pose limitations on the parameter space available for achieving ignition. In this paper, new design improvements are identified to forge a path forward for a hydro-equivalent ignition demonstration. The first is related to a new formulation of the statistical model (SM) used to accurately predict target performance directly from input parameters such as laser pulse shape and target specifications. This new SM formulation provides direct guidance on target dimensions and laser beam-to-target radius to achieve the highest fusion yield on the OMEGA laser. The second improvement comes from cooling the deuterium–tritium (DT) ice layer below the triple point right before shot time leading to lower DT vapor densities and higher convergence. Guided by these design improvements, a Bayesian optimization algorithm was used to design an implosion that is predicted to closely approach a Lawson triple product that hydrodynamically scales to ignition if equivalent laser–target coupling is achieved at laser energies typical of the National Ignition Facility.

Deuterium↗
DOE OSTI2022

Machine learning with knowledge constraints for process optimization of open-air perovskite solar cell manufacturing

Perovskite photovoltaics (PV) have achieved rapid development in the past decade in terms of power conversion efficiency of small-area lab-scale devices; however, successful commercialization still requires further development of low-cost, scalable, and high-throughput manufacturing techniques. One of the critical challenges of developing a new fabrication technique is the high-dimensional parameter space for optimization, but machine learning (ML) can readily be used to accelerate perovskite PV scaling. Herein, we present an ML-guided framework of sequential learning for manufacturing process optimization. We apply our methodology to the Rapid Spray Plasma Processing (RSPP) technique for perovskite thin films in ambient conditions. With a limited experimental budget of screening 100 process conditions, we demonstrated an efficiency improvement to 18.5% as the best-in-our-lab device fabricated by RSPP, and we also experimentally found 10 unique process conditions to produce the top-performing devices of more than 17% efficiency, which is 5 times higher rate of success than the control experiments with pseudo-random Latin hypercube sampling. Our model is enabled by three innovations: (a) flexible knowledge transfer between experimental processes by incorporating data from prior experimental data as a probabilistic constraint; (b) incorporation of both subjective human observations and ML insights when selecting next experiments; (c) adaptive strategy of locating the region of interest using Bayesian optimization first, and then conducting local exploration for high-efficiency devices. Furthermore, in virtual benchmarking, our framework achieves faster improvements with limited experimental budgets than traditional design-of-experiments methods (e.g., one-variable-at-a-time sampling). This framework shows the capability of incorporating researchers’ domain knowledge into the ML-guided optimization loop; therefore, it has the potential to facilitate the wider adoption of ML in scaling to perovskite PV manufacturing.

14 SOLAR ENERGY↗
DOE OSTI2023

2023 FORCE Development Status Update

Technical and economic analysis of integrated energy systems (IES) using software models is a complex process requiring multiple commodity market decision analysis, optimal control, process modeling, and stochastic analysis. Many assumptions used in traditional energy analysis tools do not hold in future energy markets with significant storage and variable renewable energy sources (VRE), let alone with multiple commodity markets. Capturing these intricate elements for accurate techno-economic analysis of IES led to the development of the Framework for Optimization of ResourCes and Economics (FORCE) tool suite under the U.S. Department of Energy’s Integrated Energy Systems crosscutting technology program. With the aim of a full framework release in 2025, many improvements to the FORCE tool suite were developed in fiscal year 2023 (FY23). These improvements broadly fit into three focuses for development of FORCE: capability, accessibility, and reliability. Capability refers to the ability of FORCE to accurately model the technical and economic viability of various IES. Accessibility refers to ease-of-use for new and existing IES analysts to efficiently set up, analyze, and produce results using FORCE. Reliability refers to the consistency of the software, allowing consistency to analysis regardless of erstwhile changes to the software. In addition to many smaller changes, there are three major capability improvements in FORCE in FY23. In summary, FORCE developments in FY23 have moved us close to all the capability requirements for FORCE 1.0 to be delivered in FY25. Inclusion of Bayesian optimization, resilience metrics, and levelized cost analysis expand the capability, accessibility, and reliability of FORCE.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference

Efficient machine learning implementations optimized for inference in hardware have wide-ranging benefits, depending on the application, from lower inference latency to higher data throughput and reduced energy consumption. Two popular techniques for reducing computation in neural networks are pruning, removing insignificant synapses, and quantization, reducing the precision of the calculations. In this work, we explore the interplay between pruning and quantization during the training of neural networks for ultra low latency applications targeting high energy physics use cases. Techniques developed for this study have potential applications across many other domains. We study various configurations of pruning during quantization-aware training, which we term quantization-aware pruning, and the effect of techniques like regularization, batch normalization, and different pruning schemes on performance, computational complexity, and information content metrics. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task. Further, quantization-aware pruning typically performs similar to or better in terms of computational efficiency compared to other neural architecture search techniques like Bayesian optimization. Surprisingly, while networks with different training configurations can have similar performance for the benchmark application, the information content in the network can vary significantly, affecting its generalizability.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Bayesian experimental design and parameter estimation for ultrafast spin dynamics

Abstract Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and materials physics, which often suffer from the scarcity of large-scale facility resources, such as x-ray or neutron scattering centers. To address these limitations, we introduce a methodology that leverages the Bayesian optimal experimental design paradigm to efficiently uncover key quantum spin fluctuation parameters from x-ray photon fluctuation spectroscopy (XPFS) data. Our method is compatible with existing theoretical simulation pipelines and can also be used in combination with fast machine learning surrogate models in the event that real-time simulations are unfeasible. Our numerical benchmarks demonstrate the superior performance in predicting model parameters and in delivering more informative measurements within limited experimental time. Our method can be adapted to many different types of experiments beyond XPFS and spin fluctuation studies, facilitating more efficient data collection and accelerating scientific discoveries.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Physics Discovery in Nanoplasmonic Systems via Autonomous Experiments in Scanning Transmission Electron Microscopy

Abstract Physics‐driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here, this work develops and experimentally implements a deep kernel learning (DKL) workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics‐based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization (BO) methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure–property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk‐ and edge plasmon discovery in MnPS 3 , a lesser‐known beam‐sensitive layered 2D material. This approach is universal, can be directly used as‐is with any specimen, and is expected to be applicable to any probe‐based microscopic techniques including other STEM modalities, scanning probe microscopies, chemical, and optical imaging.

42 ENGINEERING↗
DOE OSTI2026

Data‐Driven Engineering of Thermostable Collagen‐Mimetic Peptoid Triple Helices

Collagen-mimetic peptides (CMPs) are engineered molecules designed to replicate the triple-helical structure of natural collagen. A repeating x–y-Gly sequence is the defining motif of CMPs and is critical to their triple-helical structure and stability. Substitutions to the residues occupying the x and y positions present a means to modulate the CMP structure and properties. Peptoid residues—N-substituted glycine derivatives—present an attractive potential substitution due to their thermal stability, proteolytic resistance, biocompatibility, and diverse palette of non-natural side chains, but also tend to introduce a high degree of backbone flexibility that can diminish the stability of the triple helix. In this work, we report a computational active learning cycle comprising molecular dynamics simulation, Gaussian process regression, and Bayesian optimization to computationally identify a number of promising peptoid substitutions predicted to stabilize the desired quaternary structure through side chain interactions and produce stable peptoid-based collagen-like triple helices. To experimentally test the computational predictions, a top candidate identified by the screen was synthesized and imaged using scanning electron microscopy to resolve fibril-like bundles consistent with collagen-like triple helices. This work predicts a number of CMP peptoid substitutions capable of forming stable triple-helical structures, presents a generalizable design strategy for engineering desired peptoid structures, and opens new avenues for the design of peptoid-based biomimetic materials.

active learning↗
DOE OSTI2022

Building an Integrated Ecosystem of Computational and Observational Facilities to Accelerate Scientific Discovery

Future scientific discoveries will rely on flexible ecosystems that incorporate modern scientific instruments, high performance computing resources, parallel distributed data storage, and performant networks across multiple, independent facilities. In addition to connecting physical resources, such an ecosystem presents many challenges in logistics and accessibility, especially in orchestrating computations and experiments that span across leadership computing systems and experimental instruments. Past efforts have typically been application-specific or limited to interfaces for computing resources. This paper proposes a general framework for integrating computation resources and instrument operations, addressing challenges in code development/execution, data staging and collection, software stack, control mechanisms, resource authorization and governance, and hardware integration. We also describe a demonstration use case wherein a Bayesian optimization algorithm running on an edge computing resource guides a scanning probe microscope to autonomously and intelligently characterize a material sample. This science edge ecosystem framework will provide a blueprint for federating multi-institutional, disparate resources and orchestrating scientific workflows across them to enable next-generation discoveries.

Somnath, Suhas↗
DOE OSTI2025

Robustness of Deep Learning Classification to Adversarial Input on GPUs: Asynchronous Parallel Accumulation Is a Source of Vulnerability

The ability of machine learning (ML) classification models to resist small, targeted input perturbations—known as adversarial attacks—is a key measure of their safety and reliability. We show that floating-point non associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that this misclassification is particularly significant for inputs close to the decision boundary and that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We first study a linear classifier, before focusing on standard Graph Neural Network (GNN) architectures and datasets used in robustness assessments. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.

Shanmugavelu, Sanjif [Maxeler Technologies, a Groq↗
DOE OSTI2026

A multi-backend autotuning study of feature selection on GPUs

Abstract Feature selection is an important step in machine learning that can benefit from GPU acceleration. As the number of GPU vendors increases, it is imperative to adapt algorithms such as the minimum Redundancy Maximum Relevance (mRMR) feature selection method to different backends that support several GPU architectures. This work presents a multi-backend implementation of mRMR across CUDA, HIP, and SYCL, and studies its performance when combined with Bayesian optimization and transfer learning to automatically tune execution parameters for different platforms and datasets. Our experimental results show that when tuned, CUDA and HIP achieve comparable performance on NVIDIA architectures, while SYCL exhibits a moderate performance gap. Overall, this work highlights the impact of backend choice and autotuning on GPU-accelerated feature selection and provides insights into deploying mRMR across heterogeneous environments.

Beceiro, Bieito (ORCID:0000000333014890)↗
DOE OSTI2022

Machine learning in nuclear materials research

Nuclear materials are often demanded to function for extended time in extreme environments, including high radiation fluxes with associated transmutations, high temperature and temperature gradients, mechanical stresses, and corrosive coolants. They also have a wide range of microstructural and chemical makeups, resulting in multifaceted and often out-of-equilibrium interactions. Machine learning (ML) is increasingly being used to tackle these complex time-dependent interactions and aid researchers in developing models and making predictions, sometimes with better accuracy than traditional modeling that focuses on one or two parameters at a time. Conventional practices of acquiring new experimental data in nuclear materials research are often slow and expensive, limiting the opportunity for data-centric ML, but new methods are changing that paradigm. Here we review high-throughput computational and experimental data approaches, especially robotic experimentation and active learning that is based on Gaussian process and Bayesian optimization. We show ML examples in structural materials (e.g., reactor pressure vessel (RPV) alloys and radiation detecting scintillating materials) and highlight new techniques of high-throughput sample preparation and characterizations, and automated radiation/environmental exposures and real-time online diagnostics. Herein, this review suggests that ML models of material constitutive relations in plasticity, damage, and even electronic and optical responses to radiation are likely to become powerful tools as they develop. Finally, we speculate on how the recent trends of using natural language processing (NLP) to aid the collection and analysis of literature data, interpretable artificial intelligence (AI), and the use of streamlined scripting, database, workflow management, and cloud computing platforms that will soon make the utilization of ML techniques as commonplace as the spreadsheet curve-fitting practices of today.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Active operator learning with predictive uncertainty quantification for partial differential equations

With the increased prevalence of neural operators being used to provide rapid solutions to partial differential equations (PDEs), understanding the accuracy of model predictions and the associated error levels is necessary for deploying reliable surrogate models in scientific applications. Existing uncertainty quantification (UQ) frameworks employ ensembles or Bayesian methods, which can incur substantial computational costs during both training and inference. Here, we propose a lightweight predictive UQ method tailored for Deep operator networks (DeepONets) that also generalizes to other operator networks. Numerical experiments on linear and nonlinear PDEs demonstrate that the framework’s uncertainty estimates are unbiased and provide accurate out-of-distribution uncertainty predictions with a sufficiently large training dataset. Our framework provides fast inference and uncertainty estimates that can efficiently drive outer-loop analyses that would be prohibitively expensive with conventional solvers. We demonstrate how predictive uncertainties can be used in the context of Bayesian optimization and active learning problems to yield improvements in accuracy and data-efficiency for outer-loop optimization procedures. In the active learning setup, we extend the framework to Fourier Neural Operators (FNO) and describe a generalized method for other operator networks. To enable real-time deployment, we introduce an inference strategy based on precomputed trunk outputs and a sparse placement matrix, reducing evaluation time by more than a factor of five. Our method provides a practical route to uncertainty-aware operator learning in time-sensitive settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Performance Evaluation of an Offshore Wave Measurement Buoy in Monochromatic Waves

The accurate measurement of waves underpins marine energy resource characterization, device design, and project development. Datawell wave buoys are widely deployed and have long served as a trusted standard for wave measurements. We quantify the measurement performance, including wave elevation and energy flux estimation, of a Datawell DWR-MkIII buoy using prescribed monochromatic heave motions on a large-amplitude six-degree-of-freedom motion platform at the National Laboratory of the Rockies, assuming the buoy behaves as an ideal wave follower. Commanded motions were validated with an optical motion tracking system while buoy elevation and raw acceleration were recorded. Wave elevations were propagated to wave energy flux estimation using four methods, including one frequency-domain method and three time-domain methods. The Bayesian optimization was applied for design of experiments, and records from three test sites were also applied and evaluated in the present study. Results show two error regions within the nominal period range of 1.6 s to 30 s. For wave periods between 5 s and 25 s, the buoy provides accurate wave height measurements. For short periods less than 5 s, the 1.28 Hz sampling frequency induces sub-Nyquist artifacts that bias elevation and can drive maximum energy flux estimation errors above 100%. For long periods exceeding 25 s, the buoy reported elevation is underpredicted with error depending on period but relatively independent of wave height, with maximum wave height and wave energy flux errors reaching 64% and 87%, respectively. Furthermore, analysis of three field-derived cases shows that frequency-domain estimates at 1.28 Hz agree within 2% of the corresponding 100 Hz estimates, while larger method-dependent differences are observed for the Hilbert method.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗
DOE OSTI2024

Implicit neural representations for experimental steering of advanced experiments

Scattering measurements using electrons, neutrons, or photons are essential for obtaining microscopic insights into materials. However, limited facility availability and high-dimensional scattering data necessitate more efficient experimental steering techniques. Here, we report a machine learning method that guides scattering data collection and facilitates real-time estimation of model parameters, given a reliable forward model to simulate experimental signals. We employ implicit neural representations as efficient surrogates that link model parameters with simulated spectroscopies. This enables a Bayesian optimal experimental design framework to estimate the probability distributions of parameters from high-dimensional scattering data. We demonstrate the proposed method using inelastic neutron scattering with simulated and real experimental data, highlighting the method’s ability to provide real-time parameter estimation with quantified uncertainties and to deliver informed experimental guidance that reduces experimental time while maximizing scientific output. This approach paves the way for accelerated discoveries in condensed matter through scattering measurements.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Quantum Chemistry-Informed Active Learning to Accelerate the Design and Discovery of Sustainable Energy Storage Materials

Here we employed Density Functional Theory (DFT) to compute oxidation potentials of 1,400 homobenzylic ether molecules to search for the ideal sustainable redoxmer design. The generated data were used to construct an active learning model based on Bayesian optimization (BO) that targets candidates with desired oxidation potentials utilizing only a minimal number of DFT calculations. The active learning model demonstrated not only significant efficiency improvement over the random selection approach but also robust capability in identifying desired candidates in an untested set of 112,000 homobenzylic ether molecules. Our findings highlight the efficacy of quantum chemistry-informed active learning to accelerate the discovery of materials with desired properties from a vast chemical space.

25 ENERGY STORAGE↗
DOE OSTI2020

Discovery of Self-Assembling π-Conjugated Peptides by Active Learning-Directed Coarse-Grained Molecular Simulation

Electronically active organic molecules have demonstrated great promise as novel soft materials for energy harvesting and transport. Self-assembled nanoaggregates formed from π-conjugated oligopeptides composed of an aromatic core flanked by oligopeptide wings offer emergent optoelectronic properties within a water-soluble and biocompatible substrate. Nanoaggregate properties can be controlled by tuning core chemistry and peptide composition, but the sequence–structure–function relations remain poorly characterized. Here, we employ coarse-grained molecular dynamics simulations within an active learning protocol employing deep representational learning and Bayesian optimization to efficiently identify molecules capable of assembling pseudo-1D nanoaggregates with good stacking of the electronically active π-cores. We consider the DXXX-OPV3-XXXD oligopeptide family, where D is an Asp residue and OPV3 is an oligophenylenevinylene oligomer (1,4-distyrylbenzene), to identify the top performing XXX tripeptides within all 20 3 = 8000 possible sequences. By direct simulation of only 2.3% of this space, we identify molecules predicted to exhibit superior assembly relative to those reported in prior work. Spectral clustering of the top candidates reveals new design rules governing assembly. This work establishes new understanding of DXXX-OPV3-XXXD assembly, identifies promising new candidates for experimental testing, and presents a computational design platform that can be generically extended to other peptide-based and peptide-like systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗