Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “active machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

An atomic cluster expansion potential for twisted multilayer graphene

Twisted multilayer graphene, characterized by its moiré patterns arising from inter-layer rotational misalignment, serves as a rich platform for exploring quantum phenomena. Machine learning interatomic potentials (MLIPs) are a promising approach to model such systems. Our work develops a method to generate training and test datasets for fitting MLIPs that capture all possible misalignments but remain small-scale to facilitate efficient data generation and parameter estimation. To achieve this, we generate configurations with periodic boundary conditions suitable for density functional theory calculations, and then introduce an internal twist and shift within those supercell structures. Using this technique, supplemented with an active learning workflow, we fit an Atomic Cluster Expansion potential for simulating twisted multilayer graphene and test it for accuracy and robustness on a range of simulation tasks.

2D materials↗

AdenPredictor: accurate prediction of the adenylation domain specificity of nonribosomal peptide biosynthetic gene clusters in microbial genomes

Microbial natural products represent a major source of bioactive compounds for drug discovery. Among these molecules, nonribosomal peptides (NRPs) represent a diverse class that include antibiotics, immunosuppressants, anticancer agents, toxins, siderophores, pigments, and cytostatics. The discovery of novel NRPs remains a laborious process because many NRPs consist of nonstandard amino acids that are assembled by nonribosomal peptide synthetases (NRPSs). Adenylation domains (A-domains) in NRPSs are responsible for selection and activation of monomers appearing in NRPs. During the past decade, several support vector machine-based algorithms have been developed for predicting the specificity of the monomers present in NRPs. These algorithms utilize physiochemical features of the amino acids present in the A-domains of NRPSs. In this article, we benchmarked the performance of various machine learning algorithms and features for predicting specificities of NRPSs and we showed that the extra trees model paired with one-hot encoding features outperforms the existing approaches. Moreover, we show that unsupervised clustering of 453560 A-domains reveals many clusters that correspond to potentially novel amino acids. While it is challenging to predict the chemical structure of these amino acids, we developed novel techniques to predict their various properties, including polarity, hydrophobicity, charge, and presence of aromatic rings, carboxyl, and hydroxyl groups.

59 BASIC BIOLOGICAL SCIENCES↗

Applying deep learning methods to develop new models of molecular charge transfer, nonadiabatic dynamics, and nonlinear spectroscopy in the condensed phase

Photon- and field-induced charge transfer has central importance in the generation and storage of electricity, the novel properties of materials, photo-induced catalysis, and electro-optic activity (e.g., photovoltaic cells, fuel cells, and organic chromophores for use in optical fibers and light-emission diodes). These non-equilibrium electronic and chemical transformations are probed by ultrafast, nonlinear spectroscopies. Accurate simulations play a crucial role in our ability to understand, optimize, and control these transformations. This project applies modern deep learning and machine learning (ML) methods to dramatically improve models of electronic dynamics, electronic-nuclear dynamics, and spectroscopic measurements for improved simulations of chemistry in complex environments, far from equilibrium phenomena, and processes in extreme environments, such as materials exposed to strong or resonant fields. This project develops accurate neural net models that go beyond predictive capability to also provide new insight into the fundamental physics underlying electron and nuclear dynamics. To achieve its objectives, this project explores and develops customized versions of high-capacity deep learning algorithms/models. These techniques are developed with an emphasis on fundamental chemical insight, not just predictive accuracy, to assist the development of the next generation of quantum simulation methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Materials Characterization, Prediction and Control Project: Summary Report on Data Analytics Framework

This report summarizes the activities performed under the data analytics Vertex in the Materials Characterization, Prediction and Control Project funded under laboratory directed research and development at Pacific Northwest National Laboratory. The data analytics Vertex developed models for associating global or local process parameters, microstructural features, and performance properties of friction-stir-processed 316L stainless steel plates. Statistical, machine learning, and deep learning models, as well as generative artificial intelligence approaches, were used to develop the associations between the process-structure-property data streams. These associations formed the basis for predicting global properties of parts manufactured under different process envelopes, providing a basis for predicting performance using data driven as well as physics-informed and physics-constrained approaches. Additionally, the associations were used to predict local process parameters and microstructural features of the product, predictive relationships that have the potential to form the basis of a control framework that could eventually modulate a friction-stir process to maintain product quality.

316L stainless steel↗

Low Molecular Weight Volatile Organic Compounds Indicate Grazing by the Marine Rotifer Brachionus plicatilis on the Microalgae Microchloropsis salina

Microalgae produce specific chemicals indicative of stress and/or death. The aim of this study was to perform non-destructive monitoring of algal culture systems, in the presence and absence of grazers, to identify potential biomarkers of incipient pond crashes. Here, we report ten volatile organic compounds (VOCs) that are robustly generated by the marine alga, Microchloropsis salina, in the presence and/or absence of the marine grazer, Brachionus plicatilis. We cultured M. salina with and without B. plicatilis and collected in situ volatile headspace samples using thermal desorption tubes over the course of several days. Data from four experiments were aggregated, deconvoluted, and chromatographically aligned to determine VOCs with tentative identifications made via mass spectral library matching. VOCs generated by algae in the presence of actively grazing rotifers were confirmed via pure analytical standards to be pentane, 3-pentanone, 3-methylhexane, and 2-methylfuran. Six other VOCs were less specifically associated with grazing but were still commonly observed between the four replicate experiments. Through this work, we identified four biomarkers of rotifer grazing that indicate algal stress/death. This will aid machine learning algorithms to chemically define and diagnose algal mass production cultures and save algae cultures from imminent crash to make biofuel an alternative energy possibility.

headspace sampling↗

Human-in-the-Loop: The Future of Machine Learning in Automated Electron Microscopy

Machine learning (ML) methods are progressively gaining acceptance in the electron microscopy community for de-noising, semantic segmentation, and dimensionality reduction of data post-acquisition. The introduction of the application programming interfaces (APIs) by major instrument manufacturers now allows the deployment of ML workflows in microscopes, not only for data analytics but also for real-time decision-making and feedback for microscope operation. However, the number of use cases for real-time ML remains remarkably small. Furthermore, we discuss some considerations in designing ML-based active experiments and pose that the likely strategy for the next several years will be human-in-the-loop automated experiments (hAE). In this paradigm, the ML learning agent directly controls beam position and image and spectroscopy acquisition functions, and a human operator monitors experiment progression in real and feature space of the system and tunes the policies of the ML agent to steer the experiment toward specific objectives.

47 OTHER INSTRUMENTATION↗

SLAB: simultaneous labeling and binding affinity prediction for protein–ligand structures

Machine learning models are often used as scoring functions to predict the binding affinity of a protein–ligand complex. These models are trained with limited amounts of data with experimentally measured binding affinity values. A large number of compounds are labeled inactive through single-concentration screens without measuring binding affinities. These inactive compounds, along with the active ones, can be used to train binary classification models, while regression models are trained using compounds with binding affinities only. However, the classification and regression tasks are often handled separately, without sharing the learned feature representations. In this paper, we propose a novel model architecture that jointly performs regression and classification objectives, aiming to maximize data utilization and improve predictive performance by leveraging two complementary tasks. In our setup, the regression yields the binding affinity, whereas the classification task yields the label as active or inactive. We demonstrate our method using PDBbind, the standard 3D structure database, as well as a dataset of flavivirus protease compounds with binding affinity data. Our experiments show that the new joint training strategy improves the accuracy of the model, increasing applicability in various practical drug screening scenarios.

Biological and medical sciences↗

Uncertainty Quantification for Capacity Expansion Planning

This report quantifies the uncertainty in output decisions from a Capacity Expansion Planning (CEP) model. The need to understand how uncertainties within CEP models and modeling assumptions affect Quantities of Interest (QoIs) such as expansion and operating costs, as well as expansion decisions remains an ongoing challenge in scientific research and industrial operations. This area of research is particularly important for models which seek to capture how large networks will evolve and operate under increased sources of variable generation, i.e., higher penetration of renewable technologies such as solar and wind generators. Uncertainty quantification (UQ) of CEP models which estimate expansion costs and decisions, and production cost models which estimate operating costs and dispatch decisions, is a key focus of research at NREL. The Regional Energy Deployment System (ReEDS) represents a state-of-the-art CEP model and considers a range of possible grid evolutions in an attempt to identify key drivers, ramifications, and decisions which contribute to better informed investment and policy decisions. However, research to quantify how uncertainties and model assumptions, such as unit commitment (UC), within ReEDS may be affecting its outputs remains challenging due to to size and complexity of the model

24 POWER TRANSMISSION AND DISTRIBUTION↗

Verification and validation of developed short-term forecasting models

Recent advancements in machine learning (ML) and artificial intelligence (AI) technologies provide an opportunity for leveraging data-driven algorithms to predict future nuclear power plant (NPP) operating conditions by using recorded plant process data. Successfully implementing these models can lead to cost-reducing, conditioned-based predictive maintenance through optimized maintenance schedules and a reduction of unnecessary maintenance activities. This report discusses the verification and validation of short-term forecasting processes (i.e., data cleaning, feature selection, model optimization, and forecasting) developed in previous reports. The verification and validation (V&V) process demonstrates the expected precision and accuracy when the ML model encounters new datasets from different systems. Shapley additive explanations were used as the primary means of feature selection across these different data set. Individual models were trained for each data set, then validated through a cross-validation procedure. In this report, two different ML models were tasked to predict variables from three different plant process data sets with varying prediction horizons. The results indicate that support vector regression (SVR) outperformed long short-term memory (LSTM) neural networks in regard to each data set and each prediction horizon in this study, but further tuning and optimization could improve long short-term memory results. However, each forecasting model showed reduced performance as the prediction horizon was extended from 1 hour to 1 day ahead. Research is ongoing to evaluate the optimal input variable space, which is based on a given set of process parameters, to further improve forecasting accuracy.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Cloud drop number concentrations over the western North Atlantic Ocean: seasonal cycle, aerosol interrelationships, and other influential factors

Cloud drop number concentrations (N d ) over the western North Atlantic Ocean (WNAO) are generally highest during the winter (DJF) and lowest in summer (JJA), in contrast to aerosol proxy variables (aerosoloptical depth, aerosol index, surface aerosol mass concentrations, surface cloud condensation nuclei (CCN) concentrations) that generally peak inspring (MAM) and JJA with minima in DJF. Using aircraft, satellite remote sensing, ground-based in situ measurement data, and reanalysis data, we characterize factors explaining the divergent seasonal cycles and furthermore probe into factors influencing N d on seasonal timescales. The results can be summarized well by features most pronounced in DJF, including features associated with cold-air outbreak (CAO) conditions such as enhanced values of CAO index, planetary boundary layer height (PBLH),low-level liquid cloud fraction, and cloud-top height, in addition to winds aligned with continental outflow. Data sorted into high- and low-N d days in each season, especially in DJF, revealed that all of these conditions were enhanced on the high-N d days, including reduced sea level pressure and stronger wind speeds. Although aerosols may be more abundant in MAM and JJA, the conditions needed to activate those particles into cloud droplets are weaker than in colder months, which is demonstrated by calculations of the strongest (weakest) aerosol indirect effects in DJF (JJA) based on comparing N d to perturbations in four different aerosol proxy variables (total and sulfate aerosol optical depth, aerosol index, surface mass concentration of sulfate). We used three machine learning models and up to 14 input variables to infer about most influential factors related to N d for DJF and JJA, with the best performance obtained with gradient-boosted regression tree (GBRT) analysis. The model results indicated that cloud fraction was the most important input variable, followed by some combination (depending on season) of CAO index and surface mass concentrations of sulfate and organic carbon. Future work is recommended to further understand aspects uncovered here such as impacts of free tropospheric aerosol entrainment on clouds, degree of boundary layer coupling, wet scavenging, and giant CCN effects on aerosol–N d relationships, updraft velocity, and vertical structure of cloud properties such as adiabaticity that impact the satellite estimation of N d .

54 ENVIRONMENTAL SCIENCES↗

Machine learning-assisted crystal engineering of a zeolite

Abstract It is shown that Machine Learning (ML) algorithms can usefully capture the effect of crystallization composition and conditions (inputs) on key microstructural characteristics (outputs) of faujasite type zeolites (structure types FAU, EMT, and their intergrowths), which are widely used zeolite catalysts and adsorbents. The utility of ML (in particular, Geometric Harmonics) toward learning input-output relationships of interest is demonstrated, and a comparison with Neural Networks and Gaussian Process Regression, as alternative approaches, is provided. Through ML, synthesis conditions were identified to enhance the Si/Al ratio of high purity FAU zeolite to the hitherto highest level (i.e., Si/Al = 3.5) achieved via direct (not seeded), and organic structure-directing-agent-free synthesis from sodium aluminosilicate sols. The analysis of the ML algorithms’ results offers the insight that reduced Na 2 O content is key to formulating FAU materials with high Si/Al ratio. An acid catalyst prepared by partial ion exchange of the high-Si/Al-ratio FAU (Si/Al = 3.5) exhibits improved proton reactivity (as well as specific activity, per unit mass of catalyst) in propane cracking and dehydrogenation compared to the catalyst prepared from the previously reported highest Si/Al ratio (Si/Al = 2.8).

36 MATERIALS SCIENCE↗

Genomic fingerprints of the world’s soil ecosystems

Despite the explosion of soil metagenomic data, we lack a synthesized understanding of patterns in the distribution and functions of soil microorganisms. These patterns are critical to predictions of soil microbiome responses to climate change and resulting feedbacks that regulate greenhouse gas release from soils. To address this gap, we assay 1,512 manually curated soil metagenomes using complementary annotation databases, read-based taxonomy, and machine learning to extract multidimensional genomic fingerprints of global soil microbiomes. Our objective is to uncover novel biogeographical patterns of soil microbiomes across environmental factors and ecological biomes with high molecular resolution. We reveal shifts in the potential for (i) microbial nutrient acquisition across pH gradients; (ii) stress-, transport-, and redox-based processes across changes in soil bulk density; and (iii) greenhouse gas emissions across biomes. We also use an unsupervised approach to reveal a collection of soils with distinct genomic signatures, characterized by coordinated changes in soil organic carbon, nitrogen, and cation exchange capacity and in bulk density and clay content that may ultimately reflect soil environments with high microbial activity. Genomic fingerprints for these soils highlight the importance of resource scavenging, plant-microbe interactions, fungi, and heterotrophic metabolisms. Across all analyses, we observed phylogenetic coherence in soil microbiomes—more closely related microorganisms tended to move congruently in response to soil factors. Collectively, the genomic fingerprints uncovered here present a basis for global patterns in the microbial mechanisms underlying soil biogeochemistry and help beget tractable microbial reaction networks for incorporation into process-based models of soil carbon and nutrient cycling.

59 BASIC BIOLOGICAL SCIENCES↗

Machine-learning based approach to examine ecological processes influencing the diversity of riverine dissolved organic matter composition

Dissolved organic matter (DOM) assemblages in freshwater rivers are formed from mixtures of simple to complex compounds that are highly variable across time and space. These mixtures largely form due to the environmental heterogeneity of river networks and the contribution of diverse allochthonous and autochthonous DOM sources. Most studies are, however, confined to local and regional scales, which precludes an understanding of how these mixtures arise at large, e.g., continental, spatial scales. The processes contributing to these mixtures are also difficult to study because of the complex interactions between various environmental factors and DOM. Here we propose the use of machine learning (ML) approaches to identify ecological processes contributing toward mixtures of DOM at a continental-scale. We related a dataset that characterized the molecular composition of DOM from river water and sediment with Fourier-transform ion cyclotron resonance mass spectrometry to explanatory physicochemical variables such as nutrient concentrations and stable water isotopes ( 2 H and 18 O). Using unsupervised ML, distinctive clusters for sediment and water samples were identified, with unique molecular compositions influenced by environmental factors like terrestrial input and microbial activity. Sediment clusters showed a higher proportion of protein-like and unclassified compounds than water clusters, while water clusters exhibited a more diversified chemical composition. We then applied a supervised ML approach, involving a two-stage use of SHapley Additive exPlanations (SHAP) values. In the first stage, SHAP values were obtained and used to identify key physicochemical variables. These parameters were employed to train models using both the default and subsequently tuned hyperparameters of the Histogram-based Gradient Boosting (HGB) algorithm. The supervised ML approach, using HGB and SHAP values, highlighted complex relationships between environmental factors and DOM diversity, in particular the existence of dams upstream, precipitation events, and other watershed characteristics were important in predicting higher chemical diversity in DOM. Our data-driven approach can now be used more generally to reveal the interplay between physical, chemical, and biological factors in determining the diversity of DOM in other ecosystems.

54 ENVIRONMENTAL SCIENCES↗

SQMS Quantum R&D in Machine Learning, Optimization and Sensing beyond Fundamental Physics Applications

This newly formed team at SQMS under the Ecosystem Thrust is looking to develop capabilities impacting societal advances outside the core domain of HEP and condensed matter physics. We explicitly leverage the experimental and algorithmic innovations developed across all groups as well as connect to broad-scope external projects of the diverse team of PIs. As the inaugural set of projects, we are studying numerically quantum machine learning models inspired by efficiently trainable echo-state and orthogonal neural networks and developing designs for related experiments to be performed on quantum processors based on SQMS SRF cQED technology and Rigetti s transmon arrays. Investigated models exploit ideas and lessons learned from multiple prior work by SQMS team members in a variety of internal and external activities [R1]. Target initial applications include noisy signal processing, potentially captured by quantum sensors or noisy QPUs, as well as simulation and classification of healthcare data. For instance, image reconstruction of the brain s electrical properties by solving the inverse Maxwell equation problem with uncertainty [R2] through a hybrid quantum-classical physics-informed architecture for time-dependent processes [R3]. The group is also investigating the application and development of novel quantum sensors based on magnetic levitation of a superconducting sphere coupled to a superconducting qubit. This coupling enables high-precision measurements of the position of the sphere, which can be used for sensitive detection of forces, enabling practical applications such as gravimetry for geophysics analysis, or accelerometry for GPS-denied navigation [R4] [R1] Rieffel, Eleanor G., Ata Akbari Asanjan, M. Sohaib Alam, Namit Anand, David E. Bernal Neira, Sophie Block, Lucas T. Brady et al. "Assessing and advancing the potential of quantum computing: A NASA case study." Future Generation Computer Systems (2024). [R2] Yu, X., Serrall s, J.E., Giannakopoulos, I.I., Liu, Z., Daniel, L., Lattanzi, R. and Zhang, Z., 2023. Pifon-ept: Mr-based electrical property tomography using physics-informed fourier networks. IEEE Journal on Multiscale and Multiphysics Computational Techniques. [R3] Wudarski, Filip, Daniel OConnor, Shaun Geaney, Ata Akbari Asanjan, Max Wilson, Elena Strbac, P. Aaron Lott, and Davide Venturelli. "Hybrid quantum-classical reservoir computing for simulating chaotic systems." arXiv preprint arXiv:2311.14105 (2023). [R4] Higgins, Gerard, Saarik Kalia, and Zhen Liu. "Maglev for dark matter: Dark-photon and axion dark matter sensing with levitated superconductors." Physical Review D 109.5 (2024): 055024.

Venturelli, Davide↗

Opportunities for enhancing MLCommons efforts while leveraging insights from educational MLCommons earthquake benchmarks efforts

MLCommons is an effort to develop and improve the artificial intelligence (AI) ecosystem through benchmarks, public data sets, and research. It consists of members from start-ups, leading companies, academics, and non-profits from around the world. The goal is to make machine learning better for everyone. In order to increase participation by others, educational institutions provide valuable opportunities for engagement. In this article, we identify numerous insights obtained from different viewpoints as part of efforts to utilize high-performance computing (HPC) big data systems in existing education while developing and conducting science benchmarks for earthquake prediction. As this activity was conducted across multiple educational efforts, we project if and how it is possible to make such efforts available on a wider scale. This includes the integration of sophisticated benchmarks into courses and research activities at universities, exposing the students and researchers to topics that are otherwise typically not sufficiently covered in current course curricula as we witnessed from our practical experience across multiple organizations. As such, we have outlined the many lessons we learned throughout these efforts, culminating in the need for benchmark carpentry for scientists using advanced computational resources. The article also presents the analysis of an earthquake prediction code benchmark while focusing on the accuracy of the results and not only on the runtime; notedly, this benchmark was created as a result of our lessons learned. Energy traces were produced throughout these benchmarks, which are vital to analyzing the power expenditure within HPC environments. Additionally, one of the insights is that in the short time of the project with limited student availability, the activity was only possible by utilizing a benchmark runtime pipeline while developing and using software to generate jobs from the permutation of hyperparameters automatically. It integrates a templated job management framework for executing tasks and experiments based on hyperparameters while leveraging hybrid compute resources available at different institutions. The software is part of a collection called cloudmesh with its newly developed components, cloudmesh-ee (experiment executor) and cloudmesh-cc (compute coordinator).

58 GEOSCIENCES↗

Improved accuracy and transferability of molecular-orbital-based machine learning: Organics, transition-metal complexes, non-covalent interactions, and transition states

We report molecular-orbital-based machine learning (MOB-ML) provides a general framework for the prediction of accurate correlation energies at the cost of obtaining molecular orbitals. The application of Nesbet’s theorem makes it possible to recast a typical extrapolation task, training on correlation energies for small molecules and predicting correlation energies for large molecules, into an interpolation task based on the properties of orbital pairs. We demonstrate the importance of preserving physical constraints, including invariance conditions and size consistency, when generating the input for the machine learning model. Numerical improvements are demonstrated for different datasets covering total and relative energies for thermally accessible organic and transition-metal containing molecules, non-covalent interactions, and transition-state energies. MOB-ML requires training data from only 1% of the QM7b-T dataset (i.e., only 70 organic molecules with seven and fewer heavy atoms) to predict the total energy of the remaining 99% of this dataset with sub-kcal/mol accuracy. This MOB-ML model is significantly more accurate than other methods when transferred to a dataset comprising of 13 heavy atom molecules, exhibiting no loss of accuracy on a size intensive (i.e., per-electron) basis. It is shown that MOB-ML also works well for extrapolating to transition-state structures, predicting the barrier region for malonaldehyde intramolecular proton-transfer to within 0.35 kcal/mol when only trained on reactant/product-like structures. Finally, the use of the Gaussian process variance enables an active learning strategy for extending the MOB-ML model to new regions of chemical space with minimal effort. We demonstrate this active learning strategy by extending a QM7b-T model to describe non-covalent interactions in the protein backbone–backbone interaction dataset to an accuracy of 0.28 kcal/mol.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Explainability and human intervention in autonomous scanning probe microscopy

The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.

47 OTHER INSTRUMENTATION↗

Thermodynamics of high-pressure ice phases explored with atomistic simulations

Most experimentally known high-pressure ice phases have a body-centred cubic (bcc) oxygen lattice. Our large-scale molecular-dynamics simulations with a machine-learning potential indicate that, amongst these bcc ice phases, ices VII, VII' and X are the same thermodynamic phase under different conditions, whereas superionic ice VII" has a first-order phase boundary with ice VII'. Moreover, at about 300 GPa, the transformation between ice X and the Pbcm phase has a sharp structural change but no apparent activation barrier, whilst at higher pressures the barrier gradually increases. Our study thus clarifies the phase behaviour of the high-pressure ices and reveals peculiar solid–solid transition mechanisms not known in other systems.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗