Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Accelerating the discovery of battery electrode materials through data mining and deep learning models

The availability of crystalline materials databases allows for building accurate machine learning (ML) models that can accelerate the exploration of materials chemical space for energy storage applications. In this work, we screen all inorganic materials included in the Materials Project and AFLOW databases as potential metal-ion battery electrodes. We develop an efficient protocol to mine and screen raw data in current databases and provide a new database of electrode materials by considering pairs of charged and discharged electrodes. This effort leads to a new database with over 190,000 instances, in contrast to the original battery database which contains about 5000. The expanded battery data set is then used to build regression-based deep neural network models for predicting average voltages and percentage volume changes upon charging and discharging, which present improvements of at least 28% for target properties with respect to previous models, and are now able to predict anode electrodes (low voltage region) as well as electrodes that will not work in electrochemical cells (negative voltages), overcoming the challenges identified in previous ML models for battery electrodes. Additionally, a further screening of the expanded database itself allowed us to identify 35 novel electrode candidates with excellent battery performance metrics.

25 ENERGY STORAGE↗
DOE OSTI2024

Machine learning without a processor: Emergent learning in a nonlinear analog network

Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic contrastive local learning networks (CLLNs) offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here, we introduce a nonlinear CLLN—an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR (exclusive or) and nonlinear regression, without a computer. We find our decentralized system reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

Science & Technology - Other Topics↗
DOE OSTI2024

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

In additive manufacturing (AM), process defects such as keyhole pores are difficult to anticipate, affecting the quality and integrity of the AM-produced materials. Hence, considerable efforts have aimed to predict these process defects by training machine learning (ML) models using passive measurements such as acoustic emissions. This work considered a dataset in which keyhole pores of a laser powder bed fusion (LPBF) experiment were identified using X-ray radiography and then registered both in space and time to acoustic measurements recorded during the LPBF experiment. Due to AM’s intrinsic process controls, where a pore-forming event is relatively rare, the acoustic datasets collected during monitoring include more non-pores than pores. In other words, the dataset for ML model development is imbalanced. Moreover, this imbalanced and sparse data phenomenon remains ubiquitous across many AM monitoring schemes since training data is nontrivial to collect. Hence, we propose a machine learning approach to improve this dataset imbalance and enhance the prediction accuracy of pore-labeled data. Specifically, we investigate how data augmentation helps predict pores and non-pores better. This imbalance is improved using recent advances in data augmentation called Mixup, a weak-supervised learning method. Convolutional neural networks (CNNs) are trained on original and augmented datasets, and an appreciable increase in performance is reported when testing on five different experimental trials. When ML models are trained on original and augmented datasets, they achieve an accuracy of 95% and 99% on test datasets, respectively. We also provide information on how dataset size affects model performance. Lastly, we investigate the optimal Mixup parameters for augmentation in the context of CNN performance.

36 MATERIALS SCIENCE↗
DOE OSTI2025

The Artificial Intelligence Ontology: LLM-Assisted Construction of AI Concept Hierarchies

The Artificial Intelligence Ontology (AIO) is a systematization of artificial intelligence (AI) concepts, methodologies, and their interrelations. Developed via manual curation, with the additional assistance of large language models (LLMs), AIO aims to address the rapidly evolving landscape of AI by providing a comprehensive framework that encompasses both technical and ethical aspects of AI technologies. The primary audience for AIO includes AI researchers, developers, and educators seeking standardized terminology and concepts within the AI domain. We use the term “branches” for classes, and their subclasses, in our ontology that are subclasses of owl:Thing. AIO contains eight branches: Bias, Layer, Machine Learning Task, Mathematical Function, Model, Network, Preprocessing, and Training Strategy, each designed to support the modular composition of AI methods and facilitate a deeper understanding of deep learning architectures and ethical considerations in AI. AIO uses the Ontology Development Kit (ODK) for its creation and maintenance, with its content being more easily updated through AI-driven curation support. This approach not only ensures the ontology's relevance amidst the fast-paced advancements in AI but also significantly enhances its utility for researchers, developers, and educators by simplifying the integration of new AI concepts and methodologies. The ontology's utility is demonstrated through the annotation of AI methods data in a catalog of AI research publications and the integration into the BioPortal ontology resource, highlighting its potential for cross-disciplinary research. The AIO ontology is open source and is available on GitHub ( https://w3id.org/aio/ ) and BioPortal ( https://bioportal.bioontology.org/ontologies/AIO ).

Joachimiak, Marcin P. [Biosystems Data Science Dep↗
DOE OSTI2020

Machine learning uncovers independently regulated modules in the Bacillus subtilis transcriptome

The transcriptional regulatory network (TRN) of Bacillus subtilis coordinates cellular functions of fundamental interest, including metabolism, biofilm formation, and sporulation. Here, we use unsupervised machine learning to modularize the transcriptome and quantitatively describe regulatory activity under diverse conditions, creating an unbiased summary of gene expression. We obtain 83 independently modulated gene sets that explain most of the variance in expression and demonstrate that 76% of them represent the effects of known regulators. The TRN structure and its condition-dependent activity uncover putative or recently discovered roles for at least five regulons, such as a relationship between histidine utilization and quorum sensing. The TRN also facilitates quantification of population-level sporulation states. As this TRN covers the majority of the transcriptome and concisely characterizes the global expression state, it could inform research on nearly every aspect of transcriptional regulation in B. subtilis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Journey Over Destination: Differentiable Sensor Placement Enhances Generalization [Poster]

The challenge of reconstructing spatial fields that change over time from limited sensor data has been a focal point for many research studies. Various machine learning methods have been used in attempts to address this complex issue, including convolutional neural networks. All of the proposed methods share a common requirement that the user needs to manually determine the sensor positions. This requirement remains a limiting factor in the ongoing quest for efficient learning and accurate field reconstruction. This study aims to present a method that enables a model to optimize sensor positions via backpropagation, thereby facilitating the model’s exploration of the spatial domain and enhancing sensor positioning effectively. Indexing naturally incorporates discrete decisions. This operation is nondifferentiable which is a requirement for the application of gradient-based optimization methods. We showcased its effectiveness by training an attention-based neural network, which achieved top-tier performance on two separate datasets. To our knowledge, this represents the first fully end-to-end differentiable workflow for enhancing sensor placement within a neural network model.

58 GEOSCIENCES↗
DOE OSTI2022

Generalization in quantum machine learning from few training data

Modern quantum machine learning (QML) methods involve variationally optimizing a parameterized quantum circuit on a training data set, and subsequently making predictions on a testing data set (i.e., generalizing). In this work, we provide a comprehensive study of generalization performance in QML after training on a limited number N of training data points. We show that the generalization error of a quantum machine learning model with T trainable gates scales at worst as $\sqrt{T/N}$. When only K$\ll$T gates have undergone substantial change in the optimization process, we prove that the generalization error improves to $\sqrt{K/N}$. Our results imply that the compiling of unitaries into a polynomial number of native gates, a crucial application for the quantum computing industry that typically uses exponential-size training data, can be sped up significantly. We also show that classification of quantum states across a phase transition with a quantum convolutional neural network requires only a very small training data set. Other potential applications include learning quantum error correcting codes or quantum dynamical simulation. Our work injects new hope into the field of QML, as good generalization is guaranteed from few training data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Biosensor and machine learning-aided engineering of an amaryllidaceae enzyme

A major challenge to achieving industry-scale biomanufacturing of therapeutic alkaloids is the slow process of biocatalyst engineering. Amaryllidaceae alkaloids, such as the Alzheimer’s medication galantamine, are complex plant secondary metabolites with recognized therapeutic value. Due to their difficult synthesis they are regularly sourced by extraction and purification from the low-yielding daffodil Narcissus pseudonarcissus. Here, we propose an efficient biosensor-machine learning technology stack for biocatalyst development, which we apply to engineer an Amaryllidaceae enzyme in Escherichia coli. Directed evolution is used to develop a highly sensitive (EC 50 = 20 μM) and specific biosensor for the key Amaryllidaceae alkaloid branchpoint 4’-O-methylnorbelladine. A structure-based residual neural network (MutComputeX) is subsequently developed and used to generate activity-enriched variants of a plant methyltransferase, which are rapidly screened with the biosensor. Functional enzyme variants are identified that yield a 60% improvement in product titer, 2-fold higher catalytic activity, and 3-fold lower off-product regioisomer formation. A solved crystal structure elucidates the mechanism behind key beneficial mutations.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)

Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine learning for neutron reflectometry data analysis of two-layer thin films

Neutron reflectometry (NR) is a powerful tool for probing thin films at length scales down to nanometers. We investigated the use of a neural network to predict a two-layer thin film structure to model a given measured reflectivity curve. Application of this neural network to predict a thin film structure revealed that it was accurate and could provide an excellent starting point for traditional fitting methods. Employing prediction-guided fitting has considerable potential for more rapidly producing a result compared to the labor-intensive but commonly-used approach of trial and error searches prior to refinement. A deeper look at the stability of the predictive power of the neural network against statistical fluctuations of measured reflectivity profiles showed that the predictions are stable. We conclude that the approach presented here can provide valuable assistance to users of NR and should be further extended for use in studies of more complex n-layer thin film systems. This result also opens up the possibility of developing adaptive measurement systems in the future.

36 MATERIALS SCIENCE↗
DOE OSTI2025

AutoTandemML: Active Learning Enhanced Tandem Neural Networks for Inverse Design Problems

Inverse design in science and engineering involves determining optimal design parameters that achieve desired performance outcomes, a process often hindered by the complexity and high dimensionality of design spaces, leading to significant computational costs. To tackle this challenge, we propose a novel hybrid approach that combines active learning with Tandem Neural Networks to enhance the efficiency and effectiveness of solving inverse design problems. Active learning allows to selectively sample the most informative data points, reducing the required dataset size without compromising accuracy. We investigate this approach using three benchmark problems: airfoil inverse design, photonic surface inverse design, and scalar boundary condition reconstruction in diffusion partial differential equations. We demonstrate that integrating active learning with Tandem Neural Networks outperforms standard approaches across the benchmark suite, achieving better accuracy with fewer training samples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Deep learning Hamiltonians from disordered image data in quantum materials

The capabilities of image probe experiments are rapidly expanding, providing new information about quantum materials on unprecedented length- and timescales. Many such materials feature inhomogeneous electronic properties with intricate pattern formation on the observable surface. This rich spatial structure contains information about interactions, dimensionality, and disorder—a spatial encoding of the Hamiltonian driving the pattern formation. Image recognition techniques from machine learning are an excellent tool for interpreting information encoded in the spatial relationships in such images. Here, we develop a deep learning framework for using the rich information available in these spatial correlations in order to discover the underlying Hamiltonian driving the patterns. We first vet the method on a known case, scanning near-field optical microscopy on a thin film of V⁢O 2 . We then apply our trained convolutional neural network architecture to new optical microscope images of a different V⁢O 2 film as it goes through the metal-insulator transition. We find that a two-dimensional Hamiltonian with both interactions and random field disorder is required to explain the intricate, fractal intertwining of metal and insulator domains during the transition. This detailed knowledge about the underlying Hamiltonian paves the way for using the model to control the pattern formation via, e.g., tailored hysteresis protocols. Finally, we also introduce a distribution-based confidence measure on the results of a multilabel classifier, which does not rely on adversarial training. In addition, we propose a machine-learning-based criterion for diagnosing a physical system's proximity to criticality.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2020

Reification of latent microstructures: On supervised unsupervised and semi-supervised deep learning applications for microstructures in materials informatics

Machine learning (ML), including deep learning (DL), has become increasingly popular in the last few years due to its continually outstanding performance. In this context, we apply machine learning techniques to "learn" the microstructure using both supervised and unsupervised DL techniques. In particular, we focus (1) on the localization problem bridging (micro)structure (localized) property using supervised DL and (2) on the microstructure reconstruction problem in latent space using unsupervised DL. The goal of supervised and semi-supervised DL is to replace crystal plasticity finite element model (CPFEM) that maps from (micro)structure (localized) property, and implicitly the (micro)structure (homogenized) property relationships, while the goal of unsupervised DL is (1) to represent high-dimensional microstructure images in a non-linear low-dimensional manifold, and (2) to discover a way to interpolate microstructures via latent space associating with latent microstructure variables. At the heart of this report is the applications of several common DL architectures, including convolutional neural networks (CNN), autoencoder (AE), and generative adversarial network (GAN), to multiple microstructure datasets, and the quest of neural architecture search for optimal DL architectures.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Multi‐Decadal Dynamics of Wetland Methane Emissions Revealed by Knowledge‐Guided Machine Learning

Measurement of methane fluxes (FCH 4 ) from natural systems, such as wetlands, has lagged far behind carbon dioxide fluxes. Short and fragmented wetland FCH 4 data limit our ability to assess its long-term dynamics and potential climate feedbacks. Extrapolating short-term FCH 4 records to recent decades remains challenging for both process-based models and data-driven machine learning (ML) approaches. Here, we develop a knowledge-guided ML framework that integrates eddy covariance (EC) FCH 4 observations, field warming experiments, and biogeochemical knowledge to reconstruct the long-term FCH 4 budgets and trends. Focusing on the 11 longest EC monitoring sites in the AmeriFlux network, we found considerable variability in multi-decadal trends of wetland FCH 4 , with increases up to 14% per decade from 2000 to 2024. We also found that the strength of these increasing trends declines from high to low latitudes, highlighting the vulnerability of northern wetlands. This work presents novel and robust reconstructions of long-term wetland FCH 4 , offering critical benchmark datasets for bottom-up ecosystem models and advancing fundamental understanding of wetland biogeochemistry.

AmeriFlux site↗
DOE OSTI2021

ECNet is an evolutionary context-integrated deep learning framework for protein engineering

Abstract Machine learning has been increasingly used for protein engineering. However, because the general sequence contexts they capture are not specific to the protein being engineered, the accuracy of existing machine learning algorithms is rather limited. Here, we report ECNet (evolutionary context-integrated neural network), a deep-learning algorithm that exploits evolutionary contexts to predict functional fitness for protein engineering. This algorithm integrates local evolutionary context from homologous sequences that explicitly model residue-residue epistasis for the protein of interest with the global evolutionary context that encodes rich semantic and structural features from the enormous protein sequence universe. As such, it enables accurate mapping from sequence to function and provides generalization from low-order mutants to higher-order mutants. We show that ECNet predicts the sequence-function relationship more accurately as compared to existing machine learning algorithms by using ~50 deep mutational scanning and random mutagenesis datasets. Moreover, we used ECNet to guide the engineering of TEM-1 β-lactamase and identified variants with improved ampicillin resistance with high success rates.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Autonomous scanning probe microscopy investigations over WS2 and Au{111}

Abstract Individual atomic defects in 2D materials impact their macroscopic functionality. Correlating the interplay is challenging, however, intelligent hyperspectral scanning tunneling spectroscopy (STS) mapping provides a feasible solution to this technically difficult and time consuming problem. Here, dense spectroscopic volume is collected autonomously via Gaussian process regression, where convolutional neural networks are used in tandem for spectral identification. Acquired data enable defect segmentation, and a workflow is provided for machine-driven decision making during experimentation with capability for user customization. We provide a means towards autonomous experimentation for the benefit of both enhanced reproducibility and user-accessibility. Hyperspectral investigations on WS 2 sulfur vacancy sites are explored, which is combined with local density of states confirmation on the Au{111} herringbone reconstruction. Chalcogen vacancies, pristine WS 2 , Au face-centered cubic, and Au hexagonal close-packed regions are examined and detected by machine learning methods to demonstrate the potential of artificial intelligence for hyperspectral STS mapping.

36 MATERIALS SCIENCE↗
DOE OSTI2023

A deep neural network regressor for phase constitution estimation in the high entropy alloy system Al-Co-Cr-Fe-Mn-Nb-Ni

High Entropy Alloys (HEAs) are composed of more than one principal element and constitute a major paradigm in metals research. The HEA space is vast and an exhaustive exploration is improbable. Therefore, a thorough estimation of the phases present in the HEA is of paramount importance for alloy design. Machine Learning presents a feasible and non-expensive method for predicting possible new HEAs on-the-fly. A deep neural network (DNN) model for the elemental system of: Mn, Ni, Fe, Al, Cr, Nb, and Co is developed using a dataset generated by high-throughput computational thermodynamic calculations using Thermo-Calc. The features list used for the neural network is developed based on literature and freely available databases. A feature significance analysis matches the reported HEAs phase constitution trends on elemental properties and further expands it by providing so far-overlooked features. The final regressor has a coefficient of determination ( r 2 ) greater than 0.96 for identifying the most recurrent phases and the functionality is tested by running optimization tasks that simulate those required in alloy design. The DNN developed constitutes an example of an emulator that can be used in fast, real-time materials discovery/design tasks.

36 MATERIALS SCIENCE↗