Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,711 records · Page 95
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Machine learning in a data-limited regime: Augmenting experiments with synthetic data uncovers order in crumpled sheets

Machine learning has gained widespread attention as a powerful tool to identify structure in complex, high-dimensional data. However, these techniques are ostensibly inapplicable for experimental systems where data are scarce or expensive to obtain. Here, we introduce a strategy to resolve this impasse by augmenting the experimental dataset with synthetically generated data of a much simpler sister system. Specifically, we study spontaneously emerging local order in crease networks of crumpled thin sheets, a paradigmatic example of spatial complexity, and show that machine learning techniques can be effective even in a data-limited regime. This is achieved by augmenting the scarce experimental dataset with inexhaustible amounts of simulated data of rigid flat-folded sheets, which are simple to simulate and share common statistical properties. This considerably improves the predictive power in a test problem of pattern completion and demonstrates the usefulness of machine learning in bench-top experiments where data are good but scarce.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A brain-plausible neuromorphic on-the-fly learning system implemented with magnetic domain wall analog memristors

Neuromorphic computing is an approach to efficiently solve complicated learning and cognition problems like the human brain using electronics. To efficiently implement the functionality of biological neurons, nanodevices and their implementations in circuits are exploited. Here, we describe a general-purpose spiking neuromorphic system that can solve on-the-fly learning problems, based on magnetic domain wall analog memristors (MAMs) that exhibit many different states with persistence over the lifetime of the device. The research includes micromagnetic and SPICE modeling of the MAM, CMOS neuromorphic analog circuit design of synapses incorporating the MAM, and the design of hybrid CMOS/MAM spiking neuronal networks in which the MAM provides variable synapse strength with persistence. Using this neuronal neuromorphic system, simulations show that the MAM-boosted neuromorphic system can achieve persistence, can demonstrate deterministic fast on-the-fly learning with the potential for reduced circuitry complexity, and can provide increased capabilities over an all-CMOS implementation.

42 ENGINEERING↗
DOE OSTI2019

Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning

While machine learning has been making enormous strides in many technical areas, it is still massively underused in transmission electron microscopy. To address this, a convolutional neural network model was developed for reliable classification of crystal structures from small numbers of electron images and diffraction patterns with no preferred orientation. Diffraction data containing 571,340 individual crystals divided among seven families, 32 genera, and 230 space groups were used to train the network. Despite the highly imbalanced dataset, the network narrows down the space groups to the top two with over 70% confidence in the worst case and up to 95% in the common cases. As examples, we benchmarked against alloys to two-dimensional materials to cross-validate our deep-learning model against high-resolution transmission electron images and diffraction patterns. We present this result both as a research tool and deep-learning application for diffraction analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Machine learning–based extreme event attribution

The observed increase in extreme weather has prompted recent methodological advances in extreme event attribution. We propose a machine learning–based approach that uses convolutional neural networks to create dynamically consistent counterfactual versions of historical extreme events under different levels of global mean temperature (GMT). We apply this technique to one recent extreme heat event (southcentral North America 2023) and several historical events that have been previously analyzed using established attribution methods. We estimate that temperatures during the southcentral North America event were 1.18° to 1.42°C warmer because of global warming and that similar events will occur 0.14 to 0.60 times per year at 2.0°C above preindustrial levels of GMT. Additionally, we find that the learned relationships between daily temperature and GMT are influenced by the seasonality of the forced temperature response and the daily meteorological conditions. Our results broadly agree with other attribution techniques, suggesting that machine learning can be used to perform rapid, low-cost attribution of extreme events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Leveraging data mining, active learning, and domain adaptation for efficient discovery of advanced oxygen evolution electrocatalysts

Developing advanced catalysts for acidic oxygen evolution reaction (OER) is crucial for sustainable hydrogen production. This study presents a multistage machine learning (ML) approach to streamline the discovery and optimization of complex multimetallic catalysts. Our method integrates data mining, active learning, and domain adaptation throughout the materials discovery process. Unlike traditional trial-and-error methods, this approach systematically narrows the exploration space using domain knowledge with minimized reliance on subjective intuition. Then, the active learning module efficiently refines element composition and synthesis conditions through iterative experimental feedback. The process culminated in the discovery of a promising Ru-Mn-Ca-Pr oxide catalyst. Our workflow also enhances theoretical simulations with domain adaptation strategy, providing deeper mechanistic insights aligned with experimental findings. By leveraging diverse data sources and multiple ML strategies, we demonstrate an efficient pathway for electrocatalyst discovery and optimization. This comprehensive, data-driven approach represents a paradigm shift and potentially benchmark in electrocatalysts research.

Science & Technology - Other Topics↗
DOE OSTI2022

Provably efficient machine learning for quantum many-body problems

Classical machine learning (ML) provides a potentially powerful approach to solving challenging quantum many-body problems in physics and chemistry. However, the advantages of ML over traditional methods have not been firmly established. In this work, we prove that classical ML algorithms can efficiently predict ground-state properties of gapped Hamiltonians after learning from other Hamiltonians in the same quantum phase of matter. By contrast, under a widely accepted conjecture, classical algorithms that do not learn from data cannot achieve the same guarantee. We also prove that classical ML algorithms can efficiently classify a wide range of quantum phases. Extensive numerical experiments corroborate our theoretical results in a variety of scenarios, including Rydberg atom systems, two-dimensional random Heisenberg models, symmetry-protected topological phases, and topologically ordered phases.

Science & Technology - Other Topics↗
DOE OSTI2022

Quantum advantage in learning from experiments

Quantum technology promises to revolutionize how we learn about the physical world. An experiment that processes quantum data with a quantum computer could have substantial advantages over conventional experiments in which quantum states are measured and outcomes are processed with a classical computer. We proved that quantum machines could learn from exponentially fewer experiments than the number required by conventional experiments. This exponential advantage is shown for predicting properties of physical systems, performing quantum principal component analysis, and learning about physical dynamics. Furthermore, the quantum resources needed for achieving an exponential advantage are quite modest in some cases. Conducting experiments with 40 superconducting qubits and 1300 quantum gates, we demonstrated that a substantial quantum advantage is possible with today’s quantum processors.

Science & Technology - Other Topics↗
DOE OSTI2025

Quantum learning advantage on a scalable photonic platform

Recent advances in quantum technologies have demonstrated that quantum systems can outperform classical ones in specific tasks, a concept known as quantum advantage. Although previous efforts have focused on computational speedups, a definitive and provable quantum advantage that is unattainable by any classical system has remained elusive. Here, in this work, we demonstrate a provable photonic quantum advantage by implementing a quantum-enhanced protocol for learning a high-dimensional physical process. Using imperfect Einstein–Podolsky–Rosen entanglement, we achieve a sample complexity reduction of 11.8 orders of magnitude compared to classical methods without entanglement. These results show that large-scale, provable quantum advantage is achievable with current photonic technology and represent a key step toward practical quantum-enhanced learning protocols in quantum metrology and machine learning.

Liu, Zheng-Hao [Technical Univ. of Denmark, Lyngby↗
DOE OSTI2023

Toward learning Lattice Boltzmann collision operators

In this work, we explore the possibility of learning from data collision operators for the Lattice Boltzmann Method using a deep learning approach. We compare a hierarchy of designs of the neural network (NN) collision operator and evaluate the performance of the resulting LBM method in reproducing time dynamics of several canonical flows. In the current study, as a first attempt to address the learning problem, the data were generated by a single relaxation time BGK operator. We demonstrate that vanilla NN architecture has very limited accuracy. On the other hand, by embedding physical properties, such as conservation laws and symmetries, it is possible to dramatically increase the accuracy by several orders of magnitude and correctly reproduce the short and long time dynamics of standard fluid flows.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Deep Learning and Structural Imaging of Materials

Deep learning has had a transformative effect on numerous domains and is actively utilized by many scientists in data-intensive fields such as high-energy physics and cosmology. Materials science, and in particular, the structural imaging of materials with electrons and X-rays are projected to enter the age of scientific data torrents, positioning them as new application spaces for modern artificial intelligence. In this contribution, we provide a synopsis on the foundations and latest progress in deep learning and present an in-depth application of utilizing modern deep artificial neural networks in scanning transmission electron microscopy to extract structural material properties. We use this case study to expose the strengths of deep learning-based models and discuss their current limitations, in the process highlighting their potential use in other data-intensive structural imaging modalities.

Laanait, Nouamane↗
DOE OSTI2012

Scalable Regression Tree Learning on Hadoop using OpenPlanet

As scientific and engineering domains attempt to effectively analyze the deluge of data arriving from sensors and instruments, machine learning is becoming a key data mining tool to build prediction models. Regression tree is a popular learning model that combines decision trees and linear regression to forecast numerical target variables based on a set of input features. Map Reduce is well suited for addressing such data intensive learning applications, and a proprietary regression tree algorithm, PLANET, using MapReduce has been proposed earlier. In this paper, we describe an open source implement of this algorithm, OpenPlanet, on the Hadoop framework using a hybrid approach. Further, we evaluate the performance of OpenPlanet using realworld datasets from the Smart Power Grid domain to perform energy use forecasting, and propose tuning strategies of Hadoop parameters to improve the performance of the default configuration by 75% for a training dataset of 17 million tuples on a 64-core Hadoop cluster on FutureGrid.

Yin, Wei↗
DOE OSTI2019

Towards Misregistration-Tolerant Change Detection using Deep Learning Techniques with Object-Based Image Analysis

Co-registrating is a common pre-processing step for existing change detection algorithms, but registering bi-temporal images is nontrivial. The use of image patch as input for deep learning techniques provides a natural avenue to apply them in the OBIA framework, and have shown successful performance in the object-based land cover mapping and change detection applications. Even though attempts of applying deep learning techniques for change detection applications have been made with varying success, its application under OBIA framework for change detection have not been conducted and its tolerance for misregistration among temporal images are neither known. This study performed change detection under OBIA framework using deep learning techniques for the first time, and evaluated its performance regarding their tolerance of image misregistration on training and testing dataset. Our results demonstrate the proposed change detection scheme is surprisingly robust to image misregistration on the testing dataset, while classifiers trained with the training dataset containing image misregistration errors suffer from slight decrease of overall accuracy.

Liu, Tao↗
DOE OSTI2019

Scalable Causal Graph Learning through a Deep Neural Network

Learning the causal graph in a complex system is crucial for knowledge discovery and decision making, yet it remains a challenging problem because of the unknown nonlinear interaction among system components. Most of the existing methods either rely on predefined kernel or data distribution, or they focus simply on the causality between a single target and the remaining system. This work presents a deep neural network for scalable causal graph learning (SCGL) through low-rank approximation. The SCGL model can explore nonlinearity on both temporal and intervariable relationships without any predefined kernel or distribution assumptions. Through low-rank approximation, the noise influence is reduced, and better accuracy and high scalability are achieved. Experiments using synthetic and real-world datasets show that our SCGL algorithm outperforms existing state-of-the-art methods for causal graph learning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Few-shot Learning for Post-disaster Structure Damage Assessment

Automating post-disaster damage assessment with remote sensing data is critical for faster surveys of structures impacted by natural disasters. One significant obstacle to training state-of-the-art deep neural networks to support this automation is that large quantities of labelled data are often required. However, obtaining those labels is particularly unrealistic to support post-disaster damage assessment in a timely manner. Few-shot learning methods could help to mitigate this by reducing the amount of labelled data required to successfully train a model while achieving satisfactory results. To this end, we explore a feature reweighting method to the YOLOv3 object detection architecture to achieve few-shot learning of damage assessment models on the xBD dataset. Our results show that the feature reweighting approach yield improved mAP over the baseline with significantly fewer labelled samples. In addition, we use t-SNE to analyze the class-specific reweighting vectors generated by the reweighting module in order to evaluate their inter-class and intra-class similarity. We find that the vectors form clusters based on class, and that these clusters overlap with visually similar classes. Those results show the potential to employ this few-shot learning strategy for rapid damage assessment with post-event remote sensing images.

Bowman, Jordan↗
DOE OSTI2022

Virtual Cybersecurity Camps: Lessons Learned

Cybersecurity camps provide participants with an opportunity to learn about cybersecurity in a fun and safe environment. Traditionally, such camps, like many others, are held in-person. However, the COVID-19 pandemic created unique challenges and also an opportunity to counter those challenges—holding a cybersecurity camp virtually. While countless other camps, both cybersecurityrelated and others, moved to a virtual environment so that such camps could continue to be held, this paper presents some lessons learned and suggestions that may be helpful to others deciding to hold a virtual camp in the future. Some of the lessons learned may be specific to a cybersecurity camp, but most would be applicable to a broad audience.

cyber security education, student camp, STEM educa↗
DOE OSTI2022

Deep reinforcement learning with online data augmentation to improve sample efficiency for intelligent HVAC control

Deep Reinforcement Learning (DRL) has started showing success in real-world applications such as building energy optimization. Much of the research in this space utilized simulated environments to train RL-agent in an offline mode. Very few research have used DRL-based control in real-world systems due to two main reasons: 1) sample efficiency challenge---DRL approaches need to perform a lot of interactions with the environment to collect sufficient experiences to learn from, which is difficult in real systems, and 2) comfort or safety related constraints---user's comfort must never or at least rarely be violated. In this work, we propose a novel deep Reinforcement Learning framework with online Data Augmentation (RLDA) to address the sample efficiency challenge of real-world RL. We used a time series Generative Adversarial Network (TimeGAN) architecture as a data generator. We further evaluated the proposed RLDA framework using a case study of an intelligent HVAC control. With a ≈28% improvement in the sample efficiency, RLDA framework lays the way towards increased adoption of DRL-based intelligent control in real-world building energy management systems.

Kurte, Kuldeep↗
DOE OSTI2023

Tournament-Based Pretraining to Accelerate Federated Learning

Advances in hardware, proliferation of compute at the edge, and data creation at unprecedented scales have made federated learning (FL) necessary for the next leap forward in pervasive machine learning. For privacy and network reasons, large volumes of data remain stranded on endpoints located in geographically austere (or at least austere network-wise) locations. However, challenges exist to the effective use of these data. To solve the system and functional level challenges, we present an three novel variants of a serverless federated learning framework. We also present tournament-based pretraining, which we demonstrate significantly improves model performance in some experiments. Overall, these extensions to FL and our novel training method enable greater focus on science rather than ML development.

Baughman, Matt↗
DOE OSTI2024

A Deep Multimodal Representation Learning Framework for Accurate Molecular Properties Prediction

Drug discovery is a complex and challenging process, requiring the optimization of candidate compounds to identify those with the potential to become safe and effective drugs. Predicting molecular properties is an indispensable step in the drug discovery pipeline. Traditionally, this process is costly and time-intensive, involving multiple rounds of experiments and clinical trials, rendering it impractical for every candidate compound. Deep learning techniques have emerged as a promising approach to drug discovery to reduce the cost and time required to identify novel drugs. However, prevalent research in deep learning models focused on predicting molecular properties has primarily fixated on single-modal models, which utilize a single modality of data, neglecting the potential benefits of combining different data modalities. To overcome this limitation, we introduce MRL-Mol: a deep \textbf{M}ultimodal \textbf{R}epresentation \textbf{L}earning framework for accurate \textbf{Mol}ecular properties prediction. MRL-Mol harnesses three data modalities: sequence, graph, and image, augmenting the depth of comprehension. Leveraging a large-scale unlabeled dataset~($\sim$1M unique molecules), we pretrain MRL-Mol to extract inter- and intra-modal information. Our study demonstrates the superior performance of MRL-Mol in predicting molecular properties across six benchmark datasets, including both classification and regression tasks. Notably, MRL-Mol outperforms other state-of-the-art molecular properties prediction models. These findings suggest that by combining information from multiple data modalities, MRL-Mol can comprehend molecules better than single-modal deep learning models and identify molecular properties with better accuracy.

Yang, Yuxin↗