Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning tools”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗
DOE OSTI2023

Design and Operation of a Multi-Bed Catalytic Micro-Reactor for the Study of Co-Processing of Bio-Oils with VGO

An industry wide shift from fossil-based fuel to renewable fuel sources including biomass, municipal waste, and plastics will require new process monitoring methods to minimize transitional risks including off specification product formation and catalyst deactivation. This project aims to provide a machine learning based process monitoring tool composed of online, slipstream mass spectra for use in biomass refineries and co-processing in existing refineries allowing operators to monitor product qualities and adjust process conditions accordingly. In order to maximize the robustness of the tool, large volumes of data must be collected to fine tune model parameters which consists of both micro and pilot scale mass spectral data. Micro-scale data is collected with a multi-tube micro-reactor housing up to six catalysts in horizontal beds, coupled with a molecular beam mass spectrometer. A pyrolizer equipped with an auto-sampler streamlines the micro-scale data collection process. This type of pyrolizer/micro-reactor configuration does not exist on the market, and therefore had to be created for the purposes of this project. The design and commissioning of this reactor will be presented in detail. This reactor set-up is highly flexible and increases throughput of analysis. For catalyst testing, each bed can be individually selected simply by turning valves. For catalyst reduction and regeneration, simultaneous flow through all six beds is used. The reproducibility of the system was first assessed with whole biomass pyrolysis along with pyrolysis of calibration standards. Initial work on this system evaluated two FCC catalysts, equilibrium catalyst (E-cat), and a proprietary catalyst from Johnson Matthey specifically design for co-processing of bio-oil with vacuum gas oil (VGO). This work used model compounds and VGO which illuminated differences in products produced by the catalysts.

biomass↗
DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

MicroBooNE, a Liquid Argon Time Projection Chamber (LArTPC) located in the $\nu_{\mu}$-dominated Booster Neutrino Beam at Fermilab, has been studying $\nu_{e}$ charged-current (CC) interaction rates to shed light on the MiniBooNE low energy excess. The LArTPC technology employed by MicroBooNE provides the capability to image neutrino interactions with mm-scale precision. Computer vision and other machine learning techniques are promising tools for image processing that could boost efficiencies for selecting $\nu_{e}$-CC and other rare signals, reduce cosmic and beam-induced backgrounds, and improve the reconstruction of neutrino energies. The MicroBooNE experiment has been at the forefront of developing and testing such techniques for use in physics analyses. In this poster we overview deep-learning based reconstruction methods. We will showcase the use of a recurrent neural network to estimate neutrino energies and present a new reconstruction framework that uses convolutional neural networks to locate neutrino interaction vertices, tag pixels with track and shower labels, and perform particle identification on reconstructed clusters. We will present studies characterizing the performance of these new tools and demonstrate their effectiveness through their use in an inclusive $\nu_{e}$-CC event selection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back

Computational chemistry is an indispensable tool for understanding molecules and predicting chemical properties. However, traditional computational methods face significant challenges due to the difficulty of solving the Schrödinger equations and the increasing computational cost with the size of the molecular system. In response, there has been a surge of interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to in silico experiments. Integrating AI and ML into computational chemistry increases the scalability and speed of the exploration of chemical space. However, challenges remain, particularly regarding the reproducibility and transferability of ML models. This review highlights the evolution of ML in learning from, complementing, or replacing traditional computational chemistry for energy and property predictions. Starting from models trained entirely on numerical data, a journey set forth toward the ideal model incorporating or learning the physical laws of quantum mechanics. This paper also reviews existing computational methods and ML models and their intertwining, outlines a roadmap for future research, and identifies areas for improvement and innovation. Ultimately, the goal is to develop AI architectures capable of predicting accurate and transferable solutions to the Schrödinger equation, thereby revolutionizing in silico experiments within chemistry and materials science.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

ppdx : Automated modeling of protein–protein interaction descriptors for use with machine learning

This paper describes ppdx, a python workflow tool that combines protein sequence alignment, homology modeling, and structural refinement, to compute a broad array of descriptors for characterizing protein–protein interactions. The descriptors can be used to predict various properties of interest, such as protein–protein binding affinities, or inhibitory concentrations (IC 50 ), using approaches that range from simple regression to more complex machine learning models. The software is highly modular. It supports different protocols for generating structures, and 95 descriptors can be currently computed. More protocols and descriptors can be easily added. The implementation is highly parallel and can fully exploit the available cores in a single workstation, or multiple nodes on a supercomputer, allowing many systems to be analyzed simultaneously. As an illustrative application, ppdx is used to parametrize a model that predicts the IC 50 of a set of antigens and a class of antibodies directed to the influenza hemagglutinin stalk.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Advancing the Prediction of MS/MS Spectra Using Machine Learning

Tandem mass spectrometry (MS/MS) is an important tool for the identification of small molecules and metabolites where resultant spectra are most commonly identified by matching them with spectra in MS/MS reference libraries. While popular, this strategy is limited by the contents of existing reference libraries. In response to this limitation, various methods are being developed for the in silico generation of spectra to augment existing libraries. Recently, machine learning and deep learning techniques have been applied to predict spectra with greater speed and accuracy. Here, in this work, we investigate the challenges these algorithms face in achieving fast and accurate predictions on a wide range of small molecules. The challenges are often amplified by the use of generic machine learning benchmarking tactics, which lead to misleading accuracy scores. Curating data sets, only predicting spectra for sufficiently high collision energies, and working more closely with experimental mass spectrometrists are recommended strategies to improve overall prediction accuracy in this nuanced field.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Bond order predictions using deep neural networks

Machine learning is an extremely powerful tool for the modern theoretical chemist since it provides a method for bypassing costly algorithms for solving the Schrödinger equation. Already, it has proven able to infer molecular and atomic properties such as charges, enthalpies, dipoles, excited state energies, and others. Most of these machine learning algorithms proceed by inferring properties of individual atoms, even breaking down total molecular energy into individual atomic contributions. In this paper, we introduce a modified version of the Hierarchically Interacting Particle Neural Network (HIP-NN) capable of making predictions on the bonds between atoms rather than on the atoms themselves. We train the modified HIP-NN to infer bond orders for a large number of small organic molecules as computed via the Natural Bond Orbital package. We demonstrate that the trained model is extensible to molecules much larger than those in the training set by studying its performance on the COMP6 dataset. This method has applications in cheminformatics and force field parameterization and opens a promising future for machine learning models to predict other quantities that are defined between atoms such as density matrix elements, Hamiltonian parameters, and molecular reactivities.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Method to simultaneously facilitate all jet physics tasks

Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that specially constructed machine learning models trained for a specific jet classification task can improve the accuracy, precision, or speed of all other jet physics tasks. This is demonstrated by training on a particular multiclass generation and classification task and then using the learned representation for different generation and classification tasks, for datasets with a different (full) detector simulation, for jets from a different collision system ($pp$ versus $ep$), for generative models, for likelihood ratio estimation, and for anomaly detection. We consider our omnilearn approach thus as a jet-physics foundation model. It is made publicly available for use in any area where state-of-the-art precision is required for analyses involving jets and their substructure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRSDate not supplied

Machine Learning Explainability and Transferability for Path Navigation

Deep neural networks are powerful tools for machine perception. Unfortunately their decisions are difficult to explain due to the complexity and size of the networks. Previously we have alleviated this issue by using the representational portion of a deep neural network and combining it with a k-nearest neighbor (KNN) classifier. Through inspection of the decisions made by the KNN, we can directly see the training data responsible for the decisions, allowing us to determine the quality of the overall decision and the quality of the representational layer of the deep NN. While the technique worked well, it requires tens of thousands of latent vectors to be stored for classification. In addition, it lacks the ability to show how parts of an image influence the classification decision. Here we address these issues by 1) Using a radial basis function network (RBFN) in place of the KNN allowing far fewer images to be used in deployment and 2) Using an auto encoder network for explainability. In addition to these techniques, we examine the effects of transfer learning to determine that results are robust. All results are tested on a domain where an unmanned aerial vehicle (UAV) navigates a forest trail through a single camera.

explainable machine learning↗
DOE OSTI2024

Machine learning for accuracy in density functional approximations

Machine learning techniques have found their way into computational chemistry as indispensable tools to accelerate atomistic simulations and materials design. In addition, machine learning approaches hold the potential to boost the predictive power of computationally efficient electronic structure methods, such as density functional theory, to chemical accuracy and to correct for fundamental errors in density functional approaches. In this paper, recent progress in applying machine learning to improve the accuracy of density functional and related approximations is reviewed. Promises and challenges in devising machine learning models transferable between different chemistries and materials classes are discussed with the help of examples applying promising models to systems far outside their training sets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Improving Detection of Disease Re-emergence Using a Web-Based Tool (RED Alert): Design and Case Analysis Study

Background: Currently, the identification of infectious disease re-emergence is performed without describing specific quantitative criteria that can be used to identify re-emergence events consistently. This practice may lead to ineffective mitigation. In addition, identification of factors contributing to local disease re-emergence and assessment of global disease re-emergence require access to data about disease incidence and a large number of factors at the local level for the entire world. This paper presents Re-emerging Disease Alert (RED Alert), a web-based tool designed to help public health officials detect and understand infectious disease re-emergence. Objective: Our objective is to bring together a variety of disease-related data and analytics needed to help public health analysts answer the following 3 primary questions for detecting and understanding disease re-emergence: Is there a potential disease re-emergence at the local (country) level? What are the potential contributing factors for this re-emergence? Is there a potential for global re-emergence? Methods: We collected and cleaned disease-related data (eg, case counts, vaccination rates, and indicators related to disease transmission) from several data sources including the World Health Organization (WHO), Pan American Health Organization (PAHO), World Bank, and Gideon. We combined these data with machine learning and visual analytics into a tool called RED Alert to detect re-emergence for the following 4 diseases: measles, cholera, dengue, and yellow fever. We evaluated the performance of the machine learning models for re-emergence detection and reviewed the output of the tool through a number of case studies. Results: Our supervised learning models were able to identify 82%-90% of the local re-emergence events, although with 18%-31% (except 46% for dengue) false positives. This is consistent with our goal of identifying all possible re-emergences while allowing some false positives. The review of the web-based tool through case studies showed that local re-emergence detection was possible and that the tool provided actionable information about potential factors contributing to the local disease re-emergence and trends in global disease re-emergence. Conclusions: To the best of our knowledge, this is the first tool that focuses specifically on disease re-emergence and addresses the important challenges mentioned above.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Training Data Regarding Optimal Power Flow For Efficient Machine Learning

The code is intended to take the data outputted by the MATPOWER OPF solver tools and restructures the data in a way that is best suited for machine learning. The variables are read and the desired values are taken and added to an array in the correct format. This array is then converted into a python array for future use. The code also utilizes MATPOWER's ability to construct different power flow scenarios and repeat them a chosen number of times. Each iteration will add a new line to the formatted array so that the final output is a matrix has a height equal to the number of repetition used.

Seidel, RachaelI↗
DOE OSTI2020

The Role of Machine Learning in the Understanding and Design of Materials

Developing algorithmic approaches for the rational design and discovery of materials can enable us to systematically find novel materials, which can have huge technological and social impact. However, such rational design requires a holistic perspective over the full multistage design process, which involves exploring immense materials spaces, their properties, and process design and engineering as well as a techno-economic assessment. The complexity of exploring all of these options using conventional scientific approaches seems intractable. Instead, novel tools from the field of machine learning can potentially solve some of our challenges on the way to rational materials design. Here we review some of the chief advancements of these methods and their applications in rational materials design, followed by a discussion on some of the main challenges and opportunities we currently face together with our perspective on the future of rational materials design and discovery.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Using feature importance as an exploratory data analysis tool on Earth system models

Abstract. Machine learning (ML) models are commonly used to generate predictions, but these models can also support the discovery of new science. Generating accurate predictions necessitates that a model captures the structure of the underlying data. If the structure is properly extracted, ML could be a useful exploratory and evidential tool. In this paper, we present a case study that demonstrates the use of ML for exploratory data analysis (EDA) in the climate space. We apply the ML explainability method of spatiotemporal zeroed feature importance (stZFI) to understand how climate-variable associations evolve over space and time. Our analyses focus on data from ensembles of Earth system models (ESMs) which provide data on different climate states and conditions. We elect to work with ESM ensembles since they allow us to compare feature importance across alternative scenarios not available with observed data. The ensembles also account for natural variability so that we can distinguish between signal and noise due to natural climate variability when computing feature importance. The use of perturbed initial condition ensembles introduces variability mimicking the natural variability in the atmosphere; thus the signals emerging using feature importance (FI) can be evaluated against the natural variability in the climate system. For our analyses, we consider the 1991 volcanic eruption of Mount Pinatubo, which was a large stratospheric aerosol injection. We explore the climate pathway associated with the eruption from aerosols to radiation to temperature at both the near-surface and stratospheric levels. In addition to applying the method to data generated from two different ESMs, we apply stZFI to reanalysis data to compare the associations identified by stZFI. We show how stZFI tracks the importance of aerosol optical depth over time on forecasting temperatures. This case study illustrates usefulness of an ML tool (stZFI) for EDA on a well-studied climate exemplar.

Ries, Daniel (ORCID:0000000250294647)↗
DOE OSTI2022

Using machine-learning to understand complex microstructural effects on the mechanical behavior of Ti-6Al-4V alloys

Structural materials properties are highly dependent on their microstructure. Their microstructure is in turn affected by multiple fabrication and thermo-mechanical treatment parameters, all of which conform a highly-dimensional parametric space with often hidden correlations that are difficult to extract by experimentation alone. This is particularly true for alloys of the dual-phase Ti-6Al-4V family, with their greatly complex and rich microstructures, which combine several intrinsic length scales associated with multiple grain and subgrain structures, grains with different crystal lattices (α and β phases), and complex chemistry. In this paper we use a comprehensive set of machine learning techniques to develop predictive tools relating the yield strength and hardening rate of these alloys to a set of input parameters covering extensive ranges. The data generator is a finite-element crystal plasticity model for polycrystal deformation that takes into account slip anisotropy and employs standard dislocation evolution models for the α and β phases of Ti-based alloys. Our dataset includes over two thousand independent simulations and is used to train the machine learning models, which are then used to establish correlations between microstructural parameters and the alloys’ mechanical response. Our results point to the most influential parameters affecting yield strength and hardening rate, information that can then be used to guide experimental synthesis and characterization efforts to save time and resources.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Gamma-ray Spectrum Explanations (GRSE) v0.1

This code repository provides tools for producing various types of machine learning model explanations trained on gamma-ray spectra. It includes implementations of saliency mapping, Grad-CAM, LIME, and Kernel SHAP, including improvements and clarifications to some of the approaches that are specific to their use with gamma-ray spectral data. These tools are in support of the manuscript M.S. Bandstra et al., "Explaining machine-learning models for gamma-ray detection and identification," under review at PLOS ONE.

Bandstra, Mark↗