Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning tools”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83

Applying novel analytical tools for analyzing multidimensional secondary organic aerosol measurements

In the atmosphere, secondary organic aerosols (SOA) are often the major components of fine particulate matter and interact with clouds and radiation. SOA comprises a mixture of thousands of organic compounds. There is tremendous complexity and uncertainty in understanding SOA formation, since it is formed by oxidation and gas to particle conversion of a variety of sources: natural biogenic, anthropogenic (vehicles, cooking coal combustion) and biomass burning. The Aerosol Mass Spectrometer (AMS) produces multidimensional chemical information about SOA but analyzing this data to understand SOA sources relies on time consuming analyses (~months to years) such as the positive matrix factorization (PMF). PMF also becomes difficult for aircraft data where signal to noise ratio is weaker. There is a critical need to develop fast machine learning techniques that can analytically provide information about SOA sources using AMS data on the same timescales as the data is being collected (~minutes). We apply a machine learning supervised classification approach: the multinomial logistic regression to rapidly classify AMS data obtained from aircraft measurements.

47 OTHER INSTRUMENTATION↗

FALCON: Framework for Anomaly Detection in Industrial Control Systems

Industrial Control Systems (ICS) are used to control physical processes in critical infrastructure. These systems are used in a wide variety of operations such as water treatment, power generation and distribution, and manufacturing. While the safety and security of these systems are of serious concern, recent reports have shown an increase in targeted attacks aimed at manipulating physical processes to cause catastrophic consequences. This trend emphasizes the need for algorithms and tools that provide resilient and smart attack detection mechanisms to protect ICS. In this paper, we propose an anomaly detection framework for ICS based on a deep neural network. The proposed methodology uses dilated convolution and long short-term memory (LSTM) layers to learn temporal as well as long term dependencies within sensor and actuator data in an ICS. The sensor/actuator data are passed through a unique feature engineering pipeline where wavelet transformation is applied to the sensor signals to extract features that are fed into the model. Additionally, this paper explores four variations of supervised deep learning models, as well as an unsupervised support vector machine (SVM) model for this problem. The proposed framework is validated on Secure Water Treatment testbed results. This framework detects more attacks in a shorter period of time than previously published methods.

97 - MATHEMATICS AND COMPUTING↗

Authoring, Analyzing, and Monitoring Requirements for a Lift-Plus-Cruise Aircraft

Requirements specification and analysis is widely applied to ensure the correctness of industrial systems in safety critical domains. Requirements are often initially written in natural language, which is highly ambiguous, and as a second step transformed into a language with rigorous semantics for formal analysis. In this paper, we report on our experience in requirements creation and analysis, as well as run-time monitor generation using the Formal Requirement Elicitation Tool (FRET), on an industrial case study for a Lift-Plus-Cruise concept aircraft. We study the creation of requirements directly in the structured language of FRET without a prior definition of the same requirements in natural language. We focus on requirements describing state machines and discuss the challenges that we faced, in terms of creating requirements and generating monitors. We demonstrate how realizability, i.e., checking whether a requirements specification can be implemented, is crucial for understanding temporal interdependencies among requirements. Our study is the first complete attempt at using FRET to create industrial, realizable requirements and generate run-time monitors. Insight from lessons learned was materialized into new features in the FRET and JKind analysis frameworks.

Requirements engineering↗

To Fail or not to Fail: An Exploration of Machine Learning Techniques for Predictive Maintenance

Predictive maintenance refers to the ability to predict when machinery or systems need to be maintained. Making an accurate prediction is quite challenging given the costs for both over-estimating (unnecessary maintenance and reduction in availability of assets) and under-estimating (untimely breakdowns and possible loss of equipment or lives). To address these challenges researchers were able to develop new approaches for analyzing oil samples taken extracting samples from oil-wetted machinery that may provide information critical to developing predictive capabilities. We consider the problem from both supervised (though data limited) and unsupervised approaches and provide a first look into a data driven approach for identification of condition indicators. Through this work we identify a collection of candidate features that can form the basis of condition indicators for both a high level discrimination of failure vs. normal operation as well as a set for potential failure mode identification. Finally, we present an anomaly detection framework for detecting failures which can be a viable solution for an onboard analysis tool in deployed systems.

predictive maintenance, anomaly detection, Laserne↗

Facility Cybersecurity Framework Best Practices

Federal facilities are increasingly adopting automation and connecting to the Internet creating an energy-internet-of-things environment that converges operational technology (OT) and information technology (IT). Today's buildings increasingly weave together networked sensors and cyber and physical systems that enable data to be collected, aggregated, exchanged, stored and monetized in new ways. Building technological advances have created new energy technology, services, markets and value creation opportunities (e.g. transactive energy, two-way grid communications, machine learning, and increased use of renewable and distributed energy resources). But as larger data sets are being exchanged at faster speeds between an increasing number of OT systems, it becomes more difficult to protect the security of the data lifecycle and the physical equipment it interacts with. These challenges are especially difficult to overcome because the economic and environmental gain (interoperability, big data, social networks and ubiquitous information sharing) are driving these prominent trends in the digital age. Often cybersecurity is an afterthought. The U.S. Department of Energy’s (DOE) Federal Energy Management Program (FEMP) funded the Pacific Northwest National Laboratory (PNNL) to develop various cybersecurity tools, trainings, and reports to aid federal facility managers – and other building owners and operators – in better applying frameworks and lessons learned from the National Institute of Standards and Technology (NIST) Cybersecurity Framework (CSF), risk management framework (RMF), DOE’s cybersecurity capability maturity model (C2M2), and a wide variety of industry best practices and guidance documents (i.e., NIST 800 series, Department of Defense United Facilities Criteria). This set of tools, collectively known as the FEMP Facility-Related Control System Cyber Toolkit (FRCS Cyber Toolkit)2, is focused on cybersecurity concerns from facility-related control systems and other operational technology (OT), such as industrial control systems (ICS). The FRCS Cyber Toolkit can be applied across six of the sixteen critical infrastructure sectors designated by the Department of Homeland Security, including government facilities, healthcare and public health, commercial facilities (e.g., public assembly, offices, lodging), financial services (e.g., banking and insurance), emergency services (e.g., fire and police stations), and information technology. With increasingly converged IT and OT systems, it is crucial to address OT cybersecurity considerations and assess how the seam of these two systems could impact the overall cybersecurity posture of a facility. The objective of this report is to provide an overview of the best possible method to use FRCS Cyber Toolkit (section 2.0) and distilled cybersecurity best practices for the federal facilities to address growing non-linear cyber threats (section 3.0). Recommendations in this document are aggregated from several NIST and other documents (see Appendix A for additional details).

97 MATHEMATICS AND COMPUTING↗

Nerpa: A Tool for Discovering Biosynthetic Gene Clusters of Bacterial Nonribosomal Peptides

Microbial natural products are a major source of bioactive compounds for drug discovery. Among these molecules, nonribosomal peptides (NRPs) represent a diverse class of natural products that include antibiotics, immunosuppressants, and anticancer agents. Recent breakthroughs in natural product discovery have revealed the chemical structure of several thousand NRPs. However, biosynthetic gene clusters (BGCs) encoding them are known only for a few hundred compounds. Here, we developed Nerpa, a computational method for the high-throughput discovery of novel BGCs responsible for producing known NRPs. After searching 13,399 representative bacterial genomes from the RefSeq repository against 8368 known NRPs, Nerpa linked 117 BGCs to their products. We further experimentally validated the predicted BGC of ngercheumicin from Photobacterium galatheae via mass spectrometry. Nerpa supports searching new genomes against thousands of known NRP structures, and novel molecular structures against tens of thousands of bacterial genomes. The availability of these tools can enhance our understanding of NRP synthesis and the function of their biosynthetic enzymes.

59 BASIC BIOLOGICAL SCIENCES↗

A New Era of Discovery: The 2023 Long-Range Plan for Nuclear Science (V.1.2)

Nuclear science is the investigation of how protons and neutrons are formed from elementary particles and how the forces between those particles produce both nuclei and the vast variety of nuclear phenomena that occur in the universe. It has evolved into a broad field that addresses profound scientific questions: Where does the mass of visible matter come from? How do stars ignite, live, and die? How do nuclei illuminate the search for new laws of nature? This science points the way to using nuclei to build new technologies that benefit society. The 2015 Nobel Prize in physics was shared by nuclear physicists Art McDonald and Takaaki Kajita for the discovery of neutrino oscillations, which confirmed that neutrinos have mass. Our progress on big questions like this one since 2015 has been remarkable owing to new experimental tools, theoretical breakthroughs, powerful computational techniques, and the talented people who make these innovations possible. Focusing on these new tools, the Facility for Rare Isotope Beams (FRIB) at Michigan State University is already producing exciting results on decays of never-before-produced isotopes a year after it was completed on time and on budget. The energy upgrade of the Continuous Electron Beam Accelerator Facility (CEBAF) at the Thomas Jefferson National Accelerator Facility (Jefferson Lab) was also completed on schedule and on budget—new data from this facility are revealing the spectrum, structure, and dynamics of protons, neutrons, nuclei, and mesons. On the theory front, we can now calculate the distribution of quarks inside the proton from first principles. The implementation of artificial intelligence (AI) and machine learning (ML) techniques has led to improved data analysis and increased efficiency in running experiments and theoretical calculations. The impact of nuclear science goes beyond expanding the frontiers of knowledge about matter in the universe. We simultaneously develop a STEM work force that advances the security, technology, health, and wealth of our nation. Some connections are obvious. Expert scientists trained to work with radioactive nuclei are in demand in nuclear security arenas and are highly sought after by various government agencies and private industries. Graduate students and postdoctoral fellows (postdocs) obtain extensive computational, modeling, and data science skills that are similarly in high demand. Less obvious but equally important is the connection between these trained scientists and success in other professions, including medicine, energy, and entrepreneurial pursuits. The workforce that enables discovery in nuclear science also makes breakthroughs in technologies with tremendous impact on the nation’s economic advancement.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

BETO 2021 Peer Review - Inverse Bioproduct Design Through Machine Learning and Molecular Simulation

This work aims to identify performance advantaged bioproducts (PABPs) through property prediction, which will guide experimental synthesis. The impact of this work will be faster market adoption of bioproducts with greater performance relative to incumbent products. We have identified >106 bioproduct candidates, but only some will have superior performance to create a market pull. High-throughput property prediction, enabled by machine learning, and elucidation of structure-function relationships, enabled by molecular simulation, provide a hypothesis driven approach for down selection of candidate biomolecules to pursue experimentally. To enable machine learning and molecular simulation for bioproduct discovery, automated structure generation and embedding must capture relevant features for prediction, databases must cover domains applicable to biobased products, and best practices for simulation of polymer systems must be developed. To address these challenges, we have established bioproduct relevant datasets, developed high-throughput polymer structure generation, and built end-to-end neural networks that have predicted 8 properties for >1.4 x 106 biopolymers. A molecular simulation pipeline for building, running, and analyzing polymers and polymer additives is being used to predict performance and develop design principles of biobased products. In collaboration with the PABP synthesis project, these computational tools are guiding synthesis and informing design of PABPs.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Association of Diet and Antimicrobial Resistance in Healthy U.S. Adults

Antimicrobial resistance (AMR) represents a significant source of morbidity and mortality worldwide, with expectations that AMR-associated consequences will continue to worsen throughout the coming decades. Since resistance to antibiotics is encoded in the microbiome, interventions aimed at altering the taxonomic composition of the gut might allow us to prophylactically engineer microbiomes that harbor fewer antibiotic resistant genes (ARGs). Diet is one method of intervention, and yet little is known about the association between diet and antimicrobial resistance. To address this knowledge gap, we examined diet using the food frequency questionnaire (FFQ; habitual diet) and 24-h dietary recalls (Automated Self-Administered 24-h [ASA24 ® ] tool) coupled with an analysis of the microbiome using shotgun metagenome sequencing in 290 healthy adult participants of the United States Department of Agriculture (USDA) Nutritional Phenotyping Study. We found that aminoglycosides were the most abundant and prevalent mechanism of AMR in these healthy adults and that aminoglycoside-O-phosphotransferases (aph3-dprime) correlated negatively with total calories and soluble fiber intake. Individuals in the lowest quartile of ARGs (low-ARG) consumed significantly more fiber in their diets than medium- and high-ARG individuals, which was concomitant with increased abundances of obligate anaerobes, especially from the family Clostridiaceae, in their gut microbiota. Finally, we applied machine learning to examine 387 dietary, physiological, and lifestyle features for associations with antimicrobial resistance, finding that increased phylogenetic diversity of diet was associated with low-ARG individuals. These data suggest diet may be a potential method for reducing the burden of AMR.

59 BASIC BIOLOGICAL SCIENCES↗

Graph Embeddings for CEBAF Operations: Progress and Future Plans

We describe research towards leveraging deep learning on graph representations of the injector beamline at the Continuous Electron Beam Accelerator Facility (CEBAF) in order to create a tool for improving the efficiency of beam tuning tasks. Specifically, we use graphs to represent the injector beamline at any arbitrary date and time and invoke a graph neural network to extract a low-dimensional, informative representation that can be visualized in two-dimensions. By analyzing years of operational data from the CEBAF archiver, good and bad regions of parameter space can be identified. The goal is to exercise this framework as a real-time tool to aid beam tuning, which represents the dominant source of machine downtime.

Tennant, C.↗

High-Performance Computing Optimization for Aladyn – Adaptive Neural Network Molecular Dynamics Mini-Application

This report provides a description and performance evaluation of the optimization techniques for high performance computing (HPC) implementation of the open source Computational Materials mini-application Aladyn (https://github.com/nasa/aladyn). Aladyn is a basic molecular dynamics code written in FORTRAN 2003, which is designed to demonstrate the use of adaptive neural networks (ANNs) in atomistic simulations. The role of ANNs is to efficiently reproduce the very complex energy landscape resulting from the atomic interactions in materials with the accuracy of the more expensive quantum mechanics-based calculations. The ANN is trained on a large set of atomic structures calculated using the density functional theory (DFT) method. While achieving orders of magnitude faster computational performance than DFT, the ANN-based approach was still very computationally demanding compared to the conventional approach of using empirically fitted energy functions. After its initial development, Aladyn was evaluated and optimized by experts at the NASA Advanced Supercomputing (NAS) division to exploit modern supercomputer architectures. The code has been optimized for execution on multicore central processing units (CPUs), including Intel® Skylake microarchitecture, and on graphic accelerators, such as Nvidia® V100 graphic processing units (GPUs), using Open Multi-Processing (OpenMP) and Open Accelerators (OpenACC) programming interfaces. The optimization achieved a speedup of 4.7 times the baseline version on CPU performance and an additional 2.4 times on CPU+GPU performance. Atomistic computer simulations are a fundamental tool in materials research to model material properties form physics-based first principles. Atomic interaction, governed by Quantum Mechanics (QM) require sophisticated and highly computationally demanding mathematical models to calculate [1]. Classical methods use approximate functional forms, empirically fitted through a set of variable parameters to emulate atomic energies as direct functions of atomic coordinates [2]. While empirical potentials are computationally much simpler, allowing simulations of large-scale systems of up to a trillion (1012) atoms [3], they are substantially less accurate compared to quantum calculations and applicable only to very specific atomic configurations or predefined crystallographic phases. A recently suggested approach is to use heuristic machine learning methods [4], such as those based on Adaptive Neural Networks (ANNs) to predict atomic energies, after being trained on a sufficiently large database of QM-calculated structures [5,6]. This approach reduces significantly the computational complexity, allowing for simulations of orders of magnitude larger systems compared to QM-based methods without compromising accuracy. Still, compared to classical methods using empirical energy functions, ANN methods remain two- to three orders of magnitude more computationally demanding. Hence, the computational cost of simulations, together with the need for extensive training of ANNs, still makes the practical implementation of ANN-based methods quite challenging. The purpose of the Aladyn mini-application software [7], available as open source at https://github.com/nasa/aladyn, is to be a testbed for exploring possible optimization strategies to develop highly scalable parallel algorithms for ANN-based atomistic simulations. Aladyn is aimed at utilizing the architecture of the high-end modern highperformance computing (HPC) hardware based on multicore central processing units (CPUs) equipped with graphic processing unit (GPU) accelerators. Specifically, the goal is to optimize the performance on a single HPC compute node, before implementing scaling to multi-node parallelization using message passing interface (MPI). At the same time, the open source code of Aladyn can serve as a training model for students and professors in academia.

Yamakov, Vesselin I.↗

Authoring, Analyzing, and Monitoring Requirements for a Lift-Plus-Cruise Aircraft

Requirements specification and analysis is widely applied to ensure the correctness of industrial systems in safety critical domains. Requirements are often initially written in natural language, which is highly ambiguous, and as a second step transformed into a language with rigorous semantics for formal analysis. [Question/problem] In this paper, we report on our experience in requirements creation and analysis, as well as run-time monitor generation using the Formal Requirement Elicitation Tool (FRET), on an industrial case study for a Lift-Plus-Cruise concept aircraft. [Principal ideas/results] We study the creation of requirements directly in the structured language of FRET without a prior definition of the same requirements in natural language. We focus on requirements describing state machines and discuss the challenges that we faced, in terms of creating requirements and generating monitors. We demonstrate how realizability, i.e., checking whether a requirements specification can be implemented, is crucial for understanding temporal interdependencies among requirements. [Contribution] Our study is the first complete attempt at using FRET to create industrial, realizable requirements and generate run-time monitors. Insight from lessons learned was materialized into new features in the FRET and JKind analysis frameworks.

Requirements engineering↗

Computational modeling of grain boundary segregation: A review

Nearly all metals, alloys, ceramics, and their associated composites are polycrystalline in nature, with grain boundaries that separate well-defined crystalline regions that influence materials properties. In all but the most pure elemental systems, intentional solutes or impurities are present and can segregate to, or less commonly away from, the grain boundaries, in turn influencing boundary behavior, their stability, and associated materials properties. In some cases, grain-boundary segregation can also trigger “phase-like” structural transitions that dramatically alter the essential nature of the boundary. With the development of advanced electron microscopy techniques, researchers can directly observe grain-boundary structures and segregation with atomic precision. Despite such spatial resolution, the underlying mechanisms governing grain-boundary segregation remain difficult to characterize. As a result, computational modeling techniques such as density functional theory, molecular dynamics, mesoscale phase-field, continuum defect theory, and others are important complementary tools to experimental observations for studying grain-boundary segregation behavior. In conclusion, these computational methods offer the ability to explore the underlying formation mechanisms of grain-boundary segregation, elucidate complex segregation behavior, and provide insights into solutions to effectively controlling microstructure.

36 MATERIALS SCIENCE↗

Machine Learning Application to Assess Occurrence and Saturations of Methane Hydrate in Marine Deposits Offshore India

Artificial Neural Networks (ANN) were used to assess methane hydrate occurrence and saturation in marine sediments offshore India. The ANN analysis classifies the gas hydrate occurrence into three types: methane hydrate in pore space, methane hydrate in fractures, or no methane hydrate. Further, predicted saturation characterizes the volume of gas hydrate with respect to the available void volume. Log data collected at six wells, which were drilled during the India National Gas Hydrate Program Expedition 02 (NGHP-02), provided a combination of well log measurements that were used as input for machine learning (ML) models. Well log measurements included density, porosity, electrical resistivity, natural gamma radiation, and acoustic wave velocity. Combinations of well logs used in the ML models provide good overall balanced accuracy (0.79 to 0.86) for the prediction of the gas hydrate occurrence and good accuracy (0.68 to 0.92) for methane hydrate saturation prediction in the marine accumulations against reference data. The accuracy scores indicate that the ML models can successfully predict reservoir characteristics for marine methane hydrate deposits. In conclusion, the results indicate that the ML models can either augment physics-driven methods for assessing the occurrence and saturation of methane hydrate deposits or serve as an independent predictive tool for those characteristics.

58 GEOSCIENCES↗

Using remotely sensed information to support landslide hazard and exposure assessment throughout the disaster lifecycle

The global coverage and temporal frequency that satellites provide offers a unique opportunity to estimate landslide hazard and exposure throughout the disaster lifecycle, from pre-event planning and forecasting to post-event mapping and impact assessment, and finally to recovery and mitigation. The relevance of satellite-derived data and model products is largely contingent on the spatiotemporal sampling, the hazard characteristics, and the needs from the research or applications community. This work presents an advanced Landslide Hazard Assessment for Situational Awareness (LHASA) framework that brings together satellite and model products with new machine learning techniques and global inventory data to better model landslide hazard and exposure. We present several new ways to map, model, and assess landslide hazard using a range of satellite data. Two new thrusts of this work are to better account for the exacerbating impacts of fires and to provide a multi-day forecast of potential hazard. Together, these additional components blend information from a suite of satellite and model sources to improve early warning of potentially hazardous areas, identify landslide occurrence and impacts in near real-time, and better characterize the spatiotemporal patterns of landslide hazard and exposure more broadly for future awareness and planning. This suite of tools and products is open to the public and provides information to better assess the potential occurrence and impacts of landslides within different regions of the world. This presentation explores both the architecture behind this framework and examples of how the model components and products have been used by different stakeholders around the world.

Thomas Stanley↗

Characterizing Machine Learning I/O Workloads on Leadership Scale HPC Systems

High performance computing (HPC) is no longer solely limited to traditional workloads such as simulation and modeling. With the increase in the popularity of machine learning (ML) and deep learning (DL) technologies, we are observing that an increasing number of HPC users are incorporating ML methods into their workflow and scientific discovery processes, across a wide spectrum of science domains such as biology, earth science, and physics. This gives rise to a diverse set of I/O patterns than the traditional checkpoint/restart-based HPC I/O behavior. The details of the I/O characteristics of such ML I/O workloads have not been studied extensively for large-scale leadership HPC systems. This paper aims to fill that gap by providing an in-depth analysis to gain an understanding of the I/O behavior of ML I/O workloads using darshan - an I/O characterization tool designed for lightweight tracing and profiling. We study the darshan logs of more than 23, 000 HPC ML I/O jobs over a time period of one year running on Summit - the second-fastest supercomputer in the world. This paper provides a systematic I/O characterization of ML I/O jobs running on a leadership scale supercomputer to understand how the I/O behavior differs across science domains and the scale of workloads, and analyze the usage of parallel file system and burst buffer by ML I/O workloads.

Paul, Arnab↗

Neural Network Reflectance Prediction Model for Both Open Ocean and Coastal Waters

Remote sensing of global ocean color is a valuable tool for understanding the ecology and biogeochemistry of the worlds oceans, and provides critical input to our knowledge of the global carbon cycle and the impacts of climate change. Ocean polarized reflectance contains information about the constituents of the upper ocean euphotic zone, such as colored dissolved organic matter (CDOM), sediments, phytoplankton, and pollutants. In order to retrieve the information on these constituents, remote sensing algorithms typically rely on radiative transfer models to interpret water color or remote-sensing reflectance; however, this can be resource-prohibitive for operational use due to the extensive CPU time involved in radiative transfer solutions. In this work, we report a fast model based on machine learning techniques, called Neural Network Reflectance Prediction Model (NNRPM), which can be used to predict ocean bidirectional polarized reflectance given inherent optical properties of ocean waters. This supervised model is trained using a large volume of data derived from radiative transfer simulations for coupled atmosphere and ocean systems using the successive order of scattering technique (SOS-CAOS). The performance of the model is validated against another large independent test dataset generated from SOS-CAOS. The model is able to predict both polarized and unpolarized reflectances with an absolute error (AE) less than 0.004 for 99% of test cases. We have also shown that the degree of linear polarization (DoLP) for unpolarized incident light can be predicted with an AE less than 0.002 for 99% of test cases. In general, the simulation time of SOS-CAOS depends on optical depth, and required accuracy. When comparing the average speeds of the NNRPM against the SOS-CAOS model for the same parameters, we see that the NNRPM is able to predict the Ocean BRDF 6000 times faster than SOS-CAOS. Both ultraviolet and visible wavelengths are included in the model to help differentiate between dissolved organic material and chlorophyll in the study of the open ocean and the coastal zone. The incorporation of this model into the retrieval algorithm will make the retrieval process more efficient, and thus applicable for operational use with global satellite observations.

radiative transfer↗