Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Data Assessment and Assimilation for Atmospheric Radiation Measurement Data Using Dynamic Bayesian Networks

A data-driven machine learning approach to scientific sensor data assessment may automatically learn directly from the structural and temporal patterns in abundant observations. The project has developed data assessment and assimilation models to screen datasets using dynamic Bayesian network (DBN) and deep learning (DL) methods, which rely on correlations between variables, across time, and between spatial locations, to detect poor or invalid data with a high degree of confidence.

97 MATHEMATICS AND COMPUTING↗

Exploring Classification of Topological Priors With Machine Learning for Feature Extraction

In many scientific endeavors, increasingly abstract representations of data allow for new interpretive methodologies and conceptualization of phenomena. For example, moving from raw imaged pixels to segmented and reconstructed objects allows researchers new insights and means to direct their studies toward relevant areas. Thus, the development of new and improved methods for segmentation remains an active area of research. With advances in machine learning and neural networks, scientists have been focused on employing deep neural networks such as U-Net to obtain pixel-level segmentations, namely, defining associations between pixels and corresponding/referent objects and gathering those objects afterward. Topological analysis, such as the use of the Morse-Smale complex to encode regions of uniform gradient flow behavior, offers an alternative approach: first, create geometric priors, and then apply machine learning to classify. This approach is empirically motivated since phenomena of interest often appear as subsets of topological priors in many applications. Using topological elements not only reduces the learning space but also introduces the ability to use learnable geometries and connectivity to aid the classification of the segmentation target. Here, in this article, we describe an approach to creating learnable topological elements, explore the application of ML techniques to classification tasks in a number of areas, and demonstrate this approach as a viable alternative to pixel-level classification, with similar accuracy, improved execution time, and requiring marginal training data.

97 MATHEMATICS AND COMPUTING↗

Communication-Avoiding and Memory-Constrained Sparse Matrix-Matrix Multiplication at Extreme Scale

Sparse matrix-matrix multiplication (SpGEMM) is a widely used kernel in various graph, scientific computing and machine learning algorithms. In this paper, we consider SpGEMMs performed on hundreds of thousands of processors generating trillions of nonzeros in the output matrix. Distributed SpGEMM at this extreme scale faces two key challenges: (1) high communication cost and (2) inadequate memory to generate the output. Furthermore, we address these challenges with an integrated communication-avoiding and memory-constrained SpGEMM algorithm that scales to 262,144 cores (more than 1 million hardware threads) and can multiply sparse matrices of any size as long as inputs and a fraction of output fit in the aggregated memory. As we go from 16,384 cores to 262,144 cores on a Cray XC40 supercomputer, the new SpGEMM algorithm runs 10x faster when multiplying large-scale protein-similarity matrices.

97 MATHEMATICS AND COMPUTING↗

From smart homes to smart laboratories: connected instruments for materials science

The current focus on artificial intelligence and machine learning in the scientific community has the potential to greatly speed up discovery. In this article, we explore what a “smart facility” would mean for materials science. We propose to capture meta-data at every step of an experiment, including materials synthesis, sample production and characterization, simulation, and the analysis software used to extract information. Although most of this information is captured in various institutional systems and staff logbooks, more insight could be obtained by connecting this information through a system that allows automation. AI-enabled processes built on such a system would have the potential of making experiment planning easier and minimize the time between experiment and publication.

Doucet, Mathieu↗

The globus compute dataset: An open function-as-a-service dataset from the edge to the cloud

Here we present a unique function-as-a-service (FaaS) dataset capturing the use of the Globus Compute (previously funcX) platform. Globus Compute implements a federated model via which users may deploy endpoints on arbitrary remote computers, from the edge to high performance computing (HPC) cluster, and they may then invoke Python functions on those endpoints via a reliable cloud -hosted service. The dataset covers 31 weeks and includes 2121472 task submissions from 252 users executed on 580 remote computing endpoints. It includes 277386 registered functions. We describe the dataset and various observations, some that are similar to other FaaS datasets, for example, that 74% of tasks run for less than 1 s, and some that are unique to Globus Compute, for example, that endpoints are used in different ways and that the majority of functions are related to scientific computing and machine learning. To the best of our knowledge, this dataset represents the first federated FaaS dataset that includes user workloads, distributed computing endpoints, and analysis of registered function bodies. We expect the dataset to be useful for researching FaaS architectures, workload modeling, container warming, and other distributed computing architectures.

97 MATHEMATICS AND COMPUTING↗

Leveraging explainable AI to characterize floating-point exceptions in linear solvers

Linear solver packages are central to many scientific, engineering, and machine learning applications. When floating-point exceptions occur in these solvers, e.g., division by zero or overflow, numerical results are compromised and become unreliable. Existing static and dynamic analysis tools can detect such exceptions, but they do not explain why the exceptions occur in terms of the solver inputs. Here, we present a study to characterize the inputs that cause numerical exceptions in linear solver packages. Our approach uses explainable AI (XAI) to find the most relevant characteristics of input matrices that explain the occurrence of exceptions in the solvers. Since training data in this domain is scarce, we perform extensive data gathering and data augmentation to obtain exception-inducing inputs. Our approach uses a repair strategy on the features blamed by XAI to validate that such features indeed explain the exceptions. We compare the LIME and SHAP XAI techniques using a dozen matrix features with three classifiers. We evaluate the approach on three widely used linear solver packages and find that some input characteristics can explain the occurrence of exceptions 100% of the time, in specific solvers and preconditioners.

Explainable AI↗

Efficient graph representation framework for chemical molecule similarity tasks

Graph data has emerged in numerous scientific domains and machine learning techniques have been widely used for analysis and learning of diverse data for prediction and decision. Machine learning techniques can readily address complex problems by leveraging their structural information. But graphs cannot be directly used for existing machine learning algorithms unless encoded as vectors. The problem of efficient representation of graphs is a substantial challenge in graph machine learning. In this paper, we propose a novel two-stage framework for the representation of chemical molecule graphs based on the strengths of Graph Isomorphism Networks (GINs) and Siamese autoencoders. In the first stage, the GIN model is constructed and trained using the structural information of chemical molecule graphs. Node attributes, edge attributes, and edge indices are used as input data, while graph attributes are used as labels. The GIN model effectively captures the structural characteristics of graphs and can accurately predict graph attributes, i.e., molecular properties. It also generates Graph Embeddings, represented as vectors that encode the structural information of graphs. In the second stage, Graph Embedding vectors are further optimized for downstream similarity tasks while preserving the graph structural information. The Siamese autoencoder is constructed and trained, which reduces the dimensionality of the Graph Embedding vectors, while maximizing the preservation of structural information in the original high-dimensional vectors. The resulting low-dimensional Graph Embeddings can be effectively utilized for tasks such as approximate nearest neighbor search. The experimental results demonstrate the effectiveness of our proposed framework in accurately predicting graph similarity.

Ma, Jiaji↗

Cross-Cutting Software Solutions in Support of Experimental Analysis Challenges at National Scattering Facilities

Rapid improvements in instrumentation at neutron and light source combined with new capabilities in computing, data analytics, and machine learning offer unprecedented scientific capabilities to address fundamental and applied science questions in a wide area of subjects from quantum materials to biological systems. In order to make these capabilities accessible to the broader scientific community, significant developments are needed related to workflow spanning from experimental control to advanced materials simulations as well as the underpinning scientific software. In this chapter, we explore these needs and recent developments at the Spallation Neutron Source (SNS) at Oak Ridge National Laboratory. A central part of the workflow at the SNS is the data reduction framework Mantid which is developed as a partnership between European and US neutron-scattering facilities.

Proffen, Thomas↗

Thermal Management for FPGA Nodes in HPC Systems

The integration of FPGAs into large-scale computing systems is gaining attention. In these systems, real-time data handling for networking, tasks for scientific computing, and machine learning can be executed with customized datapaths on reconfigurable fabric within heterogeneous compute nodes. At the same time, thermal management, particularly battling the cooling cost and guaranteeing the reliability, is a continuing concern. The introduction of new heterogeneous components into HPC nodes only adds further complexities to thermal modeling and management. The thermal behavior of multi-FPGA systems deployed within large compute clusters is less explored. Here, we first show that the thermal behaviors of different FPGAs of the same generation can vary due to their physical locations in a rack and process variation, even though they are running the same tasks. We present a machine learning–based model to capture the thermal behavior of each individual FPGA in the cluster. We then propose two thermal management strategies guided by our thermal model. First, we mitigate thermal variation and hotspots across the cluster by proactive thermal-aware task placement. Under the tested system and benchmarks, we achieve up to 26.4° C and on average 13.3° C system temperature reduction with no performance penalty. Second, we utilize this thermal model to guide HLS parameter tuning at the task design stage to achieve improved thermal response after deployment.

97 MATHEMATICS AND COMPUTING↗

Mathematics: The Tao of Data Science

The two pieces, "Ten Research Challenge Areas in Data Science" by Jeannette M. Wing and “Challenges and Opportunities in Statistics and Data Science: Ten Research Areas” by Xuming He and Xihong Lin, provide an impressively complete list of data science challenges from luminaries in the field of data science. They have done an extraordinary job, so this response offers a complementary viewpoint from a mathematical perspective and evangelizes advanced mathematics as a key tool for meeting the challenges they have laid out. Notably, we pick up the themes of scientific understanding of machine learning and deep learning, computational considerations such as cloud computing and scalability, balancing computational and statistical considerations, and inference with limited data. We propose that mathematics is an important key to establishing rigor in the field of data science and as such has an essential role to play in its future.

97 MATHEMATICS AND COMPUTING↗

Project DarkStar: Vision for LLNL in 2030

DarkStar was a Strategic Initiative (FY2021-FY2024) to investigate applications of Artificial Intelligence (AI) and Machine Learning (ML) to scientific problems of complex hydrodynamics, shockwave physics and energetic materials. The research focused on physics and engineering design as a process that can be tremendously accelerated through merging AI with advanced physics simulation on exascale-class platforms, and to experimentally validate this revolutionary new approach through dynamic materials campaigns. A central thread of scientific inquiry was in the application of AI to enable human understanding of how to control hydrodynamic instability (which has impacts to areas such as inertial confinement fusion) via engineering features and time-dependent sources. Motivated by an unfinished line of research started by Dr. Johnny von Neumann, AI-enabled simulation approaches were developed that allowed DarkStar researchers to uncover several ground-breaking discoveries regarding hydrodynamic instability, including how to completely suppress Richtmyer-Meshkov instability (RMI). These S&T discoveries, along with other advances, have shown the way for an entirely new approach to time-dependent problems known as inverse design – the idea that complex systems can be developed directly from a final state that is to be achieved and resolve the initial design via satisfying several constraints simultaneously via AI/ML. Through experimental campaigns conducted across a wide range of facilities in the NNSA complex (the High Explosive Application Facility at LLNL, the Dynamic Compression Sector/Advanced Photon Source at Argonne National Lab, and Special Technologies Laboratory at MSTS) the radical new AI/ML approach to engineering complex material dynamics was verified, establishing a new field of study within the realm of shock physics. As advanced manufacturing capabilities continue to develop, the great importance of inverse design as a means to apply that technology effectively for NNSA missions will feature prominently over this decade. DarkStar has positioned NNSA as a world-leader in this newly emerging cross-disciplinary area of AI methods for advanced physics simulation and pioneered multiple novel approaches that have enabled the broader scientific community. By allowing us to see past the horizon, to 2030 and beyond, DarkStar has illuminated the vast potential of AI/ML to impact a wide range of new national security missions and, consequently, multiple areas of further research have already emerged across the NNSA and DOD complex.

42 ENGINEERING↗

Discovering Communicable Scientific Knowledge from Spatio-Temporal Data

This paper describes how we used regression rules to improve upon a result previously published in the Earth science literature. In such a scientific application of machine learning, it is crucially important for the learned models to be understandable and communicable. We recount how we selected a learning algorithm to maximize communicability, and then describe two visualization techniques that we developed to aid in understanding the model by exploiting the spatial nature of the data. We also report how evaluating the learned models across time let us discover an error in the data.

Schwabacher, Mark↗

Discovering Communicable Models from Earth Science Data

This chapter describes how we used regression rules to improve upon results previously published in the Earth science literature. In such a scientific application of machine learning, it is crucially important for the learned models to be understandable and communicable. We recount how we selected a learning algorithm to maximize communicability, and then describe two visualization techniques that we developed to aid in understanding the model by exploiting the spatial nature of the data. We also report how evaluating the learned models across time let us discover an error in the data.

Schwabacher, Mark↗

Artificial Intelligence Workshop Report

The 4th NASA Science Mission Directorate (SMD) Artificial Intelligence (AI) Workshop, held during March 25-27, 2024, in Huntsville, AL, highlighted the significant potential of AI and machine learning (ML) in scientific research and processes. The workshop, supported by the NASA Office of Chief Science Data Officer (OCSDO), emphasized the critical role of foundation models (FMs) and large language models (LLMs) in advancing scientific disciplines. The event brought together domain scientists, computer scientists, AI experts, program managers, program scientists, and industry partners to address key challenges and explore opportunities in applying these advanced technologies.

Manil Maskey↗

Automated detection and analysis of particle beams in laser-plasma accelerator simulations

Numerical simulations of laser-plasma wakefield (particle) accelerators model the acceleration of electrons trapped in plasma oscillations (wakes) left behind when an intense laser pulse propagates through the plasma. The goal of these simulations is to better understand the process involved in plasma wake generation and how electrons are trapped and accelerated by the wake. Understanding of such accelerators, and their development, offer high accelerating gradients, potentially reducing size and cost of new accelerators. One operating regime of interest is where a trapped subset of electrons loads the wake and forms an isolated group of accelerated particles with low spread in momentum and position, desirable characteristics for many applications. The electrons trapped in the wake may be accelerated to high energies, the plasma gradient in the wake reaching up to a gigaelectronvolt per centimeter. High-energy electron accelerators power intense X-ray radiation to terahertz sources, and are used in many applications including medical radiotherapy and imaging. To extract information from the simulation about the quality of the beam, a typical approach is to examine plots of the entire dataset, visually determining the adequate parameters necessary to select a subset of particles, which is then further analyzed. This procedure requires laborious examination of massive data sets over many time steps using several plots, a routine that is unfeasible for large data collections. Demand for automated analysis is growing along with the volume and size of simulations. Current 2D LWFA simulation datasets are typically between 1GB and 100GB in size, but simulations in 3D are of the order of TBs. The increase in the number of datasets and dataset sizes leads to a need for automatic routines to recognize particle patterns as particle bunches (beam of electrons) for subsequent analysis. Because of the growth in dataset size, the application of machine learning techniques for scientific data mining is increasingly considered. In plasma simulations, Bagherjeiran et al. presented a comprehensive report on applying graph-based techniques for orbit classification. They used the KAM classifier to label points and components in single and multiple orbits. Love et al. conducted an image space analysis of coherent structures in plasma simulations. They used a number of segmentation and region-growing techniques to isolate regions of interest in orbit plots. Both approaches analyzed particle accelerator data, targeting the system dynamics in terms of particle orbits. However, they did not address particle dynamics as a function of time or inspected the behavior of bunches of particles. Ruebel et al. addressed the visual analysis of massive laser wakefield acceleration (LWFA) simulation data using interactive procedures to query the data. Sophisticated visualization tools were provided to inspect the data manually. Ruebel et al. have integrated these tools to the visualization and analysis system VisIt, in addition to utilizing efficient data management based on HDF5, H5Part, and the index/query tool FastBit. In Ruebel et al. proposed automatic beam path analysis using a suite of methods to classify particles in simulation data and to analyze their temporal evolution. To enable researchers to accurately define particle beams, the method computes a set of measures based on the path of particles relative to the distance of the particles to a beam. To achieve good performance, this framework uses an analysis pipeline designed to quickly reduce the amount of data that needs to be considered in the actual path distance computation. As part of this process, region-growing methods are utilized to detect particle bunches at single time steps. Efficient data reduction is essential to enable automated analysis of large data sets as described in the next section, where data reduction methods are steered to the particular requirements of our clustering analysis. Previously, we have described the application of a set of algorithms to automate the data analysis and classification of particle beams in the LWFA simulation data, identifying locations with high density of high energy particles. These algorithms detected high density locations (nodes) in each time step, i.e. maximum points on the particle distribution for only one spatial variable. Each node was correlated to a node in previous or later time steps by linking these nodes according to a pruned minimum spanning tree (PMST). We call the PMST representation 'a lifetime diagram', which is a graphical tool to show temporal information of high dense groups of particles in the longitudinal direction for the time series. Electron bunch compactness was described by another step of the processing, designed to partition each time step, using fuzzy clustering, into a fixed number of clusters.

43 PARTICLE ACCELERATORS↗

A Heterogeneity-Aware Task Scheduler for Spark

Big data processing systems such as Spark are employed in an increasing number of diverse applications-such as machine learning, graph computation, and scientific computing-each with dynamic and different resource needs. These applications increasingly run on heterogeneous hardware, e.g., with out-of-core accelerators. However, big data platforms do not factor in the multi-dimensional heterogeneity of applications and hardware. This leads to a fundamental mismatch between the application and hardware characteristics, and the resource scheduling adopted in big data platforms. For example, Hadoop and Spark consider only data locality when assigning tasks to nodes, and typically disregard the hardware capabilities and suitability to specific application requirements. In this paper, we present RUPAM, a heterogeneity-aware task scheduling system for big data platforms, which considers both task-level resource characteristics and underlying hardware characteristics, as well as preserves data locality. RUPAM adopts a simple yet effective heuristic to decide the dominant scheduling factor (e.g., CPU, memory, or I/O), given a task in a particular stage. Our experiments show that RUPAM is able to improve the performance of representative applications by up to 62.3% compared to the standard Spark scheduler.

Computer Science↗

HPC-FAIR: A Framework Managing Data and AI Models for Analyzing and Optimizing Scientific Applications

The increasing reliance on machine learning (ML) to analyze and optimize large-scale scientific applications on supercomputers faces a significant bottleneck: the lack of readily available, high-quality training datasets and the difficulty in reusing existing AI models. This project was motivated by the urgent need to address the “FAIR” principles (Findability, Accessibility, Interoperability, Reusability) for both training datasets and AI models in the high-performance computing (HPC) domain. The project developed HPC-FAIR, a high-performance computing data management framework designed to centralize HPC-related datasets and AI models within a unified hub. To ensure interoperability, the framework established a standardized representation and vocabulary (ontology) for both data and models. HPC-FAIR also implemented automated workflows to streamline data processing, model access, and benchmarking. Additionally, the project focused on optimizing data harnessing efficiency through advanced techniques like deep reuse and compression-based analytics.

97 MATHEMATICS AND COMPUTING↗

QSO photometric redshifts using machine learning and neural networks

ABSTRACT The scientific value of the next generation of large continuum surveys would be greatly increased if the redshifts of the newly detected sources could be rapidly and reliably estimated. Given the observational expense of obtaining spectroscopic redshifts for the large number of new detections expected, there has been substantial recent work on using machine learning techniques to obtain photometric redshifts. Here, we compare the accuracy of the predicted photometric redshifts obtained from deep learning (DL) with the k-nearest neighbour (kNN) and the decision tree regression (DTR) algorithms. We find using a combination of near-infrared, visible, and ultraviolet magnitudes, trained upon a sample of Sloan Digital Sky Survey quasi-stellar objects, that the kNN and DL algorithms produce the best self-validation result with a standard deviation of σΔz = 0.24 (σΔz(norm) = 0.11). Testing on various subsamples, we find that the DL algorithm generally has lower values of σΔz, in addition to exhibiting a better performance in other measures. Our DL method, which uses an easy to implement off-the-shelf algorithm with neither filtering nor removal of outliers, performs similarly to other, more complex, algorithms, resulting in an accuracy of Δz < 0.1 up to z ∼ 2.5. Applying the DL algorithm trained on our 70 000 strong sample to other independent (radio-selected) data sets, we find σΔz ≤ 0.36 (σΔz(norm) ≤ 0.17) over a wide range of radio flux densities. This indicates much potential in using this method to determine photometric redshifts of quasars detected with the Square Kilometre Array.

Curran, S. J.↗