Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

OLCF’s Advanced Computing Ecosystem (ACE): FY25 Update for Ongoing Efforts

The advent of widespread use of artificial intelligence (AI) and machine learning (ML) models in science, coupled with fast data production rates of scientific instruments strain the traditional batch-oriented high-performance computing (HPC) environment. As scientific exploration continues to require more data and faster processing and analysis, new emerging technologies and capabilities to enable cross-facility and time-sensitive workflows are required for seamless integration of HPC and experimental facilities. The Advanced Computing Ecosystem (ACE) is a strategic initiative within the Oak Ridge Leadership Computing Facility (OLCF) established in 2024 to support the development of cutting-edge technologies to advance computational research and infrastructure at OLCF and across the Department of Energy (DOE). Several DOE initiatives are spearheading the evolution of the scientific landscape by blurring facility boundaries and connecting the user facilities to advance scientific capabilities and ensure energy dominance. The DOE Integrated Research Infrastructure (IRI) program is one example that is laying a foundation to support complex cross-facility workflows. The IRI program aims to integrate diverse computational resources, data infrastructures, and scientific instruments to facilitate collaboration and accelerate scientific discovery. The Interconnected Science Ecosystem (INTERSECT) initiative at Oak Ridge National Laboratory (ORNL) is another example that aims to revolutionize scientific research through AI-driven, interconnected autonomous laboratories and research facilities. Finally, the American Science Cloud (AmSC), recently announced in the “One Big Beautiful Bill”, aims to leverage prior infrastructure efforts of the IRI and automation and AI efforts of INTERSECT (and others) to build a federated, AI-augmented AmSC platform to unify the DOE’s computing, experimental, and data resources to catalyze scientific innovation.

97 MATHEMATICS AND COMPUTING↗

Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems

Physics-informed machine learning (PIML) is a set of methods and tools that systematically integrate machine learning (ML) algorithms with physical constraints and abstract mathematical models developed in scientific and engineering domains. As opposed to purely data-driven methods, {PIML} models can be trained from additional information obtained by enforcing physical laws such as energy and mass conservation. More broadly, {PIML} models can include abstract properties and conditions such as stability, convexity, or invariance. The basic premise of {PIML} is that the integration of ML and physics can yield more effective, physically consistent, and data-efficient models. This paper aims to provide a tutorial-like overview of the recent advances in {PIML} for dynamical system modeling and control. Specifically, the paper covers an overview of the theory, fundamental concepts and methods, tools, and applications on topics of: 1) physics-informed learning for system identification; 2) physics-informed learning for control; 3) analysis and verification of {PIML} models; and 4) physics-informed digital twins. The paper is concluded with a perspective on open challenges and future research opportunities.

Nghiem, Truong↗

Enhancing Electron Microscopy Image Classification Using Data Augmentation

Manual labeling for machine learning tasks such as image classification is tedious and labor-intensive; as a result, scientific datasets suitable for deep learning applications are scarce and limited. While data augmentation techniques have shown promise for extending image datasets, very little work has been done to understand the impact of combining multiple augmentation methods sequentially or the limits of their effectiveness when combined. Our work addresses this gap by examining how standard and combinatorial data augmentation affects the performance of machine learning models when trained on small datasets for label classification tasks. For our analysis, we generate single, double and quadruple-augmented datasets for a microscopy image classification task using six standard augmentation methods, and compare the resultant improvements observed in binary classification accuracy with three standard image classification models (DenseNet169, MobileNetV2, ResNet101V2). Our experiments show a non-monotonic relationship between the number of simultaneous augmentation methods and classification accuracy, indicating that there is a trade-off between the degree of augmentation and the model performance. These findings suggest that the optimal number of augmentation methods will vary by domain and use case. We also find that the order in which augmentation methods are applied to a limited dataset matters when combining augmentation schemes, with our use case showing performance differences up to 2.6% when the augmentation order is reversed for double-augmented datasets. Our work offers insights to the limits of data augmentation when working on image classification tasks with limited datasets.

Welsman, Jordan A↗

Metallization of diamond

Experimental discovery of ultralarge elastic deformation in nanoscale diamond and machine learning of its electronic and phonon structures have created opportunities to address new scientific questions. Can diamond, with an ultrawide bandgap of 5.6 eV, be completely metallized, solely under mechanical strain without phonon instability, so that its electronic bandgap fully vanishes? Through first-principles calculations, finite-element simulations validated by experiments, and neural network learning, we show here that metallization/demetallization as well as indirect-to-direct bandgap transitions can be achieved reversibly in diamond below threshold strain levels for phonon instability. We identify the pathway to metallization within six-dimensional strain space for different sample geometries. We also explore phonon-instability conditions that promote phase transition to graphite. These findings offer opportunities for tailoring properties of diamond via strain engineering for electronic, photonic, and quantum applications.

36 MATERIALS SCIENCE↗

Geometry-complete perceptron networks for 3D molecular graphs

Abstract Motivation The field of geometric deep learning has recently had a profound impact on several scientific domains such as protein structure prediction and design, leading to methodological advancements within and outside of the realm of traditional machine learning. Within this spirit, in this work, we introduce GCPNet, a new chirality-aware SE(3)-equivariant graph neural network designed for representation learning of 3D biomolecular graphs. We show that GCPNet, unlike previous representation learning methods for 3D biomolecules, is widely applicable to a variety of invariant or equivariant node-level, edge-level, and graph-level tasks on biomolecular structures while being able to (1) learn important chiral properties of 3D molecules and (2) detect external force fields. Results Across four distinct molecular-geometric tasks, we demonstrate that GCPNet’s predictions (1) for protein–ligand binding affinity achieve a statistically significant correlation of 0.608, more than 5%, greater than current state-of-the-art methods; (2) for protein structure ranking achieve statistically significant target-local and dataset-global correlations of 0.616 and 0.871, respectively; (3) for Newtownian many-body systems modeling achieve a task-averaged mean squared error less than 0.01, more than 15% better than current methods; and (4) for molecular chirality recognition achieve a state-of-the-art prediction accuracy of 98.7%, better than any other machine learning method to date. Availability and implementation The source code, data, and instructions to train new models or reproduce our results are freely available at https://github.com/BioinfoMachineLearning/GCPNet.

59 BASIC BIOLOGICAL SCIENCES↗

FAIR AI models in high energy physics

Abstract The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning models—algorithms that have been trained on data without being explicitly programmed—and more generally, artificial intelligence (AI) models, are an important target for this because of the ever-increasing pace with which AI is transforming scientific domains, such as experimental high energy physics (HEP). In this paper, we propose a practical definition of FAIR principles for AI models in HEP and describe a template for the application of these principles. We demonstrate the template’s use with an example AI model applied to HEP, in which a graph neural network is used to identify Higgs bosons decaying to two bottom quarks. We report on the robustness of this FAIR AI model, its portability across hardware architectures and software frameworks, and its interpretability.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Developing and testing capabilities for simulating cases with heterogeneous land/water surfaces in a novel atmospheric large eddy simulation code

Large eddy simulations (LES) are the primary computational tool used to simulate high Reynolds number three-dimensional turbulent flows. In the context of earth system sciences, particularly atmospheric science, LES are uniquely able to resolve the scales of atmospheric motion that are key for building process-level understanding of boundary layer turbulence, atmosphere-surface interaction, clouds, and cloud-aerosol-chemistry interaction, and are a core limited-area modeling capability. Increasing demands are being placed on LES code bases as growing high performance computing resources allow LES to address a wider range of scientific problems. In addition, LES are emerging as a source of high-quality machine learning training data. These demands necessitate an agile and extensible code base that allows the model to quickly adapt to emergent needs. However, LES have largely relied on legacy Fortran code bases that lack flexibility. A new, Python-based LES capability called Predicting INteractions of Aerosol and Clouds in Large Eddy Simulation (PINACLES) has been developed as part of the Department of Energy’s Earth System Model Development (ESMD) program area’s Enabling Aerosol-cloud interactions at Global convection-permitting scalES (EAGLES) project. PINACLES was developed from the ground up with a philosophy of maximizing scientific throughput, by attempting to optimize for both model throughput and software extensibility. The initial development of PINACLES delivered a state-of-the-art idealized LES capability solving the non-hydrostatic anelastic equations of motion with doubly periodic boundary conditions and idealized homogenous surface boundary conditions. Here we provide a final report on the outcomes of a fiscal year 2021 Seed Laboratory Directed Research Project that extended PINACLES in two key ways. First, PINACLES was coupled to a state-of-the-art land surface model enabling it to simulate spatially inhomogeneous land-atmosphere interactions that are known to control key atmospheric processes. Second, the dynamical core of PINACLES was modified to permit non-periodic boundary conditions. This model enhancement enables simulation of realistic cases with boundary conditions prescribed from atmospheric reanalysis and enables nested simulations conducted on a hierarchy of computational domains with increasing resolution. Together, these extensions to PINACLES make it a formidable modeling capability and expand its potential application to diverse components of DOE’s atmospheric science portfolio.

42 ENGINEERING↗

Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics

Data-driven constitutive modeling with neural networks has received increased interest in recent years due to its ability to easily incorporate physical and mechanistic constraints and to overcome the challenging and time-consuming task of formulating phenomenological constitutive laws that can accurately capture the observed material response. However, even though neural network-based constitutive laws have been shown to generalize proficiently, the generated representations are not easily interpretable due to their high number of trainable parameters. Sparse regression approaches exist that allow for obtaining interpretable expressions, but the user is tasked with creating a library of model forms which by construction limits their expressiveness to the functional forms provided in the libraries. Here, in this work, we propose to train regularized physics-augmented neural network-based constitutive models utilizing a smoothed version of $L^0$-regularization. This aims to maintain the trustworthiness inherited by the physical constraints, but also enables interpretability which has not been possible thus far on any type of machine learning-based constitutive model where model forms were not assumed a priori but were actually discovered. During the training process, the network simultaneously fits the training data and penalizes the number of active parameters, while also ensuring constitutive constraints such as thermodynamic consistency. We show that the method can reliably obtain interpretable and trustworthy constitutive models for compressible and incompressible hyperelasticity, yield functions, and hardening models for elastoplasticity, using synthetic and experimental data. This work aims to set a new paradigm for interpretable machine learning models in the broad area of solid mechanics where low and limited data is available along with prior knowledge of physical constraints that the learned maps need to obey. This paradigm can potentially be extended to a broader spectrum of scientific exploration.

Data-driven constitutive models↗

Applications of visualization technology in the structural sciences

The structural sciences are undergoing a transformation driven by advancements in visualization technologies that aid researchers in understanding and communicating experimental data from complex molecular systems. New applications of integrative structural biological and biophysical approaches add a wide variety of complementary information from a broad range of scientific disciplines. These approaches extend structural biophysical methodologies to enable research by the incorporation of a variety of data streams and utilization of tools like molecular graphics, virtual reality, and machine learning. To redefine how structural data—particularly from cryo-electron microscopy and x-ray crystallography—are fed forward for scientific exploration and communication, the advances in tools for data visualization and interpretation have been critical. By bringing molecular systems into an interactive three-dimensional space, these novel technologies enhance research workflows, facilitate structure-based drug design, and create engaging educational experiences. Taken together, these visualization innovations are essential tools for advancing the field by making concepts more accessible and compelling.

Eng, Edward T. [New York Structural Biology Center↗

Predicting cutoff L-shells of solar protons using the GPPSn particle dataset

Solar energetic protons (SEPs) arriving at the Earth trigger severe radiation storms in the near-Earth space, directly impacting space missions operating at various altitudes. Therefore, monitoring SEP events and predicting the penetration depths of solar protons are critical for aerospace sectors. Building on previous efforts, here we demonstrate the feasibility of using proton measurements from the Global Prompt Proton Sensor network (GPPSn), enabled by Los Alamos National Laboratory developed combined X-ray dosimeters aboard GPS satellites, to characterize and predict the penetration of solar protons into the geomagnetic field. The inclined medium-Earth-orbits (MEOs) of the global GPS constellation offer a unique advantage of allowing simultaneous measurements of penetrating solar protons inside both open- and closed-field line regions. Therefore, the L-profiles of ∼10s–100 MeV solar protons and their associated cutoff L-shells can be determined from the GPPSn dataset, using predefined threshold proton flux values rather than traditional flux ratios. After examining a list of SEP event intervals across solar cycles 23, 24 and 25—including the 2024 Mother’s Day superstorm, we showcase how the latest GPPSn proton dataset (release v1.10), reprocessed and calibrated, can not only be used to monitor solar proton distributions inside the dynamic geomagnetic field for individual events, but also to derive a new empirical model linking cutoff L-shells with several key space weather parameters. This newly developed SEPCL-MEO model demonstrates high predictive performance; for example, predictions for > 30 MeV solar protons yield a correlation coefficient of 0.85 and performance efficiency of 0.67 when validated against GPPSn observations. Results from this pilot study underscores the scientific and operational value of the GPPSn dataset, and this dataset—when paired with machine-learning techniques—can play a critical role in observing and predicting the effects of future incoming SEP events, including extreme ones.

58 GEOSCIENCES↗

Creating Training Data for Scientific Named Entity Recognition with Minimal Human Effort

Scientific Named Entity Referent Extraction is often more complicated than traditional Named Entity Recognition (NER). For example, in polymer science, chemical structure may be encoded in a variety of nonstandard naming conventions, and authors may refer to polymers with conventional names, commonly used names, labels (in lieu of longer names), synonyms, and acronyms. As a result, accurate scientific NER methods are often based on task-specific rules, which are difficult to develop and maintain, and are not easily generalized to other tasks and fields. Machine learning models require substantial expert-annotated data for training. Here we propose polyNER: a semi-automated system for efficient identification of scientific entities in text. PolyNER applies word embedding models to generate entity-rich corpora for productive expert labeling, and then uses the resulting labeled data to bootstrap a context-based word vector classifier. Evaluation on materials science publications shows that the polyNER approach enables improved precision or recall relative to a state-of-the-art chemical entity extraction system at a dramatically lower cost: it required just two hours of expert time, rather than extensive and expensive rule engineering, to achieve that result. This result highlights the potential for human-computer partnership for constructing domain-specific scientific NER systems.

Crowdsourcing↗

Physics in the Machine: Integrating Physical Knowledge in Autonomous Phase-Mapping

Application of artificial intelligence (AI), and more specifically machine learning, to the physical sciences has expanded significantly over the past decades. In particular, science-informed AI, also known as scientific AI or inductive bias AI, has grown from a focus on data analysis to now controlling experiment design, simulation, execution and analysis in closed-loop autonomous systems. The CAMEO (closed-loop autonomous materials exploration and optimization) algorithm employs scientific AI to address two tasks: learning a material system’s composition-structure relationship and identifying materials compositions with optimal functional properties. By integrating these, accelerated materials screening across compositional phase diagrams was demonstrated, resulting in the discovery of a best-in-class phase change memory material. Key to this success is the ability to guide subsequent measurements to maximize knowledge of the composition-structure relationship, or phase map. In this work we investigate the benefits of incorporating varying levels of prior physical knowledge into CAMEO’s autonomous phase-mapping. This includes the use of ab-initio phase boundary data from the AFLOW repositories, which has been shown to optimize CAMEO’s search when used as a prior.

97 MATHEMATICS AND COMPUTING↗

High-Performance Deep Learning Toolbox for Genome-Scale Prediction of Protein Structure and Function

Computational biology is one of many scientific disciplines ripe for innovation and acceleration with the advent of high-performance computing (HPC). In recent years, the field of machine learning has also seen significant benefits from adopting HPC practices. In this work, we present a novel HPC pipeline that incorporates various machine-learning approaches for structure-based functional annotation of proteins on the scale of whole genomes. Our pipeline makes extensive use of deep learning and provides computational insights into best practices for training advanced deep-learning models for high-throughput data such as proteomics data. We showcase methodologies our pipeline currently supports and detail future tasks for our pipeline to envelop, including large-scale sequence comparison using SAdLSA and prediction of protein tertiary structures using AlphaFold2.

Gao, Mu↗

Data imbalance in drug response prediction: multi-objective optimization approach in deep learning setting

Abstract Drug response prediction (DRP) methods tackle the complex task of associating the effectiveness of small molecules with the specific genetic makeup of the patient. Anti-cancer DRP is a particularly challenging task requiring costly experiments as underlying pathogenic mechanisms are broad and associated with multiple genomic pathways. The scientific community has exerted significant efforts to generate public drug screening datasets, giving a path to various machine learning models that attempt to reason over complex data space of small compounds and biological characteristics of tumors. However, the data depth is still lacking compared to application domains like computer vision or natural language processing domains, limiting current learning capabilities. To combat this issue and improves the generalizability of the DRP models, we are exploring strategies that explicitly address the imbalance in the DRP datasets. We reframe the problem as a multi-objective optimization across multiple drugs to maximize deep learning model performance. We implement this approach by constructing Multi-Objective Optimization Regularized by Loss Entropy loss function and plugging it into a Deep Learning model. We demonstrate the utility of proposed drug discovery methods and make suggestions for further potential application of the work to achieve desirable outcomes in the healthcare field.

Biochemistry & Molecular Biology↗

Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. Here, in this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Proxy App Suite Release (FY2020)

Version 4.0 of the ECP Proxy App Suite is practically unchanged from the previous release. The current set of proxies has proven useful for many aspects of benchmarking and co-design and we see little reason to alter the suite. Although there have been few changes to the ECP suite, the team has been hard at work in other areas. In the area of Machine Learning (ML) we have now created a separate proxy suite dedicated to this scientific applications of ML.

97 MATHEMATICS AND COMPUTING↗

FY20 Proxy App Suite Release: Report for ECP Proxy App Project Milestone ADCD-504-10

Version 4.0 of the ECP Proxy App Suite is practically unchanged from the previous release. The current set of proxies has proven useful for many aspects of benchmarking and co-design and we see little reason to alter the suite. Although there have been few changes to the ECP suite, the team has been hard at work in other areas. In the area of Machine Learning (ML) we have now created a separate proxy suite dedicated to this scientific applications of ML. The suite includes: miniGAN (Generative Adversarial Networks), miniRL (Reinforcement Learning), CRADL (inline inference), Cosmoflow-Benchmark (Convolutional Neural Network), and MLPerf-DeepCam (Climate Segmentation Benchmark). Section 3 contains more information about these proxies as well as the principles that are guiding the development of the suite. We have surveyed available proxies for several application domains including Computational Fluid Dynamics, Quantum Chemistry, Quantum Computing Simulation, Molecular Dynamics, Monte Carlo Transport, and Density Functional Theory to identify gaps in proxy coverage. Several new proxy apps are either already available or will be released soon to fill these gaps. Section 4 provides full details. Finally, section 5 reports on our continued collaboration with the ECP Continuous Integration (CI) effort to use proxy apps to help identify problems and roadblocks to cross-lab CI. We have also assisted the El Capitan Center of Excellence (COE) to stand up a CI system that will be used to test releases of HPE and AMD software stacks. We hope that the COE effort can serve as a model for ECP by showing how the Proxy App Team can work with AD and ST teams identify critical features, kernels, patterns, etc. and incorporate them into a CI system that will help ensure that Frontier and Aurora will provide those needed capabilities.

97 MATHEMATICS AND COMPUTING↗