Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Traditional Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Nanobody screening and machine learning guided identification of cross-variant anti-SARS-CoV-2 neutralizing heavy-chain only antibodies

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) continues to persist, demonstrating the risks posed by emerging infectious diseases to national security, public health, and the economy. Development of new vaccines and antibodies for emerging viral threats requires substantial resources and time, and traditional development platforms for vaccines and antibodies are often too slow to combat continuously evolving immunological escape variants, reducing their efficacy over time. Previously, we designed a next-generation synthetic humanized nanobody (Nb) phage display library and demonstrated that this library could be used to rapidly identify highly specific and potent neutralizing heavy chain-only antibodies (HCAbs) with prophylactic and therapeutic efficacy in vivo against the original SARS-CoV-2. In this study, we used a combination of high throughput screening and machine learning (ML) models to identify HCAbs with potent efficacy against SARS-CoV-2 viral variants of interest (VOIs) and concern (VOCs). To start, we screened our highly diverse Nb phage display library against several pre-Omicron VOI and VOC receptor binding domains (RBDs) to identify panels of cross-reactive HCAbs. Using HCAb affinity for SARS-CoV-2 VOI and VOCs (pre-Omicron variants) and model features from other published data, we were able to develop a ML model that successfully identified HCAbs with efficacy against Omicron variants, independent of our experimental biopanning workflow. This biopanning informed ML approach reduced the experimental screening burden by 78% to 90% for the Omicron BA.5 and Omicron BA.1 variants, respectively. The combined approach can be applied to other emerging viruses with pandemic potential to rapidly identify effective therapeutic antibodies against emerging variants.

Antibodies↗
DOE OSTI2020

Performance Analysis of Scientific Computing Workloads on Trusted Execution Environments

Scientific computing sometimes involves computation on sensitive data. Depending on the data and the execution environment, the HPC (high-performance computing) user or data provider may require confidentiality and/or integrity guarantees. To study the applicability of hardware-based trusted execution environments (TEEs) to enable secure scientific computing, we deeply analyze the performance impact of AMD SEV and Intel SGX for diverse HPC benchmarks including traditional scientific computing, machine learning, graph analytics, and emerging scientific computing workloads. We observe three main findings: 1) SEV requires careful memory placement on large scale NUMA machines (1x -3.4x slowdown without and 1x -1.15x slowdown with NUMA aware placement), 2) virtualization - a prerequisite for SEV - results in performance degradation for workloads with irregular memory accesses and large working sets (1x -4x slowdown compared to native execution for graph applications) and 3) SGX is inappropriate for HPC given its limited secure memory size and inflexible programming model (1.2x -126x slowdown over unsecure execution). Finally, we discuss forthcoming new TEE designs and their potential impact on scientific computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

An Investigation of LES Wall Modeling for Rayleigh–Bénard Convection via Interpretable and Physics-Aware Feedforward Neural Networks with DNS

Abstract The traditional approach of using the Monin–Obukhov similarity theory (MOST) to model near-surface processes in large-eddy simulations (LESs) can lead to significant errors in natural convection. In this study, we propose an alternative approach based on feedforward neural networks (FNNs) trained on output from direct numerical simulation (DNS). To evaluate the performance, we conduct both a priori and a posteriori tests. In the a priori (offline) tests, we compare the statistics of the surface shear stress and heat flux, computed from filtered DNS input variables, to the stress and flux obtained from the filtered DNS. Additionally, we investigate the importance of various input features using the Shapley additive explanations value and the conditional average of the filter grid cells. In the a posteriori (online) tests, we implement the trained models in the System for Atmospheric Modeling (SAM) LES and compare the LES-generated surface shear stress and heat flux with those in the DNS. Our findings reveal that vertical velocity, a traditionally overlooked flow quantity, is one of the most important input features for determining the wall fluxes. Increasing the number of input features improves the a priori test results but does not always improve the model performance in the a posteriori tests because of the differences in input variables between the LES and DNS. Last, we show that physics-aware FNN models trained with logarithmic and scaled parameters can well extrapolate to more intense convection scenarios than in the training dataset, whereas those trained with primitive flow quantities cannot. Significance Statement The traditional near-surface turbulence model, based on a shear-dominated boundary layer flow, does not represent near-surface turbulence in natural convection. Using a feedforward neural network (FNN), we can construct a more accurate model that better represents the near-surface turbulence in various flows and reveals previously overlooked controlling factors and process interactions. Our study shows that the FNN-generated models outperform the traditional model and highlight the importance of the near-surface vertical velocity. Furthermore, the physics-aware FNN models exhibit the potential to extrapolate to convective flows of various intensities beyond the range of the training dataset, suggesting their broader applicability for more accurate modeling of near-surface turbulence.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Protein remote homology detection and structural alignment using deep learning

Exploiting sequence–structure–function relationships in biotechnology requires improved methods for aligning proteins that have low sequence similarity to previously annotated proteins. We develop two deep learning methods to address this gap, TM-Vec and DeepBLAST. TM-Vec allows searching for structure–structure similarities in large sequence databases. It is trained to accurately predict TM-scores as a metric of structural similarity directly from sequence pairs without the need for intermediate computation or solution of structures. Once structurally similar proteins have been identified, DeepBLAST can structurally align proteins using only sequence information by identifying structurally homologous regions between proteins. It outperforms traditional sequence alignment methods and performs similarly to structure-based alignment methods. We show the merits of TM-Vec and DeepBLAST on a variety of datasets, including better identification of remotely homologous proteins compared with state-of-the-art sequence alignment and structure prediction methods.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Detecting depinning and nonequilibrium transitions with unsupervised machine learning

Using numerical simulations of a model disk system, we demonstrate that a machine learning generated order-parameter-like measure can detect depinning transitions and different dynamic flow phases in systems driven far from equilibrium. Furthermore, we specifically consider monodisperse passive disks with short range interactions undergoing a depinning phase transition when driven over quenched disorder. The machine learning derived order-parameter-like measure identifies the depinning transition as well as different dynamical regimes, such as the transition from a flowing liquid to a phase separated liquid-solid state that is not readily distinguished with traditional measures such as velocity-force curves or Voronoi tessellation. The order-parameter-like measure also shows markedly distinct behavior in the limit of high density where jamming effects occur. Our results should be general to the broad class of particle-based systems that exhibit depinning transitions and nonequilibrium phase transitions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Abstract Uncertainty quantification (UQ) is important to machine learning (ML) force fields to assess the level of confidence during prediction, as ML models are not inherently physical and can therefore yield catastrophically incorrect predictions. Established a-posteriori UQ methods, including ensemble methods, the dropout method, the delta method, and various heuristic distance metrics, have limitations such as being computationally challenging for large models due to model re-training. In addition, the uncertainty estimates are often not rigorously calibrated. In this work, we propose combining the distribution-free UQ method, known as conformal prediction (CP), with the distances in the neural network’s latent space to estimate the uncertainty of energies predicted by neural network force fields. We evaluate this method (CP+latent) along with other UQ methods on two essential aspects, calibration, and sharpness, and find this method to be both calibrated and sharp under the assumption of independent and identically-distributed (i.i.d.) data. We show that the method is relatively insensitive to hyperparameters selected, and test the limitations of the method when the i.i.d. assumption is violated. Finally, we demonstrate that this method can be readily applied to trained neural network force fields with traditional and graph neural network architectures to obtain estimates of uncertainty with low computational costs on a training dataset of 1 million images to showcase its scalability and portability. Incorporating the CP method with latent distances offers a calibrated, sharp and efficient strategy to estimate the uncertainty of neural network force fields. In addition, the CP approach can also function as a promising strategy for calibrating uncertainty estimated by other approaches.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

Artificial intelligence-driven approaches for materials design and discovery

Materials design is an important component of modern science and technology, yet traditional approaches rely heavily on trial and error and can be inefficient. Computational techniques, enhanced by modern artificial intelligence, have reshaped the landscape of designing new materials. Among these approaches, inverse design has shown great promise in designing materials that meet specific property requirements. Here, in this Review, we present key computational advances in materials design over the past few decades. We follow the evolution of relevant materials design techniques, from high-throughput forward machine learning methods and evolutionary algorithms, to advanced artificial intelligence strategies such as reinforcement learning and deep generative models. We highlight the paradigm shift from conventional screening approaches to inverse generation driven by deep generative models. Finally, we discuss current challenges and future perspectives of materials inverse design. This Review may serve as a brief guide to the approaches, progress and outlook of designing future functional materials with technological relevance.

computational methods↗
DOE OSTI2025

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗
DOE OSTI2022

A Boundary Topology Optimization Approach for Lightweighting Electric Machines Inspired by Additive Manufacturing

Minimizing the mass in electric machines while maintaining superior performance has become a new requirement for the advancement of drivetrains used in wind energy and electric mobility. Topology optimization for light-weighting electric machines using traditional approaches typically explore a restricted design space allowed by standard parametrizable geometry and manufacturing, while advanced methods like cell-based density approaches suffer from a lack of robust manufacturability constraints during the optimization process. To overcome these drawbacks, we explore a grid-independent, boundary TO where the outer shape of the magnet is parameterized using Bezier curves. We conduct a design of experiments (DOE) to study the effect of different magnet shapes on machine performance by varying the control points on the Bezier curves. A machine-learning based surrogate model is constructed using the data from the DOE to quantify the relationship between the control points, air-gap torque and mass. The control points are then optimized to maximize the torque density. The approach is used for minimizing electrical steel mass in the IEA-15MW radial flux direct-drive wind turbine generator. The free-from boundary TO resulted in smooth and concise shapes that can be easily additively manufactured with upto 20-ton reduction in electrical steel mass.

additive manufacturing↗
DOE OSTI2022

A New Shape Optimization Approach for Lightweighting Electric Machines Inspired by Additive Manufacturing

Minimizing the mass in electric machines while maintaining superior performance is a new requirement for the advancement of drivetrains used in wind energy and electric mobility. Topology optimization (TO) for lightweighting electric machines using traditional approaches typically explores a restricted design space allowed by standard parametrizable geometry and manufacturing, while advanced methods, such as cell-based density approaches, suffer from a lack of robust manufacturability constraints during the optimization process. To overcome these drawbacks, we explore a grid-independent, boundary optimization where the outer shape of the magnet is parameterized using Bezier curves. We conduct a design of experiments (DOE) to study the effect of different magnet shapes on machine performance by varying the control points on the Bezier curves. A machine-learning-based surrogate model is constructed using the data from the DOE to quantify the relationship between the control points, air-gap torque, and mass. The control points are then optimized to maximize the torque density. The approach is used for minimizing electrical steel mass in the International Energy Agency (IEA) 15-MW radial flux direct-drive wind turbine generator. The new approach to shape optimization resulted in smooth and concise shapes that can be easily additively manufactured with up to a 20-ton reduction in electrical steel mass.

additive manufacturing↗
DOE OSTI2022

A New Shape Optimization Approach for Lightweighting Electric Machines Inspired by Additive Manufacturing: Preprint

Minimizing the mass in electric machines while maintaining superior performance is a new requirement for the advancement of drivetrains used in wind energy and electric mobility. Topology optimization (TO) for lightweighting electric machines using traditional approaches typically explores a restricted design space allowed by standard parametrizable geometry and manufacturing, while advanced methods, such as cell-based density approaches, suffer from a lack of robust manufacturability constraints during the optimization process. To overcome these drawbacks, we explore a grid-independent, boundary TO where the outer shape of the magnet is parameterized using Bezier curves. We conduct a design of experiments (DOE) to study the effect of different magnet shapes on machine performance by varying the control points on the Bezier curves. A machine-learning-based surrogate model is constructed using the data from the DOE to quantify the relationship between the control points, air-gap torque, and mass. The control points are then optimized to maximize the torque density. The approach is used for minimizing electrical steel mass in the International Energy Agency (IEA) 15-MW radial flux direct-drive wind turbine generator. The free-form boundary TO resulted in smooth and concise shapes that can be easily additively manufactured with up to a 20-ton reduction in electrical steel mass.

additive manufacturing↗
DOE OSTI2021

Secondary porosity prediction in complex carbonate reefs using 3D CT scan image analysis and machine learning

Development and preservation of carbonate porosity and permeability are critical to characterizing reservoirs. However, secondary porosity, such as vugs and fractures, are difficult to identify and require collection of expensive wireline tools or core sampling. Wireline logs and cores have traditionally been used to identify the presence of secondary porosity but fail to quantify the contribution to total reservoir porosity. Additionally, advanced wireline logs and core are not readily available for most wells. Dual energy CT scans were collected on whole core from the A-1 Carbonate and Brown Niagaran formations drawn from six wells in northern Michigan. 3D analysis techniques were applied to identify and isolate secondary porosity features. A series of machine learning and data analytics techniques were applied to the dataset to predict secondary porosity features on basic wireline logs. The predictive model successfully predicted secondary porosity with high confidence.

02 PETROLEUM↗
DOE OSTI2026

Machine Learning Calibration of Groundwater Table Depth in ELM: Impact on Land Surface Hydrology and Land‐Atmosphere Fluxes

Accurate representation of groundwater table depth (GWTD) is crucial for simulating hydrological cycling in Earth system models (ESM). Nevertheless, there is a notable gap in the literature regarding the validation of GWTD simulations in ESMs and their subsequent impact on downstream hydrological components. This study explores the calibration of parameterization of global GWTD using machine learning within the Energy Exascale Earth System Model (E3SM) Land Model (ELM). Despite achieving significant gains in simulating GWTD through calibration, offline ELM simulations unexpectedly show that these improvements do not translate to substantial enhancements in model performance for other key hydrological variables, including soil moisture (SM), runoff, groundwater contribution to runoff or base flow index (BFI), and evapotranspiration and its partitioning. The performance in SM and runoff was even degraded in some regions, while BFI was mostly overestimated. Although there is significant improvement in GWTD within the critical range of 1–5 m, where groundwater traditionally influences land surface energy fluxes, these improvements occurred mostly in humid areas where the impact of GWTD on surface processes is minimal. Although the impacts of model calibration are generally small in offline ELM simulations, coupled land-atmosphere simulations exhibit much stronger responses to GWTD calibration, highlighting the role of land-atmosphere feedbacks in Earth system modeling. These findings underscore the need for integrated calibration strategies that simultaneously optimize multiple hydrological variables. However, if a single-variable approach is necessary, it is crucial to establish clear priorities for calibration, identifying the most critical variables that have the greatest impact on overall model performance.

Fang, Yilin [Pacific Northwest National Laboratory↗
DOE OSTI2021

Enabling deeper learning on big data for materials informatics applications

Abstract The application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A Medium‐Sized Paleo‐Tsunami Reconstruction by a Deep Neural Network Processing Sedimentary Deposits

Abstract Reconstructing the magnitude and recurrence time of tsunamis, one of the most destructive and unpredictable natural hazards impacting coastal communities, is essential. While major tsunamis are the most studied due to their disastrous impact, small/medium tsunamis (SMTs) are much more frequent and can still significantly impact the coast. Therefore, SMTs potentially provide an extensive archive of information preserved in the geological record. Analyzing the deposits of small/medium paleo‐tsunamis (SMPTs) opens a window into when their direct observation was unavailable. However, deposits of SMPTs are often degraded, traditional sediment deposition inversion models might fail. Recent research has shown that Deep Neural Networks (DNN) can effectively reconstruct the flow conditions of major tsunamis from their deposits. We evaluate the effectiveness of this approach in reconstructing the characteristics of a recent medium size tsunami (2006 Java) and of a medium paleo‐tsunami (1929 Grand Banks). We successfully reconstruct the flow characteristics of the 2006 Java event and show that an inversion of comparable quality is possible for the 1929 Grand Banks tsunami, despite greater uncertainties due to the deposit degradation. Our research shows that Machine Learning has the potential to unseal the meaning of data of thousands SMPTs.

Batubo, P.↗
DOE OSTI2025

A machine learning method of modern urban building energy modeling: A case study of Chicago

Urban-scale building energy modeling is vital for urban planning. However, it can be challenging to assimilate reliable non-geometry building data for urban-scale modeling without extensive investment. Here, this study introduces a novel approach to developing modern urban-scale building energy stock data using geographic information systems and machine learning algorithms without necessarily requiring pre-supplied non-geometric metadata. The proposed framework integrates building footprint and height data to estimate gross floor areas, and matches each building to a pool of candidate records from ComStock or ResStock—filtered to the same county and ranked by geometric similarity—demonstrate a proof-of-concept case study in Chicago for predicting energy use intensity (EUI) using scalable datasets. The model achieved a mean bias error (MBE) of 0.08 kWh/m² and root mean square error (RMSE) of 14.84 kWh/m² under full metadata input for EUI prediction. With only location inputs, the model captured 69.2 % of EUI within predicted ranges. These results demonstrate the model’s potential to support early-stage urban planning, identify candidates for energy-efficient retrofits. By removing the dependency on detailed pre-surveys or extensive building metadata, the approach overcomes a key barrier in traditional urban-scale building energy modeling, illustrating a pathway toward broader and more cost-effective application, though further multi-city validation and improved treatment of pre-1925 buildings are needed.

Energy Use Intensity↗
DOE OSTI2025

Machine learning assisted unfolding for neutrino cross-section measurements with the OmniFold technique

The choice of unfolding method for a cross-section measurement is tightly coupled to the model dependence of the efficiency correction and the overall impact of cross-section modeling uncertainties in the analysis. A key issue is the dimensionality used in unfolding, as the kinematics of all outgoing particles in an event typically affect the reconstruction performance in a neutrino detector. OmniFold is an unfolding method that iteratively reweights a simulated dataset, using machine learning to utilize arbitrarily high-dimensional information, that has previously been applied to proton-proton and proton-electron datasets. This paper demonstrates OmniFold’s application to a neutrino cross-section measurement for the first time using a public T2K near detector simulated dataset, comparing its performance with traditional approaches using a mock data study.

Machine learning↗
DOE OSTI2023

Real-Time Regional PV Spinning Reserve Estimator with AGC Look-Ahead Windows

Curtailed PV generation is a zero-marginal cost spinning reserve that can be used for a number of active power control services. However, unlike the traditional spinning reserve providers, i.e., fossil-fueled generators, who have well-defined operating characteristics, e.g., available headroom or potential high limit (PHL), PV plants have by nature variable and uncertain operating characteristics. To ensure the effective coordination between PV plants and the system operator during an active power control event, accurate forecasts of the PV PHL are essential. A novel reference-control grouping based scaling method has been proposed by NREL to estimate the PV PHL in real-time. This work further enhances the methodology by: 1) improving the model accuracy through machine learning; 2) considering look-ahead windows introduced by the computation and communication latencies; 3) applying the method to regional spinning reserve estimation. A significant performance improvement, over 99% of estimation error reduction, has been observed based on real-world data collected by CAISO and PV plant operators.

potential high limit↗