Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural network compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

144 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Transported Livengood–Wu Integral Model for Knock Prediction in Computational Fluid Dynamics Simulation

This work describes the development of a transported Livengood–Wu (L–W) integral model for computational fluid dynamics (CFD) simulation to predict autoignition and engine knock tendency. The currently employed L–W integral model considers both single-stage and two-stage ignition processes, thus can be generally applied to different fuels such as paraffin, olefin, aromatics, and alcohol. The model implementation is first validated in simulations of homogeneous charge compression ignition (HCCI) combustion for three different fuels, showing good accuracy in prediction of autoignition timing for fuels with either single-stage or two-stage ignition characteristics. Then, the L–W integral model is coupled with G-equation model to indicate end-gas autoignition and knock tendency in CFD simulations of a direct-injection spark-ignition engine. This modeling approach is about 10 times more efficient than the ones that based on detailed chemistry calculation and pressure oscillation analysis. Two fuels with same Research Octane Number (RON) but different octane sensitivity are studied, namely, Co-Optima alkylate and Co-Optima E30. Feed-forward neural network model in conjunction with multivariable minimization technique is used to generate fuel surrogates with targets of matched RON, octane sensitivity, and ethanol content. The CFD model is validated against experimental data in terms of pressure traces and heat release rate for both fuels under a wide range of operating conditions. The knock tendency—indicated by the fuel energy contained in the autoignited region—of the two fuels at different load conditions correlates well with the experimental results and the fuel octane sensitivity, implying the current knock modeling approach can capture the octane sensitivity effect and can be applied to further investigation on composition of octane sensitivity.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

A transported Livengood-Wu integral model for knock prediction in CFD simulation

This work describes the development of a transported Livengood-Wu (L-W) integral model for computational fluid dynamics (CFD) simulation to predict auto-ignition and engine knock tendency. The currently employed L-W integral model considers both single-stage and two-stage ignition processes, thus can be generally applied to different fuels such as paraffin, olefin, aromatics and alcohol. The model implementation is first validated in simulations of homogeneous charge compression ignition combustion for three different fuels, showing good accuracy in prediction of auto-ignition timing for fuels with either single-stage or two-stage ignition characteristics. Then, the L-W integral model is coupled with G-equation model to indicate end-gas auto-ignition and knock tendency in CFD simulations of a direct injection spark ignition engine. This modeling approach is about 10 times more efficient than the ones that based on detailed chemistry calculation and pressure oscillation analysis. Two fuels with same Research Octane Number (RON) but different octane sensitivity are studied, namely Co-Optima Alkylate and Co-Optima E30. Feed-forward neural network model in conjunction with multi-variable minimization technique is used to generate fuel surrogates with targets of matched RON, octane sensitivity and ethanol content. The CFD model is validated against experimental data in terms of pressure traces and heat release rate for both fuels under a wide range of operating conditions. The knock tendency indicated by the fuel energy contained in the auto-ignited region of the two fuels at different load conditions correlates well with the experimental results and the fuel octane sensitivity, implying the current knock modeling approach can capture the octane sensitivity effect and can be applied to further investigation on composition of octane sensitivity.

Yue, Zongyu↗
DOE OSTI2024

Data-driven closure modeling for hypersonic turbulent flows

The Reynolds-averaged Navier–Stokes (RANS) equations remain a workhorse technology for simulating compressible fluid flows of practical interest. Due to model-form errors, however, RANS models can yield erroneous predictions that preclude their use on mission-critical problems. This report summarizes work performed from FY22-FY24 focused on improving RANS models for hypersonic flows using data-driven modeling and scientific machine learning. In this work we: 1. Investigate the current capabilities of RANS models in Sandia’s parallel aerodynamics and re-entry code (SPARC) for hypersonic flows with a focus on shock boundary layer interactions (SBLIs), 2. Assess several established corrections that exist in the literature aimed at improving predictions for SBLIs, 3. Develop improved models for the Reynolds stress tensor using tensor-basis neural networks, 4. Develop a neural-network-based variable turbulent Prandtl number model to reduce errors in wall heating in SBLIs. 5. Begin future investigations including employing the LIFE framework to improve wall heating predictions in SBLIs as well as the ensemble Kalman filter. We find that current RANS models in SPARC are deficient for complex SBLI flows. In particular, no current model jointly predicts wall heat flux, wall shear stress, and wall pressure with reasonable accuracy. Existing corrections help, but do not alleviate this issue altogether. The development of improved models for the Reynolds stress tensor via tensor-basis neural networks results in more predictive RANS models across a suite of low-speed and high-speed cases. For hypersonic boundary layers, the inclusion of the wall-normal Reynolds stress via TBNNs has an appreciable impact on the wall-normal momentum balance and wall quantities. However, we find that improvements to the Reynolds stress tensor do not address the over-prediction in wall heat flux in SBLIs. We find that a neural-network-based variable turbulent Prandtl number model systematically and substantially improves wall heating predictions for a range of SBLI cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

FPGA-accelerated SpeckleNN with SNL for real-time X-ray single-particle imaging

We present the implementation of a specialized version of our previously published unified embedding model, SpeckleNN, for real-time speckle pattern classification in X-ray Single-Particle Imaging (SPI), using the SLAC Neural Network Library (SNL) on an FPGA platform. This hardware realization transitions SpeckleNN from a prototypic model into a practical edge solution, optimized for running inference near the detector in high-throughput X-ray free-electron laser (XFEL) facilities, such as those found at the Linac Coherent Light Source (LCLS). To address the resource constraints inherent in FPGAs, we developed a more specialized version of SpeckleNN. The original model, which was designed for broader classification across multiple biological samples, comprised ~5.6 million parameters. The new implementation, while reducing the parameter count to 64.6K (a 98.8% reduction), focuses on maintaining the model's essential functionality for real-time operation, achieving an accuracy of 90%. Furthermore, we compressed the latent space from 128 to 50 dimensions. This implementation was demonstrated on the KCU1500 FPGA board, utilizing 71% of available DSPs, 75% of LUTs, and 48% of FFs, with an average power consumption of 9.4W according to the Vivado post-implementation report. The FPGA performed inference on a single image with a latency of 45.015 microseconds at a 200 MHz clock rate. In comparison, running the same inference on an NVIDIA A100 GPU resulted in an average power consumption of ~73W and an image processing latency of around 400 microseconds. Our FPGA-accelerated version of SpeckleNN demonstrated significant improvements, achieving an 8.9 × speedup and a 7.8 × reduction in power consumption compared to the GPU implementation. Key advancements include model specialization and dynamic weight loading through SNL, which eliminates the need for time-consuming FPGA design re-synthesis, allowing fast and continuous deployment of models (re)trained online. These innovations enable real-time adaptive classification and efficient vetoing of speckle patterns, making SpeckleNN more suited for deployment in XFEL facilities. This implementation has the potential to significantly accelerate SPI experiments and enhance adaptability to evolving experimental conditions.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Three-dimensional reconstruction of inertial confinement fusion hot-spot plasma from x-ray and nuclear diagnostics on OMEGA

Multidimensional effects degrade the neutron yield and the compressed areal density of laser-direct-drive inertial confinement fusion implosions of layered deuterium–tritium cryogenic targets on the OMEGA Laser System with respect to 1D radiation-hydrodynamic simulation predictions. A comprehensive physics-informed 3D reconstruction effort is under way to infer hot-spot and shell conditions at stagnation from four x-ray and seven neutron detectors distributed around the OMEGA target chamber. Neutron diagnostics, providing measurements of the neutron yield, hot-spot flow velocity, and apparent ion-temperature distribution, are used to infer the mode-1 perturbation at stagnation. The x-ray imagers record the shape of the hot-spot plasma to diagnose mode-1 and mode-2 perturbations. A deep-learning convolutional neural network trained on an extensive set of 3D radiation-hydrodynamic simulations is used to interpret the x-ray and nuclear measurements to infer the 3D profiles of the hot-spot plasma conditions and the amount of laser energy coupled to the hot-spot plasma. A 3D simulation database shows that larger mode-1 asymmetries are correlated with higher hot-spot flow velocities and reduced laser-energy coupling and neutron yield. Three-dimensional hot-spot reconstructions from x-ray measurements indicate that higher amounts of residual kinetic energy are correlated with higher measured hot-spot flow velocities, consistent with 3D simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

42 ENGINEERING↗
DOE OSTI2021

Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics

Graph neural networks have been shown to achieve excellent performance for several crucial tasks in particle physics, such as charged particle tracking, jet tagging, and clustering. An important domain for the application of these networks is the FGPA-based first layer of real-time data filtering at the CERN Large Hadron Collider, which has strict latency and resource constraints. We discuss how to design distance-weighted graph networks that can be executed with a latency of less than one μs on an FPGA. To do so, we consider a representative task associated to particle reconstruction and identification in a next-generation calorimeter operating at a particle collider. We use a graph network architecture developed for such purposes, and apply additional simplifications to match the computing constraints of Level-1 trigger systems, including weight quantization. Using the hls4ml library, we convert the compressed models into firmware to be implemented on an FPGA. Performance of the synthesized models is presented both in terms of inference accuracy and resource usage.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Predicting the viscoplastic response of a crystallizing fluoropolymer using transient network theory

We employ a molecular theory of dynamic polymer networks to describe the viscoplastic response of rubbery FK-800, a thermoplastic copolymer of chlorotrifluoroethylene and vinylidene fluoride, over a broad range of thermal histories. The kinetics of crystallization at different annealing temperatures was modeled using a modified Avrami equation, whose parameters were found to evolve through simple relationships over the full temperature range of the rubbery state. By fitting experimental compression data, we discovered predictable trends for the physical parameters in our mechanical model over its full range of crystallinities (up to ≈20%) and provided insights based on molecular-level physics to justify them. Using this, an end-to-end model was developed to predict the yielding and post-yield behavior of rubbery FK-800 for arbitrary thermal histories. The model successfully predicted the highly nonlinear evolution of characteristic mechanical signatures (stiffness, yield point, post-yield drop) throughout the crystallization process. A statistical analysis of variance test was employed to determine that the measured variations in the mechanical behavior of rubbery FK-800 are primarily dictated by its fractional crystallinity, regardless of its exact thermal history.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Geometry-aware training of factorized layers in tensor Tucker format

Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during train and inference of large networks. Given its easy implementation and flexibility, one promising approach is layer factorization, which reshapes weight tensors into a matrix format and parameterizes it as the product of two rank-r matrices. However, this family of approaches often requires an initial full-model warm-up phase, prior knowledge of a feasible rank, and it is sensitive to parameter initialization.In this work, we introduce a novel approach to train the factors of a Tucker decomposition of the weight tensors. Our training proposal proves to be optimal in locally approximating the original unfactorized dynamics and stable for the initialization. Furthermore, the rank of each mode is dynamically updated during training.We provide a theoretical analysis of the algorithm, showing convergence, approximation and local descent guarantees. The method's performance is further illustrated through a variety of experiments, showing remarkable training compression rates and comparable or even better performance than the full baseline and alternative layer factorization strategies.

Zangrando, Emanuele [Gran Sasso Science Institute ↗
DOE OSTI2021

Using Computationally-Determined Properties for Machine Learning Prediction of Self-Diffusion Coefficients in Pure Liquids

The ability to predict transport properties of liquids quickly and accurately will greatly improve our understanding of fluid properties both in bulk and complex mixtures, as well as in confined environments. Such information could then be used in the design of materials and processes for applications ranging from energy production and storage to manufacturing processes. As a first step, we consider the use of machine learning (ML) methods to predict the diffusion properties of pure liquids. Recent results have shown that Artificial Neural Networks (ANNs) can effectively predict the diffusion of pure compounds based on the use of experimental properties as the model inputs. In the current study, a similar ANN approach is applied to modeling diffusion of pure liquids using fluid properties obtained exclusively from molecular simulations. A diverse set of 102 pure liquids is considered, ranging from small polar molecules (e.g., water) to large nonpolar molecules (e.g., octane). Self-diffusion coefficients were obtained from classical molecular dynamics (MD) simulations. Since nearly all the molecules are organic compounds, a general set of force field parameters for organic molecules was used. The MD methods are validated by comparing physical and thermodynamic properties with experiment. Computational input features for the ANN include physical properties obtained from the MD simulations as well as molecular properties from quantum calculations of individual molecules. Furthermore, fluid properties describing the local liquid structure were obtained from center of mass radial distribution functions (COM-RDFs). Feature sensitivity analysis revealed that isothermal compressibility, heat of vaporization, and the thermal expansion coefficient were the most impactful properties used as input for the ANN model to predict the MD simulated self-diffusion coefficients. The MD-based ANN successfully predicts the MD self-diffusion coefficients with only a subset (2 to 3) of the available computationally determined input features required. A separate ANN model was developed using literature experimental self-diffusion coefficients as model targets. Although this second ML model was not as successful due to a limited number of data points, a good correlation is still observed between experimental and ML predicted self-diffusion coefficients.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Database development and exploration of process–microstructure relationships using variational autoencoders

The paper demonstrates graphical representation of a large database containing process–microstructure relationships using an unsupervised machine learning algorithm. Correlating microstructural features to processing is an essential first step to answer the difficult problem of process sequence design. Here, a large database of 346,200 orientation distribution functions resulting from a variety of process sequences is constructed, where each sequence comprises up to four stages of tension, compression and rolling along different directions in various permutations. This open-source database is constructed for collaborative development of process design algorithms. The paper demonstrates a novel application of the large database: graphical representation of texture–process relationships. A variational autoencoder is used to reduce the entire database to a two dimensional latent space where variations in processes and properties can be visualized. Using proximity analysis in this latent space, we can quickly unearth multiple process solutions to the problem of texture or property design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Tutorial: Machine Learning and Artificial Intelligence in Batteries

Machine learning (ML) promises to compress the time needed to characterize battery performance, lifetime and safety. By coupling ML with physical models and metrics, that learning can bridge across materials, chemistries and cell designs. This tutorial will discuss the most popular ML techniques and resources and review recent work in the electrochemical literature. Applications include materials discovery, image recognition for quantitative microscopy analysis, fast charge algorithm development and life prediction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Neural Image Compression: Generalization, Robustness, and Spectral Biases

Recent advances in neural image compression (NIC) have resulted in models which are starting to outperform traditional codecs. While this has led to growing excitement about using these methods in real-world applications, the successful adoption of any machine learning system (including NIC) in the wild requires it to generalize (and be robust) to unseen distribution shifts at deployment time. Unfortunately, current research lacks comprehensive datasets and informative tools to evaluate and understand compression performance in real-world settings. To bridge this crucial gap, first, this paper presents a comprehensive benchmark suite to evaluate the out-of-distribution (OOD) performance of image compression methods. Specifically, we design CLIC-C and Kodak-C by introducing 15 common corruptions to popular CLIC and Kodak benchmarks. Next, we propose spectrally inspired introspection tools to gain a deeper understanding of errors introduced by image compression methods as well as their OOD performance. To this end, we carry out a detailed performance comparison of the classical codec with various variants of NIC (e.g., original, variable rate, pruned), revealing intriguing findings that challenge our current understanding of the strengths and limitations of NIC. Finally, we corroborate our empirical findings with theoretical analysis, providing an in-depth view of the OOD performance of NIC. Our benchmarks, spectral introspection tools, and findings provide a crucial bridge to the real-world adoption of NIC. We hope that our work will propel future efforts in designing more robust and generalizable NIC methods.

neural networks, Variational Autoencoder, robustne↗
DOE OSTI2022

Predicting beam transmission using 2-dimensional phase space projections of hadron accelerators

We present a method to compress the 2D transverse phase space projections from a hadron accelerator and use that information to predict the beam transmission. This method assumes that obtaining at least three projections of the 4D transverse phase space is possible and that an accurate simulation model is available for the beamline. Using a simulated model, we show that—a computer can train a convolutional autoencoder to reduce phase-space information which can later be used to predict the beam transmission. Finally, we argue that although using projections from a realistic nonlinear distribution produces less accurate results, the method still generalizes well.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Deep learning-based predictive models for laser direct drive at the Omega Laser Facility

The rich and complex physics of inertial confinement fusion provides a unique and challenging space for high-fidelity first-principles modeling. Consequently, simulation codes that are used to design experiments are computationally expensive and lack the predictive capability required for extensive parameter exploration in search of a high-performing design for laser direct drive. In this article, we present two deep-learning-based predictive models intended to address these difficulties. The first model (TL DNN) acts as a fast emulator of simulations as well as experiments at the Omega Laser Facility. This model is trained on a simulation database and subsequently calibrated on experimental data using transfer learning. To facilitate the development of this model, an autoencoder is developed to reduce the dimensionality of the input space by compressing the laser pulse input. The model predicts key experimental scalar observables of Omega experiments with high accuracy and minimal computational cost. This deep neural net enables rapid exploration of a high-dimensional input parameter space for an optimal implosion design. The second model (DNN SM+) aims to extend the statistical modeling work of Lees et al. [Phys. Rev. Lett. 127, 105001 (2021)], by increasing the complexity of the model space and allowing for coupling between degradation terms. Since the model capacity of DNN SM+ is higher than the model of Lees et al., DNN SM+ can potentially provide an improvement in predictive capability, and we use this model to provide insight into complicated degradation dependencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Understanding latent timescales in neural ordinary differential equation models of advection-dominated dynamical systems

The neural ordinary differential equation (ODE) framework has shown considerable promise in recent years in developing highly accelerated surrogate models for complex physical systems characterized by partial differential equations (PDEs). For PDE-based systems, state-of-the-art neural ODE strategies leverage a two-step procedure to achieve this acceleration: a nonlinear dimensionality reduction step provided by an autoencoder, and a time integration step provided by a neural-network based model for the resultant latent space dynamics (the neural ODE). This work explores the applicability of such autoencoder-based neural ODE strategies for PDEs in which advection terms play a critical role. More specifically, alongside predictive demonstrations, physical insight into the sources of model acceleration (i.e., how the neural ODE achieves its acceleration) is the scope of the current study. Such investigations are performed by quantifying the effects of both autoencoder and neural ODE components on latent system time-scales using eigenvalue analysis of dynamical system Jacobians. To this end, the sensitivity of various critical training parameters – de-coupled versus end-to-end training, latent space dimensionality, and the role of training trajectory length, for example – to both model accuracy and the discovered latent system timescales is quantified. Furthermore, this work specifically uncovers the key role played by the training trajectory length (the number of rollout steps in the loss function during training) on the latent system timescales: larger trajectory lengths correlate with an increase in limiting neural ODE time-scales, and optimal neural ODEs are found to recover the largest time-scales of the full-order (ground-truth) system. Demonstrations are performed across fundamentally different unsteady fluid dynamics configurations influenced by advection: (1) the Kuramoto–Sivashinsky equations (2) Hydrogen-Air channel detonations (the compressible reacting Navier–Stokes equations with detailed chemistry), and (3) 2D Atmospheric flow.

Advection-dominated dynamical systems↗
DOE OSTI2020

Numerical Investigation of Fuel Property Effects on Mixed-Mode Combustion in a Spark-Ignition Engine

In this research, lean mixed-mode combustion is numerically investigated using computational fluid dynamics (CFD) in a spark-ignition engine. A new E30 fuel surrogate is developed using a neural network model with matched octane numbers. A skeletal mechanism is also developed by automated mechanism reduction and by incorporating a NO x submechanism. A hybrid approach that couples the G-equation model and the well-stirred reactor model is employed for turbulent combustion modeling. The developed CFD model is shown to well predict pressure and apparent heat release rate (AHRR) traces compared with experiment. Two types of combustion cycles (deflagration-only and mixed-mode cycles) are observed. The mixed-mode cycles feature early flame propagation and subsequent end-gas auto-ignition, leading to two distinctive AHRR peaks. The validated CFD model is then employed to investigate the effects of NO x chemistry. The NO x chemistry is found to promote auto-ignition through the residual gas, while the deflagration phase remains largely unaffected. Sensitivity analysis is finally performed to understand effects of fuel properties, including heat of vaporization (HoV) and laminar flame speed (S L ). An increased HoV tends to suppress auto-ignition through charge cooling, while the impact of HoV on flame propagation is insignificant. In contrast, an increased S L is found to significantly promote both flame propagation and end-gas auto-ignition. The promoting effect of S L on auto-ignition is not a direct chemical effect; it is rather caused by an advancement of the combustion phasing, which increases compression heating of the end-gas.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

DeePMD-kit v2: A software package for deep potential models

DeePMD-kit is a powerful open-source software package that facilitates molecular dynamics simulations using machine learning potentials known as Deep Potential (DP) models. This package, which was released in 2017, has been widely used in the fields of physics, chemistry, biology, and material science for studying atomistic systems. The current version of DeePMD-kit offers numerous advanced features, such as DeepPot-SE, attention-based and hybrid descriptors, the ability to fit tensile properties, type embedding, model deviation, DP-range correction, DP long range, graphics processing unit support for customized operators, model compression, non-von Neumann molecular dynamics, and improved usability, including documentation, compiled binary packages, graphical user interfaces, and application programming interfaces. This article presents an overview of the current major version of the DeePMD-kit package, highlighting its features and technical details. Additionally, this article presents a comprehensive procedure for conducting molecular dynamics as a representative application, benchmarks the accuracy and efficiency of different models, and discusses ongoing developments.

97 MATHEMATICS AND COMPUTING↗