Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gradient methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Identifying common stored product insects using automated deep learning methods

Monitoring stored product insect pests is a common practice for post-harvest management of stored grain and grain-based commodities, which helps ensure product quality from harvest to final consumer. Current methods of sampling and monitoring can be time-consuming, labor-intensive, expensive and require expertise in insect identification. Therefore, this study aims to develop an image-based automated identification system for common stored product insect species using deep-learning methods. Top-down images of the common stored product adult insect species of Rhyzopertha dominica, Cryptolestes ferrugineus, Tribolium castaneum, Sitophilus oryzae, and Oryzaephilus surinamensis were acquired and analyzed. Deep learning-based, state-of-the-art Convolutional Neural Networks (CNN) models (ResNet-50, MobileNet-v2, DarkNet-53, and EfficientNet-b0) were fine-tuned with a transfer learning approach to classify the insect species. All models were able to correctly identify the insect species with at least 96% accuracy and with few misclassifications. One issue with trained CNNs is that they do not explain the reasoning for the classification and are often called a “black box”. Therefore, visualization methods called Gradient-weighted Class Activation Mapping (Grad-CAM) were implemented to explore the black box network. The Grad-CAM uses heat maps to highlight the major image features that the network focused on to make insect species predictions. The Grad-CAM verifies the network's prediction and also helps improve network performance. This study contributes to the overall goal of developing a camera-based system for monitoring stored grain insects. As a result, the developed system would empower warehouse, flour mills, and other food facilities with a tool to quickly and accurately identify insect species in stored product environments and could be implemented as part of a close to real-time monitoring system.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Spontaneous imbibition in a square tube with corner films: theoretical model and numerical simulation

Spontaneous imbibition in an angular tube with corner films is a fundamental problem in many scientific and engineering processes. In this paper, a modified interacting capillary bundle model is developed to describe the liquid imbibition dynamics in a square tube with corner films. The square tube is decomposed into several interacting sub-capillaries and the local capillary pressure in each sub-capillary is derived based on the specific shape of its meniscus. The conductance of each sub-capillary is calculated using single-phase lattice Boltzmann simulation. The modified interacting capillary bundle model and color-gradient lattice Boltzmann method are used to simulate the liquid imbibition dynamics in the square tube with different fluid properties. The predictions by the modified interacting capillary bundle model match well with the lattice Boltzmann simulation results for different conditions, demonstrating the accuracy and robustness of the interacting capillary bundle model to describe the imbibition dynamics with corner films. In addition, the interacting capillary bundle model is helpful to investigate the mechanisms during spontaneous imbibition and the influences of fluid viscosity, surface tension, wetting phase contact angle and gravity on imbibition dynamics. Finally, a universal scaling law of imbibition dynamics for the main meniscus is developed and the scaling law for arc meniscus is also analyzed.

42 ENGINEERING↗
DOE OSTI2020

High mass and halo resolution from fast low resolution simulations

Generating mocks for future sky surveys requires large volumes and high resolutions, which is computationally expensive even for fast simulations. Here we try to develop numerical schemes to calibrate various halo and matter statistics in fast low resolution simulations compared to high resolution N-body and hydrodynamic simulations. For the halos, we improve the initial condition resolution and develop a halo finder "relaxed-FoF", where we allow different linking lengths for different halo mass and velocity dispersions. We show that our relaxed-FoF halo finder improves the common statistics, such as halo bias, halo mass function, halo auto power spectrum, cross correlation coefficient with the reference halo catalog, and halo-matter cross power spectrum. We also calibrate small-scale velocities of small halos to improve the power spectrum in redshift space. For the matter statistics, we incorporate the potential gradient descent (PGD) method into fast simulations to improve the matter distribution at nonlinear scales. By building a lightcone output, we show that the PGD method significantly improves the weak lensing convergence tomographic power spectrum. With these improvements FastPM is comparable to the high resolution full N-body simulation of the same mass resolution, with two orders of magnitude fewer time steps. These techniques can be used to improve the halo and matter statistics of FastPM simulations for mock catalogs of future surveys such as DESI and LSST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Adaptive Quantum Generative Training using an Unbounded Loss Function

We propose a generative quantum learning algorithm using the Adaptive Derivative-Assembled Problem Tailored ansatz (ADAPT) framework in which the loss function to be minimized is the maximal quantum Rényi divergence of order two, an unbounded function that mitigates barren plateaus which inhibit training variational circuits. We benchmark this method against other state-of-the-art adaptive algorithms by learning random two-local thermal states. We perform numerical experiments of up to 12 qubits comparing our method learning algorithms that use linear objective functions and show that Rényi-ADAPT is capable of constructing shallow quantum circuits competitive with existing methods, while the gradients remain favorable resulting from the maximal Rényi divergence loss function.

quantum algorithms, quantum machine learning, quan↗
DOE OSTI2026

Robust Optimal Experimental Design of Infinite-Dimensional Bayesian Nonlinear Inverse Problems

Abstract. We consider robust optimal experimental design (ROED) for nonlinear Bayesian inverse problems governed by partial differential equations (PDEs). An optimal design is one that maximizes some utility quantifying the quality of the solution of an inverse problem. However, the optimal design is dependent on elements of the inverse problem such as the simulation model, the prior, or the measurement error model. ROED aims to produce an optimal design that is aware of the additional uncertainties encoded in the inverse problem and remains optimal even after variations in them. We follow a worst-case scenario approach to develop a new framework for robust optimal design of nonlinear Bayesian inverse problems. The proposed framework (a) is scalable and designed for infinite-dimensional Bayesian nonlinear inverse problems constrained by PDEs; (b) develops efficient approximations of the utility, namely the expected information gain; (c) employs eigenvalue sensitivity techniques to develop analytical forms and efficient evaluation methods of the gradient of the utility with respect to the uncertainties against which we wish to be robust; and (d) employs a probabilistic optimization paradigm that properly defines and efficiently solves the resulting combinatorial max-min optimization problem. The effectiveness of the proposed approach is illustrated for optimal sensor placement problem in an inverse problem governed by an elliptic PDE.

Chowdhary, Abhijit↗
DOE OSTI2022

Source-to-Source Automatic Differentiation of OpenMP Parallel Loops

This article presents our work toward correct and efficient automatic differentiation of OpenMP parallel worksharing loops in forward and reverse mode. Automatic differentiation is a method to obtain gradients of numerical programs, which are crucial in optimization, uncertainty quantification, and machine learning. The computational cost to compute gradients is a common bottleneck in practice. For applications that are parallelized for multicore CPUs or GPUs using OpenMP, one also wishes to compute the gradients in parallel. Here, we propose a framework to reason about the correctness of the generated derivative code, from which we justify our OpenMP extension to the differentiation model. We implement this model in the automatic differentiation tool Tapenade and present test cases that are differentiated following our extended differentiation procedure. Performance of the generated derivative programs in forward and reverse mode is better than sequential, although our reverse mode often scales worse than the input programs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

NeuroFEM

SAND2025-00525O NeuroFEM is a software tool that demonstrates a neuromorphic algorithm for solving finite element problems. It sets up a 2D finite element problem for the Poisson equation on a disk, constructs synaptic matrices, and simulates neural dynamics to solve the resulting sparse linear system. The software illustrates how the algorithm converges to the solution and plots the results, showcasing a neuromorphic counterpart to traditional methods like Conjugate Gradient or GMRES. This tool is designed to highlight the potential of neuromorphic algorithms for solving sparse linear systems, which are prevalent in various computational applications. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2020

Effects of Jacobian Matrix Regularization on the Detectability of Adversarial Samples

The well-known vulnerability of Deep Neural Networks to adversarial samples has led to a rapid cycle of increasingly sophisticated attack algorithms and proposed defenses. While most contemporary defenses have been shown to be vulnerable to carefully configured attacks, methods based on gradient regularization and out-of-distribution detection have attracted much interest recently by demonstrating higher resilience to a broad range of attack algorithms. However, no study has yet investigated the effect of combining these techniques. In this paper, we consider the effect of Jacobian matrix regularization on the detectability of adversarial samples on the CIFAR-10 image benchmark dataset. We find that regularization has a significant effect on detectability, and in some cases can make an undetectable attack on a baseline model detectable. In addition, we give evidence that regularization may mitigate the known weaknesses of detectors to high-confidence adversarial samples. The defenses we consider here are highly generalizable, and we believe they will be useful for further investigations to transfer machine learning robustness to other data domains.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

Samoa Updater: An Application of the Levenberg-Marquardt Method to Update DELFIC Predictions Using Field Measurements

The US Department of Energy (DOE) Forensics Operations (DFO) is a member of the Ground Collections Task Force (GCTF), which is responsible for sample collection of radiological debris for attribution should a nuclear detonation ever occur in the United States. The DFO runs the Defense Land Fallout Interpretive Code (DELFIC) Fallout Planning Tool to predict the deposition of fallout from a nuclear detonation. This prediction is refined using the DELFIC Updater tool, which takes ground measurements and adjusts DELFIC inputs to minimize the difference between prediction and observation, yielding improved predictions of fallout in locations both measured and not yet measured. Samoa, a framework for uncertainty analysis and optimization, is used to improve DELFIC predictive fallout modeling. This new capability using Samoa, dubbed “Samoa Updater,” is compared with the current DELFIC Updater, a brute-force sampling approach. Samoa Updater uses the Levenberg– Marquardt (LM) method, a gradient-based nonlinear least squares approach that uses the functional shape of the input space to increase optimization speed. In simulated test cases Samoa Updater yields faster and more accurate solutions than the current Updater.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2022

What Makes You Hold on to That Old Car? Joint Insights From Machine Learning and Multinomial Logit on Vehicle-Level Transaction Decisions

What makes you hold on to that old car? While the vast majority of household vehicles are still powered by conventional internal combustion engines, the progress of adopting emerging vehicle technologies will critically depend on how soon the existing vehicles are transacted out of the household fleet. Leveraging a nationally representative longitudinal data set, the Panel Study of Income Dynamics, this study examines how household decisions to dispose of or replace a given vehicle are: 1) influenced by the vehicle’s attributes, 2) mediated by households’ concurrent socio-demographic and economic attributes, and 3) triggered by key life cycle events. Coupled with a newly developed machine learning interpretation tool, TreeExplainer, we demonstrate an innovative use of machine learning models to augment traditional logit modeling to both generate behavioral insights and improve model performance. We find the two gradient-boosting-based methods, CatBoost and LightGBM, are the best performing machine learning models for this problem. The multinomial logistic model can achieve similar performance levels after its model specification is informed by TreeExplainer. Both machine learning and multinomial logit models suggest that while older vehicles are more likely to be disposed of or replaced than newer ones, such probability decreases as the vehicles serve the family longer. Pickup trucks and sport utility vehicles are less likely to be disposed of or replaced than cars, and leased vehicles are more likely to be transacted than owned vehicles. We find that married families, families with higher education levels, homeowners, and older families tend to keep their vehicles longer. Life events such as childbirth, residential relocation, and change of household composition and income are found to increase vehicle disposal and/or replacement. We provide additional insights on the timing of vehicle replacement or disposal, in particular, the presence of children and childbirth events are more strongly associated with vehicle replacement among younger parents.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Efficient Subset Simulation using Hamiltonian Neural Network enhanced Markov Chain Monte Carlo Methods

The Monte Carlo method delivers an unbiased estimate of the probability of failure. However, the variance of the estimate depends on the number of evaluated samples. This number must be very large for estimations of a low probability of failure. If the evaluation of each sample is computationally expensive, the crude Monte Carlo simulation strategy is impracticable. Therefore, subset simulations are used to reduce the required number of evaluations. Subset simulations require a Markov Chain Monte Carlo sampler, such as the random walk Metropolis-Hastings algorithm. The algorithm, however, struggles with sampling in low-probability regions, especially if they are narrow. As a consequence, advanced Markov Chain Monte Carlo simulations have been developed. In particular, the Hamiltonian Monte Carlo method explores the target distribution rapidly. Driven by the idea of Hamiltonian dynamics, this sampler provides a non-random walk through the target distribution. The incorporation of subset simulation and Hamiltonian Monte Carlo methods has shown promising results for reliability analysis. One downside of the Hamiltonian Monte Carlo method is that gradient evaluations are computationally expensive, especially when dealing with high-dimensional problems and evaluating long trajectories. We show that integrating Hamiltonian neural networks in Hamiltonian Monte Carlo simulations significantly speeds up the sampling task. Furthermore, the enhancement of adaptive trajectory length within the Hamiltonian Monte Carlo results in the efficient proposal of the following states. Based on this recent enhancement, we provide a fast sampling strategy for subset simulations using Hamiltonian neural networks to replace the evaluation of the gradient and significantly speed up the Hamiltonian Monte Carlo simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Dynamic Learning of Correlation Potentials for a Time-Dependent Kohn-Sham System

We develop methods to learn the correlation potential for a time-dependent Kohn-Sham (TDKS) system in one spatial dimension. We start from a low-dimensional two-electron system for which we can numerically solve the time-dependent Schr¨odinger equation; this yields electron densities suitable for training models of the correlation potential. We frame the learning problem as one of optimizing a least-squares objective subject to the constraint that the dynamics obey the TDKS equation. Applying adjoints, we develop efficient methods to compute gradients and thereby learn models of the correlation potential. Our results show that it is possible to learn values of the correlation potential such that the resulting electron densities match ground truth densities. We also show how to learn correlation potential functionals with memory, demonstrating one such model that yields reasonable results for trajectories outside the training set.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Adaptive Model-Free Vehicle Path-Tracking via Fast-Converging Prescribed-Time Newton-Based Extremum-Seeking Control

Model-free control (MFC) offers a simple and effective approach to automated vehicle path-tracking without requiring an explicit plant model for control law design. However, gain tuning in MFC is typically carried out through trial-and-error, which can be time-consuming and may lead to suboptimal performance. To address this limitation, extremum-seeking-based adaptive MFC has shown promise by enabling real-time adaptation of control gains, without relying on a predefined vehicle model. Nonetheless, existing ESC approaches often suffer from slow convergence. This paper integrates MFC, employing longitudinal and lateral ultra-local models of a rear-wheel-drive vehicle, with a novel prescribed-time (PT) Newton-based extremum-seeking control (ESC) strategy that ensures rapid convergence of control gains within the prescribed time. Unlike conventional gradient-based ESC methods, the PT Newton-based ESC leverages artificial delays and time-periodic gains, not only to guarantee convergence within the specified time, but also to compensate for feedback delays. Simulation results demonstrate that the proposed approach significantly improves gain adaptation speed and tracking accuracy. This work advances adaptive model-free vehicle control by offering a high-performance, delay-resilient alternative to existing ESCMFC frameworks.

Waleed khan, Muhammad [The University of Texas at ↗
DOE OSTI2023

Trapped-ion entangling gates with bichromatic pair of microwave fields and magnetic field gradient

A trapped-ion quantum logic gate and a method of operating the trapped-ion quantum logic gate are provided. The trapped-ion quantum logic gate includes at least one ion having two internal states and forming a qubit having a qubit transition frequency ω 0 0, a magnetic field gradient, and two microwave fields. Each of the two microwave fields has a respective frequency that is detuned from the qubit transition frequency ω 0 by frequency difference δ. The at least one ion has a Rabi frequency Ω μ due to the two microwave fields and a Rabi frequency Ω g due to the magnetic field gradient. The method includes applying the magnetic field gradient and the two microwave fields to the at least one ion such that a quantity Ω g /δ is in a range between zero and 5×10 −2 .

Sutherland, Robert Tyler↗
DOE OSTI2022

A Market Feedback Framework for Improved Estimates of the Arbitrage Value of Energy Storage Using Price-Taker Models

Price-taker (PT) models are often used to assess the potential value or revenue of energy arbitrage opportunities for energy storage in wholesale markets. But as greater amounts of energy storage are deployed on the grid, current PT models fail to predict the effects that energy storage itself can have on market prices. This can lead to an overestimation of the economic value of storage and an inability to capture price suppression. In this paper, we propose the use of a modified PT model to simulate the impact of increased storage deployment on energy prices and the resulting impact on revenue. Our method uses a gradient-boosting regressor to estimate the impact on prices, and we apply our method on historical price data from the PJM and California Independent System Operator wholesale markets. We use this approach to explore possible causes of electricity price suppression that occur from storage capacity additions, which is generally not possible with PT models.

arbitrage↗
DOE OSTI2023

An ecologically-constrained deep learning model for tropical leaf phenology monitoring using PlanetScope satellites

In tropical forests, leaf phenology signals leaf-on/off status and exhibits considerable variability across scales from a single tree-crown to the entire forest ecosystem. Such phenology signals importantly regulate large-scale biogeochemical cycles and regional climate. PlanetScope CubeSats data with a 3-m resolution and near-daily global coverage provide an unprecedented opportunity to monitor both fine- and ecosystem-scale phenology variability along large environmental gradients. However, a scalable method that accurately characterizes leaf phenology from PlanetScope with biophysically meaningful metrics remains lacking. Here we developed an index-guided, ecologically constrained autoencoder (IG-ECAE) method to automatically derive a deciduousness metric (i.e. percentage of tree canopies with leaf-off status within an image pixel) from PlanetScope. The IG-ECAE first estimated the reflectance spectra of leafy/leafless canopies based on their spectral indices characteristics, then used the derived reflectance spectra to guide an autoencoder deep learning method with additional ecological constraints to refine the reflectance spectra, and finally used linear spectral unmixing to estimate the relative abundance of leafless canopies (or deciduousness) per PlanetScope image pixel. We tested the IG-ECAE method at 16 tropical forest sites spanning multiple continents and a large precipitation gradient (1470-2819 mm year -1 ). Among these sites, we evaluated the PlanetScope-derived deciduousness with corresponding measures derived from WorldView-2 (n=9 sites) and local phenocams (n=9 sites). Our results show that PlanetScope-derived deciduousness agrees: 1) with that derived from WorldView-2 at the patch level (90m×90m) with r 2 =0.89 across all sites; and 2) with that derived from phenocams to quantify ecosystem-scale seasonality with r 2 ranging from 0.62-0.96. These results demonstrate the effectiveness and scalability of IG-ECAE in characterizing the wide variability in deciduousness across scales from pixels to forest ecosystems, and from a single date to the full annual cycle, indicating the potential for using high-resolution satellites to track the large-scale phenological patterns and response of tropical forests to climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

The Global LAnd Surface Satellite (GLASS) evapotranspiration product Version 5.0: Algorithm development and preliminary validation

An accurate estimation of spatially and temporally continuous global terrestrial evapotranspiration (ET) is essential in the assessment of surface energy, water and carbon cycles. The Global LAnd Surface Satellite (GLASS) ET product Version 4.0 (v4.0) based on the Bayesian model averaging (BMA) method was generated to estimate global terrestrial ET. However, certain uncertainty for the GLASS ET product v4.0 limits its application. In this study, we introduced the deep neural networks (DNN) merging framework to improve terrestrial ET estimation for GLASS ET product Version 5.0 (v5.0) generation by integrating five satellite-derived ET products [Moderate Resolution Imaging Spectroradiometer (MODIS) ET product (MOD16), Shuttleworth–Wallace dual-source ET product (SW), Priestley–Taylor-based ET product (PT-JPL), modified satellite-based Priestley–Taylor ET product (MS-PT) and simple hybrid ET product (SIM)]. We compared the performance of DNN method against other merging methods, including GLASS ET algorithm v4.0 (BMA), the gradient boosting regression tree (GBRT) method and the random forest (RF) method, based on 195 global eddy covariance (EC) flux towers covering observations from 2000 through 2015. Validations indicated that the DNN had the highest accuracy among four merging methods across different land cover types, yielding the highest average determination coefficients (R 2 , 0.62), root-mean-squared-error (RMSE, 24.1 W/m 2 ) and Kling–Gupta efficiency (KGE, 0.77) with a of 99% confidence interval. Compared with GLASS ET algorithm v4.0, the DNN improved on the R 2 by approximately 7% (p < 0.01) and the KGE by 10%. Based on the DNN, we then generated 8-day GLASS ET product v5.0 globally with a 1 km spatial resolution from 2001 to 2015 driven by GLASS vegetation and surface net radiation (R n ) datasets and Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA2) datasets. Finally, this global terrestrial ET product provides a valuable dataset for monitoring regional and global water resources and environmental changes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗