Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “iterative smoothing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Convergence analysis for a nonlocal gradient descent method via directional Gaussian smoothing

We analyze the convergence of a nonlocal gradient descent method for minimizing a class of high-dimensional non-convex functions, where a directional Gaussian smoothing (DGS) is proposed to define the nonlocal gradient (also referred to as the DGS gradient). The method was first proposed in [Zhang et al., Enabling long-range exploration in minimization of multimodal functions, UAI 2021], in which multiple numerical experiments showed that replacing the traditional local gradient with the DGS gradient can help the optimizers escape local minima more easily and significantly improve their performance. However, a rigorous theory for the efficiency of the method on nonconvex landscape is lacking. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by deterministic oscillating noise. We provide a convergence theory under which the iterates exponentially converge to a tightened neighborhood of the solution, whose size is characterized by the noise wavelength. Here, we also establish a correlation between the optimal values of the Gaussian smoothing radius and the noise wavelength, thus justifying the advantage of using moderate or large smoothing radii with the method. Furthermore, if the noise level decays to zero when approaching the global minimum, we prove that DGS-based optimization converges to the exact global minimum with linear rates, similarly to standard gradient-based methods in optimizing convex functions. Several numerical experiments are provided to confirm our theory and illustrate the superiority of the approach over those based on the local gradient.

Tran, Hoang [Oak Ridge National Laboratory (ORNL),↗
DOE OSTI2024

Minimum-Energy Conical Intersections by Compressed Multistate Pair-Density Functional Theory

Compressed multistate pair-density functional theory (CMS-PDFT) is a multistate version of multiconfiguration pair-density functional theory that can capture the correct topology of coupled potential energy surfaces (PESs) around conical intersections. Here, in this work, we develop interstate coupling vectors (ISCs) for CMS-PDFT in the OpenMolcas and PySCF/mrh electronic structure packages. Yet, the main focus of this work is using ISCs to calculate minimum-energy conical intersections (MECIs) by CMS-PDFT. This is performed using the projected constrained optimization method in OpenMolcas, which uses ISCs to restrain the iterations to the conical intersection seam. We optimize the S 1 /S 0 MECIs for ethylene, butadiene, and benzene and show that CMS-PDFT gives smooth PESs in the vicinities of the MECIs. Furthermore, the CMS-PDFT MECIs are in good agreement with the MECI calculated by the more expensive XMS-CASPT2 method.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Physics↗
DOE OSTI2025

Derivative-free stochastic optimization via adaptive sampling strategies

In this paper, we present a novel derivative-free framework for solving unconstrained stochastic optimization problems. Many problems in fields ranging from simulation optimization to reinforcement learning to quantum computing involve settings where only stochastic function values are obtained via a zeroth-order oracle, which has no available gradient information and necessitates the usage of derivative-free optimization methodologies. Our approach includes estimating gradients using stochastic function evaluations and integrating adaptive sampling techniques to control the accuracy in these stochastic approximations. Our framework encapsulates several gradient estimation techniques, including standard finite-difference, Gaussian smoothing, sphere smoothing, randomized coordinate finite-difference, and randomized subspace finite-difference methods. We provide theoretical convergence guarantees for our framework and analyze the worst-case iteration and sample complexities associated with each gradient estimation method. Finally, we demonstrate the empirical performance of the methods on logistic regression and nonlinear least squares problems.

Adaptive sampling↗
DOE OSTI2022

Inexact Newton-CG algorithms with complexity guarantees

Abstract We consider variants of a recently developed Newton-CG algorithm for nonconvex problems (Royer, C. W. & Wright, S. J. (2018) Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477) in which inexact estimates of the gradient and the Hessian information are used for various steps. Under certain conditions on the inexactness measures, we derive iteration complexity bounds for achieving $\epsilon $-approximate second-order optimality that match best-known lower bounds. Our inexactness condition on the gradient is adaptive, allowing for crude accuracy in regions with large gradients. We describe two variants of our approach, one in which the step size along the computed search direction is chosen adaptively, and another in which the step size is pre-defined. To obtain second-order optimality, our algorithms will make use of a negative curvature direction on some steps. These directions can be obtained, with high probability, using the randomized Lanczos algorithm. In this sense, all of our results hold with high probability over the run of the algorithm. We evaluate the performance of our proposed algorithms empirically on several machine learning models. Our approach is a first attempt to introduce inexact Hessian and/or gradient information into the Newton-CG algorithm of Royer & Wright (2018, Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477).

Mathematics↗
DOE OSTI2024

A comprehensive review of dwell time optimization methods in computer-controlled optical surfacing

Dwell time plays a vital role in determining the accuracy and convergence of the computer-controlled optical surfacing process. However, optimizing dwell time presents a challenge due to its ill-posed nature, resulting in non-unique solutions. To address this issue, several well-known methods have emerged, including the iterative, Bayesian, Fourier transform, and matrix-form methods. Despite their independent development, these methods share common objectives, such as minimizing residual errors, ensuring dwell time's positivity and smoothness, minimizing total processing time, and enabling flexible dwell positions. This paper aims to comprehensively review the existing dwell time optimization methods, explore their interrelationships, provide insights for their effective implementations, evaluate their performances, and ultimately propose a unified dwell time optimization methodology.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Split representation of adaptively compressed polarizability operator

The polarizability operator plays a central role in density functional perturbation theory and other perturbative treatment of first principle electronic structure theories. The cost of computing the polarizability operator generally scales as O(Ne4) where Ne is the number of electrons in the system. The recently developed adaptively compressed polarizability operator (ACP) formulation [L. Lin, Z. Xu and L. Ying, Multiscale Model. Simul. 2017] reduces such complexity to O(Ne 3 ) in the context of phonon calculations with a large basis set for the first time, and demonstrates its effectiveness for model problems. In this paper, we improve the performance of the ACP formulation by splitting the polarizability into a near singular component that is statically compressed, and a smooth component that is adaptively compressed. The new split representation maintains the O(Ne 3 ) complexity, and accelerates nearly all components of the ACP formulation, including Chebyshev interpolation of energy levels, iterative solution of Sternheimer equations, and convergence of the Dyson equations. For simulation of real materials, we discuss how to incorporate nonlocal pseudopotentials and finite temperature effects. In this work, we demonstrate the effectiveness of our method using one-dimensional model problem in insulating and metallic regimes, as well as its accuracy for real molecules and solids.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

srMO-BO-3GP: A sequential regularized multi-objective Bayesian optimization for constrained design applications using an uncertain Pareto classifier

Bayesian optimization (BO) is an efficient and flexible global optimization framework that is applicable to a very wide range of engineering applications. To leverage the capability of the classical BO, many extensions, including multi-objective, multi-fidelity, parallelization, and latent-variable modeling, have been proposed to address the limitations of the classical BO framework. In this work, we propose a novel multi-objective BO formalism, called srMO-BO-3GP, to solve multi-objective optimization problems in a sequential setting. Three different Gaussian processes (GPs) are stacked together, where each of the GPs is assigned with a different task. The first GP is used to approximate a single-objective computed from the multi-objective definition, the second GP is used to learn the unknown constraints, and the third one is used to learn the uncertain Pareto frontier. At each iteration, a multi-objective augmented Tchebycheff function is adopted to convert multi-objective to single-objective, where the regularization with a regularized ridge term is also introduced to smooth the single-objective function. Finally, we couple the third GP along with the classical BO framework to explore the convergence and diversity of the Pareto frontier by the acquisition function for exploitation and exploration. The proposed framework is demonstrated using several numerical benchmark functions, as well as a thermomechanical finite element model for flip-chip package design optimization.

42 ENGINEERING↗
DOE OSTI2021

Topology optimization of 3D photonic crystals with complete bandgaps

The design of photonic crystals with complete bandgaps has recently received considerable research focus for numerous reasons. This work leverages well-known nonlinear programming techniques to alleviate the non-smoothness caused by degenerate eigenvalues such that topology optimization problems can be solved with the open-source IPOPT software. A fully-vectorial plane wave expansion technique is used with an iterative eigensolver to efficiently predict dispersion properties of candidate structures. Nonlinear programming is employed to solve the inverse problem of designing three-dimensional periodic structures that exhibit complete two-dimensional (2D) and three-dimensional (3D) photonic bandgaps. Mesh refinement is performed to alleviate the large computational burden of designing and analyzing photonic crystals, and a periodic density filter is implemented to impose a minimum feature size for manufacturability considerations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

High-Fidelity Accelerated Design of High-performance Electrochemical Systems

Large-scale electrification is vital to addressing the climate crisis, but several scientific and technological challenges remain to fully electrify both the chemical industry and transportation. In both of these areas, new electrochemical materials will be critical, but their development currently relies heavily on human-time-intensive experimental trial and error and computationally expensive first-principles, meso-scale and continuum simulations. To accelerate this process, our team has developed the AutoMat platform. AutoMat can accelerate development of new electrochemical materials along two avenues: first, automated input generation and management of simulations at multiple lengthscales as well as “handoff” of outputs from one lengthscale as inputs to the next; and second, replacement of the most computationally intensive simulation processes with machine-learned surrogate models. The crux of our team’s effort was not “reinventing the wheel” by developing entirely new techniques, but rather building a “superhighway” that allows existing state-of-the-art techniques to run faster and more smoothly than before. AutoMat can utilize tools spanning from first-principles quantum chemistry computations to automated robotic experimentation, and is driven by design space search techniques to reduce the number of iterations through the full simulation loop by rapidly targeting promising regions of design spaces such as single-atom alloy catalysts or blends of liquid electrolytes.

25 ENERGY STORAGE↗
DOE OSTI2026

Desmearing Bonse–Hart USANS data using Bayesian Gaussian process regression

Ultra-small-angle neutron scattering (USANS) enables access to micrometer-scale structures but is intrinsically affected by strong, anisotropic resolution smearing arising from slit-geometry optics. As a result, recovery of the intrinsic scattering intensity constitutes an ill-posed inverse problem, and commonly used iterative desmearing methods lack rigorous uncertainty quantification. We present a Bayesian desmearing framework for slit-geometry USANS based on Gaussian process regression. In this approach, the scattering intensity is modeled as a smooth random function, and the instrumental point spread function is incorporated explicitly as a forward operator. The resulting formulation yields a closed-form maximum a posteriori solution with well-defined credibility intervals. Computational benchmarks and experimental validation using combined USANS and small-angle neutron scattering (SANS) measurements demonstrate that the framework enables stable desmearing, suppresses experimental noise, and preserves physically meaningful structural features under realistic conditions.

Tung, Chi-Huan [Oak Ridge National Laboratory (ORN↗
DOE OSTI2025

Efficient shallow Ritz method for 1D diffusion problems

This paper studies the shallow Ritz method for solving the one-dimensional diffusion problem. It is shown that the shallow Ritz method improves the order of approximation dramatically for non-smooth problems. To realize this optimal or nearly optimal order of the shallow Ritz approximation, we develop a damped block Newton (dBN) method that alternates between updates of the linear and non-linear parameters. Per each iteration, the linear and the non-linear parameters are updated by exact inversion and one step of a modified, damped Newton method applied to a reduced non-linear system, respectively. The computational cost of each dBN iteration is $\mathcal{O}$(n). Starting with the non-linear parameters as a uniform partition of the interval, numerical experiments show that the dBN is capable of efficiently moving mesh points to nearly optimal locations. In conclusion, to improve the efficiency of the dBN further, we propose an adaptive damped block Newton (AdBN) method by combining the dBN with the adaptive neuron enhancement (ANE) method [28].

Diffusion problems↗
DOE OSTI2022

Deblurring galaxy images with Tikhonov regularization on magnitude domain

Abstract We propose a regularization-based deblurring method that works efficiently for galaxy images. The spatial resolution of a ground-based telescope is generally limited by seeing conditions and is much worse than space-based telescopes. This circumstance has generated considerable research interest in the restoration of spatial resolution. Since image deblurring is a typical inverse problem and often ill-posed, solutions tend to be unstable. To obtain a stable solution, much research has adopted regularization-based methods for image deblurring, but the regularization term is not necessarily appropriate for galaxy images. Although galaxies have an exponential or Sérsic profile, the conventional regularization assumes the image profiles to behave linearly in space. The significant deviation between the assumption and real situations leads to blurring of the images and smoothing out the detailed structures. Clearly, regularization on logarithmic domain, i.e., magnitude domain, should provide a more appropriate assumption, which we explore in this study. We formulate a problem of deblurring galaxy images by an objective function with a Tikhonov regularization term on a magnitude domain. We introduce an iterative algorithm minimizing the objective function with a primal–dual splitting method. We investigate the feasibility of the proposed method using simulation and observation images. In the simulation, we blur galaxy images with a realistic point spread function and add both Gaussian and Poisson noise. For the evaluation with the observed images, we use galaxy images taken by the Subaru HSC-SSP. Both of these evaluations show that our method successfully recovers the spatial resolution of the deblurred images and significantly outperforms the conventional methods. The code is publicly available from the GitHub 〈https://github.com/kzmurata-astro/PSFdeconv_amag〉.

Murata, Kazumi↗
DOE OSTI2020

Extended Finite Element Based Approach in Additive Manufacturing Modeling for Optimizing Highly Complex Manifold in Protonic Ceramic Electrochemical Cells

The objective of this project is to develop a novel numerical approach based on the extended finite element method (XFEM) for modeling moving boundaries of material deposited in additive manufacturing (AM) process with improved accuracy and reduced computational cost. One of the major challenges in simulating AM processes is that the boundaries of the computational domain need to change as material is deposited. Previous approaches have achieved this by activating new finite elements, but this requires a high level of mesh refinement to capture small movement of the boundary. XFEM allows the solution boundary to move independently of the mesh, permitting smooth representation of the evolution of the boundary as material is deposited. As a result, this new method will significantly improve the simulation accuracy while reducing the computational cost compared with existing approaches. Furthermore, this project will bring significant improvements for the design and AM process optimization in an iterative fashion among numerical simuation, parameter optimization, and experimental validation. Specifically, this project will support the protonic ceramic electrochemical cells (PCEC) stack development at INL by providing insights into the density, residual stress, thermal and mechanical properties of the PCEC manifold and interconnect, which is critical in enhancing the system lifetime and reducing the overall cost. Upon the success of proposed development and validation of the proposed approach, simulation will be applied to determine an optimal set of AM process parameters for the PCEC manifold production. This project will consolidate Idaho National Laboratory (INL)’s simulation capabilities provided by the MOOSE framework and the Valhalla AM simulation application, with encouraging expansion to emerging PCEC applications and AM technologies at INL and beyond.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Exploiting the Local Parabolic Landscapes of Adversarial Losses to Accelerate Black-Box Adversarial Attack

Existing black-box adversarial attacks on image classifiers update the perturbation at each iteration from only a small number of queries of the loss function. Since the queries contain very limited information about the loss, black-box methods usually require much more queries than white-box methods. We propose to improve the query efficiency of black-box methods by exploiting the smoothness of the local loss landscape. However, many adversarial losses are not locally smooth with respect to pixel perturbations. To resolve this issue, our first contribution is to theoretically and experimentally justify that the adversarial losses of many standard and robust image classifiers behave like parabolas with respect to perturbations in the Fourier domain. Our second contribution is to exploit the parabolic landscape to build a quadratic approximation of the loss around the current state, and use this approximation to interpolate the loss value as well as update the perturbation without additional queries. Since the local region is already informed by the quadratic fitting, we use large perturbation steps to explore far areas. We demonstrate the efficiency of our method on MNIST, CIFAR-10 and ImageNet datasets for various standard and robust models, as well as on Google Cloud Vision. The experimental results show that exploiting the loss landscape can help significantly reduce the number of queries and increase the success rate. Our codes are available at https://github.com/HoangATran/BABIES.

Tran, Hoang↗
DOE OSTI2023

Stochastic projective splitting

Here, we present a new, stochastic variant of the projective splitting (PS) family of algorithms for inclusion problems involving the sum of any finite number of maximal monotone operators. This new variant uses a stochastic oracle to evaluate one of the operators, which is assumed to be Lipschitz continuous, and (deterministic) resolvents to process the remaining operators. Our proposal is the first version of PS with such stochastic capabilities. We envision the primary application being machine learning (ML) problems, with the method’s stochastic features facilitating “mini-batch” sampling of datasets. Since it uses a monotone operator formulation, the method can handle not only Lipschitz-smooth loss minimization, but also min–max and noncooperative game formulations, with better convergence properties than the gradient descent-ascent methods commonly applied in such settings. The proposed method can handle any number of constraints and nonsmooth regularizers via projection and proximal operators. We prove almost-sure convergence of the iterates to a solution and a convergence rate result for the expected residual, and close with numerical experiments on a distributionally robust sparse logistic regression problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A conservative discontinuous Galerkin algorithm for particle kinetics on smooth manifolds

A novel, conservative discontinuous Galerkin algorithm is presented for particle kinetics on manifolds. The motion of particles on the manifold is represented using both canonical and non-canonical Hamiltonian formulations. Our schemes apply to both formulations, but the canonical formulation results in a particularly efficient scheme that also conserves particle density and energy exactly. The collisionless update is coupled to a Bhatnagar-Gross-Krook (BGK) collision operator that provides a simplified model for relaxation to local thermodynamic equilibrium. An iterative scheme is constructed to ensure collisional invariants (density, momentum and energy) are preserved numerically. Rotation of the manifold is incorporated by modifying the Hamiltonian while ensuring a canonical formulation. Several test problems, including a kinetic version of the classical Sod shock problem, Kelvin-Helmholtz instability on the surfaces of a sphere and a hyperboloid, with and without rotations, are presented. A prospectus for further development of this approach to simulation of kinetic theory in general relativity is presented.

Discontinuous Galerkin↗