Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “nonlinear least squares”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Derivative-free stochastic optimization via adaptive sampling strategies

In this paper, we present a novel derivative-free framework for solving unconstrained stochastic optimization problems. Many problems in fields ranging from simulation optimization to reinforcement learning to quantum computing involve settings where only stochastic function values are obtained via a zeroth-order oracle, which has no available gradient information and necessitates the usage of derivative-free optimization methodologies. Our approach includes estimating gradients using stochastic function evaluations and integrating adaptive sampling techniques to control the accuracy in these stochastic approximations. Our framework encapsulates several gradient estimation techniques, including standard finite-difference, Gaussian smoothing, sphere smoothing, randomized coordinate finite-difference, and randomized subspace finite-difference methods. We provide theoretical convergence guarantees for our framework and analyze the worst-case iteration and sample complexities associated with each gradient estimation method. Finally, we demonstrate the empirical performance of the methods on logistic regression and nonlinear least squares problems.

Adaptive sampling↗
DOE OSTI2026

ReMU: regional minimal updating for model-based derivative-free optimization

Derivative-free optimization (DFO) problems are optimization problems where derivative information is unavailable or extremely difficult to obtain. Model-based DFO solvers have been applied extensively in scientific computing. Powell's NEWUOA (2004) [Powell, The NEWUOA software for unconstrained optimization without derivatives, in Large-Scale Nonlinear Optimization, Nonconvex Optimization and its Applications Vol. 83, G. Di Pillo and M. Roma, eds., Springer, 2006, pp. 255–297] and Wild's POUNDerS (2014) [Wild, Solving derivative-free nonlinear least squares problems with POUNDERS, in Advances and Trends in Optimization with Engineering Applications, T. Terlaky, M.F. Anjos, and S. Ahmed, eds., SIAM, 2017, pp. 529–540] explore the numerical power of the minimal norm Hessian (MNH) model for DFO and contributed to the open discussion on building better models with fewer data to achieve faster numerical convergence. Another decade later, we propose the regional minimal updating (ReMU) models, and extend the previous models into a broader class, including the H 2 norm models [Xie and Yuan, Least H 2 norm updating of quadratic interpolation models for derivative-free trust-region algorithms, IMA J. Numer. Anal. 46 (2025), pp. 21–50]. This paper shows motivation behind ReMU models, computational details, theoretical and numerical results on particular extreme points and the barycentre of ReMU's weight coefficient region, and the associated KKT matrix error and distance. Novel metrics, such as the truncated Newton step error, are proposed to numerically understand the new models' properties. A new algorithmic strategy, based on iteratively adjusting the ReMU model type, is also proposed, and shows numerical advantages by combining and switching between the barycentric model and the classic least Frobenius norm model in an online fashion.

derivative-free trust-region methods↗
DOE OSTI2020

Algorithms in Diffraction Profile Analysis

This chapter introduces several approaches to crystallographic structure refinement from diffraction profile data including the Rietveld method, global optimization methods, and Bayesian modeling. First, the Rietveld method and related nonlinear least squares approaches to analyzing diffraction profiles are briefly discussed. We then review the most common algorithms within global optimization as alternative methods of structure solution: grid search, simulated annealing, and genetic algorithms. As the Rietveld method is the standard tool for analyzing diffraction profile data, we catalogue its challenges and limitations. A Bayesian approach to diffraction profile analysis is presented as an alternative, along with several sampling algorithms including Gibbs sampling, variants of Metropolis sampling, Hamiltonian Monte Carlo, and approximate Bayesian computing. These sampling algorithms are applied to the fitting of a neutron diffraction profile from a National Institute of Standards and Technology silicon standard reference material, and their results are compared.

Singh, Susheela↗
DOE OSTI2022

Investigating the Influence of Ni, ZrO 2 , and Y 2 O 3 from SOFC Anodes on Siloxane Deposition

Siloxanes, as a type of impurity in biogas, can poison the Ni-YSZ anode of SOFCs. However, the influence of individual components of the anode, such as Ni, ZrO 2 , and Y 2 O 3 , on the siloxane deposition process has not been investigated extensively. In this study, Ni, ZrO 2 , and Y 2 O 3 pellets were exposed to H 2 + N 2 + H 2 O + D4 (octamethylcyclotetrasiloxane, 2.5 ppmv) and H 2 + N 2 + D4 (2.5 ppmv siloxane) gas mixtures at 750 °C to investigate their affinity and tolerance for siloxane degradation. Surface morphology analysis and electrochemical analysis including electrochemical impedance spectroscopy (EIS), related distribution of relaxation times (DRT) analysis and equivalent circuit modeling with complex nonlinear least square (CNLS) fitting were conducted. Here, a microstructure parameter—tortuosity factor to porosity ratio $\tau /\varepsilon $ calculated by diffusion polarization resistance was utilized for siloxane deposition evaluation. After comparing pellets surface morphology changes before and after experiments and $\tau /\varepsilon $ change following the contamination, Ni is considered as a major factor in siloxane deposition reactions in Ni-YSZ anode.

25 ENERGY STORAGE↗
DOE OSTI2021

A Novel Multiphysics Multiscale Multiporosity Shale Gas Transport Model for Geomechanics/Flow Coupling in Steady and Transient States

Summary A novel multiphysics multiscale multiporosity shale gas transport (M3ST) model was developed to investigate shale gas transport in both transient and steady states. The microscale model component contains a kerogen domain and an inorganic matrix domain, and each domain has its own geomechanical and gas transport properties. Permeabilities of various shale cores were measured in the laboratory using a pulse decay permeameter (PDP) with different pore pressure and confining stress combinations. The PDP-measured apparent permeability as a function of pore pressure under two effective stresses was fitted using the microscale M3ST model component based on nonlinear least squares fitting (NLSF), and the fitted model parameters were able to provide accurate model predictions for another effective stress. The parameters and petrophysical properties determined in the steady state were then used in the transient-state, continuum-scale M3ST model component, which performed history matching of the evolutions of the upstream and downstream gas pressures. In addition, a double-exponential empirical model was developed as a powerful alternative to the M3ST model to fit laboratory-measured apparent permeability under various effective stresses and pore pressures. The developed M3ST model and the research findings in this study provided critical insights into the role of the multiphysics mechanisms, including geomechanics, fluid dynamics and transport, and the Klinkenberg effect on shale gas transport across different spatial scales in both steady and transient states.

Engineering↗
DOE OSTI2021

TAO Users Manual (Rev. 3.15)

The Toolkit for Advanced Optimization (TAO) focuses on the development of algorithms and software for the solution of large-scale optimization problems on high-performance architectures. Areas of interest include unconstrained and bound-constrained optimization, nonlinear least squares problems, optimization problems with partial differential equation constraints, and variational inequalities and complementarity constraints. The development of TAO was motivated by the scattered support for parallel computations and the lack of reuse of external toolkits in current optimization software. Our aim is to produce high-quality optimization software for computing environments ranging from workstations and laptops to massively parallel high-performance architectures. Our design decisions are strongly motivated by the challenges inherent in the use of large-scale distributed memory architectures and the reality of working with large, often poorly structured legacy codes for specific applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Samoa Updater: An Application of the Levenberg-Marquardt Method to Update DELFIC Predictions Using Field Measurements

The US Department of Energy (DOE) Forensics Operations (DFO) is a member of the Ground Collections Task Force (GCTF), which is responsible for sample collection of radiological debris for attribution should a nuclear detonation ever occur in the United States. The DFO runs the Defense Land Fallout Interpretive Code (DELFIC) Fallout Planning Tool to predict the deposition of fallout from a nuclear detonation. This prediction is refined using the DELFIC Updater tool, which takes ground measurements and adjusts DELFIC inputs to minimize the difference between prediction and observation, yielding improved predictions of fallout in locations both measured and not yet measured. Samoa, a framework for uncertainty analysis and optimization, is used to improve DELFIC predictive fallout modeling. This new capability using Samoa, dubbed “Samoa Updater,” is compared with the current DELFIC Updater, a brute-force sampling approach. Samoa Updater uses the Levenberg– Marquardt (LM) method, a gradient-based nonlinear least squares approach that uses the functional shape of the input space to increase optimization speed. In simulated test cases Samoa Updater yields faster and more accurate solutions than the current Updater.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2022

humidogram.c1

fRH equivalent hygroscopicity parameter calculated by a nonlinear least squares fit method

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Estimating and Calibrating DER Model Parameters Using Levenberg–Marquardt Algorithm in Renewable Rich Power Grid

The proliferation of inverter-based distributed energy resources (IBDERs) has increased the number of control variables and dynamic interactions, leading to new grid control challenges. For stability analysis and designing appropriate protection controls, it is important that IBDER models are accurate. This paper focuses on the accurate estimation and parameter calibration of DER_A, a recently proposed aggregated IBDER model. In particular, we focus on the parameters of the reactive power–voltage regulation module. We formulate the problem of parameter tuning as a non-linear least square minimization problem and solve it using the Levenberg–Marquardt (LM) method. The LM method is primarily chosen due to its flexibility in adaptively selecting between the steepest descent and Gauss–Newton methods through a damping parameter. The LM approach is used to minimize the error between the actual measurements and the estimated response of the model. Further, the computational challenges posed by the numerical calculation of the Jacobian are tackled using a quasi-Newton root-finding approach. The proposed method is validated on a real feeder model in the northeastern part of the United States. The feeder is modeled in OpenDSS and the measurements thus obtained are fed to the DER_A model for calibration. The simulation results indicate that our approach is able to successfully calibrate the relevant model parameters quickly and with high accuracy, with a total sum of square error of 3.57 × 10 –7 .

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

A moving discontinuous Galerkin finite element method with interface condition enforcement for compressible flows

A variation of moving discontinuous Galerkin finite element method with interface condition enforcement (MDG-ICE) is developed for solving the compressible Euler equations. The MDG-ICE method, originating from the work of Corrigan et al. [1], [2], [3], [4], is based on the space-time DG formulation, where both flow field and grid geometry are considered as independent variables and the conservation laws are enforced both on discrete elements and element interfaces. The element conservation laws are solved in the standard discontinuous solution space to determine conservative quantities, while the interface conservation is enforced using a variational formulation in a continuous space to determine discrete grid geometry. The resulting over-determined system of nonlinear equations arising from the MDG-ICE formulation can then be solved in a least-squares sense, leading to an unconstrained nonlinear least-squares problem that is regularized and solved by Levenberg-Marquardt method. A number of numerical experiments for both 1D unsteady and 2D steady state compressible flow problems are conducted to assess the accuracy and robustness of the MDG-ICE method. Numerical results obtained indicate that the MDG-ICE method is able to implicitly detect and track all types of discontinuities via interface conservation enforcement and satisfy the conservation law on both elements and interfaces via grid movement and grid management, demonstrating that an exponential rate of convergence for Sod and Lax-Harden shock tube problems can be achieved and highly accurate solutions without overheating to both double-rarefaction wave and Noh problems can be obtained.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic average model methods

We consider the solution of finite-sum minimization problems, such as those appearing in nonlinear least-squares or general empirical risk minimization problems. We are motivated by problems in which the summand functions are computationally expensive and evaluating all summands on every iteration of an optimization method may be undesirable. Here we present the idea of stochastic average model (SAM) methods, inspired by stochastic average gradient methods. SAM methods sample component functions on each iteration of a trust-region method according to a discrete probability distribution on component functions; the distribution is designed to minimize an upper bound on the variance of the resulting stochastic model. We present promising numerical results concerning an implemented variant extending the derivative-free model-based trust-region solver POUNDERS, which we name SAM-POUNDERS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Subspace Methods in Multi-Parameter Seismic Full Waveform Inversion

In full waveform inversion (FWI) high-resolution subsurface model parameters are sought. FWI is normally treated as a nonlinear least-squares inverse problem, in which the minimum of the corresponding misfit function is found by updating the model parameters. When multiple elastic or acoustic properties are solved for, simple gradient methods tend to confuse parameter classes. This is referred to as parameter cross-talk; it leads to incorrect model solutions, poor convergence and strong dependence on the scaling of the different parameter types. Determining step lengths in a subspace domain, rather than directly in terms of gradients of different parameters, is a potentially valuable approach to address this problem. The particular subspace used can be defined over a span of different sets of data or different parameter classes, provided it involves a small number of vectors compared to those contained in the whole model space. Additionally, in a subspace method, the basis vectors are defined first, and a local minimum is found in the space spanned by these. We examine the application of the subspace method within acoustic FWI in determining simultaneously updates for velocity and density. We first discuss the choice of basis vectors to construct the spanned space, from linear updates by distinguishing only the contributions of different parameter classes towards nonlinear updates by adding the contributions of higher-order perturbations of each parameter class. The numerical character of FWI solutions generated via subspace methods involving different basis vectors is then analyzed and compared with traditional FWI methods. The subspace methods can provide better reconstructions of the model, especially for the velocity, as well as improved convergence rates, while the computational costs are still comparable with the traditional FWI methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

A Scalable Space-Time Domain Decomposition Approach for Solving Large Scale Nonlinear Regularized Inverse Ill Posed Problems in 4D Variational Data Assimilation

We address the development of innovative algorithms designed to solve the strong-constraint Four Dimensional Variational Data Assimilation (4DVar DA) problems in large scale applications. We present a space-time decomposition approach which employs the whole domain decomposition, i.e. both along the spacial and temporal direction in the overlapping case, and the partitioning of both the solution and the operator. Starting from the global functional defined on the entire domain, we get to a sort of regularized local functionals on the set of sub domains providing the order reduction of both the predictive and the Data Assimilation models. The algorithm convergence is developed. Performance in terms of reduction of time complexity and algorithmic scalability is discussed on the Shallow Water Equations on the sphere. The number of state variables in the model, the number of observations in an assimilation cycle, as well as numerical parameters as the discretization step in time and in space domain are defined on the basis of discretization grid used by data available at repository Ocean Synthesis/Reanalysis Directory of Hamburg University.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Biomass char gasification kinetic rates compared to data, including ash effects

This investigation provides both experimental and theoretical analyses of biomass char gasification kinetics over 1150–1350 °C in the environment of CO 2 , H 2 O and combinations of these reactants. The experimental data include continuous measurements of particle mass, external and internal temperatures, shape, and size with periodic but not continuous measurements of char porosity, pore size distribution, and pore surface area. The model describes heterogeneous char gasification for poplar wood, corn stover and switchgrass in different shapes and industrially relevant sizes (12.5 mm and smaller). Nonlinear least-squares regression produces optimized power-law model parameters for global kinetics that describe biomass char gasification with respect to both CO 2 and H 2 O separately and in combination. A single set of kinetic parameters describes reaction rates for all three fuels. Model simulations agree with measured data at all stages of char conversion even though the observed mass loss rates change significantly with conversion. In conclusion, the change in rate with conversion depends strongly on ash content, and this investigation provides a simple and theoretically based reaction rate expression that includes the effect of inert ash occupying an increasing portion of the reactive surface with increased conversion.

09 BIOMASS FUELS↗
DOE OSTI2021

Quasi-periodic travelling gravity–capillary waves

We present a numerical study of spatially quasi-periodic travelling waves on the surface of an ideal fluid of infinite depth. This is a generalization of the classic Wilton ripple problem to the case when the ratio of wavenumbers satisfying the dispersion relation is irrational. We propose a conformal mapping formulation of the water wave equations that employs a quasi-periodic variant of the Hilbert transform to compute the normal velocity of the fluid from its velocity potential on the free surface. We develop a Fourier pseudo-spectral discretization of the travelling water wave equations in which one-dimensional quasi-periodic functions are represented by two-dimensional periodic functions on the torus. This leads to an overdetermined nonlinear least-squares problem that we solve using a variant of the Levenberg–Marquardt method. We investigate various properties of quasi-periodic travelling waves, including Fourier resonances, time evolution in conformal space on the torus, asymmetric wave crests, capillary wave patterns that change from one gravity wave trough to the next without repeating and the dependence of wave speed and surface tension on the amplitude parameters that describe a two-parameter family of waves.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

NIR fluorescence lifetime macroscopic imaging with a time-gated SPAD camera

The performance of SwissSPAD2 (SS2), a large scale, widefield time-gated CMOS SPAD imager developed for fluorescence lifetime imaging, has recently been described in the context of visible range and fluorescence lifetime imaging microscopy (FLIM) of dyes with lifetimes in the 2.5 – 4 ns range. Here, we explore its capabilities in the NIR regime relevant for small animal imaging, where its sensitivity is lower and typical NIR fluorescent dye lifetimes are much shorter (1 ns or less). We carry out this study in a simple macroscopic imaging setup based on a compact NIR picosecond pulsed laser, an engineered diffuser-based illumination optics, and NIR optimized imaging lens suitable for well-plate or small animal imaging. Because laser repetition rates can vary between models, but the synchronization signal frequency accepted by SS2 is fixed to 20 MHz, we first checked that a simple frequency-division scheme enables data recording for different laser repetition rates. Next, we acquired data using different time gate widths, including gates with duration longer than the laser period, and analyzed the resulting data using both standard nonlinear least-square fit (NLSF) and phasor analysis. We show that the fixed synchronization rate and large gate widths characterizing SS2 (10 ns and over) are not an obstacle to accurately extracting lifetime in the 1 ns range and to distinguishing between close lifetimes. As a result, SS2 and similar very large gated SPAD imagers appear as a versatile alternative to other widefield time-resolved detectors for NIR fluorescence lifetime imaging, including preclinical molecular applications.

FLIM↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗