Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convex functions”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Time Dilated Bundt Cake Analysis of PV Output [Poster]

We present a novel method for modeling time-dependent statistics in the power signal generated by a photovoltaic (PV) system. Our white-box machine learning method is interpretable and auditable, based on principles of multiperiodic basis functions and convex optimization. Our proposed method of time dilating the daily signal to remove night time values results in a novel representation of PV power signals, evocative of a ‘Bundt cake’. The proposed model describes the marginal distribution of power output as a function of date and time. The resulting probabilistic model of a PV system can be used to perform a variety of tasks, and here, we demonstrate the application of clear sky detection.

14 SOLAR ENERGY↗
DOE OSTI2025

Triangle Method for Dense ReLU Layers [SWR-25-72]

This software is an implementation of the methods for initializing and training neural networks to be more efficient per parameter, described more fully below and in the related publication: In theory, depth should make a ReLU network EXPONENTIALLY more efficient by enabling it to produce an exponential number of piecewise linear sections in its output. This reasoning is largely based on the work of mathematicians that have hand-constructed networks that make good use of depth. In practice however, even very deep ReLU networks that have been randomly initialized will behave identically to their shallow counterparts - missing an entire exponential dimension of efficiency. The triangle method is a first attempt at realizing the exponential potential of deep networks. Instead of randomly setting weights, we force pairs of neurons in each layer learn to build triangles (i.e. functions from [0,1] -> [0,1] that look like triangles). This is a very efficient pattern for generating lots of linear pieces because composing two triangular functions doubles the number of pieces with each composition. The triangle method is more than just a different initialization, it is a new paradigm of training. Instead of making direct updates to the matrix weights, we do an extra step of backpropagation to collect the derivatives of the loss function with respect to the shapes of the triangles, training them to tilt left or right. This process essentially holds the networks hand throughout the loss landscape and forces it to always use depth effectively by producing triangular shapes internally. This can produce several orders of magnitude of improvement on convex one-dimensional regression problems. Much more theoretical work is needed to realize its full potential beyond this context, but the implementation in this repository will still work in arbitrary numbers of dimensions. The file Triangle_Method.py is a generalized form of the method that will build each neuron its own custom 1-d convex activation function (with exponential efficiency). Example usage on one dimensional problems can be found in Example_Usage.ipynb and an example of using this in a real neural network can be found in Example_VGG16_CIFAR10.ipynb.

Milkert, Max [National Renewable Energy Laboratory↗
DOE OSTI2022

Subcell limiting strategies for discontinuous Galerkin spectral element methods

Here, we present a general family of subcell limiting strategies to construct robust high-order accurate nodal discontinuous Galerkin (DG) schemes. The main strategy is to construct compatible low order finite volume (FV) type discretizations that allow for convex blending with the high-order variant with the goal of guaranteeing additional properties, such as bounds on physical quantities and/or guaranteed entropy dissipation. For an implementation of this main strategy, four main ingredients are identified that may be combined in a flexible manner: (i) a nodal high-order DG method on Legendre–Gauss–Lobatto nodes, (ii) a compatible robust subcell FV scheme, (iii) a convex combination strategy for the two schemes, which can be element-wise or subcell-wise, and (iv) a strategy to compute the convex blending factors, which can be either based on heuristic troubled-cell indicators, or using ideas from flux-corrected transport methods. By carefully designing the metric terms of the subcell FV method, the resulting methods can be used on unstructured curvilinear meshes, are locally conservative, can handle strong shocks efficiently while directly guaranteeing physical bounds on quantities such as density, pressure or entropy. We further show that it is possible to choose the four ingredients to recover existing methods such as a provably entropy dissipative subcell shock-capturing approach or a sparse invariant domain preserving approach. We test the versatility of the presented strategies and mix and match the four ingredients to solve challenging simulation setups, such as the KPP problem (a hyperbolic conservation law with non-convex flux function), turbulent and hypersonic Euler simulations, and MHD problems featuring shocks and turbulence.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN

Physics-informed neural network (PINN) has been successfully applied in solving a variety of nonlinear non-convex forward and inverse problems. However, the training is challenging because of the non-convex loss functions and the multiple optima in the Bayesian inverse problem. In this work, we propose a multi-variance replica exchange stochastic gradient Langevin dynamics method to tackle the challenge of the multiple local optima in the optimization and the challenge of the multiple modal posterior distribution in the inverse problem. Replica exchange methods are capable of escaping from the local traps and accelerating the convergence; two chains with different temperatures are designed where the low temperature chain aims for the local convergence, and the target of the high temperature chain is to travel globally and explore the whole loss function entropy landscape. However, it may not be efficient to solve mathematical inversion problems by using the vanilla replica method directly since the method doubles the computational cost in evaluating the forward solvers (likelihood functions) in the two chains. To address this issue, we propose to make different assumptions on the energy function estimation and this facilities one to use solvers of different fidelities in the likelihood function evaluation. More precisely, one can use a solver with low fidelity in the high temperature chain while using a solver with high fidelity in the low temperature chain. Our proposed method significantly lowers the computational cost in the high temperature chain, meanwhile preserving the accuracy and converging very fast. Here we give an unbiased estimate of the swapping rate and give an estimation of the discretization error of the scheme. To verify our idea, we design and solve four inverse problems which have multiple modes. The proposed method is also employed to train the Bayesian PINN to solve the forward and inverse problems; faster and more accurate convergence has been observed when compared to the stochastic gradient Langevin dynamics (SGLD) method and vanilla replica exchange methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Accelerating gradient descent and Adam via fractional gradients

Here we propose a class of novel fractional-order optimization algorithms. We define a fractional-order gradient via the Caputo fractional derivatives that generalizes integer-order gradient. We refer it to as the Caputo fractional-based gradient, and develop an efficient implementation to compute it. A general class of fractional-order optimization methods is then obtained by replacing integer-order gradients with the Caputo fractional-based gradients. To give concrete algorithms, we consider gradient descent (GD) and Adam, and extend them to the Caputo fractional GD (CfGD) and the Caputo fractional Adam (CfAdam). We demonstrate the superiority of CfGD and CfAdam on several large scale optimization problems that arise from scientific machine learning applications, such as ill-conditioned least squares problem on real-world data and the training of neural networks involving non-convex objective functions. Numerical examples show that both CfGD and CfAdam result in acceleration over GD and Adam, respectively. We also derive error bounds of CfGD for quadratic functions, which further indicate that CfGD could mitigate the dependence on the condition number in the rate of convergence and results in significant acceleration over GD.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Stochastic exciton-scattering theory of optical line shapes: Renormalized many-body contributions

Spectral line shapes provide a window into the local environment coupled to a quantum transition in the condensed phase. In this paper, we build upon a stochastic model to account for non-stationary background processes produced by broad-band pulsed laser stimulation, as distinguished from those for stationary phonon bath. In particular, we consider the contribution of pair-fluctuations arising from the full bosonic many-body Hamiltonian within a mean-field approximation, treating the coupling to the system as a stochastic noise term. Herein, using the Itô transformation, we consider two limiting cases for our model, which lead to a connection between the observed spectral fluctuations and the spectral density of the environment. In the first case, we consider a Brownian environment and show that this produces spectral dynamics that relax to form dressed excitonic states and recover an Anderson–Kubo-like form for the spectral correlations. In the second case, we assume that the spectrum is Anderson–Kubo like and invert to determine the corresponding background. Using the Jensen inequality, we obtain an upper limit for the spectral density for the background. The results presented here provide the technical tools for applying the stochastic model to a broad range of problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Computational optimal transport for molecular spectra: The fully continuous case

Computational optimal transport is used to analyze the difference between pairs of continuous molecular spectra. It is demonstrated that transport distances which are derived from this approach may be a more appropriate measure of the difference between two continuous spectra than more familiar measures of distance under many common circumstances. Associated with the transport distances is the transport map which provides a detailed analysis of the difference between two molecular spectra and is a key component of our study of quantitative differences between two continuous spectra. The use of optimal transport for comparing molecular spectra is developed in detail here with a set of model spectra, so that the discussion is self-contained. The difference between the transport distance and more common definitions of distance is elucidated for some well-chosen examples and it is shown where transport distances may be very useful alternatives to standard definitions of distance. The transport distance between a theoretical and experimental electronic absorption spectrum for SO 2 is studied and it is shown how the theoretical spectrum can be modified to fit the experimental spectrum better adjusting the theoretical band origin and the resolution of the theoretical spectrum. In conclusion, this analysis includes the calculation of transport maps between the theoretical and experimental spectra suggesting future applications of the methodology.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Iterative Linearization for Phasor-Defined Optimal Power Dispatch

Optimal power flow (OPF) problems, which dispatch power targets to controllable generating units across a network, must generally account for non-convex constraints on power flow. Furthermore, adapting those problems so as to make them solvable with convex optimization techniques is an area of much academic and operational interest. In this paper, we present a method for solving OPF as a quadratic program by iteratively refining and re-initializing a linearized model of power flow based on the outputs of an associated nonlinear solver. The linear model on which we demonstrate this method is an adapted version of an approximation designed for use with unbalanced distribution networks. As an important benefit, the model allows for the explicit inclusion of nodal voltage phasor values in both the OPF problem's objective and its constraints, which opens the door to the idea of phasor-based control (PBC) design. We show in simulations on the IEEE 13-node test feeder that our method quickly converges to a set of phasor targets that are sufficiently precise for use in operations at the distribution level.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

FedOSAA: Improving Federated Learning with One-Step Anderson Acceleration

Federated learning (FL) is a distributed machine learning approach that enables multiple local clients and a central server to collaboratively train a model while keeping the data on their own devices. First-order methods, particularly those incorporating variance reduction techniques, are the most widely used FL algorithms due to their simple implementation and stable performance. However, these methods tend to be slow and require a large number of communication rounds to reach the global minimizer. We propose FedOSAA, a novel approach that preserves the simplicity of first-order methods while achieving the rapid convergence typically associated with second-order methods. Our approach applies one Anderson acceleration (AA) step following classical local updates based on first-order methods with variance reduction, such as FedSVRG and SCAFFOLD, during local training. This AA step is able to leverage curvature information from the history points and gives a new update that approximates the Newton-GMRES direction, thereby significantly improving the convergence. We establish a local linear convergence rate to the global minimizer of FedOSAA for smooth and strongly convex loss functions. Numerical comparisons show that FedOSAA substantially improves the communication and computation efficiency of the original first-order methods, achieving performance comparable to second-order methods like GIANT.

Feng, Xue [University of California, Davis]↗
DOE OSTI2022

Iterative subspace algorithms for finite-temperature solution of Dyson equation

One-particle Green’s functions obtained from the self-consistent solution of the Dyson equation can be employed in the evaluation of spectroscopic and thermodynamic properties for both molecules and solids. However, typical acceleration techniques used in the traditional quantum chemistry self-consistent algorithms cannot be easily deployed for the Green’s function methods because of a non-convex grand potential functional and a non-idempotent density matrix. Moreover, the optimization problem can become more challenging due to the inclusion of correlation effects, changing chemical potential, and fluctuations of the number of particles. In this paper, we study acceleration techniques to target the self-consistent solution of the Dyson equation directly. We use the direct inversion in the iterative subspace (DIIS), the least-squared commutator in the iterative subspace (LCIIS), and the Krylov space accelerated inexact Newton method (KAIN). We observe that the definition of the residual has a significant impact on the convergence of the iterative procedure. Based on the Dyson equation, we generalize the concept of the commutator residual used in DIIS and LCIIS and compare it with the difference residual used in DIIS and KAIN. The commutator residuals outperform the difference residuals for all considered molecular and solid systems within both GW and GF2. For a number of bond-breaking problems, we found that an easily obtained high-temperature solution with effectively suppressed correlations is a very effective starting point for reaching convergence of the problematic low-temperature solutions through a sequential reduction of temperature during calculations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Essential barrier height and a probabilistic approach in characterizing potential landscape

In this work we propose a probabilistic approach to investigate the shape of landscapes of multi-dimensional potential functions. Under a suitable coupling scheme, two copies of the overdamped Langevin dynamics associated with the potential function are coupled, and the coupling times are collected. Assuming a set of intuitive yet technically challenging conditions on the coupling scheme, it is shown that the tail distributions of the coupling times exhibit qualitatively different dependencies on the noise magnitude for single-well versus multi-well potential functions. More specifically, for convex single-well potentials, the negative tail exponent of the coupling time distribution is uniformly bounded away from zero by the convexity parameter and is independent of the noise magnitude. In contrast, for multi-well potentials, the negative tail exponent decreases exponentially as the noise vanishes, with the decay rate governed by the essential barrier height, a quantity introduced in this paper to characterize the non-convex nature of the potential function. Numerical investigations are conducted for a variety of examples, including the Rosenbrock function, interacting particle systems, and loss functions arising in artificial neural networks. These examples not only illustrate the theoretical results in various contexts but also provide crucial numerical validation of the conjectured assumptions, which are essential to the theoretical analysis yet lie beyond the reach of standard technical tools.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Rethinking the Price Formation Problem–Part 1: Participant Incentives under Uncertainty

Operators of organized wholesale electricity markets attempt to form prices in such a way that the private incentives of market participants are consistent with a socially optimal commitment and dispatch schedule. In the U.S. context, several competing price formation schemes have been proposed to address the non-convex production cost functions characteristic of most generation technologies. Here, this paper considers how the design and analysis of price formation policies for non-convex markets are affected by the uncertainty inherent in electricity demand and supply. We argue that by excluding uncertainty, the analytical framework underlying existing policies mischaracterizes the incentives of market participants, leading to inefficient price formation and poor incentives for flexibility. We establish favorable theoretical properties of a new construct, ex ante convex hull pricing , and demonstrate the difference between this idealized benchmark and existing methods on a large-scale test system. Given increased operational uncertainty with a transition to wind and solar generation, distortions caused by poor incentives for flexibility are likely to grow without improved price formation in organized wholesale markets.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Extending a 10‐node composite tetrahedral finite element for solid mechanics

Abstract We propose to extend the composite tetrahedral finite element first introduced by Thoutireddy et al. (2002) and recently reformulated by Ostien et al. (2016). We generalize the gradient operator and mass matrix to curved domains through analytical expressions weighted by subtetrahedra Jacobians. Optimal integration weights are constructed to increase the accuracy of Gaussian quadrature. We preserve the variational structure of the formulation through a new five‐field functional with additional, independent fields for the Jacobian and the pressure. This approach not only obviates volumetric locking but also yields symmetry. A deleterious soft mode, common to both the quadratic and composite tetrahedral element with constant pressure formulations is effectively stabilized through a novel convex energy penalty function. Numerous numerical examples spanning a patch test to the impact of a Taylor bar demonstrate the accuracy, robustness, and convergence of the extended composite tetrahedral element for application to structural metals.

Foulk III, James W.↗
DOE OSTI2022

On the energy landscape of symmetric quantum signal processing

Symmetric quantum signal processing provides a parameterized representation of a real polynomial, which can be translated into an efficient quantum circuit for performing a wide range of computational tasks on quantum computers. For a given polynomial f , the parameters (called phase factors) can be obtained by solving an optimization problem. However, the cost function is non-convex, and has a very complex energy landscape with numerous global and local minima. It is therefore surprising that the solution can be robustly obtained in practice, starting from a fixed initial guess Φ 0 that contains no information of the input polynomial. To investigate this phenomenon, we first explicitly characterize all the global minima of the cost function. We then prove that one particular global minimum (called the maximal solution) belongs to a neighborhood of Φ 0 , on which the cost function is strongly convex under the condition ‖ f ‖ ∞ = O ( d − 1 ) with d = d e g ( f ) . Our result provides a partial explanation of the aforementioned success of optimization algorithms.

Wang, Jiasu↗
DOE OSTI2023

On the convexity of phase-field fracture formulations: Analytical study and comparison of various degradation functions

Efficient and accurate fracture modeling is of great importance in applications where catastrophic outcomes under extreme scenarios are possible. The phase-field (PF) approach to fracture received significant attention over the past decade, due to its capability to capture complicated fracture patterns (e.g., crack merging and branching). Specifically, crack initiation and propagation are modeled via minimization of the total energy functional, which is regularized with the aid of a phase field. Despite the promising results and modeling capabilities of the PF method in many applications, the solution of fracture problems remains computationally challenging mainly due to the non-convexity of the total energy functional with respect to the combined unknown (phase field and displacement) fields. Understanding the effects of their coupling on convexity is crucial in order to address frequently encountered hurdles in fracture modeling (e.g., inefficient solvers and non-physical crack nucleation). In this paper, we develop convexity criteria for a wide class of PF fracture formulations. For this class of formulations, the second variation of the total energy functional is expressed in terms of Hessian matrices (evaluated at individual material points). Depending on the choice of geometric crack functions and degradation functions, we classify the formulations into three categories and analytically study each one separately. To study the sign of the second variation, we derive inequalities which are satisfied at material points when the Hessian matrix is locally positive semi-definite. These inequalities provide objective criteria for comparing degradation functions. Finally, the applicability of the proposed convexity criteria is demonstrated in the context of a one-dimensional problem, solved using a conventional monolithic solver.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Optimization with Neural Network Feasibility Surrogates: Formulations and Application to Security-Constrained Optimal Power Flow

In many areas of constrained optimization, representing all possible constraints that give rise to an accurate feasible region can be difficult and computationally prohibitive for online use. Satisfying feasibility constraints becomes more challenging in high-dimensional, non-convex regimes which are common in engineering applications. A prominent example that is explored in the manuscript is the security-constrained optimal power flow (SCOPF) problem, which minimizes power generation costs, while enforcing system feasibility under contingency failures in the transmission network. In its full form, this problem has been modeled as a nonlinear two-stage stochastic programming problem. In this work, we propose a hybrid structure that incorporates and takes advantage of both a high-fidelity physical model and fast machine learning surrogates. Neural network (NN) models have been shown to classify highly non-linear functions and can be trained offline but require large training sets. In this work, we present how model-guided sampling can efficiently create datasets that are highly informative to a NN classifier for non-convex functions. We show how the resultant NN surrogates can be integrated into a non-linear program as smooth, continuous functions to simultaneously optimize the objective function and enforce feasibility using existing non-linear solvers. Overall, this allows us to optimize instances of the SCOPF problem with an order of magnitude CPU improvement over existing methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Tunable noninteracting free-energy density functionals for high-energy-density physics applications

In this work, we introduce the concept of a tunable noninteracting free-energy density functional and present two examples realized: (i) via a simple one-parameter convex combination of two existing functionals and (ii) via the construction of a generalized gradient approximation (GGA) enhancement factor that contains one free parameter and is designed to satisfy a set of incorporated constraints. Functional (i), constructed as a combination of the local Thomas–Fermi and a pseudopotential-adapted GGA for the noninteracting free-energy, has already demonstrated its practical usability for establishing the high temperature end of the equation of state of deuterium [Phys. Rev. B 104, 144104 (2021)] and CHON resin [Phys. Rev. E 106, 045207 (2022)] for inertial confinement fusion applications. Hugoniot calculations for liquid deuterium are given as another example of how the application of computationally efficient orbital-free density functional theory (OF-DFT) can be utilized with the employment of the developed functionals. Once the functionals have been tuned such that the OF-DFT Hugoniot calculation matches the Kohn–Sham solution at some low-temperature point, agreement with the reference Kohn–Sham results for the rest of the high temperature Hugoniot path is very good with relative errors for compression and pressure on the order of 2% or less.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

A Convex Data-Driven Approach for Nonlinear Control Synthesis

We consider a class of nonlinear control synthesis problems where the underlying mathematical models are not explicitly known. We propose a data-driven approach to stabilize the systems when only sample trajectories of the dynamics are accessible. Our method is built on the density-function-based stability certificate that is the dual to the Lyapunov function for dynamic systems. Unlike Lyapunov-based methods, density functions lead to a convex formulation for a joint search of the control strategy and the stability certificate. This type of convex problem can be solved efficiently using the machinery of the sum of squares (SOS). For the data-driven part, we exploit the fact that the duality results in the stability theory can be understood through the lens of Perron–Frobenius and Koopman operators. This allows us to use data-driven methods to approximate these operators and combine them with the SOS techniques to establish a convex formulation of control synthesis. The efficacy of the proposed approach is demonstrated through several examples.

97 MATHEMATICS AND COMPUTING↗