Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stochastic optimizations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Hybrid Differential Dynamic Programming with Stochastic Search

Differential dynamic programming (DDP) has been demonstrated as a viable approach to low-thrust trajectory optimization, namely with the recent success of NASA's Dawn mission. The Dawn trajectory was designed with the DDP-based Static/Dynamic Optimal Control algorithm used in the Mystic software.1 Another recently developed method, Hybrid Differential Dynamic Programming (HDDP),2, 3 is a variant of the standard DDP formulation that leverages both first-order and second-order state transition matrices in addition to nonlinear programming (NLP) techniques. Areas of improvement over standard DDP include constraint handling, convergence properties, continuous dynamics, and multi-phase capability. DDP is a gradient based method and will converge to a solution nearby an initial guess. In this study, monotonic basin hopping (MBH) is employed as a stochastic search method to overcome this limitation, by augmenting the HDDP algorithm for a wider search of the solution space.

Aziz, Jonathan↗
DOE OSTI2023

Quantifying Uncertainty of Deep Reinforcement Learning Based Decision Making for Operations and Maintenance of Nuclear Power Plant

This paper summarizes research that integrates condition monitoring and prognostics with decision making for nuclear power plant operations and maintenance. As part of this research, we have developed an online asset management tool to help reduce life-cycle maintenance and repair costs. Using the latest advancements in condition monitoring, supply chain analytics, and deep reinforcement learning, we have created a predictive maintenance tool that can optimize the maintenance and spare-part management of a repairable nuclear system. To demonstrate these methods, preliminary studies were conducted on a simple, representative maintenance system undergoing a stochastic degradation process that requires repairs or replacement to continue operation. Through Monte Carlo simulations, we were able to reduce maintenance spending by approximately 50% compared to optimized, time-based maintenance strategies. Not only does the decision maker reduce the average life-cycle costs, it also minimizes the chance of high cost scenarios, lowering the variance of the expected cost distributions, and reducing overall financial risk. Furthermore, this work also studies the ability of the decision maker to handle various levels of noise from observation uncertainty. By introducing uncertainty into the decision-making process, we have quantified the robustness and resiliency of the decision maker, as well as identified necessary levels of observability to demonstrate cost effectiveness.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1974

Estimation for bilinear stochastic systems

Three techniques for the solution of bilinear estimation problems are presented. First, finite dimensional optimal nonlinear estimators are presented for certain bilinear systems evolving on solvable and nilpotent lie groups. Then the use of harmonic analysis for estimation problems evolving on spheres and other compact manifolds is investigated. Finally, an approximate estimation technique utilizing cumulants is discussed.

Willsky, A. S.↗
DOE OSTI2023

Stochastic projective splitting

Here, we present a new, stochastic variant of the projective splitting (PS) family of algorithms for inclusion problems involving the sum of any finite number of maximal monotone operators. This new variant uses a stochastic oracle to evaluate one of the operators, which is assumed to be Lipschitz continuous, and (deterministic) resolvents to process the remaining operators. Our proposal is the first version of PS with such stochastic capabilities. We envision the primary application being machine learning (ML) problems, with the method’s stochastic features facilitating “mini-batch” sampling of datasets. Since it uses a monotone operator formulation, the method can handle not only Lipschitz-smooth loss minimization, but also min–max and noncooperative game formulations, with better convergence properties than the gradient descent-ascent methods commonly applied in such settings. The proposed method can handle any number of constraints and nonsmooth regularizers via projection and proximal operators. We prove almost-sure convergence of the iterates to a solution and a convergence rate result for the expected residual, and close with numerical experiments on a distributionally robust sparse logistic regression problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimization of nonlinear turbulence in stellarators

We present new stellarator equilibria that have been optimized for reduced turbulent transport using nonlinear gyrokinetic simulations within the optimization loop. The optimization routine involves coupling the pseudo-spectral GPU-native gyrokinetic code GX with the stellarator equilibrium and optimization code DESC . Since using GX allows for fast nonlinear simulations, we directly optimize for reduced nonlinear heat fluxes. To handle the noisy heat flux traces returned by these simulations, we employ the simultaneous perturbation stochastic approximation (SPSA) method that only uses two objective function evaluations for a simple estimate of the gradient. We show several examples that optimize for both reduced heat fluxes and good quasi-symmetry as a proxy for low neoclassical transport. Finally, we run full transport simulations using the T3D stellarator transport code to evaluate the changes in the macroscopic profiles.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1986

Dynamic remapping of parallel computations with varying resource demands

A large class of computational problems is characterized by frequent synchronization, and computational requirements which change as a function of time. When such a problem must be solved on a message passing multiprocessor machine, the combination of these characteristics lead to system performance which decreases in time. Performance can be improved with periodic redistribution of computational load; however, redistribution can exact a sometimes large delay cost. We study the issue of deciding when to invoke a global load remapping mechanism. Such a decision policy must effectively weigh the costs of remapping against the performance benefits. We treat this problem by constructing two analytic models which exhibit stochastically decreasing performance. One model is quite tractable; we are able to describe the optimal remapping algorithm, and the optimal decision policy governing when to invoke that algorithm. However, computational complexity prohibits the use of the optimal remapping decision policy. We then study the performance of a general remapping policy on both analytic models. This policy attempts to minimize a statistic W(n) which measures the system degradation (including the cost of remapping) per computation step over a period of n steps. We show that as a function of time, the expected value of W(n) has at most one minimum, and that when this minimum exists it defines the optimal fixed-interval remapping policy. Our decision policy appeals to this result by remapping when it estimates that W(n) is minimized. Our performance data suggests that this policy effectively finds the natural frequency of remapping. We also use the analytic models to express the relationship between performance and remapping cost, number of processors, and the computation's stochastic activity.

Nicol, D. M.↗
DOE OSTI2020

Multistage distributionally robust mixed-integer programming with decision-dependent moment-based ambiguity sets

We study multistage distributionally robust mixed-integer programs under endogenous uncertainty, where the probability distribution of stage-wise uncertainty depends on the decisions made in previous stages. We first consider two ambiguity sets defined by decision-dependent bounds on the first and second moments of uncertain parameters and by mean and covariance matrix that exactly match decision-dependent empirical ones, respectively. For both sets, we show that the subproblem in each stage can be recast as a mixed-integer linear program (MILP). Moreover, we extend the general moment-based ambiguity set in to the multistage decision-dependent setting, and derive mixed-integer semidefinite programming (MISDP) reformulations of stage-wise subproblems. We develop methods for attaining lower and upper bounds of the optimal objective value of the multistage MISDPs, and approximate them using a series of MILPs. We deploy the Stochastic Dual Dynamic integer Programming (SDDiP) method for solving the problem under the three ambiguity sets with risk-neutral or risk-averse objective functions, and conduct numerical studies on multistage facility-location instances having diverse sizes under different parameter and uncertainty settings. Furthermore, our results show that the SDDiP quickly finds optimal solutions for moderate-sized instances under the first two ambiguity sets, and also finds good approximate bounds for the multistage MISDPs derived under the third ambiguity set. We also demonstrate the efficacy of incorporating decision-dependent distributional ambiguity in multistage decision-making processes.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1982

Methods of sequential estimation for determining initial data in numerical weather prediction

Numerical weather prediction (NWP) is an initial-value problem for a system of nonlinear differential equations, in which initial values are known incompletely and inaccurately. Observational data available at the initial time must therefore be supplemented by data available prior to the initial time, a problem known as meteorological data assimilation. A further complication in NWP is that solutions of the governing equations evolve on two different time scales, a fast one and a slow one, whereas fast scale motions in the atmosphere are not reliably observed. This leads to the so called initialization problem: initial values must be constrained to result in a slowly evolving forecast. The theory of estimation of stochastic dynamic systems provides a natural approach to such problems. For linear stochastic dynamic models, the Kalman-Bucy (KB) sequential filter is the optimal data assimilation method, for linear models, the optimal combined data assimilation-initialization method is a modified version of the KB filter.

Cohn, S. E.↗
NASA NTRS2004

Inference of Stochastic Nonlinear Oscillators with Applications to Physiological Problems

A new method of inferencing of coupled stochastic nonlinear oscillators is described. The technique does not require extensive global optimization, provides optimal compensation for noise-induced errors and is robust in a broad range of dynamical models. We illustrate the main ideas of the technique by inferencing a model of five globally and locally coupled noisy oscillators. Specific modifications of the technique for inferencing hidden degrees of freedom of coupled nonlinear oscillators is discussed in the context of physiological applications.

Smelyanskiy, Vadim N.↗
NASA NTRS2009

3D Lunar Terrain Reconstruction from Apollo Images

Generating accurate three dimensional planetary models is becoming increasingly important as NASA plans manned missions to return to the Moon in the next decade. This paper describes a 3D surface reconstruction system called the Ames Stereo Pipeline that is designed to produce such models automatically by processing orbital stereo imagery. We discuss two important core aspects of this system: (1) refinement of satellite station positions and pose estimates through least squares bundle adjustment; and (2) a stochastic plane fitting algorithm that generalizes the Lucas-Kanade method for optimal matching between stereo pair images.. These techniques allow us to automatically produce seamless, highly accurate digital elevation models from multiple stereo image pairs while significantly reducing the influence of image noise. Our technique is demonstrated on a set of 71 high resolution scanned images from the Apollo 15 mission

Broxton, Michael J.↗
NASA NTRS2019

A Learning-Based Guidance Selection Mechanism for a Formally Verified Sense and Avoid Algorithm

This paper describes a learning-based strategy for selecting conflict avoidance maneuvers for autonomous unmanned aircraft systems. The selected maneuvers are provided by a formally verified algorithm and they are guaranteed to solve any impending conflict under general assumptions about aircraft dynamics. The decision-making logic that selects the appropriate maneuvers is encoded in a stochastic policy encapsulated as a neural network. The network’s parameters are optimized to maximize a reward function. The reward function penalizes loss of separation with other aircraft while rewarding resolutions that result in minimum excursions from the nominal flight plan. This paper provides a description of the technique and presents preliminary simulation results.

Balachandran, Swee↗
DOE OSTI2021

Data-driven optimization of mixed-integer bi-level multi-follower integrated planning and scheduling problems under demand uncertainty

The coordination of interconnected elements across the different layers of the supply chain is essential for all industrial processes and the key to optimal decision-making. Yet, the modeling and optimization of such interdependent systems are still burdensome. Here we address the simultaneous modeling and optimization of medium-term planning and short-term scheduling problems under demand uncertainty using mixed-integer bi-level multi-follower programming and data-driven optimization. Bi-level multi-follower programs model the natural hierarchy between different layers of supply chain management holistically, while scenario analysis and data-driven optimization allow us to retrieve the guaranteed feasible solutions of the integrated formulation under various demand considerations. We address the data-driven optimization of this challenging class of problems using the DOMINO framework, which was initially developed to solve single-leader single-follower bi-level optimization problems to guaranteed feasibility. This framework is extended to solve single-leader multi-follower stochastic formulations and its performance is characterized by well-known single and multi-product process scheduling case studies. Through our data-driven algorithmic approach, we present guaranteed feasible solutions to linear and nonlinear mixed-integer bi-level formulations of simultaneous planning and scheduling problems and further characterize the effects of the scheduling level complexity on the solution performance, which spans over several hundred continuous and binary variables, and thousands of constraints.

42 ENGINEERING↗
NASA NTRS1986

An optimal repartitioning decision policy

A central problem to parallel processing is the determination of an effective partitioning of workload to processors. The effectiveness of any given partition is dependent on the stochastic nature of the workload. The problem of determining when and if the stochastic behavior of the workload has changed enough to warrant the calculation of a new partition is treated. The problem is modeled as a Markov decision process, and an optimal decision policy is derived. Quantification of this policy is usually intractable. A heuristic policy which performs nearly optimally is investigated empirically. The results suggest that the detection of change is the predominant issue in this problem.

Nicol, D. M.↗
DOE OSTI2025

Stochastic Waveform Estimation at the Fundamental Quantum Limit

Although measuring the deterministic waveform of a weak classical force is a well-studied problem, estimating a random waveform, such as the spectral density of a stochastic signal field, is much less well understood despite it being a widespread task at the frontier of experimental physics. State-of-the-art precision sensors of random forces must account for the underlying quantum nature of the measurement but the optimal quantum protocol for interrogating such linear sensors is not known. We derive the fundamental precision limit: the extended-channel quantum Cramér-Rao bound. In the experimentally relevant regime in which losses dominate, we prove that non-Gaussian-state preparation and measurement are required to achieve this fundamental limit and we determine numerically the optimal non-Gaussian protocol. We discuss how this scheme could accelerate searches for signatures of quantum gravity, stochastic gravitational waves, and axionic dark matter.

Axions↗
NASA NTRS1972

Adaptive nonlinear optimization of the signal-to-noise ratio of an array subject to a constraint.

Investigation of a stochastic projected gradient algorithm which can be used to find a constrained optimum point for a concave or convex objective function subject to nonlinear constraints. When the constraint consists of only one linear equation, convergence to the constrained optimum value is proven, and bounds are obtained for the rate of convergence of the algorithm to the constrained optimum value. This algorithm is then applied to the nonlinear problem of automatically making an array of detectors form a beam in a desired direction in space when unknown interfering noise is present so as to maximize the signal-to-noise ratio subject to a constraint on the supergain ratio.

Winkler, L. P.↗
DOE OSTI2019

Quasi-Stochastic Approximation and Off-Policy Reinforcement Learning: Preprint

The Robbins-Monro stochastic approximation algorithm is a foundation of many algorithmic frameworks for reinforcement learning and often an efficient approach to solving (or approximating the solution to) complex optimal control problems. In many cases, however, practitioners are unable to apply these techniques because of an inherent high variance. This paper aims to provide a general foundation for 'quasi-stochastic approximation,' in which all of the processes under consideration are deterministic, much like quasi-Monte Carlo for variance reduction in simulation. The variance reduction can be substantial, subject to tuning of pertinent parameters in the algorithm. This paper introduces a new coupling argument to establish the optimal rate of convergence provided the gain is sufficiently large. These results are established for linear models and tested also in nonideal settings. A major application of these general results is a new class of reinforcement learning algorithms for deterministic state space models. In this setting, the main contribution is a class of algorithms for approximating the value function for a given policy, using a different policy designed to introduce exploration.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2015

Global Optimization of Interplanetary Trajectories in the Presence of Realistic Mission Contraints

Interplanetary missions are often subject to difficult constraints, like solar phase angle upon arrival at the destination, velocity at arrival, and altitudes for flybys. Preliminary design of such missions is often conducted by solving the unconstrained problem and then filtering away solutions which do not naturally satisfy the constraints. However this can bias the search into non-advantageous regions of the solution space, so it can be better to conduct preliminary design with the full set of constraints imposed. In this work two stochastic global search methods are developed which are well suited to the constrained global interplanetary trajectory optimization problem.

Design↗
DOE OSTI2026

Score-based deterministic density sampling

We propose a deterministic sampling framework using Score-Based Transport Modeling for sampling an unnormalized target density π given only its score ∇ log π. Our method approximates the Wasserstein gradient flow on KL($f_t$∥π) by learning the time-varying score ∇ log $f_t$ on the fly using score matching. While having the same marginal distribution as Langevin dynamics, our method produces smooth deterministic trajectories, resulting in monotone noise-free convergence. We prove that our method dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. Numerical experiments validate our theoretical findings: our method converges at the optimal rate, has smooth trajectories, and is often more sample efficient than its stochastic counterpart. Experiments on high-dimensional image data show that our method produces high-quality generations in as few as 15 steps and exhibits natural exploratory behavior. The memory and runtime scale linearly in the sample size.

97 MATHEMATICS AND COMPUTING↗