Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Stochastic optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Stochastic Distribution Control Theory-Its Potential Application in Risk Management in Financial Systems

Stochastic Distribution Control (SDC) theory [1], originated by the author in 1996, aims at developing modeling and control strategies for dynamic and non-Gaussian stochastic systems by controlling the shape of the probability density functions of some concerned variables and parameters in stochastic systems. It generalizes the capability of standard stochastic differential equations and can therefore be applied to generic non-Gaussian systems. Since it was established in 1996, it has found a wide spectrum of applications in non-Gaussian stochastic system control, data mining, filtering and optimization for uncertain systems. In this short opinion article, discussions will be made on potential applications of SDC theory to financial systems in terms of risk analysis and management.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2015

A Stochastic Scheduler for Integrated Arrival, Departure and Surface Operations in Los Angeles

In terminal airspace, integrating arrivals, departures, and surface operations with competing resources provides the potential of improving operational efficiency by removing barriers between different operations. This work develops a centralized stochastic scheduler for operations in a terminal area including airborne and surface operations using Non-dominated sorting genetic algorithm and Monte Carlo simulations. The scheduler handles completing resources between different flows, such as runway allocations, runway crossing, departure fixes, and other interaction way points between arrivals and departures. Meanwhile, the scheduler also takes time-varied uncertainties into account when optimizing schedules. The scheduler is run sequentially to identify the best and robust schedule for the next planning window. Resulting schedules decide the routes, speed or delays, and runway assignments with separation constraints at mergingdiverging waypoints in the air and crossing and separations on runways. The Los Angels terminal area was used as an example. The implementation of this stochastic scheduler for integrated arrival, departure and surface operations is completed. And several preliminary runs are finished for over 1,200 flights in LAX in a typical day. Sensitivity studies on various planning window sizes are presented, which shows that trade-off exits between planning window size and achievable minimum delay. Preliminary results on runway usage are also presented in this abstract. Because arrivals on the outer runways have to be followed by crossings on the inner runways, algorithmic runway allocation prefers inner runways for arrivals and outer runways for departures. More results will be presented in the final paper. And current terminal arrival and departure procedures based on first-come-first-serve procedure will also be set up and used as a baseline for comparison.

integrated scheduler↗
DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗
NASA NTRSDate not supplied

Path Planning: Differential Dynamic Programming and Model Predictive Path Integral Control on VTOL Aircraft

This paper explores two optimal control approaches, widely used in robotics, to establish their viability as real-time trajectory planners for vehicle configurations envisioned for the emerging aviation sector of Urban Air Mobility (UAM). Differential Dynamic Programming (DDP) enables planning over highly nonlinear dynamics using second-order approximations along a nominal trajectory, and displays quadratic convergence to a local solution. Model Predictive Path Integral (MPPI) is a stochastic sampling-based algorithm that can optimize for general cost criteria, including potentially highly nonlinear formulations, and supports parallel computation through the use of modern GPU hardware. In this work, DDP and MPPI were implemented using model predictive control (MPC), and the results indicate they are able to successfully transition the aircraft over different flight envelopes and generate trajectories unique to UAM vehicles.

Differential Dynamic Programming↗
DOE OSTI2016

A feasibility study of dynamic adaptive radiotherapy for nonsmall cell lung cancer

Purpose: The final state of the tumor at the end of a radiotherapy course is dependent on the doses given in each fraction during the treatment course. This study investigates the feasibility of using dynamic adaptive radiotherapy (DART) in treating lung cancers assuming CBCT is available to observe midtreatment tumor states. DART adapts treatment plans using a dynamic programming technique to consider the expected changes of the tumor in the optimization process. Methods: DART is constructed using a stochastic control formalism framework. It minimizes the total expected number of tumor cells at the end of a treatment course, which is equivalent to maximizing tumor control probability, subject to the uncertainty inherent in the tumor response. This formulation allows for nonstationary dose distributions as well as nonstationary fractional doses as needed to achieve a series of optimal plans that are conformal to the tumor over time, i.e., spatiotemporally optimal plans. Sixteen phantom cases with various sizes and locations of tumors and organs-at-risk (OAR) were generated using in-house software. Each case was planned with DART and conventional IMRT prescribing 60 Gy in 30 fractions. The observations of the change in the tumor volume over a treatment course were simulated using a two-level cell population model. Monte Carlo simulations of the treatment course for each case were run to account for uncertainty in the tumor response. The same OAR dose constraints were applied for both methods. The frequency of replanning was varied between 1, 2, 5 (weekly), and 29 times (daily). The final average tumor dose and OAR doses have been compared to quantify the potential dosimetric benefits of DART. Results: The average tumor max, min, mean, and D95 doses using DART relative to these using conventional IMRT were 124.0%–125.2%, 102.1%–114.7%, 113.7%–123.4%, and 102.0%–115.9% (range dependent on the frequency of replanning). The average relative maximum doses for the cord and esophagus, mean doses for the heart and lungs, and D05 for the unspecified tissue resulting 84%–102.4%, 99.8%–106.9%, 66.9%–85.6%, 58.2%–78.8%, and 85.2%–94.0%, respectively. Conclusions: It is feasible to apply DART to the treatment of NSCLC using CBCT to observe the midtreatment tumor state. Potential increases in the tumor dose and reductions in the OAR dose, particularly for parallel OARs with mean or dose–volume constraints, could be achieved using DART compared to nonadaptive IMRT.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Modeling Flexible Generator Operating Regions via Chance- constrained Stochastic Unit Commitment

Here, we introduce a novel chance-constrained stochastic unit commitment model to address uncertainty in renewables' production uncertainty in power systems operation. For most thermal generators,underlying technical constraints that are universally treated as "hard" by deterministic unit commitment models are in fact based on engineering judgments, such that system operators can periodically request operation outside these limits in non-nominal situations, e.g., to ensure reliability. We incorporate this practical consideration into a chance-constrained stochastic unit commitment model, specifically by in-frequently allowing minor deviations from the minimum and maximum thermal generator power output levels. We demonstrate that an extensive form of our model is computationally tractable for medium-sized power systems given modest numbers of scenarios for renewables' production. We show that the model is able to potentially save significant annual production costs by allowing infrequent and controlled violation of the traditionally hard bounds imposed on thermal generator production limits. Finally, we conduct a sensitivity analysis of optimal solutions to our model under two restricted regimes and observe similar qualitative results.

42 ENGINEERING↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

CO 2 water-alternating-gas injection for enhanced oil recovery: Optimal well controls and half-cycle lengths

We report that CO 2 water-alternating-gas (WAG) injection is an enhanced oil recovery method designed to improve sweep efficiency during CO 2 injection with the injected water to control the mobility of CO 2 and to stabilize the gas front. Optimization of CO 2 -WAG injection is widely regarded as a viable technique for controlling the CO 2 and oil miscible process. Poor recovery from CO 2 -WAG injection can be caused by inappropriately designed WAG parameters. In previous study (Chen and Reynolds, 2016), we proposed an algorithm to optimize the well controls which maximize the life-cycle net-present-value (NPV). However, the effect of injection half-cycle lengths for each injector on oil recovery or NPV has not been well investigated. In this paper, an optimization framework based on augmented Lagrangian method and the newly developed stochastic-simplex-approximate-gradient (StoSAG) algorithm is proposed to explore the possibility of simultaneous optimization of the WAG half-cycle lengths together with the well controls. Finally, the proposed framework is demonstrated with three reservoir examples.

42 ENGINEERING↗
DOE OSTI2017

Additive manufacturing: Toward holistic design

Here, additive manufacturing offers unprecedented opportunities to design complex structures optimized for performance envelopes inaccessible under conventional manufacturing constraints. Additive processes also promote realization of engineered materials with microstructures and properties that are impossible via traditional synthesis techniques. Enthused by these capabilities, optimization design tools have experienced a recent revival. The current capabilities of additive processes and optimization tools are summarized briefly, while an emerging opportunity is discussed to achieve a holistic design paradigm whereby computational tools are integrated with stochastic process and material awareness to enable the concurrent optimization of design topologies, material constructs and fabrication processes.

36 MATERIALS SCIENCE↗
DOE OSTI2014

Optimization of magnetic switches for single particle and cell transport

The ability to manipulate an ensemble of single particles and cells is a key aim of lab-on-a-chip research; however, the control mechanisms must be optimized for minimal power consumption to enable future large-scale implementation. Recently, we demonstrated a matter transport platform, which uses overlaid patterns of magnetic films and metallic current lines to control magnetic particles and magnetic-nanoparticle-labeled cells; however, we have made no prior attempts to optimize the device geometry and power consumption. Here, we provide an optimization analysis of particle-switching devices based on stochastic variation in the particle's size and magnetic content. These results are immediately applicable to the design of robust, multiplexed platforms capable of transporting, sorting, and storing single cells in large arrays with low power and high efficiency.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2017

Incorporating Wind Energy in Power System Restoration Planning

Wind energy is rapidly growing. While wind brings us clean and inexpensive energy, its inherent variability and uncertainty present challenges for the power grid. In particular, employing wind energy for power system restoration is very challenging. A fast and reliable restoration plays a vital role to achieve the self-healing power grid. This paper develops a novel offline restoration planning tool for harnessing wind energy to enhance grid resilience. The Wind-for-Restoration problem is formulated as a stochastic mixed-integer linear programming problem with generated wind energy scenarios. The problem is then decomposed into two stages and solved with the integer Lshaped algorithm. Numerical experiments have been conducted through different case studies using the modified IEEE 57-bus system. The developed tool can provide the scheduled wind power at each restoration time. The impact of wind energy is investigated from the aspects of location and inertia capability, as well as wind penetration, fluctuation, and uncertainty. Moreover, a dynamic response validation tool is developed to validate the results of optimization problem in a dynamic simulation software. Finally, simulation results demonstrate that the optimal wind harnessing strategy can help improve system restoration process and enhance system resilience.

17 WIND ENERGY↗
DOE OSTI2016

Hierarchical optimization for neutron scattering problems

We present a scalable optimization method for neutron scattering problems that determines confidence regions of simulation parameters in lattice dynamics models used to fit neutron scattering data for crystalline solids. The method uses physics-based hierarchical dimension reduction in both the computational simulation domain and the parameter space. We demonstrate for silicon that after a few iterations the method converges to parameters values (interatomic force-constants) computed with density functional theory simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Benders Decomposition Using Graph Modeling and Multi-Parametric Programming

Benders decomposition is a widely used method for solving large and structured optimization problems, but its performance is affected by the repeated solution of subproblems. We propose a flexible and modular algorithmic framework for accelerating Benders decomposition. Specifically, we express the problem structure by using a graph-theoretic modeling abstraction in which nodes represent optimization subproblems and edges represent connectivity between subproblems. A key innovation of our approach is that we embed multiparametric programming (mp) surrogates for node subproblems, which maps the exact analytical map of the subproblem solution space. The use of mp surrogates allows us to replace subproblem solves with fast look-ups and function evaluations for primal and dual variables during the iterative Benders process. We formally show the equivalence between classical Benders cuts and those derived from the mp solution. We implement our framework in the open-source PlasmoBenders.jl software package. To demonstrate the capabilities of the proposed framework, we apply it to a two-stage stochastic programming problem, which aims to make optimal capacity expansion decisions under market uncertainty. We evaluate both single-cut and multicut variants of Benders decomposition and show that the use of mp surrogates achieves substantial speedups in subproblem solve time, while preserving the convergence guarantees of Benders decomposition. We highlight advantages in solution analysis and interpretability that is enabled by mp critical region tracking; specifically, we show that these reveal how decisions evolve geometrically across the Benders search. Our results aim to demonstrate that combining surrogate modeling with graph modeling offers a promising and extensible foundation for structure-exploiting decomposition. In addition, by decomposing the problem into more tractable subproblems, the proposed approach also aims to overcome scalability issues of mp. Finally, the use of mp surrogates provides a unifying and modular optimization framework that enables the representation of heterogeneous node subproblems as modeling objects with a homogeneous structure.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1983

Minimum information modelling of structural systems with uncertain parameters

Work is reviewed wherein the design of active structural control is formulated as the mean-square optimal control of a linear mechanical system with stochastic parameters. In practice, a complete probabilistic description of model parameters can never be provided by empirical determinations, and a suitable design approach must accept very limited a priori data on parameter statistics. In consequence, the mean-square optimization problem is formulated using a complete probability assignment which is made to be consistent with available data but maximally unconstrained otherwise through use of a maximum entropy principle. The ramifications of this approach for both robustness and large dimensionality are illustrated by consideration of the full-state feedback regulation problem.

Hyland, D. C.↗
DOE OSTI2018

Predictive drug dosage control through a Fokker–Planck observer

During the years, various techniques have been presented for optimal control of tumor growth for which its stochastic behavior has rarely been considered. This article uses the well-known Gompertz stochastic model, for describing tumor growth, and by receding horizon model predictive control (RH-MPC) scheme computes the drug dosage for minimizing the tumor-cell population. To do this, a predictive control problem is defined based on the difference between the probability density function of tumor-cell population and the desired probability density function. In the model, both the drug dosage limitation and the Fokker–Planck equation have been considered as constraints. By solving this problem, the drug dosage which is considered as an external input to tumor model, is computed. In this article, the Fokker–Planck equation is used (1) as a nonlinear observer of probability density function of tumor-cell population and (2) a mapping vehicle from stochastic to deterministic domain. The Fokker–Planck equation is written for the Gompertz stochastic model of tumor growth. The equation is then solved through the path integral method. In this way, the probability density function of tumor-cell population, which contains the entire stochastic characteristics of the tumor growth, is obtained for any instance of time. The simulation results have also been presented for the evaluation of the suggested approach. The results show that the tumor-cell population can be controlled within a number of time windows (15 time windows in our case study) if an appropriate desired PDF is selected.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Stochastic Charging Station Deployment Model for Electrified Taxi Fleets in Coupled Urban Transportation and Power Distribution Networks

Metropolitansworldwide are increasingly adopting electric taxis (ET) to address concerns about transportation-related emissions. However, the widespread deployment of electric taxis presents challenges in terms of increased electricity demand and changing demand profiles. This transition impacts both the urban transportation network (TN) and the electricity power distribution network (PDN), highlighting the interdependence between these two systems. Here, in this paper, we propose a two-stage stochastic programming planning model that aims to optimize both the TN and PDN, enabling efficient deployment of charging stations and grid upgrades. Our model seeks to strike a balance between meeting ET drivers' charging preferences, minimizing the costs associated with infrastructure deployment and grid expansion, and harmonizing the coordination between the TN and PDN. Additionally, we explore the potential benefits of utilizing an autonomous ET fleet to enhance overall system performance.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Machine Learning-Assisted High-Temperature Reservoir Thermal Energy Storage Optimization: Numerical Modeling and Machine Learning Input and Output Files

This data set includes the numerical modeling input files and output files used to synthesize data, and the reduced-order machine learning models trained from the synthesized data for reservoir thermal energy storage site identification. In this study, a machine-learning-assisted computational framework is presented to identify High-Temperature Reservoir Thermal Energy Storage (HT-RTES) site with optimal performance metrics by combining physics-based simulation with stochastic hydrogeologic formation and thermal energy storage operation parameters, artificial neural network regression of the simulation data, and genetic algorithm-enabled multi-objective optimization. A doublet well configuration with a layered (aquitard-aquifer-aquitard) generic reservoir is simulated for cases of continuous operation and seasonal-cycle operation scenarios. Neural network-based surrogate models are developed for the two scenarios and applied to generate the Pareto fronts of the HT-RTES performance for four potential HT-RTES sites. The developed Pareto optimal solutions indicate the performance of HT-RTES is operation-scenario (i.e., fluid cycle) and reservoir-site dependent, and the performance metrics have competing effects for a given site and a given fluid cycle. The developed neural network models can be applied to identify suitable sites for HT-RTES, and the proposed framework sheds light on the design of resilient HT-RTES systems. All the simulations and the neural network model were done by Idaho National Laboratory. A detailed description of the work was reported in publication linked below.

15 GEOTHERMAL ENERGY↗