Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Stochastic optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Gradient-based optimization for regression in the functional tensor-train format

Predictive analysis of complex computational models, such as uncertainty quantification (UQ), must often rely on using an existing database of simulation runs. In this paper we consider the task of performing low-multilinear-rank regression on such a database. Specifically we develop and analyze an efficient gradient computation that enables gradient-based optimization procedures, including stochastic gradient descent and quasi-Newton methods, for learning the parameters of a functional tensor-train (FT). We compare our algorithms with 22 other nonparametric and parametric regression methods on 10 real-world data sets and show that for many physical systems, exploiting low-rank structure facilitates efficient construction of surrogate models. Here, we use a number of synthetic functions to build insight into behavior of our algorithms, including the rank adaptation and group-sparsity regularization procedures that we developed to reduce overfitting. Finally we conclude the paper by building a surrogate of a physical model of a propulsion plant on a naval vessel.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Locating PHEV exchange stations in V2G

Plug-in hybrid electric vehicle (PREV) is an environment friendly modem transportation method and has been rapidly penetrate the transportation system. Renewable energy is another contributor to clean power but the associated intermittence increases the uncertainty in power generation. As a foreseen benefit of a vchicle-to-grid (V2G) system, PREV supporting infrastructures like battery exchange stations can provide battery service to PREV customers as well as being plugged into a power grid as energy sources and stabilizer. The locations of exchange stations are important for these two objectives under constraints from both ,transportation system and power grid. To model this location problem and to understand and analyze the benefit of a V2G system, we develop a two-stage stochastic program to optimally locate the stations prior to the realizations of battery demands, loads, and generation capacity of renewable power sources. Based on this model, we use two data sets to construct the V2G systems and test the benefit and the performance of these systems.

25 ENERGY STORAGE↗

Battery Model Parameterization Using Manufacturer Datasheet and Field Measurement for Real-Time HIL Applications

Here, this paper presents a novel battery model parameterization method using actual field measurement and manufacturer datasheet for real-time hardware-in-the-loop (HIL) applications. It is critical that real-time HIL models can accurately reproduce field test results so that tests can be conducted on HIL testbeds instead of in the field. In the past, numerical heuristic optimization algorithms were often used to derive parameters for battery models. However, the deterministic algorithms often reach a locally optimal solution and stochastic heuristic searching strategies suffer from low searching efficiency. Therefore, in this paper, we propose a global-local searching enhanced genetic algorithm (GL-SEGA). By applying the generalized opposition-based learning mechanism, GL-SEGA can efficiently explore the global solution space. By using the trust-region-reflective method to perform the local search, the GL-SEGA can improve the accuracy and convergence in its local exploitations. Field measurements and manufactory datasheets are used to test and validate the accuracy and robustness of the GL-SEGA algorithm.

25 ENERGY STORAGE↗

SU-G-201-09: Evaluation of a Novel Machine-Learning Algorithm for Permanent Prostate Brachytherapy Treatment Planning

Purpose: A novel, automated, algorithm for permanent prostate brachytherapy (PPB) treatment planning has been developed. The novel approach uses machine-learning (ML), a form of artificial intelligence, to substantially decrease planning time while simultaneously retaining the clinical intuition of plans created by radiation oncologists. This study seeks to compare the ML algorithm against expert-planned PPB plans to evaluate the equivalency of dosimetric and clinical plan quality. Methods: Plan features were computed from historical high-quality PPB treatments (N = 100) and stored in a relational database (RDB). The ML algorithm matched new PPB features to a highly similar case in the RDB; this initial plan configuration was then further optimized using a stochastic search algorithm. PPB pre-plans (N = 30) generated using the ML algorithm were compared to plan variants created by an expert dosimetrist (RT), and radiation oncologist (MD). Planning time and pre-plan dosimetry were evaluated using a one-way Student’s t-test and ANOVA, respectively (significance level = 0.05). Clinical implant quality was evaluated by expert PPB radiation oncologists as part of a qualitative study. Results: Average planning time was 0.44 ± 0.42 min compared to 17.88 ± 8.76 min for the ML algorithm and RT, respectively, a significant advantage [t(9), p = 0.01]. A post-hoc ANOVA [F(2,87) = 6.59, p = 0.002] using Tukey-Kramer criteria showed a significantly lower mean prostate V150% for the ML plans (52.9%) compared to the RT (57.3%), and MD (56.2%) plans. Preliminary qualitative study results indicate comparable clinical implant quality between RT and ML plans with a trend towards preference for ML plans. Conclusion: PPB pre-treatment plans highly comparable to those of an expert radiation oncologist can be created using a novel ML planning model. The use of an ML-based planning approach is expected to translate into improved PPB accessibility and plan uniformity.

60 APPLIED LIFE SCIENCES↗

Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance

The ability of neural networks to provide ‘best in class’ approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural networks comes to naught if one is unable to effectively train (choose) the parameters defining the network. In general, neural networks are trained by gradient descent type optimization methods,a stochastic variant thereof. In practice, such methods result in the loss function decreases rapidly at the beginning of training but then, after a relatively small number of steps, significantly slow down. The loss may even appear to stagnate over the period of a large number of epochs, only to then suddenly start to decrease fast again for no apparent reason. This so-called plateau phenomenon manifests itself in many learning tasks. The present work aims to identify and quantify the root causes of plateau phenomenon.analysis is carried out in the setting of univariate ReLU networks. No assumptions are made on the number of neurons relative to the number of training data, and our results hold for both the lazy and adaptive regimes. Here, the main findings are: plateaux correspond to periods during which activation patterns remain constant, where activation pattern refers to the number of data points that activate a given neuron; quantification of convergence of the gradient flow dynamics; and, characterization stationary points in terms solutions of local least squares regression lines over subsets of the training data. Based on these conclusions, we propose a new iterative training method, the Active Neuron Least Squares (ANLS), characterised by the explicit adjustment of the activation pattern at each step, which is designed to enable a quick exit from a plateau. Illustrative numerical examples are included throughout.

97 MATHEMATICS AND COMPUTING↗

ProgressiveHedging.jl

Implementation of the Progressive Hedging algorithm written in Julia utilizing JuMP. This is used to solve multistage optimization problems (e.g. Stochastic programs or capacity expansion models). The user uses the StructJuMP extension of JuMP to create the model and build up the scenario tree.

Sigler, Devon↗

DEEP Solar: Data DrivEn Modeling and Analytics for Enhanced System Layer ImPlementation

Realizing the SETO 2030 mission of reducing solar energy costs to 3-5 c/kWh will require innovative enabling research on effective, cost-efficient integration of local PV within distribution systems. However, the intermittent and variable nature of PVs compels operators to impose conservative hosting capacity constraints. Given the extremely high variability of (intermittent and unpredictable) solar energy generation, relaxing the capacity constraints (which are currently around 15%) and achieving 100% or greater integration of renewables will require a fundamental transformation of the power grid via the utilization of exponentially larger amounts of AMI enabled fine-grained data. To address the challenges in increasing the penetration of renewable energy based DERs, this project envisions an Enhanced System Layer (ESL) at the distribution network level that is reliable, cost-effective and scalable to millions of Distributed Energy Resources (DERs)/devices. This includes developing: 1) Transformative and highly scalable machine learning based predictive analytics tools that plug into distribution system planning and provide real-time situational awareness at the distribution level for short and long-term operational planning. The tools will be built using novel data-driven energy models of millions of active nodes with AMI, 2) Adaptive stochastic analysis and optimization algorithms for real-time grid operations, 3) Dynamic Scenario Analysis using parallel Cloudenabled implementations with < 1 minute computational cycle times.

14 SOLAR ENERGY↗

Dynamic Network Analysis of Nuclear Science Literature for Research Influence Assessment

Analyzing nuclear science literature via data-driven methods is a critical step for assessing research influence and technology advancements. Indicators of scholarly activities may be buried in large volumes of nuclear research publications and collaboration networks over time. Mining for relevant scholarly influence trends in large volumes of text can be computationally challenging; however, open-source information on research collaborations over time can offer opportunities to extract meaningful insights. While network centrality analysis of scholarly research provides topology-based insights, additional emphasis on dynamics associated with the diffusion of information through these networks is important. Here this paper represents a step in that direction through the development of a novel dynamic network analysis framework and computational engine to identify key entities and capabilities over time within global scholarly nuclear science collaboration networks. Network theoretic, stochastic simulation, and optimization methods are leveraged to address variability in scholarly interactions, influence propagation, and collaboration patterns via network connections. A topic-aware influence maximization algorithm is developed to address the goal of identifying key influential authors in diverse research topics over time. Efficient parallelized implementation of the algorithm is applied to reduce computational costs. A proof-of-concept case study using open-source Scopus data with 33,517 published nuclear research papers from 2000-2019 is presented and representative analytic insights are generated. Broad implications of these insights are discussed and future research directions are also identified.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

A parallel hub-and-spoke system for large-scale scenario-based optimization under uncertainty

Practical solution of stochastic programming problems generally requires the use of parallel computing resources. Here, we describe the open source package mpi-sppy, in which efficient and scalable parallelization is a central feature. We report computational experiments that demonstrate the ability to solve very large stochastic programming problems - including mixed-integer variants - in minutes of wall clock time, efficiently leveraging significant parallel computing resources. We report results for the largest publicly available instances of stochastic mixed-integer unit commitment problems, solving to provably tight optimality gaps. In addition, we introduce a novel software architecture that facilitates combinations of methods for accelerating convergence that can be combined in plug-and-play manner. Finally, the mpi-sppy package is written in Python, leverages the widely used Pyomo (http://www.pyomo.org) library for modeling mathematical programs, builds on existing MPI implementations to ensure efficiency and scalability, and is available via http://github.com/Pyomo/mpi-sppy.

97 MATHEMATICS AND COMPUTING↗

Sensitivity analysis and optimization of the nuclear fuel cycle

A sensitivity study has been conducted to assess the robustness of the conclusions presented in the MIT Fuel Cycle Study. The Once Through Cycle (OTC) is considered as the base-line case, while advanced technologies with fuel recycling characterize the alternative fuel cycles. The options include limited recycling in LWRs and full recycling in fast reactors and in high conversion LWRs. Fast reactor technologies studied include both oxide and metal fueled reactors. The analysis allowed optimization of the fast reactor conversion ratio with respect to desired fuel cycle performance characteristics. The following parameters were found to significantly affect the performance of recycling technologies and their penetration over time: Capacity Factors of the fuel cycle facilities, Spent Fuel Cooling Time, Thermal Reprocessing Introduction Date, and in core and Out-of-core TRU Inventory Requirements for recycling technology. An optimization scheme of the nuclear fuel cycle is proposed. Optimization criteria and metrics of interest for different stakeholders in the fuel cycle (economics, waste management, environmental impact, etc.) are utilized for two different optimization techniques (linear and stochastic). Preliminary results covering single and multi-variable and single and multi-objective optimization demonstrate the viability of the optimization scheme. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Force-Free Identification of Minimum-Energy Pathways and Transition States for Stochastic Electronic Structure Theories

Here, the accurate mapping of potential energy surfaces (PESs) is crucial to our understanding of the numerous physical and chemical processes mediated by atomic rearrangements, such as conformational changes and chemical reactions, and the thermodynamic and kinetic feasibility of these processes. Stochastic electronic structure theories, e.g., Quantum Monte Carlo (QMC) methods, enable highly accurate total energy calculations that in principle can be used to construct the PES. However, their stochastic nature poses a challenge to the computation and use of forces and Hessians, which are typically required in algorithms for minimum-energy pathway (MEP) and transition state (TS) identification, such as the nudged elastic band (NEB) algorithm and its climbing image formulation. Here, we present strategies that utilize the surrogate Hessian line-search method, previously developed for QMC structural optimization, to efficiently identify MEP and TS structures without requiring force calculations at the level of the stochastic electronic structure theory. By modifying the surrogate Hessian algorithm to operate in path-orthogonal subspaces and at saddle points, we show that it is possible to identify MEPs and TSs by using a force-free QMC approach. We demonstrate these strategies via two examples, the inversion of the ammonia (NH 3 ) molecule and the nucleophilic substitution (S N 2) reaction F – + CH 3 F → FCH 3 + F – . We validate our results using Density Functional Theory (DFT)- and Coupled Cluster (CCSD, CCSD(T))-based NEB calculations. We then introduce a hybrid DFT-QMC approach to compute thermodynamic and kinetic quantities, free energy differences, rate constants, and equilibrium constants that incorporates stochastically optimized structures and their energies, and show that this scheme improves upon DFT accuracy. Our methods generalize straightforwardly to other systems and other high-accuracy theories that similarly face challenges computing energy gradients, paving the way for highly accurate PES mapping, transition state determination, and thermodynamic and kinetic calculations at significantly reduced computational expense.

Iyer, Gopal R.↗

Optimal Allocation of Water Resources Using a Two-Stage Stochastic Programming Method with Interval and Fuzzy Parameters

Efficient water allocation is one of the most prominent issues in water resources management. In this research, a two-stage interval-parameter stochastic fuzzy programming with type 2 membership functions was used to allocate water resources optimally to different users under uncertainty. This method can handle uncertainties expressed as probability distributions, discrete intervals, and fuzzy sets. The model considers treated wastewater as an allocable water resource in a scenario, in addition to water that is extracted from wellheads, springs, and qanats. Moreover, the loss rate of water during distribution, surplus water in the reservoir in the previous and the next period, and treated wastewater parameters have been incorporated into the model. This model was applied to a case study of water resources allocation within a multi-user and multi-reservoir context in the Zarand region of Kerman, Iran. The results indicate that reasonable solutions have been generated, in the form of interval and fuzzy information under different scenarios that could help managers provide optimal water resources allocation plans. The results also demonstrate that establishing a wastewater treatment station increases the system net benefits, surplus water in wellheads for the next period, and system reliability (level of satisfying the fuzzy goal and constraints), and decreases encountering water shortages.

58 GEOSCIENCES↗

Randomized Federated Learning Methods for Nonsmooth, Nonconvex, and Hierarchical Optimization (Final Technical Report)

This final technical report summarizes the outcomes of a DOE-funded project on federated scientific machine learning (FL) under nonsmooth, nonconvex, and hierarchical optimization settings. The project develops new mathematical models, algorithms, and theoretical guarantees for decentralized stochastic, bilevel, and minimax optimization problems arising in DOE mission-relevant applications. A unified framework of randomized and zeroth-order federated optimization methods is introduced, providing provable convergence, communication efficiency, and sample-complexity guarantees. The report documents algorithmic design, theoretical analysis, and empirical validation of the proposed federated learning methods. The project also contributes to workforce development through graduate training and dissemination of results via publications and seminars.

97 MATHEMATICS AND COMPUTING↗

Generating moment matching scenarios using optimization techniques

An optimization based method is proposed to generate moment matching scenarios for numerical integration and its use in stochastic programming. The main advantage of the method is its flexibility: it can generate scenarios matching any prescribed set of moments of the underlying distribution rather than matching all moments up to a certain order, and the distribution can be defined over an arbitrary set. This allows for a reduction in the number of scenarios and allows the scenarios to be better tailored to the problem at hand. The method is based on a semi-infinite linear programming formulation of the problem that is shown to be solvable with polynomial iteration complexity. A practical column generation method is implemented. The column generation subproblems are polynomial optimization problems; however, they need not be solved to optimality. It is found that the columns in the column generation approach can be efficiently generated by random sampling. The number of scenarios generated matches a lower bound of Tchakaloff's. The rate of convergence of the approximation error is established for continuous integrands, and an improved bound is given for smooth integrands. Extensive numerical experiments are presented in which variants of the proposed method are compared to Monte Carlo and quasi-Monte Carlo methods on both numerical integration problems and stochastic optimization problems. The benefits of being able to match any prescribed set of moments, rather than all moments up to a certain order, is also demonstrated using optimization problems with 100-dimensional random vectors. Here, empirical results show that the proposed approach outperforms Monte Carlo and quasi-Monte Carlo based approaches on the tested problems.

97 MATHEMATICS AND COMPUTING↗

Scenario Creation and Power-Conditioning Strategies for Operating Power Grids with Two-Stage Stochastic Economic Dispatch

A significant difficulty associated with the use of stochastic programming to solve optimal power flow problems on a 5-minute timescale is the quality of renewable energy scenarios input by the user. This is especially true when considering power systems with high penetrations of renewable energy, e.g. wind power. This paper introduces the use of stochastic programming to solve the DC optimal power flow problem with scenarios drawn directly from high-fidelity data sets. Hence, the proposed method avoids the problem of lost physics by finding high-fidelity analogs that can describe future states of the system. Furthermore, this method can be simply extended to output multi-period scenarios to the stochastic program. We demonstrate the effectiveness of this technique by simulating dispatch operations on a synthetic test system over the course of a week.

analog forecast↗

Optimizing event selection with the random grid search

In this paper, the random grid search (RGS) is a simple, but efficient, stochastic algorithm to find optimal cuts that was developed in the context of the search for the top quark at Fermilab in the mid-1990s. The algorithm, and associated code, have been enhanced recently with the introduction of two new cut types, one of which has been successfully used in searches for supersymmetry at the Large Hadron Collider. The RGS optimization algorithm is described along with the recent developments, which are illustrated with two examples from particle physics. One explores the optimization of the selection of vector boson fusion events in the four-lepton decay mode of the Higgs boson and the other optimizes SUSY searches using boosted objects and the razor variables.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Development Optimization and Uncertainty Analysis Methods for Oil and Gas Reservoirs

Uncertainty complicates the development optimization of oil and gas exploration and production projects, but methods have been devised to analyze uncertainty and its impact on optimal decision-making. This paper compares two methods for development optimization and uncertainty analysis: Monte Carlo (MC) simulation and stochastic programming. Two example problems for a gas field development and an oilfield development are solved and discussed to elaborate the advantages and disadvantages of each method. Development optimization involves decisions regarding the configuration of initial capital investment and subsequent operational decisions. Uncertainty analysis involves the quantification of the impact of uncertain parameters on the optimum design concept. The gas field development problem is designed to highlight the differences in the implementation of the two methods and to show that both methods yield the exact same optimum design. The results show that both MC optimization and stochastic programming provide unique benefits, and that the choice of method depends on the goal of the analysis. While the MC method generates more useful information, along with the optimum design configuration, the stochastic programming method is more computationally efficient in determining the optimal solution. Reservoirs comprise multiple compartments and layers with multiphase flow of oil, water, and gas. We present a workflow for development optimization under uncertainty for these reservoirs, and solve an example on the design optimization of a multicompartment, multilayer oilfield development.

02 PETROLEUM↗