Effectiveness of Warm-Start PPO for Guidance with Highly Constrained Nonlinear Fixed-Wing Dynamics
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Purpose: In a situation where a couch shift for patient positioning is not preferred or prohibited (e.g., MR-linac), segment aperture morphing (SAM) can address target dislocation and deformation. For IMRT/VMAT with flattening-filter-free (FFF) beams, however, SAM method would lead to an adverse translational dose effect due to the beam unflattening. Here the authors propose a new two-step process to address both the translational effect of FFF beams and the target deformation. Methods: The replanning method consists of an offline and an online step. The offline step is to create a series of preshifted-plans (PSPs) obtained by a so-called “warm start” optimization (starting optimization from the original plan, rather than from scratch) at a series of isocenter shifts. The PSPs all have the same number of segments with very similar shapes, since the warm start optimization only adjusts the MLC positions instead of regenerating them. In the online step, a new plan is obtained by picking the closest PSP or linearly interpolating the MLC positions and the monitor units of the closest PSPs for the shift determined from the image of the day. This two-step process is completely automated and almost instantaneous (no optimization or dose calculation needed). The previously developed SAM algorithm is then applied for daily deformation. The authors tested the method on sample prostate and pancreas cases. Results: The two-step interpolation method can account for the adverse dose effects from FFF beams, while SAM corrects for the target deformation. Plan interpolation method is effective in diminishing the unflat beam effect and may allow reducing the required number of PSPs. The whole process takes the same time as the previously reported SAM process (5–10 min). Conclusions: The new two-step method plus SAM can address both the translation effects of FFF beams and target deformation, and can be executed in full automation except the delineation of target contour required by the SAM process.
We present a real-time optimization strategy based on warm-start for solving a moving horizon of multi period AC optimal power flow (ACOPF) problems. In each horizon, ACOPFs are temporally interlinked via generator ramp constraints, and we assume that each horizon needs to be solved every few seconds or minutes. We introduce two approximate tracking schemes that closely follow a solution path consisting of strongly regular points. We present theoretical results bounding the tracking error by the square of the parameter changes between time periods. Experimental results for networks of sizes up to 9K buses show a fast computation time while maintaining a good solution quality, thus making our approach well suited for real-time circumstances.
In several areas of economics and engineering, it is often necessary to fit discrete data points or approximate nonlinear functions with continuous functions. Piecewise linear (PWL) functions are a convenient way to achieve this. PWL functions can be modeled in mathematical problems using only linear and integer variables. Moreover, there is a computational benefit in using PWL functions that have the least possible number of segments. This work proposes a novel hierarchical mixed integer linear programming (MILP) formulation that identifies a continuous PWL approximation with minimum number of linear segments for a given target maximum error. The proposed MILP formulation also identifies the solution with the least maximum error among the solutions with minimum number of segments. Then, this work proposes a fast iterative algorithm that identifies non necessarily continuous PWL approximations by solving O(S log N) linear programming (LP) problems, where N is the number of data points and S is the minimum number of segments in the non necessarily continuous case. This work demonstrates that tight bounds for the MILP problem can be derived from these approximations. Next, a fast algorithm is introduced to transform a non necessarily continuous PWL approximation into a continuous one. Finally, the tight bounds and the continuous PWL approximations are used to tighten and warm start the MILP problem. The tightened formulation is shown in experimental results to be more efficient, especially for large data sets, with a solution time that is up to two orders of magnitude less than the existing literature.
Not Available
In this work, a class of non-negative matrix factorization (NMF) technique known as alternating non-negative least squares, combined with the projected gradient method, is used to analyze twenty-five [{sup 11}C]-DTBZ dynamic PET/CT brain data. For each subject, a two-factor model is assumed and two factors representing the striatum (factor 1) and the non-striatum (factor 2) tissues are extracted using the proposed NMF technique and commercially available factor analysis software “Pixies”. The extracted factor 1 and 2 curves represent the binding site of the radiotracer and describe the uptake and clearance of the radiotracer by soft tissues in the brain, respectively. The proposed NMF technique uses prior information about the dynamic data to obtain sample time-activity curves representing the striatum and the non-striatum tissues. These curves are then used for “warm” starting the optimization. Factor solutions from the two methods are compared graphically and quantitatively. In healthy subjects, radiotracer uptake by factors 1 and 2 are approximately 35–40% and 60–65%, respectively. The solutions are also used to develop a factor-based metric for the detection of early, untreated Parkinson's disease. The metric stratifies healthy subjects from suspected Parkinson's patients (based on the graphical method). The analysis shows that both techniques produce comparable results with similar computational time. The “semi-automatic” approach used by the NMF technique allows clinicians to manually set a starting condition for “warm” starting the optimization in order to facilitate control and efficient interaction with the data.
Residential biomass combustion is a source of carbonaceous aerosol. Inefficient combustion, particularly of solid fuels produces large quantities of black and brown carbon (BC and BrC). These particle types are important as they have noted effects on climate forcing and human health. One method of measuring these quantities is by measurement of aerosol light-absorption and scattering, which can be performed using an aethalometer and nephelometer, respectively. These instruments are widely deployed in the study of ambient air and are frequently used in air quality modeling and source apportionment studies. In this study, we will describe (1) a method for measuring primary BC and BrC emissions from two residential log-fired wood hydronic heaters and (2) the BC and BrC emission from these devices over a wide range of operating conditions, such as cold-starts, warm-starts, four different levels of output ranging from 15% to 100% maximum rated output, and periods of repeated cycling. The range in flue-gas BC concentrations, measured using an aethalometer at the 880 nanometer (nm) wavelength, were between 5.09 × 10 2 and 2.24 × 10 4 micrograms per cubic meter (µg/m 3 ) while the scattering coefficient of the flue-gas, measured by a nephelometer at 880 nm, ranged between 2.20 × 10 3 and 8.56 × 10 5 inverse megameters (Mm –1 ). The BrC concentrations, measured using the 370 nm wavelength of an aethalometer, were between 9.10 × 10 1 and 3.56 × 10 4 µg/m 3 . The calculated Angstrom Absorption Exponent (AAE) of the flue-gas aerosol ranged between 1.54 and 3.63. Performing a comparison between the measured BC concentration and an external particulate matter (PM) concentration showed that overall BC makes up roughly a quarter of the PM emitted by either of the two appliances. Further for both appliances, the cold-start and the test phase immediately following it had the highest BC and BrC concentrations, the highest measured scattering coefficient, as well as a low AAE.
It is widely understood that the thermal state of a light-duty vehicle at the beginning of a trip influences the vehicle performance throughout the drive cycle. Cold starts, or initial states with component temperatures near ambient conditions, are strongly correlated with reduced vehicle performance and energy efficiency and with increased emissions. Despite this understanding, there is little literature available that characterizes initial thermal states beyond empirical studies and simplified analyses of dwell times. We introduce a framework that considers vehicle activity patterns, including the previous drive event, duration of the previous dwell event, and relevant ambient conditions occurring during these events. Moreover, the framework allows for technologies to influence the prominence of cold starts and warm starts. An example application of the methodology using a powertrain model approach revealed the significance of all constituent factors: controlling for ambient conditions, the shortest dwell lengths corresponded to an average initial oil temperature of 70°C versus 23°C for the long dwell length simulations, and aggressive conditioning cycles corresponded to an average initial oil temperature of 51°C versus only 35°C for a short conditioning cycle. The coldest ambient conditions evaluated (-30°C to -10°C) produced a weighted initial oil temperature of 19°C versus 57°C for the warmest ambient conditions evaluated (30°C to 50°C). Finally, we introduce weighting factors that account for the real-world frequency of conditioning cycle and dwell length combinations. The procedure outlined may be useful for discovering real-world performance improvements associated with vehicle technologies, a topic of interest to both vehicle manufacturers and regulators.
We present spectrophotometry spanning 1–5 μm of 51 Eridani b, a 2–10 M{sub Jup} planet discovered by the Gemini Planet Imager Exoplanet Survey. In this study, we present new K1 (1.90–2.19 μm) and K2 (2.10–2.40 μm) spectra taken with the Gemini Planet Imager as well as an updated L {sub P} (3.76 μm) and new M {sub S} (4.67 μm) photometry from the NIRC2 Narrow camera. The new data were combined with J (1.13–1.35 μm) and H (1.50–1.80 μm) spectra from the discovery epoch with the goal of better characterizing the planet properties. The 51 Eri b photometry is redder than field brown dwarfs as well as known young T-dwarfs with similar spectral type (between T4 and T8), and we propose that 51 Eri b might be in the process of undergoing the transition from L-type to T-type. We used two complementary atmosphere model grids including either deep iron/silicate clouds or sulfide/salt clouds in the photosphere, spanning a range of cloud properties, including fully cloudy, cloud-free, and patchy/intermediate-opacity clouds. The model fits suggest that 51 Eri b has an effective temperature ranging between 605 and 737 K, a solar metallicity, and a surface gravity of log(g) = 3.5–4.0 dex, and the atmosphere requires a patchy cloud atmosphere to model the spectral energy distribution (SED). From the model atmospheres, we infer a luminosity for the planet of −5.83 to −5.93 (logL/L{sub ⊙}), leaving 51 Eri b in the unique position of being one of the only directly imaged planets consistent with having formed via a cold-start scenario. Comparisons of the planet SED against warm-start models indicate that the planet luminosity is best reproduced by a planet formed via core accretion with a core mass between 15 and 127 M{sub ⊕}.
Scientific machine learning (SciML) often operates in ill-conditioned, weakly identifiable regimes due to limited data or indirect observations. In such settings, optimization and inference are highly sensitive to the starting point, making initialization--often under-reported--a consequential degree of freedom. Random initialization is not a neutral default as it induces an implicit prior over candidate solutions and can systematically bias the result, producing large run-to-run variability. Here, we formalize this view by treating initialization as a hidden confounder in SciML and develop a unifying theory for structure-aware initialization via numerical continuation, constructing warm starts from related problem instances. Across representative tasks, including physics-informed neural networks, maximum likelihood estimation, and variational inference, warm starts have been shown to consistently reduce optimization effort and improve reliability.
Purpose: In a situation that couch shift for patient positioning is not preferred or prohibited (e.g., MR-Linac), segment aperture morphing (SAM) can address target dislocation and deformation. For IMRT/VMAT with flattening filter free (FFF) beams, however, SAM method would lead to an adverse translational dose effect due to the beam unflattening. Here we propose a new 2-step process to address both the translational effect of FFF beams and the target deformation. Methods: The replanning method consists of an offline and an online steps. The offline step is to create a series of pre-shifted plans (PSP) obtained by a so called “warm start” optimization (starting optimization from the original plan, rather from scratch) at a series of isocenter shifts with fixed distance (e.g. 2 cm, at x,y,z = 2,0,0 ; 2,2,0 ; 0,2,0; …;− 2,0,0). The PSPs all have the same number of segments with very similar shapes, since the warm-start optimization only adjusts the MLC positions instead of regenerating them. In the online step, a new plan is obtained by linearly interpolating the MLC positions and the monitor units of the closest PSPs for the shift determined from the image of the day. This two-step process is completely automated, and instantaneously fast (no optimization or dose calculation needed). The previously-developed SAM algorithm is then applied for daily deformation. We tested the method on sample prostate and pancreas cases. Results: The two-step interpolation method can account for the adverse dose effects from FFF beams, while SAM corrects for the target deformation. The whole process takes the same time as the previously reported SAM process (5–10 min). Conclusion: The new two-step method plus SAM can address both the translation effects of FFF beams and target deformation, and can be executed in full automation requiring no additional time from the SAM process. This research was supported by Elekta inc. (Crawley, UK)
Recently, the makespan-minimization problem of compiling a general class of quantum algorithms into near-term quantum processors has been introduced to the AI community. The research demonstrated that temporal planning is a strong solution approach for the studied class of quantum circuit compilation (QCC) problems. In this paper, we explore the use of methods from operations research, specifically constraint programming (CP), as an alternative and complementary approach to temporal planning. We also extend previous work by introducing two new problem variations that incorporate important characteristics identified by the quantum computing community. We apply temporal planning and CP to the baseline and extended QCC problems as both stand-alone and hybrid approaches. The hybrid method uses solutions found by temporal planning to warm-start CP, leveraging the ability of temporal planning to find satisficing solutions to problems with a high degree of task optionality, an area that CP typically struggles with. These solutions are then used to seed the CP formulation which significantly benefits from inferred bounds on planning horizon and task counts provided by the warm-start. Our extensive empirical evaluation indicates that while stand-alone CP is not competitive with temporal planning, except for the smallest problems, CP in a hybrid setting is beneficial for all temporal planners in all problem classes.
This article reports the study of algorithms for non-negative matrix factorization (NMF) in various applications involving smoothly varying data such as time or temperature series diffraction data on a dense grid of points. Utilizing the continual nature of the data, a fast two-stage algorithm is developed for highly efficient and accurate NMF. In the first stage, an alternating non-negative least-squares framework is used in combination with the active set method with a warm-start strategy for the solution of subproblems. In the second stage, an interior point method is adopted to accelerate the local convergence. The convergence of the proposed algorithm is proved. The new algorithm is compared with some existing algorithms in benchmark tests using both real-world data and synthetic data. Furthermore, the results demonstrate the advantage of the algorithm in finding high-precision solutions.
The overall operating temperature and efficiency of solar-powered Stirling engines can be improved by adding an alkali metal pool boiler heat transport system to supply heat more uniformly to the heater head tubes. One issue with liquid metal pool boilers is unstable boiling. Stable boiling is obtained with an enhanced boiling surface containing nucleation sites that promote continuous boiling. Over longer time periods, it is possible that the boiling behavior of the system will change. An 800-h life test was conducted to verify that pool boiling with the chosen fluid/surface combination remains stable as the system ages. The apparatus uses NaK boiling on a - 100 + 140 stainless steel sintered porous layer, with the addition of a small amount of xenon. Pool boiling remained stable to the end of life test. The pool boiler life test included a total of 82 cold starts, to simulate startup each morning, and 60 warm restarts, to simulate cloud cover transients. The behavior of the cold and warm starts showed no significant changes during the life test. In the experiments, the fluid/surface combination provided stable, high-performance boiling at the operating temperature of 700 C. Based on these experiments, a pool boiler was designed for a full-scale 25-kWe Stirling system.
We consider a continuous-valued simulation optimization (SO) problem, where a simulator is built to optimize an expected performance measure of a real-world system while parameters of the simulator are estimated from streaming data collected periodically from the system. At each period, a new batch of data is combined with the cumulative data and the parameters are re-estimated with higher precision. The system requires the decision variable to be selected in all periods. Therefore, it is sensible for the decision-maker to update the decision variable at each period by solving a more precise SO problem with the updated parameter estimate to reduce the performance loss with respect to the target system. We define this decision-making process as the multi-period SO problem and introduce a multi-period stochastic approximation (SA) framework that generates a sequence of solutions. Two algorithms are proposed: Re-start SA (ReSA) reinitializes the stepsize sequence in each period, whereas Warm-start SA (WaSA) carefully tunes the stepsizes, taking both fewer and shorter gradient-descent steps in later periods as parameter estimates become increasingly more precise. We show that under suitable strong convexity and regularity conditions, ReSA and WaSA achieve the best possible convergence rate in expected sub-optimality either when an unbiased or a simultaneous perturbation gradient estimator is employed, while WaSA accrues significantly lower computational cost as the number of periods increases. In addition, we present the regularized ReSA, which obviates the need to know the strong convexity constant and achieves the same convergence rate at the expense of additional computation.
The Phase I of this project confirmed the technical feasibility of a Combined Cycle integrated Thermal Energy Storage “CiTES” system, calculated the key performance parameters like power efficiency and costs, and proved its commercial value with full-year simulations for several US electricity markets with high degree of variable renewable generation and volatile hourly electricity prices. The core element of this project is the Electro Thermal Energy Storage (ETES) technology from Siemens Gamesa Renewable Energy GmbH, using thermally stable and inexpensive volcanic rocks as storage material and air as heat transfer medium. This technology is backed by more than 10 years of experience and a 440MMBTU (130MWh-th) pilot plant in Hamburg, Germany, which is in operation since 2019. The integration of this thermal storage in an existing combined cycle power plant (CCPP) is typical power plant technology without any major technology risks. It allows the storage of inexpensive renewable energy during times of surplus renewable generation and the discharge of this energy in times of high energy demand when the fossil plant is in operation. This supplements the fossil power generation with CO2-emission-free energy. The secondary effect of the CiTES system is that a small part of the stored thermal energy is used to keep the heat recovery steam generator (HRSG) and steam turbine (ST) of the combined cycle power plant in hot and ready-to-start condition. This enables the plant to start rapidly when fossil generation is required to satisfy demand as soon variable generation drops off in the evenings or during cloud cover and calm wind periods. Without pre-warming of the HRSG and ST, the CCPP would need several hours for a cold or warm start, burn a lot of gas and release high NOx emissions during start and wouldn’t be able to use the short times of high energy prices in an efficient or economical manner. The economic parameters of CiTES were determined by a full year “8760” simulation using a data set calculation for each of the hours of the year, and historical electricity and gas prices. For consistency, the simulations were focused on the pre-COVID year 2019. The financially most attractive markets were in the Energy Reliability Council of Texas (ERCOT) region, which allowed substantial value generation with arbitrage (charge with cheap energy during renewable surplus times and discharge when energy is needed and expensive). The improvement of flexibility with the CiTES system by pre-heating and warm-keeping of the CCPP allowed for additional power generation during short time periods when demand is high but renewable generation is down; when the hourly energy prices are highest in these markets. The simulations are based on 2019 data, when ERCOT had 27GW of installed photovoltaic (PV) and wind generation. They showed that the created revenue with the prototypically sized CiTES system of 1,000MMBTU (300MWh-th) falls a little bit short of what is expected from a commercially viable investment. The system has optimization opportunities for cost reduction and increased effectiveness which will be realized during a potential Phase II Pre-FEED study following this project. Furthermore, it is safe to assume that a lot of renewable generation capacity will be added all over the US in the coming years. As an example, ERCOT is predicting to more than double its renewable generation from 27GW in 2019 to a forecasted 63GW in 2023. This will increase the amount of renewable overproduction exponentially. This rapid increase of local overproduction and the need to curtail renewable generation is well documented by the California ISO (www.CAISO.com / managing oversupply). However, the simulations also revealed a weakness in the structure of the electricity markets in the US. More specifically, when electricity prices are very low and approaching negative levels, the owners of Variable Renewable Energy (VRE) will curtail a part of their facility to stabilize the price by reducing supply. This results in a situation in which storage facilities, which are integrated in existing fossil assets and don’t have the behind-the-meter benefit of a VRE, won’t be able to purchase low cost – otherwise curtailed – renewable energy off the grid. A special tariff, which motivates VRE owners to sell otherwise curtailed renewable energy to storage facilities (Hydrogen, thermal, pumped hydro, etc.) can solve this issue. The implementation of such a regulating tariff by Independent System Operators, thus avoiding renewable curtailment, is a pre condition for successful commercialization for renewable energy storage technologies. With this advancement of design and technology and improvements in the market environment, it can be expected that the Combined Cycle integrated Thermal Energy Storage proves itself as an important innovation to keep highly efficient, natural gas-based power generation economically successful and relevant for the power industry in the United States of America.
Context. We previously reported the direct detection of a low mass companion at a projected separation of 55+/-2 astronomical units around the B9 type star kappa Andromedae. The properties of the system (mass ratio, separation) make it a benchmark for the understanding of the formation and evolution of gas giant planets and brown dwarfs on wide-orbits. Aims. We present new angular differential imaging (ADI) images of the system at 2.146 (K(sub s)), 3.776 (L'), 4.052 (NB 4.05) and 4.78 micrometers (M') obtained with Keck/NIRC2 and LBTI/LMIRCam, as well as more accurate near-infrared photometry of the star with the MIMIR instrument. We aim to determine the near-infrared spectral energy distribution (SED) of the companion and use it to characterize the object. Methods. We used analysis methods adapted to ADI to extract the companion flux. We compared the photometry of the object to reference young/old objects and to a set of seven PHOENIX-based atmospheric models of cool objects accounting for the formation of dust. We used evolutionary models to derive mass estimates considering a wide range of plausible initial conditions. Finally, we used dedicated formation models to discuss the possible origin of the companion. Results. We derive a more accurate J = 15.86 +/- 0.21, H = 14.95 +/- 0.13, K(sub s) = 14.32 +/- 0.09 mag for kappa And b. We redetect the companion in all our high contrast observations. We confirm previous contrasts obtained at K(sub s) and L' band. We derive NB 4.05 = 13.0 +/- 0.2 and M' = 13.3 +/- 0.3 mag and estimate Log(base 10)(L/solar luminosity) = −3.76 +/- 0.06. Atmospheric models yield T(sub eff) = 1900(+100/−200) K. They do not set constrains on the surface gravity. "Hot-start" evolutionary models predict masses of 14(+25/−2) Jupiter mass based on the luminosity and temperature estimates, and considering a conservative age range for the system (30(+120/−10) million years). "warm-start" evolutionary tracks constrain the mass to M greater than or equal to 11 Jupiter mass. Conclusions. The mass of kappa Andromedae b mostly falls in the brown-dwarf regime, due to remaining uncertainties in age and mass-luminosity models. According to the formation models, disk instability in a primordial disk could account for the position and a wide range of plausible masses of kappa and b.
Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.