Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Policy gradient”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Towards catchment classification in data-scarce regions

Assessing spatial variation in hydrologic processes can help to inform freshwater management and advance ecological understanding, yet many areas lack sufficient flow records on which to base classifications. Seeking to address this challenge, we apply concepts developed in data-rich settings to public, global data in order to demonstrate a broadly replicable approach to characterizing hydrologic variation. The proposed approach groups the basins associated with reaches in a river network according to key environmental drivers of hydrologic conditions. This initial study examines Colorado (USA), where long-term streamflow records permit comparison to previously distinguished flow regime types, and the Republic of Ecuador, where data limitations preclude such analysis. The flow regime types assigned to gages in Colorado corresponded reasonably well to the classes distinguished from environmental features. The divisions in Ecuador reflected major known biophysical gradients while also providing a higher resolution supplement to an existing depiction of freshwater ecoregions. Although freshwater policy and management decisions occur amidst uncertainty and imperfect knowledge, this classification framework offers a rigorous and transferrable means to distinguish catchments in data-scarce regions. The maps and attributes of the resulting ecohydrologic classes offer a departure point for additional study and data collection programs such as the placement of stations in under-monitored classes, and the divisions may serve as a preliminary template with which to structure conservation efforts such as environmental flow assessments.

13 HYDRO ENERGY↗

Asymmetric responses of primary productivity to precipitation extremes: A synthesis of grassland precipitation manipulation experiments

Climatic changes are altering Earth's hydrological cycle, resulting in altered precipitation amounts, increased interannual variability of precipitation, and more frequent extreme precipitation events. These trends will likely continue into the future, having substantial impacts on net primary productivity (NPP) and associated ecosystem services such as food production and carbon sequestration. Frequently, experimental manipulations of precipitation have linked altered precipitation regimes to changes in NPP. Yet, findings have been diverse and substantial uncertainty still surrounds generalities describing patterns of ecosystem sensitivity to altered precipitation. Additionally, we do not know whether previously observed correlations between NPP and precipitation remain accurate when precipitation changes become extreme. We synthesized results from 83 case studies of experimental precipitation manipulations in grasslands worldwide. Here, we used meta-analytical techniques to search for generalities and asymmetries of aboveground NPP (ANPP) and belowground NPP (BNPP) responses to both the direction and magnitude of precipitation change. Sensitivity (i.e., productivity response standardized by the amount of precipitation change) of BNPP was similar under precipitation additions and reductions, but ANPP was more sensitive to precipitation additions than reductions; this was especially evident in drier ecosystems. Additionally, overall relationships between the magnitude of productivity responses and the magnitude of precipitation change were saturating in form. The saturating form of this relationship was likely driven by ANPP responses to very extreme precipitation increases, although there were limited studies imposing extreme precipitation change, and there was considerable variation among experiments. Finally, this highlights the importance of incorporating gradients of manipulations, ranging from extreme drought to extreme precipitation increases into future climate change experiments. Additionally, policy and land management decisions related to global change scenarios should consider how ANPP and BNPP responses may differ, and that ecosystem responses to extreme events might not be predicted from relationships found under moderate environmental changes.

54 ENVIRONMENTAL SCIENCES↗

Improved assessment of mangrove forests in Sundarbans East Wildlife Sanctuary using WorldView 2 and TanDEM-X high resolution imagery

Recent developments of remote sensing techniques which can capture both the structure and function of the ecosystem provide a more representative view of the landscape. These unique Earth observations were used to help improve traditional forestry surveys by providing species-specific land cover classes for mangrove forests in the Sundarbans East Wildlife Sanctuary. By combining optical data from WorldView2 (WV2; 2 m pixel) and a canopy height model derived using radar data from TanDEM-X (TDX; 12 m pixel), we identified nine mangrove and five non-mangrove classes by following an Iterative Self-Organizing Data Analysis Algorithm. Three dominant mangrove species accounted for nearly 50% of the sanctuary. Heritieria fomes disproportionately covered the largest area at 43%, overturning previous field-based estimates of Excoecaria agallocha dominance. E. agallocha and Sonneratia apetala, covered 3% and 1.47% of the sanctuary, respectively. Four mixed species classes were also identified with clear vegetation zonation patterns that trended toward species homogeneity with increasing distance from shore. The overall land cover accuracy (WV2: 89.33%; WV2-TDX: 89.89%), the Kappa Coefficient (WV2:0.88; WV2-TDX: 0.89) and change statistics between WV2 and WV2-TDX landcover classifications indicate that the WV2 imagery can separate mangrove community types without structural data. The combination of the land cover classifications and the canopy height model indicated that H. fomes were not only the most dominant forest but also, on average, the tallest (12.3 m) among the other eight mangrove types. Our large-scale mapping with high resolution optical and radar platforms can capture subtle changes in mangrove vegetation and canopy structural gradients more accurately and be used to monitor biodiversity changes and Aichi Biodiversity Targets and Indicators, which would contribute to biodiversity policy updating.

Md Mizanur Rahman↗

Hyperspectral leaf reflectance of grasses varies with evolutionary lineage more than with site

Abstract To predict ecological responses at broad environmental scales, grass species are commonly grouped into two broad functional types based on photosynthetic pathway. However, closely related species may have distinctive anatomical and physiological attributes that influence ecological responses, beyond those related to photosynthetic pathway alone. Hyperspectral leaf reflectance can provide an integrated measure of covarying leaf traits that may result from phylogenetic trait conservatism and/or environmental conditions. Understanding whether spectra‐trait relationships are lineage specific or reflect environmental variation across sites is necessary for using hyperspectral reflectance to predict plant responses to environmental changes across spatial scales. We measured hyperspectral leaf reflectance (400–2400 nm) and 12 structural, biochemical, and physiological leaf traits from five grass‐dominated sites spanning the Great Plains of North America. We assessed if variation in leaf reflectance spectra among grass species is explained more by evolutionary lineage (as captured by tribes or subfamilies), photosynthetic pathway (C 3 or C 4 ), or site differences. We then determined whether leaf spectra can be used to predict leaf traits within and across lineages. Our results using redundancy analysis ordination (RDA) show that grass tribe identity explained more variation in leaf spectra (adjusted R 2 = 0.12) than photosynthetic pathway, which explained little variation in leaf spectra (adjusted R 2 = 0.00). Furthermore, leaf reflectance from the same tribe across multiple sites was more similar than leaf reflectance from the same site across tribes (adjusted R 2 = 0.12 and 0.08, respectively). Across all sites and species, trait predictions based on spectra ranged considerably in predictive accuracies ( R 2 = 0.65 to <0.01), but R 2 was >0.80 for certain lineages and sites. The relationship between Vc max , a measure of photosynthetic capacity, and spectra was particularly promising. Chloridoideae, a lineage more common at drier sites, appears to have distinct spectra‐trait relationships compared with other lineages. Overall, our results show that evolutionary relatedness explains more variation in grass leaf spectra than photosynthetic pathway or site, but consideration of lineage‐ and site‐specific trait relationships is needed to interpret spectral variation across large environmental gradients.

Pau, Stephanie [Department of Geography University↗

IntraShuffler: A Privacy Preserving Framework for Heterogeneous DP Federated Learning

Heterogeneous Differential Privacy (HDP) in Federated Learning (FL) allows clients to select individual privacy budgets () according to institutional policies and data sensitivity. In practice, many HDP-FL systems employ -aware server aggregation to improve model utility by re-weighting client updates according to their declared privacy budgets. However, gradient updates in FL retain structural patterns induced by non-independent and identically-distributed (non-IID) data, and these additional signals exposed by -aware aggregation create new opportunities for inference by an honest-but-curious server. In this work, we first show that a server equipped with gradient denoising and surrogate modeling can mount a Privacy Inference Attack that infers distributional attributes of clients and links updates from the same client across training rounds, measured via surrogate inference accuracy and linkage success, under realistic knowledge constraints. The Shuffle-Model has been widely studied as a defense against such inference risks by anonymizing update sources, but it is fundamentally incompatible with HDP-FL -aware aggregation. To address this challenge, we propose IntraShuffler, a middleware defense framework designed for HDP-FL systems. IntraShuffler introduces a privacy-aware shuffling mechanism that groups clients into privacy-compatible buckets and performs parameter-level shuffling within each bucket to disrupt persistent gradient structure while preserving -aware aggregation. Experiments across four different datasets show that IntraShuffler reduces gradient recoverability by over 60% and decreases surrogate inference accuracy from 0.78 to 0.33 while maintaining comparable model utility across multiple FL aggregation rules.

Riya, Farhin Farhad [ORNL]↗

Long-term patterns of post-fire harvest diverge among ownerships in the Pacific West, U.S.A.

Abstract Post-fire harvest (PFH) is a forest management practice designed to salvage value from burned timber, mitigate safety hazards from dead trees, reduce long-term fuels, and prepare sites for replanting. Despite public controversy and extensive ecological research, little is known about how much PFH occurs on private and public lands in the U.S. Pacific West, or how practices changed with shifting forest policy and increasing area burned over the last three decades. We mapped PFH across 2.2M burned hectares in California, Oregon, and Washington between 1986-2017 and used time series intervention analysis to compare trends in area, rate (% of burned area harvested), and mean patch size between private (0.5M ha) and federal (1.6M ha) forest land and across a gradient of burn severity. Harvest rates varied by ownership (4.9% federal, 18.6% private, 8.0% overall), and practices evolved and diverged over the study period. PFH area and rate declined across all ownerships in the mid-1990s during a period of reduced fire activity. As area burned increased between the early 2000s and late 2010s, PFH area rebounded and surpassed late-1980s levels, while rates remained relatively low. On federal lands, PFH practices shifted in the early-to-mid 1990s towards lower rates (10.3% to 3.8%) and smaller patches (6.0 to 3.3 ha), following policy changes and increased litigation. PFH rates on federal lands decreased at all levels of burn severity, with the largest decreases (6.2% to 1.2%) in forests with low tree mortality (i.e. fire refugia). Conversely, private PFH rates and mean patch sizes more than doubled in forests burned at very low-to-moderate severity. Our results highlight how PFH practices have shifted with policy, socio-economic pressure, and increasing area burned over 31 years in the Pacific West. A similar area of post-fire harvest is now dispersed over larger fires, with practices diverging substantially between ownerships.

Zuspan, Aaron (ORCID:0000000315833710)↗

Modeling household online shopping demand in the U.S.: a machine learning approach and comparative investigation between 2009 and 2017

Despite the rapid growth of online shopping and research interest in the relationship between online and in-store shopping, national-level modeling and investigation of the demand for online shopping with a prediction focus remain limited in the literature. Here, this paper differs from prior work and leverages two recent releases of the U.S. National Household Travel Survey (NHTS) data for 2009 and 2017 to develop machine learning (ML) models, specifically gradient boosting machine (GBM), for predicting household-level online shopping purchases. The NHTS data allow for not only conducting nationwide investigation but also at the level of households, which is more appropriate than at the individual level given the connected consumption and shopping needs of members in a household. We follow a systematic procedure for model development including employing Recursive Feature Elimination algorithm to select input variables (features) in order to reduce the risk of model overfitting and increase model explainability. Among several ML models, GBM is found to yield the best prediction accuracy. Extensive post-modeling investigation is conducted in a comparative manner between 2009 and 2017, including quantifying the importance of each input variable in predicting online shopping demand, and characterizing value-dependent relationships between demand and the input variables. In doing so, two latest advances in machine learning techniques, namely Shapley value-based feature importance and Accumulated Local Effects plots, are adopted to overcome inherent drawbacks of the popular techniques in current ML modeling. The modeling and investigation are performed at the national level, with a number of findings obtained. The models developed and insights gained can be used for online shopping-related freight demand generation and may also be considered for evaluating the potential impact of relevant policies on online shopping demand.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Cold Neutrons Trapped in External Fields

The properties of inhomogeneous neutron matter are crucial to the physics of neutron-rich nuclei and the crust of neutron stars. Advances in computational techniques now allow us to accurately determine the binding energies and densities of many neutrons interacting via realistic microscopic interactions and confined in external fields. We perform calculations for different external fields and across several shells to place important constraints on inhomogeneous neutron matter, and hence the large isospin limit of the nuclear energy density functionals that are used to predict properties of heavy nuclei and neutron star crusts. We find important differences between microscopic calculations and current density functionals; in particular, the isovector gradient terms are significantly more repulsive than in traditional models, and the spin-orbit and pairing forces are comparatively weaker.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Unifying soil organic matter formation and persistence frameworks: the MEMS model

Soil organic matter (SOM) dynamics in ecosystem-scale biogeochemical models have traditionally been simulated as immeasurable fluxes between conceptually defined pools. This greatly limits how empirical data can be used to improve model performance and reduce the uncertainty associated with their predictions of carbon (C) cycling. Recent advances in our understanding of the biogeochemical processes that govern SOM formation and persistence demand a new mathematical model with a structure built around key mechanisms and biogeochemically relevant pools. Here, we present one approach that aims to address this need. Our new model (MEMS v1.0) is developed from the Microbial Efficiency-Matrix Stabilization framework, which emphasizes the importance of linking the chemistry of organic matter inputs with efficiency of microbial processing and ultimately with the soil mineral matrix, when studying SOM formation and stabilization. Building on this framework, MEMS v1.0 is also capable of simulating the concept of C saturation and represents decomposition processes and mechanisms of physico-chemical stabilization to define SOM formation into four primary fractions. After describing the model in detail, we optimize four key parameters identified through a variance-based sensitivity analysis. Optimization employed soil fractionation data from 154 sites with diverse environmental conditions, directly equating mineral-associated organic matter and particulate organic matter fractions with corresponding model pools. Finally, model performance was evaluated using total topsoil (0–20 cm) C data from 8192 forest and grassland sites across Europe. Despite the relative simplicity of the model, it was able to accurately capture general trends in soil C stocks across extensive gradients of temperature, precipitation, annual C inputs and soil texture. The novel approach that MEMS v1.0 takes to simulate SOM dynamics has the potential to improve our forecasts of how soils respond to management and environmental perturbation. Ensuring these forecasts are accurate is key to effectively informing policy that can address the sustainability of ecosystem services and help mitigate climate change.

54 ENVIRONMENTAL SCIENCES↗

A Comparison of Machine Learning Methods for Frequency Nadir Estimation in Power Systems: Preprint

An increasing penetration level of inverter-based renewable energy resources changes the inertia of power systems, posing challenges for maintaining the desired system frequency stability. An accurate frequency nadir estimation is crucial for power system operators to prepare preventive actions against large frequency excursions. In this paper, five machine learning methods - linear regression, gradient boosting, support vector regression, an artificial neural network, and XGBoost - are applied to two different sets of preprocess data for the prediction of the frequency nadir in the Western Electricity Coordinating Council 240-bus system with high renewable penetration levels. The training and testing data sets are collected by extensive generation scheduling simulations on the Multi-timescale Integrated Dynamic and Scheduling (MIDAS) toolbox. Numerical results show that all five machine learning methods can achieve high performance accuracy for power system nadir frequency estimation. Among them, the gradient boosting and the XGBoost are clear winners by providing the best prediction accuracy.

data driven↗

Evaluating WRF-Chem aerosol indirect effects in Southeast Pacific marine stratocumulus during VOCALS-REx

We evaluate a regional-scale simulation with the WRF-Chem model for the VAMOS (Variability of the American Monsoon Systems) Ocean-Cloud-Atmosphere-Land Study Regional Experiment (VOCALS-REx), which sampled the Southeast Pacific's persistent stratocumulus deck. Evaluation of VOCALS-REx ship-based and aircraft observations focuses on analyzing how aerosol loading affects marine boundary layer (MBL) dynamics and cloud microphysics. We compare local time series and campaign averaged longitudinal gradients, and highlight differences in model simulations with (W) and without wet (NW) deposition processes. The higher aerosol loadings in the NW case produce considerable changes in MBL dynamics and cloud microphysics, in accordance with the established conceptual model of aerosol indirect effects. These include increase in cloud albedo, increase in MBL and cloud heights, drizzle suppression, increase in liquid water content, and increase in cloud lifetime. Moreover, better statistical representation of aerosol mass and number concentration improves model fidelity in reproducing observed spatial and temporal variability in cloud properties, including top and base height, droplet concentration, water content, rain rate, optical depth (COD) and liquid water path (LWP). Together, these help to quantify confidence in WRF-Chem's modeled aerosol-cloud interactions, while identifying structural and parametric uncertainties including: irreversibility in rain wet removal; overestimation of marine DMS and sea salt emissions and accelerated aqueous sulfate conversion. Our findings suggest that WRF-Chem simulates marine cloud-aerosol interactions at a level sufficient for applications in forecasting weather and air quality and studying aerosol climate forcing, including the reliability required for policy analysis and geo-engineering applications.

54 ENVIRONMENTAL SCIENCES↗

Sulfur dioxide emissions in China and sulfur trends in East Asia since 2000.

With the rapid development of the economy, the sulfur dioxide (SO{sub 2}) emission from China since 2000 is of increasing concern. In this study, we estimate the annual SO{sub 2} emission in China after 2000 using a technology-based methodology specifically for China. From 2000 to 2006, total SO{sub 2} emission in China increased by 53%, from 21.7 Tg to 33.2 Tg, at an annual growth rate of 7.3%. Emissions from power plants are the main sources of SO{sub 2} in China and they increased from 10.6 Tg to 18.6 Tg in the same period. Geographically, emission from north China increased by 85%, whereas that from the south increased by only 28%. The emission growth rate slowed around 2005, and emissions began to decrease after 2006 mainly due to the wide application of flue-gas desulfurization (FGD) devices in power plants in response to a new policy of China's government. This paper shows that the trend of estimated SO{sub 2} emission in China is consistent with the trends of SO{sub 2} concentration and acid rain pH and frequency in China, as well as with the increasing trends of background SO{sub 2} and sulfate concentration in East Asia. A longitudinal gradient in the percentage change of urban SO{sub 2} concentration in Japan is found during 2000-2007, indicating that the decrease of urban SO{sub 2} is lower in areas close to the Asian continent. This implies that the transport of increasing SO{sub 2} from the Asian continent partially counteracts the local reduction of SO{sub 2} emission downwind. The aerosol optical depth (AOD) products of Moderate Resolution Imaging Spectroradiometer (MODIS) are found to be highly correlated with the surface solar radiation (SSR) measurements in East Asia. Using MODIS AOD data as a surrogate of SSR, we found that China and East Asia excluding Japan underwent a continuous dimming after 2000, which is in line with the dramatic increase in SO{sub 2} emission in East Asia. The trends of AOD from both satellite retrievals and model over East Asia are also consistent with the trend of SO2 emission in China, especially during the second half of the year, when sulfur contributes the largest fraction of AOD. The arrested growth in SO{sub 2} emissions since 2006 is also reflected in the decreasing trends of SO{sub 2} and SO{sub 4}{sup 2-} concentrations, acid rain pH values and frequencies, and AOD over East Asia.

54 ENVIRONMENTAL SCIENCES↗

Assessing carbon storage capacity and saturation across six central US grasslands using data–model integration

Abstract. Future global changes will impact carbon (C) fluxes and pools in most terrestrial ecosystems and the feedback of terrestrial carbon cycling to atmospheric CO2. Determining the vulnerability of C in ecosystems to future environmental change is thus vital for targeted land management and policy. The C capacity of an ecosystem is a function of its C inputs (e.g., net primary productivity – NPP) and how long C remains in the system before being respired back to the atmosphere. The proportion of C capacity currently stored by an ecosystem (i.e., its C saturation) provides information about the potential for long-term C pools to be altered by environmental and land management regimes. We estimated C capacity, C saturation, NPP, and ecosystem C residence time in six US grasslands spanning temperature and precipitation gradients by integrating high temporal resolution C pool and flux data with a process-based C model. As expected, NPP across grasslands was strongly correlated with mean annual precipitation (MAP), yet C residence time was not related to MAP or mean annual temperature (MAT). We link soil temperature, soil moisture, and inherent C turnover rates (potentially due to microbial function and tissue quality) as determinants of carbon residence time. Overall, we found that intermediates between extremes in moisture and temperature had low C saturation, indicating that C in these grasslands may trend upwards and be buffered against global change impacts. Hot and dry grasslands had greatest C saturation due to both small C inputs through NPP and high C turnover rates during soil moisture conditions favorable for microbial activity. Additionally, leaching of soil C during monsoon events may lead to C loss. C saturation was also high in tallgrass prairie due to frequent fire that reduced inputs of aboveground plant material. Accordingly, we suggest that both hot, dry ecosystems and those frequently disturbed should be subject to careful land management and policy decisions to prevent losses of C stored in these systems.

58 GEOSCIENCES↗

FREQUENCY OF SOLAR-LIKE SYSTEMS AND OF ICE AND GAS GIANTS BEYOND THE SNOW LINE FROM HIGH-MAGNIFICATION MICROLENSING EVENTS IN 2005-2008

We present the first measurement of the planet frequency beyond the 'snow line', for the planet-to-star mass-ratio interval -4.5 < log q < -2, corresponding to the range of ice giants to gas giants. We find (d{sup 2}N{sub pl})/(d log q d log s) = (0.36{+-}0.15) dex{sup -2} at the mean mass ratio q = 5 x 10{sup -4} with no discernible deviation from a flat (Oepik's law) distribution in log-projected separation s. The determination is based on a sample of six planets detected from intensive follow-up observations of high-magnification (A>200) microlensing events during 2005-2008. The sampled host stars have a typical mass M{sub host} {approx} 0.5 M {sub sun}, and detection is sensitive to planets over a range of planet-star-projected separations (s {sup -1}{sub max} R {sub E}, s{sub max} R {sub E}), where R {sub E} {approx} 3.5 AU(M{sub host}/M{sub sun}){sup 1/2} is the Einstein radius and s {sub max} {approx} (q/10{sup -4.3}){sup 1/3}. This corresponds to deprojected separations roughly three times the 'snow line'. We show that the observations of these events have the properties of a 'controlled experiment', which is what permits measurement of absolute planet frequency. High-magnification events are rare, but the survey-plus-follow-up high-magnification channel is very efficient: half of all high-mag events were successfully monitored and half of these yielded planet detections. The extremely high sensitivity of high-mag events leads to a policy of monitoring them as intensively as possible, independent of whether they show evidence of planets. This is what allows us to construct an unbiased sample. The planet frequency derived from microlensing is a factor 8 larger than the one derived from Doppler studies at factor {approx}25 smaller star-planet separations (i.e., periods 2-2000 days). However, this difference is basically consistent with the gradient derived from Doppler studies (when extrapolated well beyond the separations from which it is measured). This suggests a universal separation distribution across 2 dex in planet-star separation, 2 dex in mass ratio, and 0.3 dex in host mass. Finally, if all planetary systems were 'analogs' of the solar system, our sample would have yielded 18.2 planets (11.4 'Jupiters', 6.4 'Saturns', 0.3 'Uranuses', 0.2 'Neptunes') including 6.1 systems with two or more planet detections. This compares to six planets including one two-planet system in the actual sample, implying a first estimate of 1/6 for the frequency of solar-like systems.

79 ASTRONOMY AND ASTROPHYSICS↗

Optimal startup control of a jacketed tubular reactor.

The optimal startup policy of a jacketed tubular reactor, in which a first-order, reversible, exothermic reaction takes place, is presented. A distributed maximum principle is presented for determining weak necessary conditions for optimality of a diffusional distributed parameter system. A numerical technique is developed for practical implementation of the distributed maximum principle. This involves the sequential solution of the state and adjoint equations, in conjunction with a functional gradient technique for iteratively improving the control function.

Hahn, D. R.↗

MULTI-OBJECTIVE REINFORCEMENT LEARNING FOR LOW-THRUST TRANSFER DESIGN BETWEEN LIBRATION POINT ORBITS

Multi-Reward Proximal Policy Optimization (MRPPO) is a multi-objective reinforcement learning algorithm used to construct low-thrust transfers between periodic orbits in multi-body systems. Previous implementations of MRPPO have relied on a predefined reference transfer to successfully train each policy. In this paper, an algorithmic modification labeled the ‘moving reference’, is introduced to autonomously construct these reference trajectories during training. With this modification, MRPPO is used to recover various low-thrust transfers between two periodic orbits in the Earth-Moon circular restricted three-body problem to solve a multi-objective optimization problem. These results are then compared with the solutions recovered via a gradient descent optimization scheme to validate the performance of MRPPO with the moving reference modification.

Christopher J. Sullivan↗

MULTI-OBJECTIVE REINFORCEMENT LEARNING FOR LOW-THRUST TRANSFER DESIGN BETWEEN LIBRATION POINT ORBITS

Multi-Reward Proximal Policy Optimization (MRPPO) is a multi-objective reinforcement learning algorithm used to construct low-thrust transfers between periodic orbits in multi-body systems. Previous implementations of MRPPO have relied on a predefined reference transfer to successfully train each policy. In this paper, an algorithmic modification labeled the ‘moving reference’, is introduced to autonomously construct these reference trajectories during training. With this modification, MRPPO is used to recover various low-thrust transfers between two periodic orbits in the Earth-Moon circular restricted three-body problem to solve a multi-objective optimization problem. These results are then compared with the solutions recovered via a gradient descent optimization scheme to validate the performance of MRPPO with the moving reference modification

Christopher John Sullivan↗