Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Policy gradient”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Multi-objective Reinforcement Learning for Low-thrust Transfer Design Between Libration Point Orbits

Multi-Reward Proximal Policy Optimization (MRPPO) is a multi-objective rein- forcement learning algorithm used to construct low-thrust transfers between pe- riodic orbits in multi-body systems. Previous implementations of MRPPO have relied on a predefined reference transfer to successfully train each policy. In this paper, an algorithmic modification labeled the ‘moving reference’, is introduced to autonomously construct these reference trajectories during training. With this modification, MRPPO is used to recover various low-thrust transfers between two periodic orbits in the Earth-Moon circular restricted three-body problem to solve a multi-objective optimization problem. These results are then compared with the solutions recovered via a gradient descent optimization scheme to validate the performance of MRPPO with the moving reference modification.

Anderson, Rodney L.↗

Multi-objective Reinforcement Learning for Low-thrust Transfer Design Between Libration Point Orbits

Multi-Reward Proximal Policy Optimization (MRPPO) is a multi-objective rein- forcement learning algorithm used to construct low-thrust transfers between pe- riodic orbits in multi-body systems. Previous implementations of MRPPO have relied on a predefined reference transfer to successfully train each policy. In this paper, an algorithmic modification labeled the ‘moving reference’, is introduced to autonomously construct these reference trajectories during training. With this modification, MRPPO is used to recover various low-thrust transfers between two periodic orbits in the Earth-Moon circular restricted three-body problem to solve a multi-objective optimization problem. These results are then compared with the solutions recovered via a gradient descent optimization scheme to validate the performance of MRPPO with the moving reference modification.

Anderson, Rodney L↗

Optimizing process-based models to predict current and future soil organic carbon stocks at high-resolution

From hillslope to small catchment scales (< 50 km 2 ), soil carbon management and mitigation policies rely on estimates and projections of soil organic carbon (SOC) stocks. Here we apply a process-based modeling approach that parameterizes the MIcrobial-MIneral Carbon Stabilization (MIMICS) model with SOC measurements and remotely sensed environmental data from the Reynolds Creek Experimental Watershed in SW Idaho, USA. Calibrating model parameters reduced error between simulated and observed SOC stocks by 25%, relative to the initial parameter estimates and better captured local gradients in climate and productivity. The calibrated parameter ensemble was used to produce spatially continuous, high-resolution (10 m 2 ) estimates of stocks and associated uncertainties of litter, microbial biomass, particulate, and protected SOC pools across the complex landscape. Here, subsequent projections of SOC response to idealized environmental disturbances illustrate the spatial complexity of potential SOC vulnerabilities across the watershed. Parametric uncertainty generated physicochemically protected soil C stocks that varied by a mean factor of 4.4 × across individual locations in the watershed and a – 14.9 to + 20.4% range in potential SOC stock response to idealized disturbances, illustrating the need for additional measurements of soil carbon fractions and their turnover time to improve confidence in the MIMICS simulations of SOC dynamics.

54 ENVIRONMENTAL SCIENCES↗

Say where you sample: Increasing site selection transparency in urban ecology

Urban ecological studies have the potential to extend our understanding of socio-ecological systems beyond that of an individual city or region. Cross-comparative empirical work and synthesis are imperative to develop a general urban ecological theory. This can be achieved only if studies are replicable and generalizable. Transparency in methods reporting facilitates generalizability and replicability by documenting the decisions scientists make during the various steps of research design; this is particularly true for sampling design and selection because of their impact on both internal and external validity and the potential to unintentionally introduce bias. Three interdependent aspects of sample design are study sample selection (e.g., specific organisms, soils, or water), sample specification (measurement of specific variable of interest), and site selection (locations sampled). Of these, documentation of site selection—the where component of sample design—is underrepresented in the urban ecology literature. Using a stratified random sample of 158 papers from 12 major urban ecology journals, we investigated how researchers selected study sites in urban ecosystems and evaluated whether their site selection methods were transparent. We extracted data from these papers using a 50-question, theory-based questionnaire and a multiple-reviewer approach. Our sample represented almost 45 years of urban ecology research across 40 different countries. We found that more than 80% of the papers read were not transparent in their site selection methodology. We do not believe site selection methods are replicable for 70% of the papers read. Key weaknesses include incomplete descriptions of populations and sampling frames, urban gradients, sample selection methods, and property access. Low transparency in reporting the where methodology limits urban ecologists’ ability to assess the internal and external validity of studies’ findings and to replicate published studies; it also limits the generalizability of existing studies. The challenges of low transparency are particularly relevant in urban ecology, a field where standard protocols for site selection and delineation are still being developed. These limitations interfere with the fields’ ability to build theory and inform policy. We conclude by offering a set of recommendations to increase transparency, replicability, and generalizability.

54 ENVIRONMENTAL SCIENCES↗

Incorporating Stakeholder Decision Support Needs into an Integrated Regional Earth System Model

A new modeling effort exploring the opportunities, constraints, and interactions between mitigation and adaptation at regional scale is utilizing stakeholder engagement in an innovative approach to guide model development and demonstration, including uncertainty characterization, to effectively inform regional decision making. This project, the integrated Regional Earth System Model (iRESM), employs structured stakeholder interactions and literature reviews to identify the most relevant adaptation and mitigation alternatives and decision criteria for each regional application of the framework. The information is used to identify important model capabilities and to provide a focus for numerical experiments. This paper presents the stakeholder research results from the first iRESM pilot region. The pilot region includes the Great Lakes Basin in the Midwest portion of the United States as well as other contiguous states. This geographic area (14 states in total) permits cohesive modeling of hydrologic systems while also providing gradients in climate, demography, land cover/land use, and energy supply and demand. The results from the stakeholder research indicate that iRESM should prioritize addressing adaptation alternatives in the water resources, urban infrastructure, and agriculture sectors, such as water conservation, expanded water quality monitoring, altered reservoir releases, lowered water intakes, urban infrastructure upgrades, increased electric power reserves in urban areas, and land use management/crop selection changes. Regarding mitigation alternatives, the stakeholder research shows a need for iRESM to focus on policies affecting the penetration of renewable energy technologies, and the costs and effectiveness of energy efficiency, bioenergy production, wind energy, and carbon capture and sequestration.

54 ENVIRONMENTAL SCIENCES↗

Addressing Technical and Policy Challenges for Remediation of Metals and Radionuclides within Complex Deep Vadose Zone Environments - 17028

Deep vadose zone contamination is a significant issue in many regions of the world. Although remediation of deep vadose zone contamination is important, direct contact with the contamination does not threaten human health or the environment due to its depth in the vadose zone. Yet, movement of contamination from the deep vadose zone to the groundwater creates the potential for exposure and risk to receptors. Therefore, the challenge of deep vadose zone remediation is to maintain a minimal rate of contaminant transport such that flux from the contaminated vadose zone is limited to protect groundwater resources and down-gradient receptors. We herein review major processes for viable remediation of deep vadose zone metal and radionuclides that form the practical constraints on remedial actions and provide an overview of how these principles are being practiced to address key contaminants at the U.S. Department of Energy Hanford Site. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Assessment of New Approaches in Geothermal Exploration Decision Making: Preprint

Geothermal exploration projects have significant amount of risk associated with uncertainties encountered in the discovery of the geothermal resource. Understanding when and how to proceed in an exploration program, and when to walk away from a site, are two of the largest challenges for increased geothermal deployment. Current methodologies for exploration decision making is left to subjective by subjective expert opinion which can be incorrectly biased by expertise (e.g. geochemistry, geophysics), geographic location of focus, and the assumed conceptual model. The aim of this project is to develop a methodology for more objective geothermal exploration decision making at a given location, including go-no-go decision points to help developers and investors decide when to give up on a location. In this scope, two different approaches are investigated: 1) value of information analysis (VOIA) which is used for evaluating and quantifying the value of a data before they are purchased, and 2) enthalpy-based exploration targeting based on reservoir size, temperature gradient estimates, and internal rate of return (IRR). The first approach, VOIA, aims to identify the value of a particular data when making decisions with an uncertain outcome. This approach targets the pre-drilling phase of exploration. These estimated VOIs are highly affected by the size of the project and still have a high degree of subjectivity in assignment of probabilities. The second approach, exploration targeting, is focused on decision making during the drilling phase. It starts with a basic geothermal project definition that includes target and minimum required production capacity and initial budgeting for exploration phases. Then, it uses average temperature gradient, reservoir temperature estimates, and production capacity to define targets and go/no-go limits. The decision analysis in this approach is based on achieving a minimum IRR at each phase of the project. This second approach was determined to be less subjective, since it requires less subjectivity in the input values.

15 GEOTHERMAL ENERGY↗

Electricity Market Dynamics as Smooth Games: Organizing Gradient-Based Learners and Identifying Instabilities

Wide-spread adoption of machine learning and distributed energy production has the potential to fundamentally change behaviours of energy markets. In practice, along with the benefits, previously unseen instabilities have begun emerging in markets currently embracing these technologies. In this report we study electricity markets as dynamical systems arising in smooth games, where energy consumers are modelled as gradient-based learners who interact on a graph through distributed energy producers. We give simple examples of how instabilities can arise even when the consumers’ local objectives seem reasonable and how these instabilities can be remedied. Drawing insights from these examples, we begin developing a general model of optimization-driven electricity market dynamics and give conditions for ensuring local Nash equilibria of the system are stable fixed points of the dynamics. Our results demonstrate how models of electricity market dynamics might be used for deriving formal guarantees in order to stabilize markets and enact regulatory policies in the face of market shocks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A Market Feedback Framework for Improved Estimates of the Arbitrage Value of Energy Storage Using Price-Taker Models

Price-taker (PT) models are often used to assess the potential value or revenue of energy arbitrage opportunities for energy storage in wholesale markets. But as greater amounts of energy storage are deployed on the grid, current PT models fail to predict the effects that energy storage itself can have on market prices. This can lead to an overestimation of the economic value of storage and an inability to capture price suppression. In this paper, we propose the use of a modified PT model to simulate the impact of increased storage deployment on energy prices and the resulting impact on revenue. Our method uses a gradient-boosting regressor to estimate the impact on prices, and we apply our method on historical price data from the PJM and California Independent System Operator wholesale markets. We use this approach to explore possible causes of electricity price suppression that occur from storage capacity additions, which is generally not possible with PT models.

arbitrage↗

Supplementary active stabilization of nonrigid gravity gradient satellites

The use of active control for stability augmentation of passive gravity gradient satellites is investigated. The reaction jet method of control is the main interest. Satellite nonrigidity is emphasized. The reduction in the Hamiltonian H is used as a control criteria. The velocities, relative to local vertical, of the jets along their force axes are shown to be of fundamental significance. A basic control scheme which satisfies the H reduction criteria is developed. Each jet is fired when its velocity becomes appropriately large. The jet is de-energized when velocity reaches zero. Firing constraints to preclude orbit alteration may be needed. Control is continued until H has been minimized. This control policy is investigated using impulse and rectangular pulse models of the jet outputs.

Keat, J. E.↗

The temporal and spatial distribution of carbon dioxide emissions from fossil-fuel use in North America

Refinements in the spatial and temporal resolution of North American fossil-fuel carbon dioxide (CO{sub 2}) emissions provide additional information about anthropogenic aspects of the carbon cycle. In North America, the seasonal and spatial patterns are a distinctive component to characterizing anthropogenic carbon emissions. The pattern of fossil-fuel-based CO{sub 2} emissions on a monthly scale has greater temporal and spatial variability than the flux aggregated to the national annual level. For some areas, monthly emissions can vary by as much as 85% for some fuels when compared with monthly estimates based on a uniform temporal and spatial distribution. The United States accounts for the majority of North American fossil carbon emissions, and the amplitude of the seasonal flux in emissions in the United States is greater than the total mean monthly emissions in both Canada and Mexico. Nevertheless, Canada and Mexico have distinctive seasonal patterns as well. For the continent, emissions were aggregated on a 5{sup o} x 10{sup o} latitude-longitude grid. The monthly pattern of emissions varies on both a north-south and east-west gradient and evolves through the time period analyzed (1990-2007). For many areas in North America, the magnitude of the month-to-month variation is larger than the total annual emissions from land use change, making the characterization of emissions patterns essential to understanding humanity's influence on the carbon cycle.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Constructing the Pareto front for multi-objective Markov chains handling a strong Pareto policy approach

In this paper, we present a multi-objective solution of the Pareto front for a certain class of discrete-time ergodic controllable Markov chains. For solving the problem, we transform the original multi-objective problem into an equivalent nonlinear programming problem implementing the Lagrange principle. We then propose to adopt the Tikhonov’s regularization method to ensure the convergence of the cost functions to a unique point into the Pareto front. We show that Pareto policies are characterized as optimal policies. One of the fundamental problems related to the construction of the Pareto front is the existence and characterization of both, Pareto and strong Pareto policies. By introducing the Tikhonov’s regularizator, we ensure the existence of strong Pareto policies. This paper proposes a real solution to this problem. We formulate the original problem considering several constraints: (a) we employ the c-variable method for the introduction of linear constraints over the nonlinear problem and (b) restrict the cost functions, allowing points in the Pareto front to have a small distance from one another. The constraints imposed by the c-variable method make the problem computationally tractable and the restriction imposed by the small distance change ensures the continuation of the Pareto front. The resulting equation in this nonlinear system is an optimization problem for which the necessary and efficient condition of a minimum is solved using the projected gradient method. Moreover, we also prove the convergence conditions and compute the estimate rate of convergence of variables corresponding to the Lagrange principle and the Tikhonov’s regularization, respectively. We provide all the details needed to implement the proposed method in an efficient way. The usefulness of the method is successfully demonstrated by a numerical example.

97 MATHEMATICS AND COMPUTING↗

Real-Time Fine Grained Occupancy Estimation Using Depth Sensors on ARM Embedded Platforms

Occupancy estimation is an important primitive for a wide range of applications including building energy efficiency, safety, and security. In this paper, we explore the potential of using depth sensors to detect, estimate, identify, and track occupants in buildings. While depth sensors have been widely used for human detection and gesture recognition, computer vision algorithms are typically run on a powerful computer like XBOX or Intel R CoreTM i7 processor. In this work, we develop a prototype system called FORK using off-the-shelf components that performs the entire depth data processing on a cheaper and low power ARM processor in real-time. As ARM processors are extremely weak in running computer vision algorithms, FORK is designed to detect humans and track them in a very efficient way by leveraging a novel lightweight model based approach instead of traditional approaches based on histogram of oriented gradients (HOG) features. Unlike other camera based approaches, FORK is much less privacy invasive (even if the sensor is compromised). Based on a complete implementation, real-world deployment, and extensive evaluation at realistic scenarios, we observe that FORK achieves over 99% accuracy in real-time (4-9 FPS) in occupancy estimation.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Data-Driven Wind Farm Optimization Incorporating Effects of Turbulence Intensity

We demonstrate assimilation of field data from a nacelle-mounted lidar and meteorological tower into a medium-fidelity Reynolds-Averaged Navier Stokes wind farm flow model to better predict the effects of atmospheric stability. Increased predictability under a variety of atmospheric conditions can lead to more effective control design and optimization of a wind farm. In particular, atmospheric stability affects wind turbine wake propagation and, therefore, aspects of wind farm control and performance, such as active wind farm control, layout optimization, and power output. Accurately modeling wakes in different stability conditions remains a persistent challenge. This paper presents an optimization framework that leverages high-fidelity field or simulation data to correct a lower-fidelity flow model. Optimal model corrections are found by solving a regularized, high-dimensional, gradient-based optimization problem using adjoint flowfield information. We validate the trained model against large eddy simulation results, and perform separate gradient-based layout optimizations of a simulated utility-scale wind farm to maximize power. Using the data-driven model corrections, we find that atmospheric stability significantly impacts layout optimization and power production: the optimal layout for stable conditions produced 9.1% more power than the optimal layout for unstable conditions, and the optimal layout for neutral conditions underperformed by 8.5% in unstable conditions.

17 WIND ENERGY↗

INTEX-NA: Intercontinental Chemical Transport Experiment - North America

INTEX-NA is an integrated atmospheric chemistry field experiment to be performed over North America using the NASA DC-8 and P-3B aircraft as its primary platforms. It seeks to understand the exchange of chemicals and aerosols between continents and the global troposphere. The constituents of interest are ozone and its precursors (hydrocarbons, NOX and HOX), aerosols, and the major greenhouse gases (CO2, CH4, N2O). INTEX-NA will provide the observational database needed to quantify inflow, outflow, and transformations of chemicals over North America. INTEX-NA is to be performed in two phases. Phase A will take place during the period of May-August 2004 and Phase B during March-June 2006. Phase A is in summer when photochemistry is most intense and climatic issues involving aerosols and carbon cycle are most pressing, and Phase B is in spring when Asian transport to North America is at its peak. INTEX-NA will coordinate its activities with concurrent measurement programs including satellites (e. g. Terra, Aura, Envisat), field activities undertaken by the North American Carbon Program (NACP), and other U.S. and international partners. However, it is being designed as a 'stand alone' mission such that its successful execution is not contingent on other programs. Synthesis of the ensemble of observation from surface, airborne, and space platforms, with the help of global/regional models is an important It is anticipated that approximately 175 flight hours for each of the aircraft (DC-8 and P-3B) will be required for each Phase. Principal operational sites are tentatively selected to be Bangor, ME; Wallops Island, VA; Seattle, WA; Rhinelander, WI; Lancaster, CA; and New Orleans, LA. These coastal and continental sites can support large missions and are suitable for INTEX-NA objectives. The experiment will be supported by forecasts from meteorological and chemical models, satellite observations, surface networks, and enhanced O3,-sonde releases. In addition to characterizing Atlantic-outflow and Pacific-inflow, INTEX-NA will characterize air masses transported between the U.S., Canada, and Mexico. INTEX-NA will be the first continental scale inflow, outflow, and transformation experiment to be performed over North America. It will provide the most comprehensive observational data set to date to understand the O3/NOX/HOX/aerosol photochemical system and the carbon cycle. One of the critical needs of the carbon cycle research is to obtain large-scale vertical and horizontal concentration gradients of CO2, throughout the troposphere over continental source/sink regions. INTEX-NA is ideally suited to perform this role. Coastal and continental operational sites will allow us to develop a curtain profile of greenhouse gases (e. g. CO2,) and other key pollutants across North America. Such information is central to our quantitative understanding of chemical budgets on the continental scale. We expect to provide a number of satellite under-flights over land and water to test and validate observations from the appropriate satellite platform (e. g. Aura). We plan to develop strong collaborations with other national and international observational programs. Results from INTEX-NA should directly benefit the development of environmental policy for air quality and climate change.

Singh, Hanwant B.↗

University of Hawai‘i, Shallow Geothermal Resources: Energy Technology Innovation Partnership Project (Final Report)

Scientists at Lawrence Berkeley National Laboratory (Berkeley Lab) have teamed up with the University of Hawai‘i at Manoa (UH Manoa) through the U.S. Department of Energy’s Energy Technology Innovation Partnership Project to evaluate the technological and market feasibility of shallow geothermal heat exchanger (GHE) technology. UH requested this analysis to evaluate opportunities in building cooling, energy efficiency, and emissions reduction applications in Hawai‘i. UH has an abundance of geologic and geothermal data and is looking to the national labs’ expertise to execute this analysis. UH is also interested in investigating policy, regulatory, and business conditions advantageous for implementation of a pilot project and more broad deployment of this technology in Hawai‘i. In many locations around the world, the demands for heating and cooling are roughly balanced over the course of the year, so GHEs do not cause significant long-term changes in subsurface temperature. This is not the case in Hawai’i, where the demand for heating is very small, meaning that, over time, GHEs will add heat to the subsurface. If temperatures increase significantly, GHE systems will not work as designed. Regional groundwater flow has the potential to sweep heated water away from boreholes, thereby maintaining the functionality of the GHE system. Significant regional groundwater flow requires two things: a sufficiently large driving hydraulic head gradient (usually closely related to surface topography), and sufficient porosity and permeability to enable groundwater to flow in large enough quantities to enable near-borehole temperatures to be maintained at ambient values. Hawai‘i’s volcanic terrain offers ample surface topographic variation. The lava itself shows an extremely large range of porosity and permeability, so sites with large enough values of these properties must be selected. Numerical modeling of coupled groundwater and heat flow can be used to determine how large is large enough. Primarily, closed-loop systems have been investigated. Other options considered are open-loop systems and using cool seawater as the chilling source. Project work investigated the feasibility of GHE technology at two scales. At the island scale, GIS layers of various attributes relevant for GHE were combined to develop an overall favorability map for employing GHE in Hawai‘i. At the local scale, a hydrogeologic model for the subsurface component of a closed-loop system was developed for the Stan Sheriff Center at the UH Manoa campus. This site is considered promising because the rock below and immediately downgradient of the borefield is highly permeable, consisting of a subsurface karst system (limestone containing high-permeability open channels), which is underlain by a thick, high-permeability fractured basalt. Moreover, the site is near the base of the Ko‘olau Range, providing a large hydraulic head gradient. Thus, groundwater flow through the site is expected to be large, enabling efficient removal of heated groundwater. A full-GHE-system model of the site was also developed, with a simplified representation of the subsurface, in which groundwater flow is not considered and heat transfer is purely by conduction. Using the building cooling load data provided by UH, simulation results show that with groundwater flow present, a GHE can operate successfully for at least 10 years, but with no groundwater flow, the subsurface begins to heat up after only one year of operation, making the GHE unviable within 2-6 years. The team also developed a techno-economic model for this site to compare the cost of cooling using a GHE system with the costs of operating the current air-conditioning system. The GHE system is advantageous economically if favorable tax incentives and interest rates can be obtained.

15 GEOTHERMAL ENERGY↗

Shallow Geothermal Resources for Cooling Applications at the University of Hawai‘i

Scientists at Berkeley Lab have teamed up with The University of Hawai‘i at Manoa (UH Manoa) through the U.S. Department of Energy’s (DOE’s) Energy Transitions Initiative Partnership Project (ETIPP) to evaluate the technological and market feasibility of shallow geothermal heat exchanger (GHE) technology for building cooling, energy efficiency, and emissions reduction applications in Hawai’i. The team is assessing the data necessary to model the feasibility of deploying this technology, the actual models that will be used, and what hurdles need to be overcome to install a demonstration case. UH has an abundance of geologic and geothermal data and is looking to the national labs’ expertise to execute this analysis. UH is also interested in investigating policy, regulatory, and business conditions advantageous for implementation of a pilot project, and more broad deployment of this technology in Hawai‘i. In many locations around the world, the demands for heating and cooling are roughly balanced over the course of the year, so GHEs do not cause significant long-term changes in subsurface temperature. This is not the case in Hawai’i, where the demand for heating is very small, meaning that over time, GHEs will add heat to the subsurface. If temperatures increase significantly, GHE systems will not work as designed. Regional groundwater flow has the potential to sweep heated water away from boreholes, thereby maintaining the functionality of the GHE system. Significant regional groundwater flow requires two things: a sufficiently large driving hydraulic head gradient (usually closely related to surface topography), and sufficient porosity and permeability to enable groundwater to flow in large enough quantities to enable near-borehole temperatures to be maintained at ambient values. Hawai‘i’s volcanic terrain offers ample surface topographic variation. The lava itself shows an extremely large range of porosity and permeability, making it crucial to select sites with large enough values of these properties. Numerical modeling of coupled groundwater and heat flow can be used to determine how large is large enough. Both closed-loop and open-loop systems are being investigated. Another option being considered is using cool seawater as the source of chill. Currently work is progressing on two fronts. A hydrogeologic model for a closed-loop system is being developed for the Stan Sheriff Center at the UH Manoa campus, where a subsurface karst system immediately downgradient of the borefield may provide efficient removal of heated groundwater. The team will also develop a technoeconomic model for this site to compare the cost of cooling using a GHE system with the costs of operating the current air conditioning system. At the state scale, geographic information system (GIS) layers of various attributes relevant for GHE are being combined to develop an overall favorability map for employing GHE in Hawai‘i.

Doughty, C↗