Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed reinforcement learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

258 records · Page 15

Appurtenance Influence on Type III Hanford Single-Shell Tank Structural Integrity

The interim stabilized Hanford Single Shell Tanks (SSTs) are currently undergoing a state of the art analysis to assess the structural integrity of the waste storage tanks, for cleanup and closure operations, considering their adverse thermal histories and an updated seismic hazard for the Hanford Site near Richland, Washington. The SSTs contain a variety of ancillary pits, piping, piping supports, risers, equipment, and penetrations known as appurtenances. These appurtenances may alter the structural response and ultimately could affect the structural integrity of the SSTs. An important challenge to the structural analysis of the SSTs is determining the impact of these appurtenances on structural integrity. To achieve this, the various appurtenances were reviewed and bounding appurtenance configurations for SST Types II and III tank designs were analyzed using finite element software. The bounding configurations for the Type II tanks considered four heavy offset pits with a central pit with and without a 36-inch diameter central post-construction penetration and four 42-inch diameter offset penetrations. The bounding configuration for the Type III tanks is a tank with two heavy offset pits and one heavy central pit. For each bounding configuration two finite element models are developed: a seismic analysis model and a thermal and operating loads analysis (TOLA) model. The TOLA models include a Type II or III thermal history, concrete cracking and thermal degradation, reinforcement yielding, and soil plasticity. Additionally, operating loads such as internal waste pressure and concentrated and distributed soil surface loads are applied to the TOLA model. The seismic model treats the tank concrete as linear elastic based on the present day degraded concrete properties. Also, in the seismic model the soil is treated as linear elastic while special techniques are used in the soil above the tank dome and along the tank wall to avoid soil arching and achieve the proper soil pressure on the tank walls. Seismic time histories (in the horizontal and vertical directions) are applied to the seismic model. The American Concrete Institute (ACI) has code requirements for nuclear safety-related concrete structures (ACI-349-06) that are used to evaluate the structural integrity of the SSTs. ACI-349-06 recommends evaluating factored load combinations against reduced tank section capacities to account for both loading and material uncertainties. From both the TOLA and seismic models the structural demands (forces and moments) are extracted from sections throughout the tank under the appropriate load combinations. These demands are compared against the ACI-349-06 capacities at each of the sections. This ratio of demand to capacity is reported as a measure of structural integrity. The Type II and Type III appurtenances configurations are found to increase the demand to capacity ratios in local regions near the appurtenances. Away from the appurtenances the influence on structural integrity is minor. A comparison of the Type II and III results show that even though the Type II offset pits weight less, they have a larger impact on the structural integrity of the tanks due to their locations. Finally, lessons learned from the Type II and Type III appurtenance analysis, and their application to the more complex Type IV structural integrity analysis, will be discussed.

appurtenances↗

Future SDN-HPON Control Plane Architecture and Protocol for On-Demand Terabit End-to-End Extreme-Scale Science Applications

The UC Davis team conducted architecture, algorithm and experimental studies for next generation ultra-high-bandwidth optical networks in support of extreme-scale science applications. In particular, over the course of this 3-year project, the UC Davis team achieved the following main accomplishments: Designed a network architecture for automated multi-domain service provisioning: Designed information exchange schemes between autonomous systems (domains) Designed application programming interfaces to orchestrate resource reservation in data centers, high performance computation facilities, and communication networks belonging to different autonomous systems Experimentally assessed the multi-domain network architecture in a distributed field trial set-up connecting premises in three continents Designed a broker-based orchestrated multi-domain optical networking framework supporting observe-analyze-act cycle based cognitive inter-domain service provisioning: Designed an inter-domain operation mechanism, e.g., topology abstraction, for the broker plane and domain managers to collaboratively accomplish inter-domain service provisioning while respecting the autonomy of each administrative domain Designed an alien wavelength monitoring scheme and an artificial neural network based optical signal-to-noise ratio estimator for quality-of-transmission aware inter-domain service provisioning Developed a deep neural network based multi-domain traffic predictor and a cognitive routing and spectrum assignment algorithm to realize knowledge-based autonomous traffic engineering in multi-domain networks Implemented an optical signal-to-noise ratio estimator and traffic predictor with Google’s Tensor flow and assessed their performance using experimental and synthetic data Demonstrated the benefits of the proposed cognitive schemes with experiments ona two-domain elastic optical network testbed and evaluations on a six-domain simulation framework Built a two-domain software-defined networking control plane testbed using the ONOS platform and demonstrated the workflow of broker and domain manager cooperation, including hierarchical performance monitoring, topology abstraction and cognitive resource allocation. Designed self-learning cognitive service provisioning schemes using advanced machine learning technologies: Designed a self-taught anomaly detection framework for optical networks, including an unsupervised data clustering module for pattern recognition, a supervised data regression and classification module for online anomaly detection, and the system operation principle Experimentally validated the effectiveness of the self-taught anomaly detection framework with various failure/anomaly scenarios. Designed a self-learning autonomic routing, modulation and spectrum assignment (RMSA) provisioning framework for elastic optical networks with deep reinforcement learning. Designed effective modeling and training mechanisms for the self-learning RMSA framework Conducted extensive numerical simulations with various traffic patterns and topologies to verify the advantage of the self-learning RMSA framework.

42 ENGINEERING↗

Stopping criteria for ending autonomous, single detector radiological source searches

While the localization of radiological sources has traditionally been handled with statistical algorithms, such a task can be augmented with advanced machine learning methodologies. The combination of deep and reinforcement learning has provided learning-based navigation to autonomous, single-detector, mobile systems. However, these approaches lacked the capacity to terminate a surveying/search task without outside influence of an operator or perfect knowledge of source location (defeating the purpose of such a system). Two stopping criteria are investigated in this work for a machine learning navigated system: one based upon Bayesian and maximum likelihood estimation (MLE) strategies commonly used in source localization, and a second providing the navigational machine learning network with a “stop search” action. A convolutional neural network was trained via reinforcement learning in a 10 m × 10 m simulated environment to navigate a randomly placed detector-agent to a randomly placed source of varied strength (stopping with perfect knowledge during training). The network agent could move in one of four directions (up, down, left, right) after taking a 1 s count measurement at the current location. During testing, the stopping criteria for this navigational algorithm was based upon a Bayesian likelihood estimation technique of source presence, updating this likelihood after each step, and terminating once the confidence of the source being in a single location exceeded 0.9. A second network was trained and tested with similar architecture as the previous but which contained a fifth action: for self-stopping. The accuracy and speed of localization with set detector and source initializations were compared over 50 trials of MLE-Bayesian approach and 1000 trials of the CNN with self-stopping. The statistical stopping condition yielded a median localization error of ~1.41 m and median localization speed of 12 steps. The machine learning stopping condition yielded a median localization error of 0 m and median localization speed of 17 steps. This work demonstrated two stopping criteria available to a machine learning guided, source localization system.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Artificial Intelligence/Machine Learning Technology in Power System Applications

The primary purpose of this report is to provide an overview of the advancement in artificial intelligence and machine learning (AI/ML) technologies and their applications in power systems. It offers a foundation for understanding the transformative role of AI/ML in power systems and aims to stimulate further research and development in this area. This report begins with a historical perspective of AI/ML technologies, then explores their advancement to today’s prominence. The document highlights key contributors to the success of AI/ML technologies, including increased computational power, greater data availability, innovative algorithms, and advanced tools. It further introduces various AI/ML techniques, including supervised, unsupervised and reinforcement learning, graph neural networks, and generative AI. It also emphasizes the critical importance of ensuring the safety, security, and trustworthiness of these AI/ML techniques within this sector. The report reviews the recent representative advancements in various power system applications enhanced by AI/ML techniques, underscoring key developments and their transformative impact as evidenced by numerous studies. It also explores both the opportunities and challenges associated with the application of AI/ML technologies to improve power system applications. While the report extensively covers AI/ML applications in power systems, focusing primarily on the technical and operational aspects, it may not thoroughly explore the sociopolitical, economic, and broader regulatory implications of AI/ML integration in power systems. AI/ML techniques hold significant potential for enhancing power system applications; however, they are not omnipotent. It is crucial to acknowledge their limitations and understand that they may not be able to address all challenges in the power system domain. Various factors must be considered that influence the implementation, adoption, and effectiveness of AI/ML solutions, including but not limited to safety, security, transparency, and trustworthiness. Additionally, the incorporation of advanced human–machine interfaces is essential, as it enables humans to validate the effectiveness of AI/ML solutions while remaining actively engaged, fostering trust in AI/ML deployment. Finally, the report summarizes AI/ML research activities supported by the Department of Energy (DOE) Office of Electricity (OE) through the Advanced Grid Modeling (AGM) program. The work aligns with the interests and mission of DOE-OE AGM, with the report serving as a resource for identifying existing progress and for pinpointing future applications within AI/ML that need further exploration and support.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Computational Needs for the Next Generation Electric Grid Proceedings

The April 2011 DOE workshop, 'Computational Needs for the Next Generation Electric Grid', was the culmination of a year-long process to bring together some of the Nation's leading researchers and experts to identify computational challenges associated with the operation and planning of the electric power system. The attached papers provide a journey into these experts' insights, highlighting a class of mathematical and computational problems relevant for potential power systems research. While each paper defines a specific problem area, there were several recurrent themes. First, the breadth and depth of power system data has expanded tremendously over the past decade. This provides the potential for new control approaches and operator tools that can enhance system efficiencies and improve reliability. However, the large volume of data poses its own challenges, and could benefit from application of advances in computer networking and architecture, as well as data base structures. Second, the computational complexity of the underlying system problems is growing. Transmitting electricity from clean, domestic energy resources in remote regions to urban consumers, for example, requires broader, regional planning over multi-decade time horizons. Yet, it may also mean operational focus on local solutions and shorter timescales, as reactive power and system dynamics (including fast switching and controls) play an increasingly critical role in achieving stability and ultimately reliability. The expected growth in reliance on variable renewable sources of electricity generation places an exclamation point on both of these observations, and highlights the need for new focus in areas such as stochastic optimization to accommodate the increased uncertainty that is occurring in both planning and operations. Application of research advances in algorithms (especially related to optimization techniques and uncertainty quantification) could accelerate power system software tool performance, i.e. speed to solution, and enhance applicability for new and existing real-time operation and control approaches, as well as large-scale planning analysis. Finally, models are becoming increasingly essential for improved decision-making across the electric system, from resource forecasting to adaptive real-time controls to online dynamics analysis. The importance of data is thus reinforced by their inescapable role in validating, high-fidelity models that lead to deeper system understanding. Traditional boundaries (reflecting geographic, institutional, and market differences) are becoming blurred, and thus, it is increasingly important to address these seams in model formulation and utilization to ensure accuracy in the results and achieve predictability necessary for reliable operations. Each paper also embodies the philosophy that our energy challenges require interdisciplinary solutions - drawing on the latest developments in fields such as mathematics, computation, economics, as well as power systems. In this vein, the workshop should be viewed not as the end product, but the beginning of what DOE seeks to establish as a vibrant, on-going dialogue among these various communities. Bridging communication gaps among these communities will yield opportunities for innovation and advancement. The papers and workshop discussion provide the opportunity to learn from experts on the current state-of-the-art on computational approaches for electric power systems, and where one may focus to accelerate progress. It has been extremely valuable to me as I better understand this space, and consider future programmatic activities. I am confident that you too will enjoy the discussion, and certainly learn from the many experts. I would like to thank the authors of the papers for sharing their perspectives, as well as the paper discussants, session recorders, and participants. The meeting would not have been as successful without your commitment and engagement. I also would like to thank Joe Eto and Bob Thomas for their vision and leadership in bringing together such a well-structured and productive forum.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Investigating uncertainties in human adaptation and their impacts on water scarcity in the Colorado river Basin, United States

The Colorado River Basin (CRB) supports the water supply for seven states and forty million people in the Western United States (US) and has been suffering an extensive drought for more than two decades. As climate change continues to reshape water resources distribution in the CRB, its impact can differ in intensity and location, resulting in variations in human adaptation behaviors. The feedback from human systems in response to the environmental changes and the associated uncertainty is critical to water resources management, especially for water-stressed basins. This paper investigates how human adaptation affects water scarcity uncertainty in the CRB and highlights the uncertainties in human behavior modeling. Our focus is on agricultural water consumption, as approximately 80% of the water consumption in the CRB is used in agriculture. We adopted a coupled agent-based and water resources modeling approach for exploring human-water system dynamics, in which an agent is a human behavior model that simulates a farmer’s water consumption decisions. We examined uncertainties at the system, agent, and parameter levels through uncertainty, clustering, and sensitivity analyses. The uncertainty analysis results suggest that the CRB water system may experience 13 to 30 years of water shortage during the 2019–2060 simulation period, depending on the paths of farmers’ adaptation. The clustering analysis identified three decision-making classes: bold, prudent, and forward-looking, and quantified the probabilities of an agent belonging to each class. The sensitivity analysis results indicated agents whose decision-making models require further investigation and the parameters with the higher uncertainty reduction potentials. Here, by conducting numerical experiments with the coupled model, this paper presents quantitative and qualitative information about farmers’ adaptation, water scarcity uncertainties, and future research directions for improving human behavior modeling.

Agent-based modeling↗