Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “GAMES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57

The Long-Baseline Neutrino Facility [Slides]

The Long-Baseline Neutrino Facility is the infrastructure necessary to send a powerful beam of neutrinos 800 miles through the earth, and measure them deep underground at South Dakota’s Sanford Underground Research Facility (SURF). LBNF supports DUNE. The Deep Underground Neutrino Experiment will be a game-changing experiment for neutrino science, potentially transforming our understanding of why the universe exists as it does. The LBNF/DUNE project will be the first internationally conceived, constructed, and operated mega-science project hosted by the Department of Energy in the United States.

43 PARTICLE ACCELERATORS↗

Blackcomb 2: Hardware-Software Co-design for Non-Volatile Memory in Exascale Systems

Fast non-volatile memory (NVM) offers game-changing opportunities to rethink traditional system architectures; to address their energy inefficiencies; to leverage the new devices for greater performance; and to exploit the new capability of non-volatility to enhance system resilience, even in the face of larger system scale and degraded reliability of components. However, the devil is in the details, and to reap such potential benefits we must address challenges that arise when the contents of memory survive application failure, when software becomes responsible for managing the contents of CPU caches, and when memory accelerators blur the line between hardware and software. Building on the accomplishments of the Blackcomb Project, funded in 2010, we have identified, evaluated, and optimized promising NVM hardware and software technologies that are essential to provide the necessary memory capacity, performance, resilience, and energy efficiency in exascale systems.

97 MATHEMATICS AND COMPUTING↗

Integrated Cyber/Physical Grid Resiliency Modeling

This project explored coupling modeling and analysis methods from multiple domains to address complex hybrid (cyber and physical) attacks on mission critical infrastructure. Robust methods to integrate these complex systems are necessary to enable large trade-space exploration including dynamic and evolving cyber threats and mitigations. Reinforcement learning employing deep neural networks, as in the AlphaGo Zero solution, was used to identify "best" (or approximately optimal) resilience strategies for operation of a cyber/physical grid model. A prototype platform was developed and the machine learning (ML) algorithm was made to play itself in a game of 'Hurt the Grid'. This proof of concept shows that machine learning optimization can help us understand and control complex, multi-dimensional grid space. A simple, yet high-fidelity model proves that the data have spatial correlation which is necessary for any optimization or control. Our prototype analysis showed that the reinforcement learning successfully improved adversary and defender knowledge to manipulate the grid. When expanded to more representative models, this exact type of machine learning will inform grid operations and defense - supporting mitigation development to defend the grid from complex cyber attacks! This same research can be expanded to similar complex domains.

97 MATHEMATICS AND COMPUTING↗

Blackcomb2: Hardware-Software Co-design for Nonvolatile Memory in Exascale Systems

The impending shift from DRAM and Flash to fast nonvolatile memory (NVM) provides a game-changing opportunity to rethink traditional system architectures; to address their energy inefficiencies; to leverage the new devices for greater performance; and to exploit the new capability of nonvolatility to enhance system resilience, even in the face of larger system scale and degraded reliability of components. Building on the accomplishments of the Blackcomb Project, funded in 2010, we propose to identify, evaluate, and optimize the most promising NVM hardware and software technologies, which are essential to provide the necessary memory capacity, performance, resilience, and energy efficiency in exascale systems. Capacity and energy are the key drivers. Today's pressing application requirements mandate exascale computing; the datasets and concurrency of these applications demand memory capacity that cannot be met by traditional memory technologies. Although charge-based DRAM will endure, experts foresee only modest DRAM capacity gains. And, as moving data between levels of the hierarchy, and storing and accessing data on disk consumes significant energy, a more energy efficient solution is required. We, therefore, posit that exascale systems will need high density, energy-efficient storage technologies, such as nonvolatile memory for access, transformation, and management of exascale data. To address these issues, in this renewal request, we propose continue the work of our vertically- integrated, focussed team, where our co-design is informed by proxies and interactions with DOE co-design centers. In software, we start by assuming that nonvolatility requires new abstractions and new infrastructure, allowing persistent objects, and delivering performance gains for _le systems and robustness through fast checkpointing. In architecture, we evaluate the tradeoffs of performance and energy by employing a scalable and accurate simulation system to analyze realistic proxy applications. Through this co-design process, we rethink the whole memory architecture, from the cell to the array to the device, with ECC, wearout management, and interfaces. Finally, we provide open-source software models of these NVM-based memory systems.

97 MATHEMATICS AND COMPUTING↗

Using Machine Learning in Adversarial Environments

Cyber defense is an asymmetric battle today. We need to understand better what options are available for providing defenders with possible advantages. Our project combines machine learning, optimization, and game theory to obscure our defensive posture from the information the adversaries are able to observe. The main conceptual contribution of this research is to separate the problem of prediction, for which machine learning is used, and the problem of computing optimal operational decisions based on such predictions, coupled with a model of adversarial response. This research includes modeling of the attacker and defender, formulation of useful optimization models for studying adversarial interactions, and user studies to measure the impact of the modeling approaches in realistic settings.

97 MATHEMATICS AND COMPUTING↗

Biological Electron Transfer and Catalysis Energy Frontier Research Center

The overarching objective of the BETCy EFRC project is to investigate the mechanisms and structural basis controlling electron bifurcation, electron-ion coupling, and redox catalysis in model enzymes. This detailed understanding of how biological systems control and exploit electron flow will be used to develop modular biochemical conversions for the production of hydrocarbon and hydrogen biofuels. This project will conduct an in-depth study of metalloenzyme structure-function relationships that enable retuning of catalytic sites, electron-transfer thermodynamics, and the evolution of tailor-designed chemical reactions involving two model enzymes: hydrogenase and nitrogenase. The center is organized into three major thrust areas: Thrust 1. Energy conservation and electron flow; Thrust 2. ATP-coupled electron transfer; and Thrust 3. Atomic level determinants of enzymatic redox properties and their relationship to catalytic bias. We will develop a collective knowledge in metalloenzymes by applying physical science and computational tools to characterize biochemical reactions catalyzed specifically by multi-subunit enzymes harboring iron-sulfur clusters and flavin cofactors, as models for redox reactions throughout biology. Understanding these mechanisms is central to overcoming the key thermodynamic barriers currently limiting production of reduced products and biofuels. The research plan of the BETCy EFRC has been designed to provide the basis to exploit unique biochemical mechanisms that have yet to be explored substantively in the context of bioenergy but have the potential for game-changing advancement. The successful outcome of this research will be a collection of fundamental principles that serve as a blueprint to enable the tailored re-design of biological systems and enzymes to control matter and energy at the level of electrons and molecules to provide the foundations to create new energy technologies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

The 2019 Nonlinear Mechanics and Dynamics Research Institute

The 2019 Nonlinear Mechanics and Dynamics (NOMAD) Research Institute was successfully held from June 17 to August 1, 2019. NOMAD brings together participants with diverse technical backgrounds to work in small teams to cultivate new ideas and approaches in engineering mechanics and dynamics research. NOMAD provides an opportunity for researchers especially early career researchers - to develop lasting collaborations that go beyond what can be established from the limited interactions at their institutions or at annual conferences. A total of 20 students came to Albuquerque, New Mexico to participate in the seven-week long program held at the Mechanical Engineering building on the University of New Mexico campus. The students collaborated on one of seven research projects that were developed by various mentors from Sandia National Laboratories, the University of New Mexico, and academic institutions. In addition to the research activities, the students attended weekly technical seminars, various tours, and socialized at various off-hour events including an Albuquerque Isotopes baseball game. At the end of the summer, the students gave a final technical presentation on their research findings. Many of the research discoveries made at NOMAD are published as proceedings at technical conferences and have direct alignment with the critical mission work performed at Sandia.

42 ENGINEERING↗

Development and Integration of Predictive Models for Manufacturing and Structural Performance of Carbon Fiber Composites in Automotive Applications (Final Report)

The goal of this project is to develop integrated, state-of-the-art, computational modeling tools based on an integrated computational materials engineering (ICME) methodology that are critically needed to enable structural carbon fiber (CF) applications in automobiles. These tools are designed to predict the manufacturing and structural performance of CF composites, including stochastic effects. During the first phase of the project, the manufacturing and structural performance tools, including a stochastic driver, were developed, calibrated, and validated against coupon and component level test results. For this project, from the manufacturing side, the development efforts were focused on high-pressure resin transfer molding (HP-RTM), a potentially game-changing manufacturing technology that is capable of reaching the 3-5-minute cycle time that is necessary for volume manufacturing of composites. On the structural performance side, the crashworthiness of CF structures was studied. Accordingly, computational tools were developed and validated. The resulting differences between the numerical predictions and the experimental results in the first phase of the project were required to be less than 15%. During the second phase of the project, the manufacturing and performance tools were integrated by mapping the manufacturing outcomes (e.g., fiber angles, residual stresses, degree of cure, and defects) into the structural models. Also, using the tools developed in this project, an automotive assembly currently manufactured in steel (2016 GM Malibu) was redesigned to be manufactured using advanced CF composites with the objective of comparing the performance of the CF assembly with the developed ICME model predictions. The weight, performance and cost of the CF assembly were compared with the baseline steel assembly as well. The CF underbody design was shown to be 30% lighter than the corresponding steel design. The CF underbody assembly withstood the side pole impact with less than half the intrusion for the steel assembly. Excellent correlations within 10% for both peak impact load and intrusion were observed between the numerical predictions and experimental results. Based on the cost models developed in the project, the cost increase per Kg saved was determined to be $22. The project team has prepared a total of 28 publications in various journals and conferences and broadly communicated the project results for the benefit of the composite industry. The project team has also prepared a total of 10 patent applications that were submitted to the US Patent and Trademark office.

36 MATERIALS SCIENCE↗

Wildlife Management Plan for the Oak Ridge Reservation

This document outlines a plan for management of the wildlife resources on the US Department of Energy’s (DOE’s) Oak Ridge Reservation. Management includes wildlife population control through restoration of wildlife species; preservation, management, and enhancement of wildlife habitats; coordination of wildlife studies and characterization of areas, hunting, trapping, removal, and habitat manipulation; wildlife damage control; and law enforcement. Wildlife resources are divided into several categories, each with a specific set of objectives and procedures to attain the goals. These objectives are management of (1) wildlife habitats to ensure that all resident wildlife species exist on the Reservation in viable numbers; (2) focal species to promote healthy populations on designated land units; (3) game species for research, education, recreation, and public safety; (4) the Three Bend Scenic and Wildlife Management Refuge Area; (5) nuisance wildlife, including nonnative species, to achieve adequate population control for the maintenance of health and safety on the Reservation; (6) sensitive species (i.e., state or federally listed as Endangered, Threatened, Of Special Concern, or In Need of Management) through preservation and protection of both the species and habitats critical to the survival of those species; and (7) wildlife disease. Achievement of the objectives is a joint effort between Oak Ridge National Laboratory and the Tennessee Wildlife Resources Agency (TWRA) through agreements between DOE and the University of Tennessee (UT)-Battelle LLC, between TWRA and DOE, and between Tennessee Department of Environment and Conservation (TDEC) and DOE. Implementation of objectives is to be informed by leading-edge research and a strong collaborative approach between UT-Battelle and TWRA.

54 ENVIRONMENTAL SCIENCES↗

Towards Integrated Strategic Deterrence - 3rd Annual Cross-Domain Deterrence Seminar, 2016

Lawrence Livermore National Laboratory (LLNL) hosted the 3rd Annual Cross-Domain Deterrence (CDD) Seminar on November 15-17, 2016 in Livermore, CA. The seminar was sponsored by LLNL’s National Security Office (NSO) and Center for Global Security Research (CGSR). The primary purpose of the seminar was to utilize the crossdomain framework to work toward an approach for comprehensive and integrated strategic deterrence. The seminar focused on a tabletop exercise (TTX) to explore the dynamics of escalation and de-escalation in cyber space, outer space, and the nuclear domain. The TTX explored a scenario involving a NATO Article V defense of a Baltic ally. To help inform game “play,” the exercise was preceded by expert panel discussions about the CDD strategies of Russia and NATO. The exercise was followed by two panels on lessons learned and next steps, to help derive lessons from the TTX.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Medium-duty Urban Range Extended Connected Powertrain (Final Scientific Technical Report)

The project goal is to develop and demonstrate a Class 4 delivery vehicle that reduces fuel consumption by 50% or more when compared to an equivalent vehicle with a conventional internal combustion engine (ICE) powertrain driven on a comparable duty cycle. This shall be achieved using a plug-in hybrid electric (PHEV) configuration that optimizes the efficiency of the ICE. The proposed solution aims to be commercially viable for fleets to procure the system without additional incentives. To accomplish this, commercialized light-duty vehicle electric drive components shall be used. A PHEV configuration for Class 4 MD/HD vocational vehicles using commercialized light-duty vehicle electric drive components will require the development of a new topology for the hybrid powertrain applicable to medium-duty vehicles. Prior research indicates that an identified novel power split device could provide a fuel economy improvement potential of up to 51%. Utilizing an advanced battery management algorithm that will allow operation closer to the battery limits, the project will minimize battery capacity to achieve the desired performance results while maintaining the cost target for high commercialization. The solution must be cost effective to purchase and integrate into a Class 4 delivery vehicle, deliver the intended fuel economy improvement, and perform as well, or better, than the current conventional ICE powertrain. MD/HD vehicles typically require higher power, which is one factor that results in high costs for the powerful electric machines typical in larger HEV/PHEV systems. A key element in the project is the use of commercially available Bosch electric drive components (e.g., motors, batteries, and power electronics) from the larger passenger vehicle segment. This leverages the economies of scale to decrease cost and improve reliability beyond current purpose-built low-volume commercial vehicle components. These innovations will result in a game-changing PHEV system that meets DOE’s 50% fuel consumption reduction target, grants the vehicle an uncompromised driving range, and achieves the cost point for widespread adoption.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Testing the possibility of magnetic domain imaging based on circular & linear dichroism using photoemission electron microscopy

In recent years, an increasing number of memory and spintronic devices have been developed exploiting the combination of ferromagnetic (FM) and anti-ferromagnetic (aFM) materials. Consequently, magnetic imaging based on continuous-wave (CW) ultraviolet (UV) (λ = 266nm and longer wavelength) photoemission electron microscopy (PEEM) is gaining considerable attention due to the possibility of determining magnetizations for FM and aFM materials with 10 nm lateral resolution at video rate image acquisition. This PEEM-based approach exploits the polarization-dependent photoemission yield, which is subject to the polarization vector and the FM or aFM magnetization direction. Because of this unique attribute, magnetic imaging using PEEM when coupled to a laser with multiple illumination geometries allows for characterizing in-plane and out-of-plane magnetizations. This concept, however, has not been tested using a deep-UV laser (λ = 213nm), which has a much broader application space than the longer wavelength excitation used in previous reports. The purpose of this project in FY17 was to show the proof-of-concept of magnetic circular dichroism (MCD)-PEEM imaging using a λ = 210nm pulsed laser. Our results demonstrated the feasibility of in-plane and out-of-plane magnetic imaging with the limitations in the lateral resolution, data acquisition time, and signal-to-noise ratio anticipated for using a pulsed laser of moderate power. The project goal for FY18 is to construct the automated polarization-controlled data acquisition, and to establish the new lab facility in anticipation of acquiring a state-of-the-art high-power 213nm CW laser, planned to be installed in FY19. We successfully demonstrate the former by measuring dielectric stacks with polarization-dependent photoemission yield. Extrapolating from our result, we conclude that the capability of PEEM-based magnetic imaging using a CW deep UV laser could be a potential game-changer for scientific investigations and technological developments of magnetic materials and spintronic devices. In addition, polarization controlled PEEM imaging shows the potential for ellipsometry imaging of embedded nanomaterials exploiting their subtle differences in optical constants with respect to their surrounding dielectrics.

36 MATERIALS SCIENCE↗

The 2018 Nonlinear Mechanics and Dynamics Research Institute

The 2018 Nonlinear Mechanics and Dynamics (NOMAD) Research Institute was successfully held from June 18 to August 2, 2018. NOMAD brings together participants with diverse technical backgrounds to work in small teams to cultivate new ideas and approaches in engineering mechanics and dynamics research. NOMAD provides an opportunity for researchers -- especially early career researchers - to develop lasting collaborations that go beyond what can be established from the limited interactions at their institutions or at annual conferences. A total of 17 students came to Albuquerque, New Mexico to participate in the seven-week long program held at the Mechanical Engineering building on the University of New Mexico campus. The students collaborated on one of six research projects that were developed by various mentors from Sandia National Laboratories, University of New Mexico, and academic institutions. In addition to the research activities, the students attended weekly technical seminars, various tours, and socialized at various off-hour events including an Albuquerque Isotopes baseball game. At the end of the summer, the students gave a final technical presentation on their research findings. Many of the research discoveries made at NOMAD are published as proceedings at technical conferences and have direct alignment with the critical mission work performed at Sandia.

42 ENGINEERING↗

Veterans Commissioning Training Program for Commercial Healthcare Facilities [Final Report]

This project was designed to develop a building energy training curriculum, with hopes for creating immediate, long-term job opportunities for returning Iraq and Afghanistan veterans as energy commissioning agents within both public and private sectors. As part of this proposed effort, an online version of the curricula were developed in part using a modified version of the Association of Energy Engineers (AEE) Certified Building Commissioning Professional program. Case studies utilizing building information modeling (BIM) data from actual retrocommissioning projects and 3D gaming platform were developed as part of the curriculum. In addition to using materials from existing literature, the PIs also conducted scientific-based research on energy retrofitting related to building envelopes using numerical simulation approach. And the research findings and recommendation were incorporated in the developed curriculum. Currently, there are a lack of scientific-based guidance on envelope retrofitting practice. Many retrofitting practices were based on experiences, and are limited by the lack of scientific evidence. Existing retrofitting training material have limitations due to this missing piece. We have succeeded in producing some scientific evidence to guide and provide solutions to some building envelope design and retrofitting. The results from some of our research using computational fluid dynamics have demonstrated the feasibility of using our recommended solutions. Research results and findings were disseminated to the research and industry communities through conference papers (Appendix B) and presentations (Appendix C), and some results and findings were incorporated into the developed training curriculum (Appendix A).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

How AI Predicts the Untrained and Unseen

Focus Area: Model predictability improvements (Primary); Data optimization (secondary); Data complexity insights (secondary). The Scientific Challenge: If we believe that a future under extreme conditions will look very differently from today, we can likely agree that ML/AI models trained on past and present datasets will not be adequate to make reliable predictions into the future. This is true for water cycling, as well as biogeochemistry and other Earth system components and behaviors. Additionally, ML/AI models are inherently non-physical. Despite the flourishing success of ML/AI in many applications, such as computer vision, natural language process, and gaming, even the most sophisticated AI models don’t understand the very basic physical laws. Therefore, a natural question is: Can we trust ML/AI based predictions of Earth system behaviors that are fundamentally driven by physical laws? So, are physics models with meticulous process representation a better choice? Not exactly. Physical models, when firmly rooted in first principles, work great at predicting behaviors of systems with a well-defined set of boundary conditions and variables. However, as a complex system, the number of parameters and the degree of complexity and dynamics in processes, coupling, and scale dependent emergent behaviors make the Earth system behaviors very challenging to predict with physical models composed of deterministic laws. In addition, due to the lack of fundamental understandings, physical models often implement empirical correlations derived from observations with biases from locality of data generation. Because correlation is not necessarily causation or comply with first principles, scaling of model predictions beyond locality is often invalid. Beyond the limitation of models, physics or AI, our knowledge of the Earth system is limited by the lack of observational technologies and resources. Insufficient data density, dimensionality and diversity only offer a sliced (or projected) view of the Earth system, e.g. Plato’s Cave analogy, limiting our capability to better understand and represent fundamental processes in models.

54 ENVIRONMENTAL SCIENCES↗

The CanBikeCO Mini Pilot: Preliminary Results and Lessons Learned

In fall 2020, the Colorado Energy Office, as part of the State of Colorado’s “Can Do Colorado” initiative, initiated a project aimed at encouraging energy-efficient transportation during the COVID-19 pandemic. The initial mini-pilot provided e-bikes to 13 low-income households under an individual ownership model. This report assesses the impact of providing this additional mobility option on the travel behavior of participants. It also outlines the lessons learned from deploying a continuous monitoring platform to track the travel behavior. These lessons will influence the evaluation component for the full pilot, which will cover multiple geographic regions, starting in summer 2021, and run for 2 years. The continuous data collection was enabled by a customized version of the open-source e-mission platform, called CanBikeCO, configured with a behavioral gamification feature. The Colorado Energy Office used this system to collect a unique data set consisting of 3 months of partially automated travel diaries, combining sensed and surveyed data and linked with demographic information, from 12 participants. The data collection process worked well overall: users generally liked the app, appreciated the game, and did not complain about battery life. The long tracking period introduced behavioral challenges in user engagement, which we plan to address using repeated patterns and automated status checks for the full pilot. The analysis results, based on the subset of trips with user-reported labels (68%), indicate that the e-bike was the dominant commute mode share (31%), in sharp contrast to the census bicycle commute mode share (<1%). E-bike trips primarily replaced single-occupancy vehicle (SOV) trips (28%), followed closely by walking (24%) and regular bike (20%). The nonmotorized mode replacement corresponds to lower travel time and increased productivity enabled by the program. The emissions impact analysis of the program, computed using trip-level energy intensity factors, indicates savings of 1,367 lbs. of CO 2 . Although the results are strongly positive, the narrow demographic profile of study participants, their limited mobility alternatives, and nonuniform labeling indicate caution in broader interpretation. These preliminary results do suggest that such programs, supported by real-time education and support from program managers, can simultaneously meet equity and sustainability goals. The planned full pilot, addressing the data collection challenges and broadening the geographic scope, will provide additional insights into the generality of this approach.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Deployment of Community Scale Solar Photovoltaic Energy Systems in Forest County Potawatomi Community

FCPC is a federally recognized tribe with a membership of more than 1400. The Tribe has a reservation in Forest County, Wisconsin, and tribal trust lands in Milwaukee County, Wisconsin, with significant governmental and business operations in both areas. Until relatively recently, FCPC was one of the poorest tribes in the Midwest. Due to its successful gaming operations, it now has improved its financial situation and is using its resources to benefit both its members and the general public through environmental and other community efforts, many of which are long-delayed because of the extended period of abject poverty. The Tribe has developed the long-term goal of becoming energy independent using renewable resources. In order to meet this goal, the Tribe has taken several important steps. As an interim step, FCPC is purchasing nearly 55 million kilowatt hours of renewable energy certificates (“RECs”) annually from certified renewable energy facilities in the United States. These RECs offset more than 100% of the present electricity use at all of the Tribe’s facilities. The Tribe is currently ranked number 11 on the Environmental Protection Agency’s (EPA) Top 20 Local Government list of green power purchasers and is a member of the EPA’s Green Power Partnership. The original project suggested small scale installations on tribal homes and certain government buildings. The Tribe suggested that the project would result in 730 kw installed. By revising the project, the Tribe was able to install 938 kw and owns the largest installation in Northern Wisconsin with the 800 kW installation at the Potawatomi Carter Casino Hotel (“PCCH”). The PCCH installation, in summer months, can provide near 90% of the use during daylight hours. Although tracking was not immediately available FCPC has since confirmed significant production with certain quarterly savings nearing $210,000. These installations provide a reduction in facility costs providing more revenue for Tribal services including cultural, health, sustainable food sources, environmental preservation and other Tribal initiatives. Further, the installations have resulted in positive steps towards the Tribe’s goals of energy sovereignty and the use of 100% renewable clean energy. Finally, the PCCH project, specifically, has had the additional benefit of providing a likely area for a microgrid. The size of the installation and offset achieved have pushed the Tribe toward planning for storage and a microgrid system that would both further the Tribe’s energy goals and provide a safe haven for the Community during times of need.

14 SOLAR ENERGY↗

Optimizing and Extending the Functionality of EXARL for Scalable Reinforcement Learning [Slides]

The main goal of the Co-Design Summer School 2021 is to provide algorithmic improvements to EXARL framework by improving performance and by adding functionalities. This presentation includes an introduction to reinforcement learning and to EXARL. The researchers expanded the capability of EXARL by including additional agents like (Asynchronized) Advantage Actor Critic (A2C/A3C) and Twin Delayed Deep Deterministic Policy Gradient (TD3). They also explored algorithmic improvements such as v-trace and Prioritized Experience Replay. They found that A2C/A3C performed best with v-trace and outperformed Deep Q-Network (DQN) on both the CartPole game and the ExaBooster scientific environment. Additionally, they found that TD3 performed as good as the existing Deep Deterministic Policy Gradient (DDPG) agent and that adding Prioritized Experience Replay to DDPG accelerated convergence.

97 MATHEMATICS AND COMPUTING↗