Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “actor critic”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Sequence-Based Anomaly Detection in Critical Infrastructure Networks

United States critical infrastructure faces new cyber threats from adversarial nation-state actors in the form of malware-free attacks. Traditional cybersecurity techniques use rules-based methods to identify indicators of compromise on networks, often missing these sophisticated attacks. Our approach leverages multiple state of the art machine learning models in a pipeline to identify abnormal network events through sequential analysis. We combine both device and packet-level information into individual events to characterize anomalous network actions. The model is trained and tested on real network traffic from the Idaho National Lab High Performance Computing (HPC) with greater than 98% precision. It is capable of flagging malicious tactics used by adversaries in malware-free attacks, severe changes to the network, and abnormal user activity by network devices.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Investigate the Security of Electric Vehicle (EV) Ecosystem Applications

Apps that run on mobile devices are one of critical components of the electric vehicle (EV) ecosystem and pose possible threat actor points of entry that may impact the trust and security of EV charging systems in the future. Mobile apps often rely on communication between cloud servers and users, thereby creating potential points of entry for cyberattacks. Although app stores such as Apple App Store or Google Play Store generally test the security of apps, the cyber aspects may not be sufficient for many entities including DOD, federal fleets, and commercial entities. A more thorough inspection and the ability to influence developers is imminently needed. This research studies security attributes and vulnerabilities of a sample of mobile applications that support key user functions in the EV ecosystem. The study shows that all analyzed apps have security risks, categorized as either high or medium or both and a comprehensive cybersecurity guideline for developing mobile apps is necessary.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

A Typology for Characterizing Human Action in MultiSector Dynamics Models

The role of individual and collective human agency is increasingly recognized as a prominent and arguably paramount determinant in shaping the behavior, trajectory, and vulnerability of multisector systems. This human influence operates at multiple scales: from short-term (hourly to daily) to long-term (annually to centennial) timescales, and from the local to the global, pushing systems towards either desirable or undesirable outcomes. However, the effort to represent human systems in multisector models has been fragmented across philosophical, methodological, and disciplinary lines. To cohere insights across diverse modeling approaches, we present a new typology for classifying how human actors are represented in the broad suite of coupled human-natural system models that are applied in MultiSector Dynamics (MSD) research. The typology conceptualizes a “sector” as a system-of-systems that includes a diverse group of human actors, defined across individual to collective social levels, involved in governing, provisioning, and utilizing products, goods, or services towards some human end. We trace the salient features of modeled representations of human systems by organizing the typology around three key questions: 1) Who are the modeled actors in MSD systems? 2) What are their modeled actions? 3) How and for what purpose are these actors and actions operationalized in a computational model? The typology is used to critically examine existing models and chart the frontier of future human systems modeling for MSD research.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

The Value of Motorcycle Electrification in Mexico City

Local and city governments worldwide are looking to adopt emerging technologies and mobility approaches, such as electric vehicles, as they pursue their environmental and economic goals. Commercially available electric vehicles include buses, automobiles, motorcycles, and other two-wheelers. Electrifying transportation is critical to achieve urban sustainability, but it also raises challenges for private and public actors. Jurisdictions across countries are seeking to answer big questions, including how the transportation sector can be electrified, what impacts are priorities, and how such impacts can be obtained.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Domestic Extremism (Executive Summary)

Domestic extremism (DE) has been a growing concern in the U.S. in recent months, as illustrated in multiple bulletins from the Department of Homeland Security (DHS) warning law enforcement partners of the heightened threat. As concerns about these actors grows, it is important that facilities in the U.S. and internationally that protect critical assets, such as sensitive information, hazardous materials, or critical infrastructure, have effective methods in place to secure those assets. DE has challenged security systems through the danger of insider attack and violence, creating a new threat to be countered. In this effort, therefore, we used a literature review and focus group discussions with experts in critical asset security and extremism to understand the nature of the domestic extremist threat, to identify best practices in securing assets, recognize potential gaps in security measures to be corrected, and recommend actions for the Office of Radiological Security (ORS) to address DE with its partners.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2023

Distributed Power Allocation for 6-GHz Unlicensed Spectrum Sharing via Multi-agent Deep Reinforcement Learning

We consider the problem of power allocation over the 6 GHz Unlicensed National Information Infrastructure (UNII)- 5 spectrum. We propose a novel deep Reinforcement Learning (DRL)-based distributed power allocation scheme which utilizes the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. In particular, we model the base stations (BSs) as DRL agents that simultaneously determine the transmit powers to their scheduled user equipment (UE) in a synchronized manner. The power decision of each BS is based on its own observation of the radio environment, which consists of several local interference measurements and a limited amount of information obtained from other BSs. One advantage of the proposed scheme is that it addresses the single-agent non-stationarity problem of RL in the multi-agent scenario by incorporating the actions and observations of other BSs into each BS’s own critic which helps it to gain a more accurate perception of the overall radio environment. A centralized-training-distributed execution framework is used to train the policies where the critics are trained over the joint actions and observations of all BSs while the actor of each BS only takes the local observation as input in order to produce the transmit power. Simulation shows that the proposed power allocation scheme can achieve better throughput performance than several state-of-the-art approaches.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

Discovery of Probabilistic Dirichlet-to-Neumann Maps on Graphs

Dirichlet-to-Neumann maps enable the coupling of multiphysics simulations across computational subdomains by ensuring continuity of state variables and fluxes at artificial interfaces. We present a novel method for learning Dirichlet-to-Neumann maps on graphs using Gaussian processes, specifically for problems where the data obey a conservation law arising from an underlying partial differential equation. Our approach combines discrete exterior calculus and nonlinear optimal recovery to infer relationships between vertex and edge values. This framework yields data-driven predictions with uncertainty quantification across the entire graph, even when observations are limited to a subset of vertices and edges. By minimizing the reproducing kernel Hilbert space norm while penalizing kernel complexity through maximum likelihood estimation, our method ensures that the resulting surrogate strictly enforces conservation laws without overfitting. We demonstrate our method on two representative applications: subsurface flow in fracture networks and arterial blood flow. Finally, the results demonstrate that the method maintains high accuracy and well-calibrated uncertainty estimates even under severe data scarcity, highlighting its potential for scientific applications where limited data and reliable uncertainty quantification are critical.

Dirichlet-to-Neumann map↗
DOE OSTI2025

Enterprise Artificial Intelligence Strategy for Los Alamos National Laboratory

In the 1984 martial arts drama film, The Karate Kid, a young Daniel LaRusso is unexpectedly placed in an adversarial environment unable to eYectively adapt to a series of new threats and limitations. Fortunately for the main character, once placed under the tutelage of a Mr. Miyagi, he finds resiliency not through the adoption of new tools, but a re-focused set of fundamentals. Much in the same way that Daniel learns waxing on and buYing oY car wax by hand has rewards for Karate, LANL is choosing the harder path of self-hosting Large Language Models (LLMs) for enterprise use instead of only relying on buying access to a hosted AI service like Azure’s OpenAI Application Programming Interface (API). We also are not willing to wait for software-as-a-service (SAAS) AI services to meet us where we need to be from a FedRAMP accreditation standpoint. Our operations regularly depend on access at CUI, UCNI, ITAR and other FIPS-199 moderate-impact data levels and hosting our own services gives us the right security and compliance posture to be useful across the broad range of our work at LANL. With the rise in threats to critical infrastructure, cloud service providers (CSPs), and supply chain attacks from both state and non-state actors, we are not placing the bet that SAAS hosted AI services will be available when we need them. Should a major event occur, we do not want our staY and operations left without a pathway for us to fix the problem and resume the use of AI tools.

42 ENGINEERING↗
DOE OSTI2022

Cyber Attack Sequences Generation for Electric Power Grid

Security assessment of cyber-physical energy systems (CPESs) such as the electric power grid is a critical operation to maintain availability, reliability, and quality of service in the presence of persistent threats from malicious cyber actors. Existing security assessment approaches such as penetration testing and red teaming rely on subject matter expert experience and forensic cyber analysis of historical events to perform realistic, threat-informed assessments of CPES defense. CPESs have a large attack surface because of the heterogeneity and complexity of underlying topology, devices, measurements, and vulnerabilities. The aforementioned approaches lead to partial coverage of the attack surface with a large set of unknown but possible exploits. There is a need to automate the CPES attack surface discovery and contextualize it for relevant, highly probable, real-world attack scenarios. We propose a methodology and framework to facilitate the discovery of the CPES attack surface. We present a multilayer attack graph with ranked attack sequences to describe CPES failure scenarios. We present a work-in-progress framework that lists key components to automate the attack modeling and sequence generation. We demonstrate the published National Electric Sector Cybersecurity Organization Resource CPES failure scenario to highlight the trustworthiness of generated attack sequences.

Dutta, Ashutosh↗
DOE OSTI2023

Blockchain for Fault-Tolerant Grid Operations Version 2.0

This report explores the potential of distributed ledger technology (DLT) as a transformative tool to enhance fault-tolerant operations in electrical distribution systems. Leveraging DLT's core attributes, including an immutable decentralized ledger, distributed consensus mechanisms, and state replication capabilities, this study focuses on three critical use cases. A central aspect of this research centers on the utilization of a consensus-driven ledger, providing actors within the system, such as distributed resources, with access to a reliable data repository. This empowers these actors to collaborate effectively and make informed decisions, all securely recorded on the blockchain. The first use case concentrates on data configuration, utilizing mathematical criteria---particularly, the chi-squared test for gross error detection---to identify trustworthy sensors for advanced decision-making. Building upon this foundation of trust, the second use case, topology identification, accurately determines circuit breaker states, unveiling the distribution network's topology. Ultimately, the third use case leverages this trust to execute switching actions, reconfiguring feeders and restoring power to disconnected customers after fault events. The concept of trust serves as a cornerstone in this approach, marking a departure from traditional fault location, isolation, and service restoration (FLISR) methods. Additionally, the blockchain-based architecture introduces decentralization, empowering disconnected areas to make autonomous decisions, even when communication with a central control center is disrupted. The primary contributions of this report are twofold: (1) a novel approach for evaluating distribution system voltage areas while preserving data ownership and (2) the implementation of interactions between distribution network areas using the actor model. Unlike the previous sequential approach for evaluating the area connection voltages, which required a radial network topology, this study's area model reduction enables a more versatile approach. The area model reduction addresses issues of prolonged data waiting times and multiple points of failure within the previous approach. Notably, the presented evaluation for the reduced network model area connection reveals a significant increase in the differences in voltage magnitudes. Simulation and evaluation of area agents across four distinct cases elucidate the area-level interaction behavior during a fault event. Simulations demonstrate that the proposed distributed FLISR (DFLISR) approach can successfully restore service to an affected area. Varying message delays and message loss probabilities in each simulation case underscore their impacts on restoration times, ranging from 3 min and 32 s to 6 min and 19 s. In contrast, power is not restored in an area in one of our simulation cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Water narratives in local newspapers within the United States

Sustainable use of water resources continues to be a challenge across the globe. This is in part due to the complex set of physical and social behaviors that interact to influence water management from local to global scales. Analyses of water resources have been conducted using a variety of techniques, including qualitative evaluations of media narratives. This study aims to augment these methods by leveraging computational and quantitative techniques from the social sciences focused on text analyses. Specifically, we use natural language processing methods to investigate a large corpus (approx. 1.8M) of newspaper articles spanning approximately 35 years (1982–2017) for insights into human-nature interactions with water. Focusing on local and regional United States publications, our analysis demonstrates important dynamics in water-related dialogue about drinking water and pollution to other critical infrastructures, such as energy, across different parts of the country. Our assessment, which looks at water as a system, also highlights key actors and sentiments surrounding water. Extending these analytical methods could help us further improve our understanding of the complex roles of water in current society that should be considered in emerging activities to mitigate and respond to resource conflicts and climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Preliminary Benchmark Uncertainties for Deimos, a HALEU-Fueled and Graphite-Moderated Advanced Reactor Testbed

Many advanced reactor concepts will make use of various uranium fuels with levels of enrichment higher than previously seen in current light water reactors. In particular, High-Assay Low Enriched Uranium (HALEU), that is uranium enriched to 235 U ≈ 20 w/o%, is planned to be used in over ten new reactor concepts. HALEU is attractive for advanced reactors as it enables longer intervals between refueling. Unfortunately, little to no experience with HALEU is available in experimental literature raising concerns for not only licensing advanced re actors but also fabrication and transportation of HALEU fuels. This is where Deimos, a Los Alamos National Laboratory internal project, comes in. Deimos is a new critical experiment scheduled for FY24 at the National Criticality Experiments Research Center (NCERC). Deimos is a graphite moderated, graphite and beryllium reflected critical experiment making use of HALEU TRi-structural ISOtropic (TRISO) fuel from the Compact Nuclear Power System (CNPS). This transaction entails a brief description of efforts to benchmark Deimos for inclusion into the International Criticality Safety

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

Opportunities in AI/ML for the Rubin LSST Dark Energy Science Collaboration

The Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) will produce unprecedented volumes of heterogeneous astronomical data (images, catalogs, and alerts) that challenge traditional analysis pipelines. The LSST Dark Energy Science Collaboration (DESC) aims to derive robust constraints on dark energy and dark matter from these data, requiring methods that are statistically powerful, scalable, and operationally reliable. Artificial intelligence and machine learning (AI/ML) are already embedded across DESC science workflows, from photometric redshifts and transient classification to weak lensing inference and cosmological simulations. Yet their utility for precision cosmology hinges on trustworthy uncertainty quantification, robustness to covariate shift and model misspecification, and reproducible integration within scientific pipelines. This white paper surveys the current landscape of AI/ML across DESC's primary cosmological probes and cross-cutting analyses, revealing that the same core methodologies and fundamental challenges recur across disparate science cases. Since progress on these cross-cutting challenges would benefit multiple probes simultaneously, we identify key methodological research priorities, including Bayesian inference at scale, physics-informed methods, validation frameworks, and active learning for discovery. With an eye on emerging techniques, we also explore the potential of the latest foundation model methodologies and LLM-driven agentic AI systems to reshape DESC workflows, provided their deployment is coupled with rigorous evaluation and governance. Finally, we discuss critical software, computing, data infrastructure, and human capital requirements for the successful deployment of these new methodologies, and consider associated risks and opportunities for broader coordination with external actors.

Aubourg, Eric [APC, Paris] (ORCID:000000025592023X↗
DOE OSTI2021

Evolution and Trends of Industrial Control System Cyber Incidents since 2017

The industrial control systems (ICSs) that manage our critical infrastructure are increasingly converging with corporate networks and the Internet as technology and businesses prioritize digital connectivity. These connections make them more vulnerable and available to malicious cyber actors who traditionally targeted the companies’ more public-facing information technology (IT) networks. This paper will review select publicly reported cyber incidents to highlight the continued and growing threat to ICS devices and operational technology (OT) environments. It will summarize the incident and when available, will provide information on the cyber actors, the vulnerabilities they exploited, and any publications the U.S. Government (USG) provided in response. Data belonging to the Department of Homeland Security (DHS) will be used to highlight quantitative trends concerning ICS incidents. This paper builds on “History of Industrial Control System Cyber Incidents” (Hemsley & Fisher 2018), a paper that highlighted select noteworthy threats and incidents to ICS systems up to 2017. This paper will similarly review select incidents occurring after the last previously reviewed incident, Triton/HatMan, December 2017, and will note ICS incident trends including IT/OT convergence and advances in cyber-threat actors’ capabilities in observed in the examined incidents.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Can Common Pool Resource Theory Catalyze Stakeholder-Driven Solutions to the Freshwater Salinization Syndrome?

Freshwater salinity is rising across many regions of the United States as well as globally, a phenomenon called the freshwater salinization syndrome (FSS). The FSS mobilizes organic carbon, nutrients, heavy metals, and other contaminants sequestered in soils and freshwater sediments, alters the structures and functions of soils, streams, and riparian ecosystems, threatens drinking water supplies, and undermines progress toward many of the United Nations Sustainable Development Goals. There is an urgent need to leverage the current understanding of salinization’s causes and consequences–in partnership with engineers, social scientists, policymakers, and other stakeholders–into locally tailored approaches for balancing our nation’s salt budget. In this feature, we propose that the FSS can be understood as a common pool resource problem and explore Nobel Laureate Elinor Ostrom’s social-ecological systems framework as an approach for identifying the conditions under which local actors may work collectively to manage the FSS in the absence of top-down regulatory controls. We adopt as a case study rising sodium concentrations in the Occoquan Reservoir, a critical water supply for up to one million residents in Northern Virginia (USA), to illustrate emerging impacts, underlying causes, possible solutions, and critical research needs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Automated Cyber Security Testing Platform for Industrial Control Systems

Nuclear Power Plants (NPPs) are a complex system of coupled physics controlled by a network of Programmable Logic Controllers (PLCs). These PLCs communicate process data across the network to coordinate control actions with each other and inform the operators of process variables and control decisions. Networking the PLCs allows more effective process control and provides the operator more information which results in more efficient plant operation. This interconnectivity creates new security issues, as operators have more access to the plant controls, so will bad actors. As plant networks become more digitized and encompass more sophisticated controllers, the network surface exposed to cyber interference grows. Understanding the dynamics of these coupled systems of physics, control logic, and network communications is critical to their protection. The research into the cybersecurity of the Operational Technologies of NPPs is developing and requires a platform that can allow high fidelity physics simulations to interact with digital networks of controllers. This will require three main components: a network simulation environment, a physics simulator, and virtual PLCs (vPLC) that represent typical industry hardware. A platform that incorporates these three components to provide the most accurate representation of actual NPP networks and controllers is developed in this paper.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Gap Analysis of Global Climate Agreements

As climate change progresses, there will be greater pressure on state and nonstate actors to mitigate associated harms. This pressure will encourage aggressive action on climate change, including more ambitious emissions reduction goals and the use of both conventional and novel environmental modification techniques. Existing international agreements—including the Paris Agreement and the Environmental Modification Convention (ENMOD)—are critical to ensuring that climate change mitigation is achieved through peaceful, meaningful, and sustainable methods. With this in mind, this paper provides an overview of the Paris Agreement and ENMOD and identifies updates required for these agreements to meet the evolving challenges of climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

INDUSTRY ENGAGEMENT TO ESTABLISH A ROBUST SECURITY APPROACH TO MOBILE RADIOACTIVE SOURCES

The mobile radiological sources used in the well-logging and radiography industries are of sufficient curie quantities to be categorized as desirable material for malicious actors. Beyond the security risk posed by these sources, there is also an understanding of the potential damage, both reputational and monetary, that a lost source would have on the licensee and the industry overall. Identifying and communicating the risk these sources pose with impacted stakeholders is a critical first step in developing a security approach. Common day-to-day operations within both industries drive the unique security challenge of mobile sources. From storage facilities, transportation vehicles, temporary storage locations, and use in the field, each phase creates challenges regarding source control and accountability. All aspects of the operational use of these sources needs to be fully understood in order to address security equipment enhancements, policies, procedures, and training. This paper will leverage more than ten years of experience that ORS has gained working closely with industry partners and mobile radiological source users across the well-logging and radiography industries. It will identify the risk posed by mobile radiological sources, clearly define the operational phases of each industry, identify security best practices of mobile sources, and discuss what long-term, sustainable security looks like within these industries. In addition, it will explore areas of a robust security approach that are not commonly given priority in these industries, such as alarm adjudication and response.

Office of Radiological Security↗