Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “actor critic”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Domestic Extremism: Countering the Threat Posed to Critical Assets

Domestic extremism has been a growing concern in the United States in recent months, as illustrated in multiple bulletins from the Department of Homeland Security (DHS) warning law enforcement partners of the heightened threat. As concerns about these actors grows, it is important that facilities in the U.S. and internationally that protect critical assets, such as sensitive information, hazardous materials, or critical infrastructure, have effective methods in place to secure those assets. DE has challenged security systems through the threat of insider attack and violence, creating a new threat to be countered in the Office of Radiological Security’s radiological source security mission. In this effort, we used a literature review and focus group discussions with experts in critical asset security and extremism to understand the nature of the domestic extremist threat, to identify best practices in securing assets, recognize potential gaps in security measures to be corrected, and recommend actions and next steps. Twenty-two subject matter experts participated in a series of five focus group sessions. Questions focused on definitions of domestic extremism, potential changes in the threat, best practices in securing facilities, assets, and personnel, and any perceived gaps. Upon completion of the focus groups, notes were analyzed thematically to identify any recurring patterns in the results. In addition, a review of academic, industry, and government literature was conducted to understand the threat, describe the process of radicalization to extremism, and to identify empirically informed practices in prevention and response. Results of this project demonstrated that further work is needed to define domestic extremism in law, regulation, and policy, to help the U.S. develop a consistent response to the threat within organizations. This is especially important, as SMEs emphasized the need for early intervention in prevention efforts, noting that organizations need clear guidance on when and how to intervene. In addition, the need for social media monitoring was discussed, although challenges remain to do so with appropriate respect for privacy and civil liberties concerns.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

Sovereign Credit Rating Processes Adapted to Critical Infrastructure Cyber Risk Assessment

United States critical infrastructure entities are increasingly targeted by motivated and capable threat actors and must be prepared to assess and treat a diverse range of cyber risks. Consequently, this necessitates some form of analytical process to evaluate risks and inform cyber security investment decisions. A potential solution for structuring cyber risk evaluation exists within the field of sovereign credit ratings – where agencies employ mature approaches that integrate quantitative and qualitative data to produce a singular value of assessment. Adapting such approaches, we present a novel criterion and methodology for measuring and communicating the likelihood element of cyber risk. The methodology is composed of three sequential phases: a quantitative baseline organized by distinct capability frames, a bounded qualitative adjustment per frame, and a greater-bounded qualitative adjustment spanning the entire process. The process culminates in publication of a cyber capability rating that communicates a critical infrastructure entity’s ability and willingness to mitigate discontinuous function due to cyberattack.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2024

Sequence-Based Anomaly Detection in Critical Infrastructure Networks

United States critical infrastructure faces new cyber threats from adversarial nation-state actors in the form of malware-free attacks. Traditional cybersecurity techniques use rules-based methods to identify indicators of compromise on networks, often missing these sophisticated attacks. Our approach leverages multiple state of the art machine learning models in a pipeline to identify abnormal network events through sequential analysis. We combine both device and packet-level information into individual events to characterize anomalous network actions. The model is trained and tested on real network traffic from the Idaho National Lab High Performance Computing (HPC) with greater than 98% precision. It is capable of flagging malicious tactics used by adversaries in malware-free attacks, severe changes to the network, and abnormal user activity by network devices.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Investigate the Security of Electric Vehicle (EV) Ecosystem Applications

Apps that run on mobile devices are one of critical components of the electric vehicle (EV) ecosystem and pose possible threat actor points of entry that may impact the trust and security of EV charging systems in the future. Mobile apps often rely on communication between cloud servers and users, thereby creating potential points of entry for cyberattacks. Although app stores such as Apple App Store or Google Play Store generally test the security of apps, the cyber aspects may not be sufficient for many entities including DOD, federal fleets, and commercial entities. A more thorough inspection and the ability to influence developers is imminently needed. This research studies security attributes and vulnerabilities of a sample of mobile applications that support key user functions in the EV ecosystem. The study shows that all analyzed apps have security risks, categorized as either high or medium or both and a comprehensive cybersecurity guideline for developing mobile apps is necessary.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

A Typology for Characterizing Human Action in MultiSector Dynamics Models

The role of individual and collective human agency is increasingly recognized as a prominent and arguably paramount determinant in shaping the behavior, trajectory, and vulnerability of multisector systems. This human influence operates at multiple scales: from short-term (hourly to daily) to long-term (annually to centennial) timescales, and from the local to the global, pushing systems towards either desirable or undesirable outcomes. However, the effort to represent human systems in multisector models has been fragmented across philosophical, methodological, and disciplinary lines. To cohere insights across diverse modeling approaches, we present a new typology for classifying how human actors are represented in the broad suite of coupled human-natural system models that are applied in MultiSector Dynamics (MSD) research. The typology conceptualizes a “sector” as a system-of-systems that includes a diverse group of human actors, defined across individual to collective social levels, involved in governing, provisioning, and utilizing products, goods, or services towards some human end. We trace the salient features of modeled representations of human systems by organizing the typology around three key questions: 1) Who are the modeled actors in MSD systems? 2) What are their modeled actions? 3) How and for what purpose are these actors and actions operationalized in a computational model? The typology is used to critically examine existing models and chart the frontier of future human systems modeling for MSD research.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRSDate not supplied

Experiments for Securing Air Traffic Against Cyber-Physical System Attacks

This presentation describes experiments conducted with single board computers to investigate methods for creating trust for enabling the development of cyber-resilient air transportation systems. Methods included secure communication to prevent unauthorized access to data, consistency of data obtained via sensors and by processing, and built-in safeguards to prevent mission failure. The motivation for this work are the following. The future air transportation system needs to ensure availability, integrity, confidentiality and safety of operations. Safety of vehicles and operations is paramount for successful integration of Urban Air Mobility (UAM), Unmanned Aerial Systems (UAS), supersonic aircraft and launch vehicles with conventional aviation operations in the National Airspace System. Security is becoming critical because the sensors, networks and computers are far more vulnerable to bad actors than their mechanical or human predecessors. The goal therefore is to design and develop cyber-resilient systems that continue to function even in degraded states. The main findings are (1) off-the-shelf hardware can support development of cyber-resilient onboard flight computers and (2) trust in system design and implementation can be accomplished by integrating layers in depth (detail) and in breadth (scope).

cyber-resilient autonomy, trust, secure communicat↗
DOE OSTI2021

Domestic Extremism (Executive Summary)

Domestic extremism (DE) has been a growing concern in the U.S. in recent months, as illustrated in multiple bulletins from the Department of Homeland Security (DHS) warning law enforcement partners of the heightened threat. As concerns about these actors grows, it is important that facilities in the U.S. and internationally that protect critical assets, such as sensitive information, hazardous materials, or critical infrastructure, have effective methods in place to secure those assets. DE has challenged security systems through the danger of insider attack and violence, creating a new threat to be countered. In this effort, therefore, we used a literature review and focus group discussions with experts in critical asset security and extremism to understand the nature of the domestic extremist threat, to identify best practices in securing assets, recognize potential gaps in security measures to be corrected, and recommend actions for the Office of Radiological Security (ORS) to address DE with its partners.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2023

Distributed Power Allocation for 6-GHz Unlicensed Spectrum Sharing via Multi-agent Deep Reinforcement Learning

We consider the problem of power allocation over the 6 GHz Unlicensed National Information Infrastructure (UNII)- 5 spectrum. We propose a novel deep Reinforcement Learning (DRL)-based distributed power allocation scheme which utilizes the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. In particular, we model the base stations (BSs) as DRL agents that simultaneously determine the transmit powers to their scheduled user equipment (UE) in a synchronized manner. The power decision of each BS is based on its own observation of the radio environment, which consists of several local interference measurements and a limited amount of information obtained from other BSs. One advantage of the proposed scheme is that it addresses the single-agent non-stationarity problem of RL in the multi-agent scenario by incorporating the actions and observations of other BSs into each BS’s own critic which helps it to gain a more accurate perception of the overall radio environment. A centralized-training-distributed execution framework is used to train the policies where the critics are trained over the joint actions and observations of all BSs while the actor of each BS only takes the local observation as input in order to produce the transmit power. Simulation shows that the proposed power allocation scheme can achieve better throughput performance than several state-of-the-art approaches.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

Discovery of Probabilistic Dirichlet-to-Neumann Maps on Graphs

Dirichlet-to-Neumann maps enable the coupling of multiphysics simulations across computational subdomains by ensuring continuity of state variables and fluxes at artificial interfaces. We present a novel method for learning Dirichlet-to-Neumann maps on graphs using Gaussian processes, specifically for problems where the data obey a conservation law arising from an underlying partial differential equation. Our approach combines discrete exterior calculus and nonlinear optimal recovery to infer relationships between vertex and edge values. This framework yields data-driven predictions with uncertainty quantification across the entire graph, even when observations are limited to a subset of vertices and edges. By minimizing the reproducing kernel Hilbert space norm while penalizing kernel complexity through maximum likelihood estimation, our method ensures that the resulting surrogate strictly enforces conservation laws without overfitting. We demonstrate our method on two representative applications: subsurface flow in fracture networks and arterial blood flow. Finally, the results demonstrate that the method maintains high accuracy and well-calibrated uncertainty estimates even under severe data scarcity, highlighting its potential for scientific applications where limited data and reliable uncertainty quantification are critical.

Dirichlet-to-Neumann map↗
DOE OSTI2025

Enterprise Artificial Intelligence Strategy for Los Alamos National Laboratory

In the 1984 martial arts drama film, The Karate Kid, a young Daniel LaRusso is unexpectedly placed in an adversarial environment unable to eYectively adapt to a series of new threats and limitations. Fortunately for the main character, once placed under the tutelage of a Mr. Miyagi, he finds resiliency not through the adoption of new tools, but a re-focused set of fundamentals. Much in the same way that Daniel learns waxing on and buYing oY car wax by hand has rewards for Karate, LANL is choosing the harder path of self-hosting Large Language Models (LLMs) for enterprise use instead of only relying on buying access to a hosted AI service like Azure’s OpenAI Application Programming Interface (API). We also are not willing to wait for software-as-a-service (SAAS) AI services to meet us where we need to be from a FedRAMP accreditation standpoint. Our operations regularly depend on access at CUI, UCNI, ITAR and other FIPS-199 moderate-impact data levels and hosting our own services gives us the right security and compliance posture to be useful across the broad range of our work at LANL. With the rise in threats to critical infrastructure, cloud service providers (CSPs), and supply chain attacks from both state and non-state actors, we are not placing the bet that SAAS hosted AI services will be available when we need them. Should a major event occur, we do not want our staY and operations left without a pathway for us to fix the problem and resume the use of AI tools.

42 ENGINEERING↗
DOE OSTI2022

Cyber Attack Sequences Generation for Electric Power Grid

Security assessment of cyber-physical energy systems (CPESs) such as the electric power grid is a critical operation to maintain availability, reliability, and quality of service in the presence of persistent threats from malicious cyber actors. Existing security assessment approaches such as penetration testing and red teaming rely on subject matter expert experience and forensic cyber analysis of historical events to perform realistic, threat-informed assessments of CPES defense. CPESs have a large attack surface because of the heterogeneity and complexity of underlying topology, devices, measurements, and vulnerabilities. The aforementioned approaches lead to partial coverage of the attack surface with a large set of unknown but possible exploits. There is a need to automate the CPES attack surface discovery and contextualize it for relevant, highly probable, real-world attack scenarios. We propose a methodology and framework to facilitate the discovery of the CPES attack surface. We present a multilayer attack graph with ranked attack sequences to describe CPES failure scenarios. We present a work-in-progress framework that lists key components to automate the attack modeling and sequence generation. We demonstrate the published National Electric Sector Cybersecurity Organization Resource CPES failure scenario to highlight the trustworthiness of generated attack sequences.

Dutta, Ashutosh↗
DOE OSTI2023

Blockchain for Fault-Tolerant Grid Operations Version 2.0

This report explores the potential of distributed ledger technology (DLT) as a transformative tool to enhance fault-tolerant operations in electrical distribution systems. Leveraging DLT's core attributes, including an immutable decentralized ledger, distributed consensus mechanisms, and state replication capabilities, this study focuses on three critical use cases. A central aspect of this research centers on the utilization of a consensus-driven ledger, providing actors within the system, such as distributed resources, with access to a reliable data repository. This empowers these actors to collaborate effectively and make informed decisions, all securely recorded on the blockchain. The first use case concentrates on data configuration, utilizing mathematical criteria---particularly, the chi-squared test for gross error detection---to identify trustworthy sensors for advanced decision-making. Building upon this foundation of trust, the second use case, topology identification, accurately determines circuit breaker states, unveiling the distribution network's topology. Ultimately, the third use case leverages this trust to execute switching actions, reconfiguring feeders and restoring power to disconnected customers after fault events. The concept of trust serves as a cornerstone in this approach, marking a departure from traditional fault location, isolation, and service restoration (FLISR) methods. Additionally, the blockchain-based architecture introduces decentralization, empowering disconnected areas to make autonomous decisions, even when communication with a central control center is disrupted. The primary contributions of this report are twofold: (1) a novel approach for evaluating distribution system voltage areas while preserving data ownership and (2) the implementation of interactions between distribution network areas using the actor model. Unlike the previous sequential approach for evaluating the area connection voltages, which required a radial network topology, this study's area model reduction enables a more versatile approach. The area model reduction addresses issues of prolonged data waiting times and multiple points of failure within the previous approach. Notably, the presented evaluation for the reduced network model area connection reveals a significant increase in the differences in voltage magnitudes. Simulation and evaluation of area agents across four distinct cases elucidate the area-level interaction behavior during a fault event. Simulations demonstrate that the proposed distributed FLISR (DFLISR) approach can successfully restore service to an affected area. Varying message delays and message loss probabilities in each simulation case underscore their impacts on restoration times, ranging from 3 min and 32 s to 6 min and 19 s. In contrast, power is not restored in an area in one of our simulation cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2021

NOAA-17 Break-up Engineering Investigation Board Final Report

The NOAA-17 break-up was found to be a single, localized debris event producing about100 trackable pieces; there is likely no catastrophic damage to the whole spacecraft. NOAA-17 debris is very much in family with NOAA-16 debris, and DMSP F11 and F13 are very much in family with each other and share similarities with NOAA-16 and 17; it is likely all four (“The Big Four”) share the same debris source. DMSP F13 break-up occurred simultaneous with a known battery overcharge and therefore battery rupture is most likely intermediate cause of the Big Four break-ups; this is a low confidence assessment since other debris sources cannot be definitively ruled out. No root cause was found as the NOAA-17 batteries were all confirmed to have been disconnected from the charge path as intended. Possible conditions for reconnection are all unlikely including short circuits and commanding from a “bad actor.” All related spacecraft pose a risk of similar break-ups for decades to come and are a threat to the critical 800-850 km polar orbit regime; even appropriately decommissioned spacecraft appear to be at risk. Recommendations include an update to the decommissioning procedure and consideration of further investigations and active debris removal, consistent with national policy.

Maggie Atkinson↗
DOE OSTI2023

Water narratives in local newspapers within the United States

Sustainable use of water resources continues to be a challenge across the globe. This is in part due to the complex set of physical and social behaviors that interact to influence water management from local to global scales. Analyses of water resources have been conducted using a variety of techniques, including qualitative evaluations of media narratives. This study aims to augment these methods by leveraging computational and quantitative techniques from the social sciences focused on text analyses. Specifically, we use natural language processing methods to investigate a large corpus (approx. 1.8M) of newspaper articles spanning approximately 35 years (1982–2017) for insights into human-nature interactions with water. Focusing on local and regional United States publications, our analysis demonstrates important dynamics in water-related dialogue about drinking water and pollution to other critical infrastructures, such as energy, across different parts of the country. Our assessment, which looks at water as a system, also highlights key actors and sentiments surrounding water. Extending these analytical methods could help us further improve our understanding of the complex roles of water in current society that should be considered in emerging activities to mitigate and respond to resource conflicts and climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Preliminary Benchmark Uncertainties for Deimos, a HALEU-Fueled and Graphite-Moderated Advanced Reactor Testbed

Many advanced reactor concepts will make use of various uranium fuels with levels of enrichment higher than previously seen in current light water reactors. In particular, High-Assay Low Enriched Uranium (HALEU), that is uranium enriched to 235 U ≈ 20 w/o%, is planned to be used in over ten new reactor concepts. HALEU is attractive for advanced reactors as it enables longer intervals between refueling. Unfortunately, little to no experience with HALEU is available in experimental literature raising concerns for not only licensing advanced re actors but also fabrication and transportation of HALEU fuels. This is where Deimos, a Los Alamos National Laboratory internal project, comes in. Deimos is a new critical experiment scheduled for FY24 at the National Criticality Experiments Research Center (NCERC). Deimos is a graphite moderated, graphite and beryllium reflected critical experiment making use of HALEU TRi-structural ISOtropic (TRISO) fuel from the Compact Nuclear Power System (CNPS). This transaction entails a brief description of efforts to benchmark Deimos for inclusion into the International Criticality Safety

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
NASA NTRSDate not supplied

Informing New Concepts for UAS and Autonomous System Safety Management using Disaster Management and First Responder Scenarios

As emerging flight operations become more prevalent and increasingly automated and distributed, the capabilities for managing safety of vehicles and operations will also need to evolve. To address this challenge, the National Academies has envisioned an In-Time Aviation Safety Management System (IASMS) capability for a wide range of aviation operations including current commercial operations as well as new entrants envisioned with advanced air mobility (AAM). The suite of IASMS services, functions, and capabilities (SFCs) would be implemented in a federated approach and would address trends as well as individual operations. Through predictive modeling and data analysis, IASMS is envisioned to identify arising risks so that they can be mitigated, in-time, before a safety incident occurs. IASMS and its requisite set of SFCs must leverage a wide range of information to perform. To better understand these new needs, FSF worked with the aviation and humanitarian communities to develop and validate scenarios that include traditional aviation operations and UAS operations intermingled as they are deployed for disaster management and first responder (DMFR) situations. The three scenarios developed include: • Post Natural disaster response, such as a hurricane, involving multiple parties utilizing traditional aviation and UAS to support rescue operations, surveil damage, and locate survivors needing assistance. • Wildfire fighting in remote locations with traditional aircraft for transport and fire-retardant delivery combined with UAS for surveillance of fire locations as well as to track individual firefighter locations. • Medical Operations and AAM in Urban Environments including passenger-carrying helicopters and AAM vehicles, medical missions (such as transport of radio-pharmaceuticals), and other UAS delivery operations (such as the delivery of defibrillators). Each scenario was developed and validated by representatives with expertise in humanitarian operations, urban and rural emergency response, air traffic management, UAS operations, and traditional flight operations. The scenario definitions address roles and responsibilities of individual actors, the appropriate utilization of UAS, and the actions taken by those actors to appropriately manage risks associated with the mission and environment. The risks to aviation traffic and to people on the ground explored included potential risks arising from incompatibilities in calculating reference altitudes (eg, differing uses of AGL, MSL, barometric, or GPS-derived values), loss of command and control (C2) communications, rapid changes in weather and winds, and physical interference. For each risk, IASMS SFCs were postulated in the context of monitoring services, risk assessment capabilities, and identifying appropriate mitigation strategies. The identified SFC capabilities were envisioned from known services postulated for IASMS and for UTM. For these unique environments, IASMS SFCs are needed to address conditions such as hazardous payloads, micro-climates and urban canyons, and the need to keep uninvolved air traffic out of the area where DMFR operations are being conducted. The second phase of analysis focused on inferring the specific information needs and the SFCs for IASMS, utilizing a structure of 16 information classes to organize requirements. For each of the risks identified in the workshops, it was postulated what data sources would be necessary to monitor critical aspects of the risk (eg, surrounding air traffic, ground population, terrain, etc). to be directly measured as well as data that would be derived, which implies additional SFCs for different actors to understand what information would likely be exchanged between parties. For an IASMS to be effective, additional research is needed to develop the advanced algorithms that can address the increasingly autonomous and complex operations in differing environments and to develop means of identifying unknown risks. Looking at these scenarios highlighted a number of research issues. These include the ability to quickly "cordon off" airspace thru temporary flight restrictions (TFRs) or other means, developing clear definitions to enable automation-based algorithms for prioritizing operations, defining airspace density metrics, standardization of altitude reporting, and establishing a basis for safety data metrics definition and collection. This paper seeks to outline the development of an IASMS in the context of the DMFR scenarios and resulting demonstrations. Utilizing this contextual approach, NASA will generate recommendations for an assured safety framework for AAM operations that enables AAM operations to safely access the NAS.

In Time Aviation Safety Management System↗
DOE OSTI2026

Opportunities in AI/ML for the Rubin LSST Dark Energy Science Collaboration

The Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) will produce unprecedented volumes of heterogeneous astronomical data (images, catalogs, and alerts) that challenge traditional analysis pipelines. The LSST Dark Energy Science Collaboration (DESC) aims to derive robust constraints on dark energy and dark matter from these data, requiring methods that are statistically powerful, scalable, and operationally reliable. Artificial intelligence and machine learning (AI/ML) are already embedded across DESC science workflows, from photometric redshifts and transient classification to weak lensing inference and cosmological simulations. Yet their utility for precision cosmology hinges on trustworthy uncertainty quantification, robustness to covariate shift and model misspecification, and reproducible integration within scientific pipelines. This white paper surveys the current landscape of AI/ML across DESC's primary cosmological probes and cross-cutting analyses, revealing that the same core methodologies and fundamental challenges recur across disparate science cases. Since progress on these cross-cutting challenges would benefit multiple probes simultaneously, we identify key methodological research priorities, including Bayesian inference at scale, physics-informed methods, validation frameworks, and active learning for discovery. With an eye on emerging techniques, we also explore the potential of the latest foundation model methodologies and LLM-driven agentic AI systems to reshape DESC workflows, provided their deployment is coupled with rigorous evaluation and governance. Finally, we discuss critical software, computing, data infrastructure, and human capital requirements for the successful deployment of these new methodologies, and consider associated risks and opportunities for broader coordination with external actors.

Aubourg, Eric [APC, Paris] (ORCID:000000025592023X↗
DOE OSTI2021

Evolution and Trends of Industrial Control System Cyber Incidents since 2017

The industrial control systems (ICSs) that manage our critical infrastructure are increasingly converging with corporate networks and the Internet as technology and businesses prioritize digital connectivity. These connections make them more vulnerable and available to malicious cyber actors who traditionally targeted the companies’ more public-facing information technology (IT) networks. This paper will review select publicly reported cyber incidents to highlight the continued and growing threat to ICS devices and operational technology (OT) environments. It will summarize the incident and when available, will provide information on the cyber actors, the vulnerabilities they exploited, and any publications the U.S. Government (USG) provided in response. Data belonging to the Department of Homeland Security (DHS) will be used to highlight quantitative trends concerning ICS incidents. This paper builds on “History of Industrial Control System Cyber Incidents” (Hemsley & Fisher 2018), a paper that highlighted select noteworthy threats and incidents to ICS systems up to 2017. This paper will similarly review select incidents occurring after the last previously reviewed incident, Triton/HatMan, December 2017, and will note ICS incident trends including IT/OT convergence and advances in cyber-threat actors’ capabilities in observed in the examined incidents.

99 GENERAL AND MISCELLANEOUS↗