Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Actors”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Risk-Aware Reinforcement Learning Framework for User-Centric O-RAN

The evolution of Open Radio Access Networks (O-RAN) presents an opportunity to enhance network performance by enabling dynamic orchestration of configuration and optimization parameters (COPs) through online learning methods. However, leveraging this potential requires overcoming the limitations of traditional cell-centric RAN architectures, which lack the necessary flexibility. On the other hand, despite their recent popularity, the practical deployment of online learning frameworks, such as Deep Reinforcement Learning (DRL)-based COP optimization solutions, remains limited due to their risk of deteriorating network performance during the exploration phase. In this article, we propose and analyze a novel risk-aware DRL framework for user-centric RAN (UC-RAN), which offers both the architectural flexibility and COP optimization to exploit this flexibility. We investigate and identify UC-RAN COPs that can be optimized via a soft actor-critic algorithm implementable as an O-RAN application (rApp) to jointly maximize latency satisfaction, reliability satisfaction, area spectral efficiency, and energy efficiency. We use the offline learning on UC-RAN to reliably accelerate DRL training, thus minimizing the risk of DRL deteriorating cellular network performance. Results show that our proposed solution approaches near-optimal performance in just a few hundred iterations with a decrease in risk score by a factor of ten.

6G and beyond↗
DOE OSTI2025

FIRE: A Failure-Adaptive RL Framework for Edge Computing Migrations

In edge computing, users' service profiles are migrated between edge servers due to user mobility. Reinforcement Learning (RL) frameworks have been proposed to do so, often trained on simulated data. However, existing RL frameworks overlook occasional server failures, which although rare, impact latency-sensitive applications like AR/VR and real- time obstacle detection. These rare failures, being not adequately represented in historical training data, pose a challenge for data-driven RL algorithms. We introduce FIRE, a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment. We propose FIRE-ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function. FIRE considers delay, migration, failure, and backup placement costs across individual and shared service profiles. We prove FIRE-ImRE's boundedness and convergence to optimality. Next, we introduce novel deep Q-learning (FIRE-ImDQL) and actor critic (FIRE-ImACRE) versions of our algorithm to enhance scalability. Here, we extend our framework to accommodate users with varying risk tolerances of rare failure events. Through trace-driven experiments, we show that FIRE reduces edge computing costs compared to vanilla RL and the greedy baseline in the event of failures.

Edge computing↗
DOE OSTI2024

Resilient Control of Networked Microgrids Using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Improving system-level resiliency of networked microgrids against adversarial cyber-attacks is an important aspect in the current regime of increased inverter-based resources (IBRs). To achieve that, this paper contributes in designing a hierarchical control layer, in conjunction with the existing control layers, resilient to adversarial attack signals. Considering model complexities, unknown dynamical behaviors of IBRs, and privacy issues regarding data sharing in multi-party-owned microgrids, designing such a control layer is non-trivial. Here, to tackle these issues, a novel federated reinforcement learning (Fed-RL) method is proposed. To grasp the interconnected dynamics of networked microgrids, the paper develops Federated Soft Actor-Critic (FedSAC) algorithm following the vertical structure of implementing Fed-RL. Next, utilizing the OpenAI Gym interface, we built a custom set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark comprising 3 interconnected microgrids. Finally, the learned policies in the simulation are transferred to the real-time hardware-in-the-loop (HIL) test-bed developed using the high-fidelity Hypersim platform. Finally, experiments show that the simulator-trained RL controllers achieve desirable performance with the test-bed platform, validating the minimization of the sim-to-real gap.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

On the Abuse and Detection of Polyglot Files

A polyglot is a file that is valid in two or more formats. Polyglot files pose a problem for file-upload and generative AI web interfaces that rely on format identification to determine how to securely handle incoming files. In this work we found that existing file-format and embedded-file detection tools, even those developed specifically for polyglot files, fail to reliably detect polyglot files used in the wild. To address this issue, we studied the use of polyglot files by malicious actors in the wild, finding 30 polyglot samples and 15 attack chains that leveraged polyglot files. Using knowledge from our survey of polyglot usage in the wild---the first of its kind---we created a novel data set based on adversary techniques. We then trained a machine learning detection solution, PolyConv, using this data set. PolyConv achieves a precision-recall area-under-curve score of 0.999 with an F1 score of 99.20% for polyglot detection and 99.47% for file-format identification, significantly outperforming all other tools tested. We developed a content disarmament and reconstruction tool, ImSan, that successfully sanitized 100% of the tested image-based polyglots, which were the most common type found via the survey. Our work provides concrete tools and suggestions to enable defenders to better defend themselves against polyglot files, as well as directions for future work to create more robust file specifications and methods of disarmament.

Oesch, T [ORNL] (ORCID:0000000269091022)↗
DOE OSTI2025

DIMPLES: Distributed Influence Maximization for Pandemic pLanning on Exascale Systems

We study exascale parallel algorithms for the selection of intervention or monitoring strategies in massive realistic socio-technical networks through scalable Influence Maximization (InfMax) algorithms. We employ novel techniques to enable efficient scaling on up to 8k nodes of OLCF Frontier, with 65k AMD GPUs and 458k AMD CPU cores. Current state-of-the-art InfMax tools are limited to networks with only a few million actors (vertices) and a few hundred million interactions (edges). By overcoming these limitations, we show that our approach is capable of processing a realistic social contact network of the United States with 285 million nodes and about 8 billion edges. This two orders-of-magnitude improvement over the previous state-of-the-art is obtained by leveraging algorithmic advancements for the InfMax problem and designing several problem-specific approaches to overlap communication with computation, improve GPU efficiency, and lower the application’s memory requirements. We evaluate strong scaling for computing 10k most influential seeds using up to 8k nodes of an exascale system, and weak scaling from 128 to 8k system nodes for seed sets ranging from 625 to 40k seeds. We achieve the fastest-known runtime of 25 minutes while performing 48 million diffusion simulations totaling 2.31 petabytes to identify 40k influential seeds using 8k nodes, and take 5.75 minutes to identify 10k seeds while using 4k nodes.

Minutoli, Marco [Pacific Northwest National Labora↗
DOE OSTI2024

Modeling Framework for the Assessment of a Sustainable Hydrogen Production and Supply Chain Network in California

The cost-effective and sustainable deployment of hydrogen supply and demand networks, especially in large economic regions like California, can be challenging considering the spatial-temporal availability and variability of the different actors across the network such as production processes, distribution modes, and end-users. In this presentation, we will provide an overview and demonstration of a modeling framework used to assess the environmental, economic, and human health impacts of plausible hydrogen production and supply chain networks in California. Scenarios focus on green hydrogen production pathways using water electrolysis and biomass gasification. End-use applications included in the model are transit, medium and heavy-duty trucking, port authorities, and power and aviation companies that currently consume natural gas, diesel, and aviation fuel for their day-to-day operation. Representative locations for hydrogen production and end-use are based on recent projections of the hydrogen economy in California. All mass and energy flows, as well as estimated emissions, are based on H2A process model designs and projections of technology performance, literature review, and LBNL process, economic and life cycle modeling, and not on company data for the sake of this presentation. Human health impacts are included following methodologies developed for the University of California Irvine HyDeal project. Life cycle phases associated with hydrogen production include feedstock preparation (water and biomass), energy production and consumption (renewable, grid, and combination of renewable and grid electricity), maintenance (chemical utilization in electrolysis and natural gas combustion in gasification), carbon sequestration, hydrogen storage (compression and liquefaction), and distribution (truck and pipeline). We apply the framework utilizing California specific emission factors, financial data, and human health damages and explore the impact of network characteristics on results. Example variations include: the inclusion of policy incentives or not, different representations of the electricity grid and source, electrolysis versus gasification versus combinations of both for production, liquefaction versus compression based on producer capacity cutoffs, transportation truck versus pipeline based on existing infrastructure, and ultimate end use. Comparison of these different scenarios can help inform future projects by demonstrating the trade-offs among environmental, economic, and human health impacts. This model, automated in R, is a starting platform upon which new analysis, modeling capabilities, locations, and emission factors can be rapidly tested and integrated.

Zaki, Mohammed Tamim↗
DOE OSTI2025

CyRRL (Cyber Resilient Reinforcement Learning for grid voltage control) [SWR-24-115]

This codebase contains a multi-agent, actor-critic reinforcement learning implementation for cyber-resilient grid voltage control. It uses a 123-bus OpenDSS system as the environment, with three-phase power flow translating nodal power injections into solved nodal voltages. The reward function penalizes deviations from nominal voltage as well as reactive power dispatch, while encouraging agents to take actions that result in fast convergence to nominal conditions. The codebase models false data injection attacks and includes functionality for training, testing, hyper-parameter tuning, and visualization.

Murphy, Sinnott [National Renewable Energy Laborat↗
DOE OSTI2024

Towards a Workforce Roadmap for Distributed Wind: Phase 1 - Identifying Needs and Barriers

Despite recent policy, research efforts, and resources available for utility-scale and offshore wind workforce development, the distributed wind (DW) industry has yet to make similar advances to address its workforce challenges. This report initiates the phased development of a DW workforce roadmap to provide a foundation for roadmap development. Phase 1, contained in this report, uses desk-based research on clean energy workforce approaches and a DW interested-party survey to discuss needs and barriers hindering the workforce from expanding. Findings are summarized into goals and solutions for Phase 2 of roadmap development, which identifies actors responsible for implementing solutions identified in Phase 1. Phase 1 results suggest that the sector's small size and limited growth motivate the short-term need for skilled workers or candidates with well-rounded and multifaceted abilities. Long-term expansion plans must diversify positions while capitalizing on existing utility-scale wind and offshore workforce efforts, as well as other renewable sector best practices where possible. The findings in this report can advance workforce development in the DW sector by aligning interested industry parties around common goals to address challenges. Training providers, installers, operators, manufacturers, federal agencies, national laboratories, academic partners, and labor unions can utilize the findings to help promote sustainable growth of the DW sector.

17 WIND ENERGY↗
DOE OSTI2024

Investigate the Security of Electric Vehicle (EV) Ecosystem Applications

Apps that run on mobile devices are one of critical components of the electric vehicle (EV) ecosystem and pose possible threat actor points of entry that may impact the trust and security of EV charging systems in the future. Mobile apps often rely on communication between cloud servers and users, thereby creating potential points of entry for cyberattacks. Although app stores such as Apple App Store or Google Play Store generally test the security of apps, the cyber aspects may not be sufficient for many entities including DOD, federal fleets, and commercial entities. A more thorough inspection and the ability to influence developers is imminently needed. This research studies security attributes and vulnerabilities of a sample of mobile applications that support key user functions in the EV ecosystem. The study shows that all analyzed apps have security risks, categorized as either high or medium or both and a comprehensive cybersecurity guideline for developing mobile apps is necessary.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Large-Scale Hydrogen Storage Cyber Risk Assessment

Hydrogen storage systems may become more widely deployed throughout the country, and so it is possible that individual and interconnected systems will be exposed to cyber-attacks. These events can cause physical and financial harm to employees, people in the vicinity of the facility, and the company that owns the facility. The two main ways bad actors may access information or control from a hydrogen storage facility are through information technology and operations technology devices, the former of which refers to data and information from networked devices and the latter of which refers to onsite controls for the physical system. Both types of entryways into the system should be considered when companies conduct cyber risk assessments and when regulators develop or revise relevant codes and standards. This report analyzes cybersecurity risks associated with a generic hydrogen storage system by outlining the system's purpose and the importance of its cybersecurity. The hydrogen storage system architecture and communication protocols are provided to understand potential cyber vulnerabilities. Later, an event tree analysis is performed on hydrogen operation to identify system weaknesses by outlining potential attack scenarios. This report also identifies critical cyber assets related to different hydrogen operations followed by an examination of potential threats, and the impact of cyber assets on those operational assets.

08 HYDROGEN↗
DOE OSTI2024

Large-Scale Hydrogen Storage Cyber Risk Assessment

Hydrogen storage systems are becoming more widely deployed throughout the country, and as their presence continues to grow, it is possible that individual and interconnected systems will be exposed to cyber-attacks. These events can cause physical and financial harm to employees, people in the vicinity, and to the company that owns the facility. The two main mechanisms malicious actors may access information or control from a hydrogen storage facility are through information technology and operations technology devices, the former of which refers to data and information from networked devices and the latter of which refers to onsite controls for the physical system. Both types of entryways into the system should be considered when facility managers conduct cyber risk assessments and when regulators develop or revise relevant codes and standards. This report analyzes cybersecurity risks applicable to a wide variety of hydrogen storage systems by outlining the system's purpose and the importance of its cybersecurity. The hydrogen storage system architecture and communication protocols are provided to understand potential cyber vulnerabilities. Later, an event tree analysis is performed on hydrogen operation to identify system weaknesses by outlining potential attack scenarios. This report also identifies critical cyber assets related to different hydrogen operations followed by an examination of potential threats, and the impact of cyber assets on those operational assets.

08 HYDROGEN↗
DOE OSTI2024

Marine Energy Commercialization Review: Evaluation of the Transition From Public to Private Capital

The mission of the U.S. Department of Energy's Water Power Technologies Office (WPTO) is to advance marine energy technologies through research, testing, and commercialization. This paper explores the barriers and potential solutions for marine energy commercialization by evaluating publicly available literature, feedback from public and private actors, and historical WPTO actions. A key finding is the absence of standardized metrics to measure marine energy commercialization progress and the lack of targeted success goals. This paper aims to define those metrics, informed by public and private goals and the challenges developers experience, and to further evaluate targets offered by public funders and private capital providers. Recommendations to address barriers in marine energy commercialization include enhancing public-private communication, refining commercialization requirements, leveraging technology transfer programs, and exploring novel funding mechanisms like green bonds and contracts for difference. Addressing these challenges through proposed adjustments could facilitate the transition of marine energy technologies from public funding to sustainable private investment, ultimately advancing their commercialization.

13 HYDRO ENERGY↗
DOE OSTI2024

Performance Year 1 Technical Report - OPEN COG Grid: Extendable Coherent Models-Datasets for Cognitive Power Grids

The OPEN COG Grid project is a collaborative effort between LLNL, NREL, and Texas A&M University (TAMU) to develop synthetic power system datasets that (i) contain all technical information that would be available in a real system, allowing to conduct studies ranging from dynamic simulation to long term planning studies; ii) are accessible to researchers from the broader data sciences community, as oppossed to power system experts only; and (iii) This report summarizes the work conducted during the first 15 months of execution of the project. These activities encompassed: 1. Conduct a survey of existing open data sets and open source power systems simulators, their supported use cases, and accessibility (Chapter 1). 2. Define a new extensible specification for power system data, covering all parameters necessary for most computational use cases (Chapter 2). 3. Collecting real technical system data to complete missing parameters in existing open source datasets (Chapter 3). 4. Develop models that capture the behavior of emergent actors in power grids, neglected by existing datasets; aggregated residential demand response (Chapter 4) and demand response of cryptocurrency miners (Chapter 5). 5. Collect detailed spatial information on distributed energy resources, particular, solar photovoltaic facilities (Chapter 6). The following chapters provide detailed descriptions of these tasks, the assumptions taken, and their findings. In conducting these tasks, the project team produced: two (accepted) conference papers; one journal paper under submission; one draft journal paper pending submission; released one repository with the developed power system data specification, with documentation and examples; and one extended dataset for the Texas power grid under review for release. The team hopes these contributions will enhance access to power system data and remove barriers to the development of new computational techniques for power systems, particularly, those inspired by cognitive sciences.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Proactive Regulatory Approaches to Electrification and Load Growth: Workshop Report

On July 10 and 11, 2024, Pacific Northwest National Laboratory and RMI led a workshop in Aurora, Colorado, to explore novel and proactive approaches to electrification and load growth while minimizing risks and costs to customers. Over the next decade, a unique opportunity exists to invest strategically in the electricity system to enable electrification across the transportation, industrial, and building sectors and respond to data and technology-based load growth. However, current utility and regulatory planning practices are insufficient to identify and enable the right investments, and work must be done to reduce the risk and decisional uncertainty faced by utility regulatory commissions and utilities. Ensuring timely electrification investments may require new approaches to address risk, uncertainty, prudence, and cost recovery. Understanding the decision-making process and information needs of utilities and regulators is critical. New policies (or application of policies), financial tools, systems analysis, regulatory mechanisms, and enhanced process transparency may be required. The workshop's goal was to identify proactive regulatory approaches for electrification and load growth that minimize costs and risks to customers. Our intention was that the conversations and the resulting solutions and takeaways would be specific and tactical rather than general and theoretical and that together we would create actionable next steps for key actors in the system, including utilities, regulators, thought leaders, researchers, and the U.S. Department of Energy (DOE). This report is intended to provide workshop attendees with a record and summary of the discussion and proposals raised at the workshop and to provide interested entities who did not attend, such as other regulators, policymakers, utilities, and U.S. DOE offices, with an understanding of what was discussed and with ideas to explore in their organizations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Taxonomy and Feature set for Server-Side Identification of Proxies

Malicious actors frequently use proxies and VPNs to evade detection and hide their origin. Current challenges to information security include the use of residential proxies to blend in with normal traffic and Man-in-the-Middle phishing proxies that are used to compromise accounts protected with mult-factor authentication. We advance a taxonomy and feature set for the identification of proxied traffic based on the network layer where proxying occurs. We describe how these features apply to common proxy types and how to use these features in the classification of the proxied traffic. Collection of these additional features is feasible using existing network sensors and web servers, while only adding about 30% volume to commonly deployed network sensor logs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Enterprise Artificial Intelligence Strategy for Los Alamos National Laboratory

In the 1984 martial arts drama film, The Karate Kid, a young Daniel LaRusso is unexpectedly placed in an adversarial environment unable to eYectively adapt to a series of new threats and limitations. Fortunately for the main character, once placed under the tutelage of a Mr. Miyagi, he finds resiliency not through the adoption of new tools, but a re-focused set of fundamentals. Much in the same way that Daniel learns waxing on and buYing oY car wax by hand has rewards for Karate, LANL is choosing the harder path of self-hosting Large Language Models (LLMs) for enterprise use instead of only relying on buying access to a hosted AI service like Azure’s OpenAI Application Programming Interface (API). We also are not willing to wait for software-as-a-service (SAAS) AI services to meet us where we need to be from a FedRAMP accreditation standpoint. Our operations regularly depend on access at CUI, UCNI, ITAR and other FIPS-199 moderate-impact data levels and hosting our own services gives us the right security and compliance posture to be useful across the broad range of our work at LANL. With the rise in threats to critical infrastructure, cloud service providers (CSPs), and supply chain attacks from both state and non-state actors, we are not placing the bet that SAAS hosted AI services will be available when we need them. Should a major event occur, we do not want our staY and operations left without a pathway for us to fix the problem and resume the use of AI tools.

42 ENGINEERING↗
DOE OSTI2024

Collaborative Enhancements to Unlock Interregional Transmission

Multiple recent studies highlight significant benefits associated with expansion of interregional transfer capabilities for efficiently adapting to these changes. Despite these proposed benefits, few, if any, interregional transmission projects have been built in recent memory . The objective of this report is to confront the barriers to interregional transmission that exist today and address them with potential reforms and collaborative solutions. This report identifies solutions with the potential for immediate beneficial impact on interregional transmission with the ultimate goal of allowing more effective identification and advancement of interregional transmission projects that create the most positive net value to the participating systems . It distinguishes between states, federal government, and planning regions as key actors in implementing solutions designed to be flexible to accommodate regional differences.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Sovereign Credit Rating Processes Adapted to Critical Infrastructure Cyber Risk Assessment

United States critical infrastructure entities are increasingly targeted by motivated and capable threat actors and must be prepared to assess and treat a diverse range of cyber risks. Consequently, this necessitates some form of analytical process to evaluate risks and inform cyber security investment decisions. A potential solution for structuring cyber risk evaluation exists within the field of sovereign credit ratings – where agencies employ mature approaches that integrate quantitative and qualitative data to produce a singular value of assessment. Adapting such approaches, we present a novel criterion and methodology for measuring and communicating the likelihood element of cyber risk. The methodology is composed of three sequential phases: a quantitative baseline organized by distinct capability frames, a bounded qualitative adjustment per frame, and a greater-bounded qualitative adjustment spanning the entire process. The process culminates in publication of a cyber capability rating that communicates a critical infrastructure entity’s ability and willingness to mitigate discontinuous function due to cyberattack.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗