Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Practitioner”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

AI-Ready Data Pilot Project Report

The proliferation of artificial intelligence in scientific research has created an urgent need to define "AI-ready data" for researchers and, more importantly, provide resources to help them produce AI-ready data. At Pacific Northwest National Laboratory, we conducted a pilot study with three data scientists evaluating three CSV datasets from different scientific domains, followed by semi-structured interviews capturing assessment practices. Our findings reveal that AI-readiness evaluation is intuition-based, with practitioners asking "How fast can I go from raw data to my machine learning pipeline?" Data scientists consistently prioritized workflow efficiency, human interpretability, and quality stewardship signals. From these insights, we developed a practical evaluation framework comprising data requirements, metadata standards, and validation tests that provides actionable criteria for producing and curating AI-ready datasets, addressing the gap between theoretical understanding and practical implementation.

97 MATHEMATICS AND COMPUTING↗

When and Why Hiring and Training Occur: A Survey of Building Efficiency Contractors

As state and local governments, along with homeowners, work to enhance building energy performance, a sufficiently skilled energy-efficiency workforce is essential for implementing solutions at scale. In recent years, employers in the energy efficiency sector, including building performance contractors, have frequently reported challenges finding qualified workers. Additionally, as building performance technologies evolve and become more sophisticated, current workers need to acquire new skills and practices to maintain high-quality work. This study surveyed 209 contractors involved in building energy performance projects across 37 U.S. states, Washington, D.C., Guam, and the Marshall Islands. The survey aimed to understand their experiences in the labor market, their business practices, and their decision-making processes when hiring new workers and training existing employees. The findings highlight contractors' motivations and considerations in hiring and training, as well as the most significant challenges they face in the labor market. These insights can help workforce development practitioners design training programs that are aligned with workforce needs and responsive to employer concerns.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Outcomes of PAX sapiens-Supported Global Wildlife Data Sharing Conferences for Enhanced One Health Security (GWDSC)

Across two consecutive Global Wildlife Data Sharing Conferences supported by PAX sapiens—Year 1 (May 2024) at Pacific Northwest National Laboratory and Year 2 (2025) in Ciudad Real, Spain—the initiative converted wildlife data sharing from aspiration into operational reality, producing measurable impacts in platform development, data mobilization, standards harmonization, and international partnership formation. The conferences addressed a critical gap in global health security: while 75% of emerging infectious diseases affect both humans and animals and over 60% originate in wildlife, wildlife health surveillance has historically lagged behind human and agricultural sectors due to fragmented databases, inconsistent terminology, uneven capacity, and limited cross-border coordination. By convening practitioners, government agencies, international organizations, academic institutions, and NGOs, the GWDSC catalyzed trust-based relationships and practical workflows that enable earlier detection, better risk assessment, and more effective prevention of threats at the wildlife–domestic animal–human–environment interface.

54 ENVIRONMENTAL SCIENCES↗

A Systematic Review and Integrated Approach to Modeling of Aging Utility Scale PV Systems

The growing deployment of utility-scale photovoltaic (PV) systems has increased the importance of techno-economic modeling operational photovoltaic (PV) systems for predicting energy yield, optimizing asset management, and informing financial decisions. Through a systematic review of literature and current industry practices, we review the different common modeling practices of a system's configuration and age, performance and degradation, operation and maintenance (O&M), while also focusing on specific considerations for repowering, revamping, and decommissioning. Building on the synthesis, we develop a structured framework for techno-economic modeling of operating PV systems that integrate performance and degradation analysis, a decommissioning and repowering cost model that estimates the system's end-of-life costs to reduce uncertainty quantifications and improve consistency across the sector. This research contributes to improved modeling methodologies and potentially to reduced financial performance requirements by providing practitioners with input resources and practical approaches to estimate performance, degradation, and costs associated with continued operation, revamping, repowering, or decommissioning decisions.

14 SOLAR ENERGY↗

Customer-Focused Key Performance Indicators for Electric Vehicle Charging

To systematically improve the public charging experience, EV charging industry stakeholders need to define and measure it precisely. Many stakeholders currently measure aspects of the charging experience, but they typically employ metrics that are either operational in nature, such as charger uptime and mean time between failures, or composite customer satisfaction indices. To improve the customer experience most effectively, the industry needs metrics that define the charging experience from the perspective of the customer, not business operations. Furthermore, industry practitioners need granular metrics to know what specific aspects of the charging experience need improvement. This report defines such customer-focused metrics, called key performance indicators (KPIs).

33 - ADVANCED PROPULSION SYSTEMS↗

Precursor Analysis Report: Ryuk Ransomware Attack on Universal Health Services 2020

The Ryuk Ransomware Attack on Universal Health Services (UHS) 2020 Precursor Analysis Report leverages publicly available information about the 2020 UHS cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. UHS manages over 400 hospitals and is one of the largest healthcare providers in the United States with 3.5 million patients each year. On 27 September 2020, UHS suffered a widespread ransomware attack that resulted in a denial of service to critical internet-dependent healthcare systems including workstations, phones, and data centers. Employees resorted to filing patient details with pen and paper, while other facilities had to redirect ambulances and urgent patients to other facilities for adequate care. Adversaries carried out the attack with Ryuk, a ransomware that encrypts data and generates a RyukReadMe.txt ransom note with the ransom fee to decrypt the data, varying from 15 Bitcoin (BTC) to 50 BTC, equivalent to roughly $\$$353,892 to $\$$964,617. UHS did not pay the ransom and was able to recover data through backups, but still reported an impact of $\$$67 million dollars in recovery costs. On 29 October, one month after the attack, UHS made an official statement that their systems had been restored and they were resuming normal operations. Researchers and analysts identified 18 unique techniques (used in a sequence of 19 steps) utilized during the attack with a total of 185 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Fourteen of the identified techniques used during the UHS cyber attack were precursors to the triggering event. Analysis identified 106 observables associated with these precursor techniques, 82 of which were assessed to have an increased likelihood of being perceived in the 30 days to two hours preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: JBS Foods Ransomware Attack 2021

The JBS Foods 2021 Ransomware Attack Precursor Analysis Report leverages publicly available information about the JBS ransomware cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. In late May 2021, one of the world’s largest meat producers, JBS Foods, announced they had fallen victim to a worldwide ransomware attack, later found to be REvil ransomware. In the United States alone, JBS Foods accounts for nearly 25% of beef and roughly 20% of pork production. Adversaries initially launched a Distributed Denial of Service (DDoS) attack on the company’s Information Technology (IT) networks in Australia, but the attack impacted operations in Brazil, Canada, and the United States, as well. The attack caused plant operations in all four countries to shut down for at least one day. All nine of the U.S. meatpacking plants temporarily shut down because of the attack. The adversaries initially demanded a $\$$22 million ransom for the company’s data, but later negotiated the ransom down to $\$$11 million even after JBS Foods restored most of their systems. JBS Foods eventually paid the $\$$11 Million for reassurance from the adversaries that none of their customers’ data would be compromised in the future. Despite its short duration, the attack still caused large stocks of meat to spoil. The incident also underscored how adversaries can simultaneously compromise and move laterally through global subsidiaries of an organization. Researchers and analysts identified 22 unique techniques (in a sequence of 21 steps) utilized during the attack with a total of 361 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Sixteen of the identified techniques used during the JBS Foods cyber attack were precursors to the triggering event. Analysis identified 308 observables associated with these precursor techniques, 163 of which were assessed to have an increased likelihood of being perceived in the 75 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: SQL Slammer Worm Infection of Davis-Besse Nuclear Power Plant 2003

The SQL Slammer Worm Infection of Davis-Besse Nuclear Power Plant 2003 Precursor Analysis Report leverages publicly available information about Davis-Besse’s 2003 cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. On 25 January 2003, the SQL Slammer worm infected more than 90% of vulnerable hosts and crashed the internet in 10 to 15 minutes, making it one of the fastest spreading worms in history. SQL Slammer is a fileless, memory-resident worm that remotely exploits a stack-based buffer overflow vulnerability on local hosts to intensively scan and rapidly self-propagate across the internet. The worm infected approximately 300,000 unpatched hosts running Microsoft Structured Query Language (SQL) Server 2000 or Microsoft Desktop Engine (MSDE) 2000 with SQL Server Resolution Service. The SQL Slammer worm indirectly infected FirstEnergy’s Davis-Besse nuclear power plant by first infecting a consultant’s company network server and then propagating through an external misconfigured connection into Davis-Besse’s site network. The infection caused major network congestion, slow performance, data overloads, and the inability of local hosts to communicate with each other, which eventually caused a loss of availability and a loss of view when the Safety Parameter Display System (SPDS) and Plant Process Computer (PPC) crashed. At the time of the infection, the plant was already offline, the digital monitoring systems had redundant analog backups, and the plant control and safety functions were not affected, so there were no concerns of a safety breach. However, this incident resulted in many lessons learned and spawned important discussions about cybersecurity’s role in nuclear safety and electric power reliability regulation, policy, and guidance. Researchers and analysts identified 10 unique techniques utilized during the attack with a total of 640 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Eight of the identified techniques used during Davis-Besse cyber attack were precursors to the triggering event. Analysis identified 596 observables associated with these precursor techniques, 428 of which were assessed to have an increased likelihood of being perceived in the 331 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Blackmatter Ransomware Attack on New Cooperative 2021

The BlackMatter Ransomware Attack on New Cooperative 2021 Precursor Analysis Report leverages publicly available information about the New Cooperative cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. The BlackMatter ransomware was first identified in July 2021 and is reported to have infected more than 50 corporations around the world. , The Iowa-based grain cooperative, New Cooperative, was impacted by the BlackMatter ransomware on or before 18 September 2021. The adversary likely resided on New Cooperative’s networks for 15 days prior to encrypting its network and demanding New Cooperative pay $5.9 million in ransom by 25 September to unlock systems and prevent 1 terabyte (TB) of sensitive data from being publicly released. It is not clear if New Cooperative paid the ransom. The full impact of the ransomware attack is not known; however, according to New Cooperative’s general manager, the attack caused the company’s automated processes to revert back to processes used in the 1970s. , As of 6 October, only 50 percent of New Cooperative’s operations were utilizing automated processes. The company took eight weeks to rebuild the entire network and information technology (IT) systems from the ground up, which puts the date of fully recovery around 13 November. Researchers and analysts identified 20 unique techniques utilized during the attack with a total of 404 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Seventeen of the identified techniques used during the New Cooperative cyber attack were precursors to the triggering event. Analysis identified 360 observables associated with these precursor techniques, 284 of which were assessed to have an increased likelihood of being perceived in the 15 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Industroyer2 and Wiper Malware Targeting Ukrainian Energy Provider 2022

The Industroyer2 and Wiper Malware Targeting Ukrainian Energy Provider 2022 Precursor Analysis Report leverages publicly available information about the Industroyer2 cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. An adversary attempted to cause a blackout in Ukraine in April 2022 by using the Industroyer2 malware against a regional Ukrainian energy provider. The adversary targeted eight high-voltage electrical substations and utilized the malware in tandem with disk wipers for Windows, Linux, and Solaris operating systems in an attempt to make response and recovery efforts more difficult. The adversary reused a piece of the original Industroyer malware designed to open circuit breakers and de-energize target substations. The adversary gained initial access to the victim’s enterprise network through unknown means in February 2022 and was able to perform reconnaissance, pivot to the operations network, and reside in the system for at least 51 days. This gave the adversary a detailed understanding of the environment and allowed them to customize the Industroyer2 malware to the victim’s operations network. However, defenders detected and stopped the attack before the adversary could achieve their intended impact. Had the Industroyer2 attack been successful, it could have caused a blackout for more than two million people during the early stages of Russia’s invasion of Ukraine. Researchers and analysts identified 22 unique techniques (used in a sequence of 31 steps) utilized during the attack with a total of 297 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-three of the identified techniques used during the Industroyer2 cyber attack were precursors to the triggering event. Analysis identified 224 observables associated with these precursor techniques, 122 of which were assessed to have an increased likelihood of being perceived in the 51 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Industroyer Targeting Ukraine Electric Power Transport Utility (Ukrenergo) 2016

The Industroyer Targeting Ukraine Electric Power Transport Utility (Ukrenergo) 2016 Precursor Analysis Report leverages publicly available information about the December 2016 cyber attack against the Ukrainian Ukrenergo electric transmission utility and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. Industroyer is a modular malware framework designed to deploy several Industrial Control System (ICS) protocol-specific attack payloads to disrupt electricity distribution. Adversaries deployed Industroyer within the target network on a Microsoft Windows endpoint capable of directly manipulating or communicating with ICS. Industroyer abuses the functionality of a targeted ICS’s legitimate control system to achieve its intended impact. Adversaries likely first gained access to Ukrenergo enterprise networks in early 2016 after a successful spearphishing campaign against organizations in the electric power sector. Adversaries then began capturing credentials beginning on 1 December 2016. This allowed access to the ICS environment at the Pivnichna electric transmission substation outside Kyiv through a device dual-homed on the Information Technology (IT) and ICS networks. Adversaries conducted discovery, targeting, and access to this device using information and previously captured credentials from compromised enterprise IT machines. Finally, the adversaries deployed and launched the Industroyer malware just before midnight on 17 December. By midnight, Ukrenergo had lost control of a targeted substation, resulting in electric power outages for over an hour in the city of Kyiv and the Kyiv region. Researchers and analysts identified 31 unique techniques (used in a sequence of 33 steps) utilized during the attack with a total of 846 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-nine of the identified techniques used during the Industroyer cyber attack were precursors to the triggering event. Analysis identified 548 observables associated with these precursor techniques, 353 of which were assessed to have an increased likelihood of being perceived in the 300 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Conti Ransomware Attack on the Health Service Executive of Ireland 2021

The Conti Ransomware Attack on the Health Service Executive (HSE) of Ireland 2021 Precursor Analysis Report leverages publicly available information about the attack and catalogs anomalous observables for each technique employed by the adversary. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. The HSE provides public healthcare corporate services and operational services throughout Ireland, with critical functions including the acute national ambulance service, acute hospital service, and community healthcare service. On 14 May 2021, Conti ransomware encrypted 80 percent of the HSE’s Information Technology (IT) infrastructure across corporate, hospital, community, and electronic health record services. Conti is a ransomware-as-a-service operation that encrypts local files, uses double extortion against victims, and is facilitated by many intrusion tools. The attack forced the HSE to shut down its entire IT infrastructure to contain the ransomware, forcing employees to revert to pen and paper recordkeeping and leading to the cancellation of many appointments and procedures. The adversary also exfiltrated 700 GB of data, compromising the confidentiality of patients’ protected health information. Had the adversary targeted the COVID-19 cloud systems or operational technology assets, such as Internet of Medical Things medical devices or smart building management systems, the impact of the attack would almost certainly have been far more severe. Researchers and analysts identified 21 unique techniques (used in a sequence of 23 steps) likely utilized during the attack with a total of 1,185 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-one of the identified techniques used during the attack on the HSE were precursors to the triggering event. Analysis identified 1,086 observables associated with these precursor techniques, 850 of which were assessed to have an increased likelihood of being perceived in the 57 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Cyber Attack on Thyssenkrupp Blast Furnace 2014

The Cyber Attack on Thyssenkrupp Blast Furnace 2014 Precursor Analysis Report leverages publicly available information about the Thyssenkrupp Steel Mill cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. In December 2014, the German Government’s Federal Office for Information Security (BSI) released a report detailing a cyber attack on a German steel mill that occurred earlier that year, though exact dates and details of the attack were not revealed. While the report did not specify the name of the company, multiple sources identified the victim as one of Europe’s largest steel manufacturers, Thyssenkrupp AG. Further, Thyssenkrupp announced on 16 May of that year that Europe’s largest blast furnace, “Schwelgern 2,” located at its facility in Duisburg, Germany, would be offline for several weeks for repairs and upgrades, suggesting Schwelgern 2 was likely the target of the attack. The attack began in early 2014, when adversaries infiltrated the victim steel mill’s Information Technology (IT) network via a spearphishing campaign, then worked their way into the Operational Technology (OT) environment, where they executed software that caused denial of service, denial of control, and eventually a loss of control. This led to the blast furnace shutting down without proper safety procedures, resulting in catastrophic physical damage. No lives were lost in the incident, but ThyssenKrupp suffered $4 million in damage to the blast furnace and an additional $6 million in lost revenue. The adversaries required specialized knowledge and expertise in steel production, which enabled them to compromise a variety of internal systems and components across both IT and OT networks. The attack also demonstrated detailed knowledge of the industrial control systems (ICS) and production processes being used. This combination resulted in one of the earliest known publicly reported cybersecurity incidents resulting in physical damage to ICS equipment. Researchers and analysts identified 19 unique techniques (used in a sequence of 20 steps) utilized during the attack with a total of 454 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Fifteen of the identified techniques used during the Thyssenkrupp cyber attack were precursors to the triggering event. Analysis identified 369 observables associated with these precursor techniques, 316 of which were assessed to have an increased likelihood of being perceived in the 120 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Doppelpaymer Ransomware Attack on Petroleos Mexicanos (PEMEX) 2019

The DoppelPaymer Ransomware Attack on Petroleos Mexicanos (PEMEX) 2019 Precursor Analysis Report leverages publicly available information about the PEMEX cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. The 2019 DoppelPaymer ransomware attack on PEMEX, Mexico’s nationalized petroleum corporation, highlights a unique threat that ransomware and cybercriminal extortion poses to Operational Technology (OT) environments in critical infrastructure. The incident began with an employee downloading commodity malware that allowed adversaries to gain initial access to PEMEX’s enterprise environment. After conducting privilege escalation, tool ingress, and data exfiltration, the adversaries deployed DoppelPaymer ransomware throughout the PEMEX enterprise environment, resulting in the company having to take dozens of systems offline for at least several days. Although PEMEX stated that their operations were not affected, the data exfiltrated from PEMEX was made available for download on DoppelPaymer’s leak site, as well as on other illicit criminal forums. This stolen data included not only company information, but also sensitive OT-specific configuration data. This incident showcases how cybercriminal exfiltration and posting of sensitive OT architecture documentation can pose security concerns for the targeted organization for years due to the long lifespan of OT assets and architectures. Researchers and analysts identified 18 unique techniques utilized during the attack with a total of 190 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Fifteen of the identified techniques used during the DoppelPaymer ransomware attack were precursors to the triggering event. Analysis identified 163 observables associated with these precursor techniques, 34 of which were assessed to have an increased likelihood of being perceived in the 60 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Precursor Analysis Report: Remote Access Attack on Oldsmar Water Treatment Facility 2021

The Remote Access Attack on Oldsmar Water Treatment Facility 2021 Precursor Analysis Report leverages publicly available information about the Oldsmar cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. On 5 February 2021, an adversary gained unauthorized remote access to Bruce T. Haddock Water Treatment Plant in Oldsmar, Florida, which provides treated water to 15,000 customers. The adversary accessed the facility’s Supervisory Control and Data Acquisition (SCADA) workstation and human machine interface (HMI) to change the chemical concentration of sodium hydroxide, commonly referred to as lye and used to regulate acidity levels, from 100 parts per million (PPM) to 11,100 PPM. The chemical was raised to lethal levels that if ingested could lead to serious soft tissue damage, burns, or even death. The facility, however, had redundancies and alarms in place to alert personnel of dangerous chemical levels, and facility officials stated it would have taken 24 to 36 hours for the chemical changes to affect the water supply. Researchers and analysts identified six unique techniques utilized during the attack with a total of 23 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Four of the identified techniques used during the Oldsmar cyber attack were precursors to the triggering event. Analysis identified 21 observables associated with these precursor techniques, 20 of which were assessed to have an increased likelihood of being perceived in the minutes preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

What Is the Agent Doing? Visualizing Agentic AI Querying Workflows

We explore how visualizations can help users understand what an AI agent is doing as it builds and runs queries over data. As part of the LinkQ system, a natural language interface for querying knowledge graphs with a large language model (LLM), we designed two complementary views: A State Diagram that shows where the agent is within a larger workflow, and a Live Action Display that gives real-time updates about the agent's current task. In a study with 14 practitioners, we found that these visuals helped participants build stronger mental models of the agent's behavior while also increasing their confidence in the system. However, we also observed that users sometimes trusted incorrect outputs simply because the agent appeared to be doing the "right" thing. Our findings point to both the value and risk of visualizing agent behavior in interactive AI systems.

97 MATHEMATICS AND COMPUTING↗

Support for the Core Research Activities and Studies of the Computer Science and Telecommunications Board (CSTB)

The Computer Science and Telecommunications Board of the National Research Council considers technical and policy issues pertaining to computer science (CS), telecommunications, and information technology (IT). The functions of the board include: (1) monitoring and promoting the health of the CS, IT, and telecommunications fields, including attention as appropriate to issues of human resources and funding levels and program structures for research; (2) initiating studies involving CS, IT, and telecommunications as critical resources and sources of national economic strength; (3) responding to requests from the government, non-profit organizations, and private industry for expert advice on CS, IT, and telecommunications issues; and to requests from the government for expert advice on computer and telecommunications systems planning, utilization, and modernization; (4) fostering interaction among CS, IT, and telecommunications researchers and practitioners, and with other disciplines; and providing a base of expertise in the National Research Council in the areas of CS, IT, and telecommunications. This award has supported the overall operation of CSTB. Reports resulting from the Board's efforts have been widely disseminated in both electronic and print form, and all CSTB reports are available at its World Wide Web home page at cstb.org. The following reports, resulting from projects that were separately funded by a wide array of sponsors, were completed and released during the award period: 2007: * Summary of a Workshop on Software-Intensive Systems and Uncertainty at Scale * Social Security Administration Electronic Service Provision: A Strategic Assessment * Toward a Safer and More Secure Cyberspace * Software for Dependable Systems: Sufficient Evidence? * Engaging Privacy and Information Technology in a Digital Age * Improving Disaster Management: The Role of IT in Mitigation, Preparedness, Response, and Recovery 2006: * Renewing U.S. Telecommunications Research * Letter Report on Electronic Voting * Summary of a Workshop on the Technology, Policy, and Cultural Dimensions of Biometric System 2005: * Catalyzing Inquiry at the Interface of Computing and Biology * Summary of a Workshop on Using IT to Enhance Disaster Management * Asking the Right Questions About Electronic Voting * Building an Electronic Records Archive at NARA: Recommendations for a Long-Term Strategy * Signposts in Cyberspace: The Domain Name System and Internet Navigation 2004: * ITCP: Information Technology and Creative Practices (brochure) * Radio Frequency Identification (RFID) Technologies: A Workshop Summary * Getting up to Speed: The Future of Supercomputing * Summary of a Workshop on Software Certification and Dependability * Computer Science: Reflections on the Field, Reflections from the Field CSTB conducted numerous briefings of these reports and transmitted copies of these reports to researchers and key decision makers in the public and private sectors. It developed articles for journals based on several of these reports. As requested, and in fulfillment of its congressional charter to act as an independent advisor to the federal government, it arranged for congressional testimony on several of these reports. CSTB also convenes a number of workshops and other events, either as part of studies or in conjunctions with meetings of the CSTB members. These events have included the following: two 2007 workshops explored issues and challenges related to state voter registration databases, record matching, and database interoperability. A Sept. 2007 workshop, Trends in Computing Performance, explored fundamental trends in areas such as power, storage, programming, and applications. An Oct. 2007, workshop presented highlights of CSTB's May 2007 report, Software for Dependable Systems: Sufficient Evidence?, along with several panels discussing the report's conclusions and their implications. A Jan. 2007 workshop, Uncertainty at Scale, explored engineering uncertainty, system complexity, and scale issues in developing large software systems. A Feb. 2007 workshop explored China's and India's roles in the IT R&D ecosystem; observations about the ecosystem over the long term; perspectives from serial entrepreneurs about the evolution of the ecosystem; and a cross-industry, global view of the R&D ecosystem. A Nov. 2006 event brought together participants from government, industry, and academia to share their perspectives on the health of the ecosystem, patterns of funding and investment, and the Potomac-area IT startup environment. A symposium entitled 2016, held in Oct. 2006, featured a number of distinguished speakers who shared their views on how computer science and telecommunications will look in 10 years. This well-attended event was also the subject of an Oct. 31, 2006, feature essay in the New York Times, "Computing, 2016: What Won't Be Possible?"

97 MATHEMATICS AND COMPUTING↗

Final Report-Optimization Under Uncertainty and Nonconvexity: Algorithms and Software

The goal of this research was to develop new algorithmic techniques for solving large-scale numerical optimization problems, focusing on problems classes that have proven to be among the most challenging for practitioners: those involving uncertainty and those involving nonconvexity. This research advanced the state-of-the-art in solving mixed integer linear programs containing symmetry, mixed integer nonlinear programs, and stochastic optimization problems.

97 MATHEMATICS AND COMPUTING↗