K-anonymization
New algorithms to de-identify geolocations. A unique feature of our algorithms is the ability to compare de-identified values that are similar but not the same
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
New algorithms to de-identify geolocations. A unique feature of our algorithms is the ability to compare de-identified values that are similar but not the same
This is a combined experimental and theoretical study of the compound Cs 2 PuO 2 Cl 4 , using X-ray Photoelectron Spectroscopy (XPS) and Relativistic Discrete Variation (RDV) calculations.
This is a combined experimental and theoretical study of the compound UO 2 , UO 2 (NO 3 ) 2 (H 2 0) 6 and UO 0.75 Pu 0.25 O 2 , using resonant inelastic x-ray scattering (RXIS), high resolution x-ray absorption (XAS) and LDA and LDA-U calculations.
Explore the source record for details and available documents.
Presentation slides describing the GISA Tool, a product of EDX4CCS task 46.
The National Renewable Energy Laboratory (NREL) has deep experience developing secure, national data repositories, which it augments with analysis, technology and market expertise, high-performance computing, and innovative data visualization. By adapting the architecture of existing mobility databases (i.e., Fleet DNA, the Transportation Secure Data Center, the National Fuel Cell Technology Evaluation Center), NREL can build a powerful aviation data clearinghouse - Flight DNA - that helps stakeholders navigate the web of pitfalls and possibilities generated by new aviation technologies.
Carbon Storage BIL Workshop, Santa Fe, NM, May 1-3, 2023
Background & Purpose: RIRAS is a web-based information system deployed on the Veterans Health Administration intranet in early 2014 to collect adverse events and good catch data; analyze the causes and contributing factors; and find ways to prevent future occurrences. Material and Methods: Incident learning consists of a feedback loop which starts with reporting an event, followed by analysis of contributing factors, and culminates in the development of a patient safety work product (PSWP) to prevent recurrence. RIRAS permits both anonymous and non-anonymous reporting. Each report is analyzed by a team of medical physicists who are independent of the reporting facility. The analysts usually contact the reporting facilities for additional information. We analyzed all reports and held telephonic interviews (when necessary) with the reporters. We then generated PSWPs with corrective/preventive and learning actions. Anonymous reporting is handled in the same manner, except without the ability to further interview the reporter. Results: In a significant number of reports, the causes and recommended preventive actions were considerably altered by the independent analysis and additional information from the facility. 130 reports have been entered in RIRAS; 9 misadministrations, 83 good catches, 3 anonymous good catches, and 35 earlier reported incidents from FY2005-14. 45% of the reported incidents occurred in the treatment delivery stages, 19% in on-treatment management, and 16% in pre-treatment verification. 80% of the good catches were found in the treatment delivery workflow. Majority of these incidents were due to inconsistent patient setup instructions or documentation, nonadherence to policies and procedures, lax time-out policy, distracted RTTs, and inadequate RTT staffing. Conclusion: RIRAS has identified many areas for improvement and elevated the quality and safety of radiation treatments in the VHA. We found that the ability to learn is significantly diminished when the analysts do not have the ability to request additional information.
Vehicle data consist of electric vehicle performance data collected directly from the vehicle during standard operations. Data were collected using onboard data loggers that were either installed by the project team or preinstalled by the original equipment manufacturer. Data recorded by the data loggers were made accessible via an online web portal or an application programming interface. Different data loggers were used (HEM, ViriCiti, and Geotab), and the method for each vehicle is defined in the vehicle attributes file. Some systems collected data on a “trip-level” basis, in which each row of a table represents a single trip (the period between a key-on and key-off event), whereas other data were collected on a per-day basis, in which each row represents a single day of operation. Data were collected over a range of data collection periods, depending on the project. Data have been anonymized by removing information or decreasing information resolution as necessary so that fleets are not identifiable. Due to the wide range of vehicle types represented and variation in data collection, data parameters and frequencies differ between vehicles and fleets The **Performance Data Daily/Trip Data Dictionaries** contain definitions for each available parameter associated with a vehicle’s operations, aggregated at either a daily or trip level. The parameters available will vary from vehicle to vehicle, but every possible parameter will be defined. The **Vehicle Attributes Data Dictionary** contains definitions for each available parameter associated with a vehicle’s physical and functional attributes and fleet context. The **Vehicle Attributes** table contains specific vehicle characteristics, coded to an anonymous Vehicle ID. This Vehicle ID can be used as a key between vehicle data and vehicle attribute tables. The **Vehicle Data** tables contain the data from each vehicle’s operations, aggregated at either a daily or trip level, coded to an anonymous Vehicle ID. This Vehicle ID can be used as a key between vehicle data and vehicle attribute tables. Data is being uploaded quarterly through 2023 and subject to change until the conclusion of the project.
Charging data are collected from one of three sources, each with varying levels of additional information. These sources, in approximate order from most to least additional information, are: • The electric vehicle supply equipment (charger) • Onboard the vehicle itself • From a utility submeter. Many chargers provide software that allows for the collection and reporting of charging session data. If unavailable, data may be recorded by the charging vehicle’s onboard systems. If neither of these options is available, data can be acquired from utility submeters that simply track the energy flowing to one or more chargers. Data collected directly from the electric vehicle supply equipment (EVSE) are typically the most accurate and highest frequency. However, it is not always possible to discern which exact vehicle is being charged during any one session. EVSE-side data can be identified where a single charger ID but a range of vehicle IDs are present (e.g., CH001, EV001-EV005). Data collected from the vehicle’s onboard systems usually does not provide information on which exact charger is being used. Vehicle-side data can be identified where a single Vehicle ID but a range of Charger IDs are present (e.g., EV001, CH001-CH005). Data collected from utility submeters provide no information on which specific vehicle is charging or which specific charger is in use. Submeter data can be identified where multiple Vehicle IDs and multiple Charger IDs are present, but only a single Fleet ID is present (e.g., EV001-EV005, CH001-CH005, Fleet01). The **Charge Data Daily/Session Dictionaries** contains definitions for each available parameter collected as part of an individual charging session, aggregated at either a daily or session level. The parameters available will vary between vehicles and chargers. The **Charger Attributes** table contains specific charger characteristics, coded to at least one anonymous Charger ID and linked to either a single or a range of Vehicle IDs. Vehicle ID can be used as a key between charging data and vehicle attribute tables. The **Charger Attributes Data Dictionary** contains definitions for each available parameter collected on the physical and operational characteristics of the charging hardware itself. The **Vehicle Attributes Data Dictionary** contains definitions for each available parameter associated with a vehicle’s physical and functional attributes and fleet context. The **Vehicle Attributes** table contains specific vehicle characteristics, coded to an anonymous Vehicle ID. This Vehicle ID can be used as a key between vehicle data and vehicle attribute tables, and in cases where charging data are supplied, links a vehicle with the charger(s) that supplied it power. The **Charging Data** tables contain the data from each charger’s operations, coded to at least one anonymous Charger ID and linked to either a single or a range of Vehicle IDs. Vehicle ID can be used as a key between charging data and vehicle attribute tables. Data is being uploaded quarterly through 2023 and subject to change until the conclusion of the project.
Overview: Anonymized job-level records from the Kestrel HPC system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, utilization, energy estimates, and efficiency metrics. Sensitive fields (user, account, job name, submit line, working directory, submit script, and job type) are replaced with 7-character cryptographic hashes. System & Timeframe: Kestrel is located at the NLR campus. Standard compute nodes have 104 cores and 256 GB RAM; bigmem nodes have 2,000 GB. GPU nodes (gpu-h100 partition) use NVIDIA H100 GPUs. Data covers jobs submitted August 2023 through December 2025. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.kestrel.job-anon.zip — Anonymized job records (Hive-partitioned Parquet) datacard.md — Full dataset documentation ~11 million rows, 50 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct with timezone-aware export (SLURM_TIME_FORMAT="%Y-%m-%dT%H:%M:%S%z"), loaded into PostgreSQL. Calculated columns updated via database triggers and batch functions. All timestamps use timestamptz and correctly handle DST transitions. Preprocessing: Anonymization of name, user, account, submit_line, work_dir, submit_script, and job_type via 7-char hex hashes Derived columns: queue_wait, cpu_eff, max/min/avg_mem_eff, energy estimates Simplified job state mapping (e.g., "CANCELLED by 132357" → "CANCELLED") Boolean flags: python_job, reframe_job Temporal decomposition: year, month, day, day_of_week, hour, minute from submit_time Shared node tracking: shared_job_count, nodes_shared, jobs_shared Key Variables: Scheduling: job_id, partition, state_simple, submit_time, start_time, end_time, queue_wait Resources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max/min/avg_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, consumed_energy_raw_joules, consumed_energy_raw_watt_hours Sharing: shared_job_count, nodes_shared, jobs_shared Partitions: short, standard, debug, gpu-h100 Job States: CANCELLED, COMPLETED, FAILED, PENDING, RUNNING QoS Levels: normal, high Important Notes: Timestamps include timezone offsets; DST transitions are handled correctly, though adding intervals across DST boundaries requires offset adjustment shared_job_count reflects physical node co-residency, not use of the shared partition Job step records and raw Slurm JSONB fields are excluded Do not attempt to re-identify individuals from hashed fields
Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields
Purpose: Over the past 30 years, much work in treatment of breast cancer has contributed to improvement of cosmetic and functional outcomes. The goal of breast-conservation treatment (BCT) is avoidance of mastectomy through use of lumpectomy and adjuvant radiation. Modern data demonstrate 'excellent' or 'good' cosmesis in >90% of patients treated with BCT. Methods and Materials: Patient-reported data were gathered via a convenience sample frame from breast cancer survivors using a publically available, free, Internet-based tool for creation of survivorship care plans. During use of the tool, breast cancer survivors are queried as to the cosmetic appearance of the treated breast, as well as perceived late effects. All data have been maintained anonymously with internal review board approval. Results: Three hundred fifty-four breast cancer survivors having undergone BCT and voluntarily using this tool were queried with regard to breast cosmesis and perceived late effects. Median diagnosis age was 48 years, and median current age 52 years. 'Excellent' cosmesis was reported by 27% (n = 88), 'Good' by 44% (n = 144), 'Fair' by 24% (n = 81), and 'Poor' by 5% (n = 18). Of the queries posted to survivors after BCT, late effects most commonly reported were cognitive changes (62%); sexual concerns (52%); changes in texture and color of irradiated skin (48%); chronic pain, numbness, or tingling (35%); and loss of flexibility in the irradiated area (30%). Survivors also described osteopenia/osteoporosis (35%), cardiopulmonary problems (12%), and lymphedema (19%). Conclusions: This anonymous tool uses a convenience sample frame to gather patient reported assessments of cosmesis and complications after breast cancer. Among the BCT population, cosmetic assessment by survivors appears less likely to be 'excellent' or 'good' than would be expected, with 30% of BCT survivors reporting 'fair' or 'poor' cosmesis. Patient reported incidence of chronic pain, as well as cognitive and sexual changes, also appears higher than expected.
The analysis of vast amounts of data and the processing of complex computational jobs have traditionally relied upon high performance computing (HPC) systems, which offer reliable and efficient management of large-scale computational and data resources. Understanding these analyses’ needs is paramount for designing solutions that can lead to better science, and similarly, understanding the characteristics of the user behavior on those systems is important for improving user experiences on HPC systems. A common approach to gathering data about user behavior is to extract workload characteristics from system log data available only to system administrators. Recently at Oak Ridge Leadership Computing Facility (OLCF), however, we unveiled user behavior about the Summit supercomputer by collecting data from a user’s point of view with ordinary Unix commands.In this paper, we discuss the process, challenges, and lessons learned while preparing this dataset for publication and submission to an open data challenge. The original dataset contains personal identifiable information (PII) about the users of OLCF which needed be masked prior to publication, and we determined that anonymization, which scrubs PII completely, destroyed too much of the structure of the data to be interesting for the data challenge. We instead chose to pseudonymize the dataset, which reduced the linkability of the dataset to the users’ identities. Pseudonymization is significantly more computationally expensive than anonymization, and the size of our dataset, which is approximately 175 million lines of raw text, necessitated the development of a parallelized workflow that could be reused on different HPC machines. We demonstrate the scaling behavior of the workflow on two leadership class HPC systems at OLCF, and we show that we were able to bring the overall makespan time from an impractical 20+ hours on a single node down to around 2 hours. As a result of this work, we release the entire pseudonymized dataset and make the workflows and source code publicly available.
In this paper we introduce a location deidentification procedure that uses road network structure to protect against certain types of inference-based attacks. Our target is large databases containing vehicle locations. Previous anonymization approaches are inappropriate because location generalization and perturbation of geopositions could negatively affect development of safety-critical applications that require precise position information. Furthermore, k-anonymity-based clustering approaches would lead to significant data suppression. Our algorithm attempts to balance privacy protection and data utility, while protecting against re-identification attacks. Our data is from the first connected vehicle model deployment in the United States.
Power outages disrupt critical infrastructure and cause billions of dollars in economic losses annually in the United States. Accurate and granular outage reporting is vital for effective restoration and mitigation. This paper examines the integration of the Hexagonal Hierarchical Geospatial Indexing System (H3) to enhance power outage reporting, leveraging its uniform grid structure, scalable resolutions, and support for privacy-preserving analysis. Using high-resolution LandScan Global population data and K-anonymization techniques, this work achieves a balance between data granularity and privacy. Results show that lower privacy thresholds (e.g., K-anonymity = 2) enable higher resolution, while stricter thresholds (e.g., >15 people per hex) reduce granularity, potentially affecting localized responses. State-and county-level resolution case studies demonstrate H3’s adaptability and the trade-offs between precision and privacy. The proposed H3-based framework offers a scalable and efficient solution for geospatial data integration within the energy sector, such as outage data, aiding utilities and regulators in improving resilience and response efforts, particularly in disaster-prone regions.
Cyber Threat Intelligence (CTI) is a key resource for developing defensive strategies against potential cyber adversaries. Entities typically access CTI through open-source platforms, national agencies, or specialized commercial services. However, the bi-directional exchange of CTI is hindered by organizational trust boundaries, which complicate the sharing processes between entities and CTI providers. Centralized CTI services benefit from receiving suspicious cyber observables such as IP addresses, domain names, and email addresses from various entities. The aggregation allows for the correlation of widespread adversarial activities to enhance the alert and response mechanisms across the network of involved parties. Despite these benefits, openly sharing such observables incurs potential legal, regulatory, and reputational risks for the disclosing entities.This paper introduces a system designed to facilitate the secure exchange of cyber observables across trust boundaries without compromising the anonymity of the sharing entities. Here, we propose an architecture that leverages common web protocols alongside zero-knowledge proofs to authenticate members while maintaining anonymity. Additionally, we outline a privacy model tailored for STIX (Structured Threat Information eXpression) cyber observables to minimize the risk of inadvertently disclosing private information. Through our threat models, we assess the privacy implications of our proposed system and demonstrate its potential to enhance collaborative cyber defense efforts without exposing entities to undue risk.
Cyber-physical systems have enabled the collection of massive amounts of data in an unprecedented level of spatial and temporal granularity. Publishing these data can prosper big data research, which, in turn, helps improve overall system efficiency and resiliency. The main challenge in data publishing is to ensure the usefulness of published data while providing necessary privacy protection. In our previous work (Jia et al. 2017a), we presented a privacy-preserving data publishing framework (referred to as PAD hereinafter), which can guarantee k-anonymity while achieving better data utility than traditional anonymization techniques. PAD learns the information of interest to data users or features from their interactions with the data publishing system and then customizes data publishing processes to the intended use of data. However, our previous work is only applicable to the case where the desired features are linear in the original data record. In this article, we extend PAD to nonlinear features. Our experiments demonstrate that for various data-driven applications, PAD can achieve enhanced utility while remaining highly resilient to privacy threats.