Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data harmonization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

SoDaH: the SOils DAta Harmonization database, an open-source synthesis of soil data from research networks, version 1.0

Data collected from research networks present opportunities to test theories and develop models about factors responsible for the long-term persistence and vulnerability of soil organic matter (SOM). Synthesizing datasets collected by different research networks presents opportunities to expand the ecological gradients and scientific breadth of information available for inquiry. Synthesizing these data is challenging, especially considering the legacy of soil data that have already been collected and an expansion of new network science initiatives. To facilitate this effort, here we present the SOils DAta Harmonization database (SoDaH; https://lter.github.io/som-website, last access: 22 December 2020), a flexible database designed to harmonize diverse SOM datasets from multiple research networks. SoDaH is built on several network science efforts in the United States, but the tools built for SoDaH aim to provide an open-access resource to facilitate synthesis of soil carbon data. Moreover, SoDaH allows for individual locations to contribute results from experimental manipulations, repeated measurements from long-term studies, and local- to regional-scale gradients across ecosystems or landscapes. Finally, we also provide data visualization and analysis tools that can be used to query and analyze the aggregated database. The SoDaH v1.0 dataset is archived and available at https://doi.org/10.6073/pasta/9733f6b6d2ffd12bf126dc36a763e0b4 (Wieder et al., 2020).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗
DOE OSTI2022

Multiview Incomplete Knowledge Graph Integration with application to cross-institutional EHR data harmonization

Objective: The growing availability of electronic health records (EHR) data opens opportunities for integrative analysis of multi-institutional EHR to produce generalizable knowledge. A key barrier to such integrative analyses is the lack of semantic interoperability across different institutions due to coding differences. We propose a Multiview Incomplete Knowledge Graph Integration (MIKGI) algorithm to integrate information from multiple sources with partially overlapping EHR concept codes to enable translations between healthcare systems. Methods: The MIKGI algorithm combines knowledge graph information from (i) embeddings trained from the co-occurrence patterns of medical codes within each EHR system and (ii) semantic embeddings of the textual strings of all medical codes obtained from the Self-Aligning Pretrained BERT (SAPBERT) algorithm. Due to the heterogeneity in the coding across healthcare systems, each EHR source provides partial coverage of the available codes. MIKGI synthesizes the incomplete knowledge graphs derived from these multi-source embeddings by minimizing a spherical loss function that combines the pairwise directional similarities of embeddings computed from all available sources. MIKGI outputs harmonized semantic embedding vectors for all EHR codes, which improves the quality of the embeddings and enables direct assessment of both similarity and relatedness between any pair of codes from multiple healthcare systems. Results: With EHR co-occurrence data from Veteran Affairs (VA) healthcare and Mass General Brigham (MGB), MIKGI algorithm produces high quality embeddings for a variety of downstream tasks including detecting known similar or related entity pairs and mapping VA local codes to the relevant EHR codes used at MGB. Based on the cosine similarity of the MIKGI trained embeddings, the AUC was 0.918 for detecting similar entity pairs and 0.809 for detecting related pairs. For cross-institutional medical code mapping, the top 1 and top 5 accuracy were 91.0% and 97.5% when mapping medication codes at VA to RxNorm medication codes at MGB; 59.1% and 75.8% when mapping VA local laboratory codes to LOINC hierarchy. When trained with 500 labels, the lab code mapping attained top 1 and 5 accuracy at 77.7% and 87.9%. MIKGI also attained best performance in selecting VA local lab codes for desired laboratory tests and COVID-19 related features for COVID EHR studies. Compared to existing methods, MIKGI attained the most robust performance with accuracy the highest or near the highest across all tasks. Conclusions: The proposed MIKGI algorithm can effectively integrate incomplete summary data from biomedical text and EHR data to generate harmonized embeddings for EHR codes for knowledge graph modeling and cross-institutional translation of EHR codes.

Zhou, Doudou↗
NASA NTRSDate not supplied

Maximizing Earth Science Observations with Data Harmonization: Harmonized Landsat/Sentinel-2

In August 2021, NASA released science quality harmonized Landsat/Sentinel-2 products for both cloud-based access and direct download from the Land Processes Distributed Active Archive Center (LP DAAC). These 30-meter products, HLSS30 (Sentinel-2 component) and HLSL30 (Landsat component) are placed on the same grid and are generated and distributed fully in the cloud. Similarly, ESA is prototyping harmonized Landsat Sentinel-2 products at 10-meter resolution. The data production and science teams from NASA and ESA will present technical details on the data production system, data product status and availability, and the benefit of harmonizing instruments with similar sensing characteristics between the agencies.

Earth Science↗
DOE OSTI2022

Harmonic Modeling, Data Generation and Analysis of Power Electronics-Interfaced Residential Loads

The share of electronics-based residential load is expected to rise as devices such as variable frequency drives (VFDs), electric vehicle chargers, and inverter-based distributed energy resources (DERs), e.g., photovoltaic (PV) systems become more common. These loads may introduce significant harmonics into power networks that need to be closely studied in order to perform accurate load modeling and forecasting. However, it can be difficult to obtain harmonic-rich voltage and current data - necessary for identifying accurate load models - for residential electrical loads. Recognizing this need, we identify and model a set of electronics-based end-use loads and DERs in an electromagnetic transients program (EMTP) tool for a residence with a single- phase split-phase supply. Further, a procedure is developed to model harmonic interactions between end-use loads connected to the same non-ideal supply voltage in a residential setting. Finally, a harmonic-rich dataset produced via the proposed procedure is utilized to identify frequency coupling matrix (FCM) based load model. Numerical results demonstrate the accuracy of the model, and explore model identifiability with limited data points.

harmonics, power quality, load modeling↗
DOE OSTI2025

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

Automated medical encoding↗
DOE OSTI2023

Harmonized geospatial data to support infrastructure siting feasibility planning for energy system transitions

Climate change, energy system transitions, and socioeconomic change are compounding influences affecting the growth of electricity demand. While energy efficiency initiatives and distributed resources can address a significant amount of this demand, the United States will likely still need new utility-scale generation resources. The energy sector uses capacity expansion planning models to determine the aggregate need for new generation, but these models are typically at the state or regional scale and are not equipped to address the wide range of location- and technology-specific issues that are increasingly a factor in power plant siting. To help address these challenges, we have developed the Geospatial Raster Input Data for Capacity Expansion Regional Feasibility (GRIDCERF) data package, a high-resolution product to evaluate siting suitability for renewable and non-renewable power plants in the conterminous United States. GRIDCERF offers 264 suitability layers for use with 56 power plant technologies in a harmonized format that can be easily ingested by geospatially-enabled modeling software allowing for customization to robustly address science objectives when evaluating varying future conditions.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Impacts of Experimentally Obtained Harmonic Spectrums of Residential Appliances on Distribution Feeder

Owing to the increased use of power electronic based appliances and energy-efficient home equipment such as modern lighting loads, the percentage of nonlinear loads has increased in the distribution system, which can impact system performance, loss, and stability. Thus a comprehensive knowledge of their power quality and harmonic analysis is essential to improve the load models, voltage stability assessment, determination of possible interactions at harmonic frequencies, protection planning, and the effect of system impedance. This paper focuses on harmonic load flow analysis for multiple residential load types to determine the expected impacts on an modeled distribution secondary. For this study, the household appliances include lighting, power electronic, resistive, and motor loads with a nominal supply voltage of 120 V single-phase or 240 V split phase at a nominal frequency of 60 Hz. The harmonic spectrums, as obtained from the experimental evaluation of real loads, are used to inform the harmonic load flow for a detailed secondary network, including the distribution service transformer extracted from a real distribution feeder's model. Additionally, the impact of voltage harmonics on a three-phase test load is presented with experimental results. The harmonic data of the transformer terminal voltage, as obtained from the former load flow study, are scaled to generate the source voltage for the three-phase configuration of the grid simulator to make the test setup very similar to a typical secondary design in the U.S.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗
DOE OSTI2021

Accelerated Assessment of Critical Infrastructure in Aiding Recovery Efforts During Natural and Human-made Disaster

Relief and recovery from disasters (both natural and human-made) require a coordinated approach across several federal and state government agencies. In order to achieve optimal resource allocation and deployment of first responders, accurate and timely assessment of the impact and extent of destruction are the cornerstones to any recovery effort. Ideally, this knowledge should be gathered and shared within the first 0-24 hours (termed as "Acute Phase" by the U.S. CDC guideline) for informed decision-making. But achieving this poses significant challenges for the data collection and data harmonization processes, particularly when voluminous data are being generated from diverse and distributed sources during the disaster responses. To this end, this work developed a scalable and efficient workflow to dynamically collect and harmonize crowd-sourced geographic multi-modal data, and then assess critical infrastructure (CI) damaged during disaster events. We demonstrate the application of our framework with two real-world experiences in addressing post-disaster recovery efforts - for the Bahamas (Natural - due to Hurricane Dorian, 2019) and Beirut (Human-made - due to explosion caused by the ammonium nitrate stored in a warehouse, 2020). We have illustrated that a coordinated effort is needed for planning as well as for execution to achieve informed decision making.

Thakur, Gautam Malviya↗
DOE OSTI2025

Computational tools and data integration to accelerate vaccine development: challenges, opportunities, and future directions

The development of effective vaccines is crucial for combating current and emerging pathogens. Despite significant advances in the field of vaccine development there remain numerous challenges including the lack of standardized data reporting and curation practices, making it difficult to determine correlates of protection from experimental and clinical studies. Significant gaps in data and knowledge integration can hinder vaccine development which relies on a comprehensive understanding of the interplay between pathogens and the host immune system. In this review, we explore the current landscape of vaccine development, highlighting the computational challenges, limitations, and opportunities associated with integrating diverse data types for leveraging artificial intelligence (AI) and machine learning (ML) techniques in vaccine design. We discuss the role of natural language processing, semantic integration, and causal inference in extracting valuable insights from published literature and unstructured data sources, as well as the computational modeling of immune responses. Furthermore, we highlight specific challenges associated with uncertainty quantification in vaccine development and emphasize the importance of establishing standardized data formats and ontologies to facilitate the integration and analysis of heterogeneous data. Through data harmonization and integration, the development of safe and effective vaccines can be accelerated to improve public health outcomes. Looking to the future, we highlight the need for collaborative efforts among researchers, data scientists, and public health experts to realize the full potential of AI-assisted vaccine design and streamline the vaccine development process.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

LungMAP Portal Ecosystem: Systems-level Exploration of the Lung

An improved understanding of the human lung necessitates advanced systems models informed by an ever-increasing repertoire of molecular omics, cellular, imaging, and pathological datasets. To centralize and standardize information across broad lung research efforts we expanded the LungMAP.net website into a new gateway portal. This portal connects a broad spectrum of research networks, bulk and single-cell multi-omics data and a diverse collection of image data that span mammalian lung development, and disease. The data are standardized across species and technologies using harmonized data and metadata models that leverage recent advances including those from the Human Cell Atlas, diverse ontologies, and the LungMAP CellCards initiative. To cultivate future discoveries, we have aggregated a diverse collection of single-cell atlases for multiple species (human, rhesus, mouse), to enable consistent queries across technologies, cohorts, age, disease, and drug treatment. These atlases are provided as independent and integrated queryable datasets, with an emphasis on dynamic visualization, figure generation, re-analysis, cell-type curation, and automated reference-based classification of user-provided single-cell genomics datasets (Azimuth). Finally, as this resource grows, we intend to increase the breadth of available interactive interfaces, supported data types, data portals and datasets from LungMAP and external research efforts.

59 BASIC BIOLOGICAL SCIENCES↗