Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data harmonization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

SoDaH: the SOils DAta Harmonization database, an open-source synthesis of soil data from research networks, version 1.0

Data collected from research networks present opportunities to test theories and develop models about factors responsible for the long-term persistence and vulnerability of soil organic matter (SOM). Synthesizing datasets collected by different research networks presents opportunities to expand the ecological gradients and scientific breadth of information available for inquiry. Synthesizing these data is challenging, especially considering the legacy of soil data that have already been collected and an expansion of new network science initiatives. To facilitate this effort, here we present the SOils DAta Harmonization database (SoDaH; https://lter.github.io/som-website, last access: 22 December 2020), a flexible database designed to harmonize diverse SOM datasets from multiple research networks. SoDaH is built on several network science efforts in the United States, but the tools built for SoDaH aim to provide an open-access resource to facilitate synthesis of soil carbon data. Moreover, SoDaH allows for individual locations to contribute results from experimental manipulations, repeated measurements from long-term studies, and local- to regional-scale gradients across ecosystems or landscapes. Finally, we also provide data visualization and analysis tools that can be used to query and analyze the aggregated database. The SoDaH v1.0 dataset is archived and available at https://doi.org/10.6073/pasta/9733f6b6d2ffd12bf126dc36a763e0b4 (Wieder et al., 2020).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗
DOE OSTI2022

Multiview Incomplete Knowledge Graph Integration with application to cross-institutional EHR data harmonization

Objective: The growing availability of electronic health records (EHR) data opens opportunities for integrative analysis of multi-institutional EHR to produce generalizable knowledge. A key barrier to such integrative analyses is the lack of semantic interoperability across different institutions due to coding differences. We propose a Multiview Incomplete Knowledge Graph Integration (MIKGI) algorithm to integrate information from multiple sources with partially overlapping EHR concept codes to enable translations between healthcare systems. Methods: The MIKGI algorithm combines knowledge graph information from (i) embeddings trained from the co-occurrence patterns of medical codes within each EHR system and (ii) semantic embeddings of the textual strings of all medical codes obtained from the Self-Aligning Pretrained BERT (SAPBERT) algorithm. Due to the heterogeneity in the coding across healthcare systems, each EHR source provides partial coverage of the available codes. MIKGI synthesizes the incomplete knowledge graphs derived from these multi-source embeddings by minimizing a spherical loss function that combines the pairwise directional similarities of embeddings computed from all available sources. MIKGI outputs harmonized semantic embedding vectors for all EHR codes, which improves the quality of the embeddings and enables direct assessment of both similarity and relatedness between any pair of codes from multiple healthcare systems. Results: With EHR co-occurrence data from Veteran Affairs (VA) healthcare and Mass General Brigham (MGB), MIKGI algorithm produces high quality embeddings for a variety of downstream tasks including detecting known similar or related entity pairs and mapping VA local codes to the relevant EHR codes used at MGB. Based on the cosine similarity of the MIKGI trained embeddings, the AUC was 0.918 for detecting similar entity pairs and 0.809 for detecting related pairs. For cross-institutional medical code mapping, the top 1 and top 5 accuracy were 91.0% and 97.5% when mapping medication codes at VA to RxNorm medication codes at MGB; 59.1% and 75.8% when mapping VA local laboratory codes to LOINC hierarchy. When trained with 500 labels, the lab code mapping attained top 1 and 5 accuracy at 77.7% and 87.9%. MIKGI also attained best performance in selecting VA local lab codes for desired laboratory tests and COVID-19 related features for COVID EHR studies. Compared to existing methods, MIKGI attained the most robust performance with accuracy the highest or near the highest across all tasks. Conclusions: The proposed MIKGI algorithm can effectively integrate incomplete summary data from biomedical text and EHR data to generate harmonized embeddings for EHR codes for knowledge graph modeling and cross-institutional translation of EHR codes.

Zhou, Doudou↗
DOE OSTI2022

Harmonic Modeling, Data Generation and Analysis of Power Electronics-Interfaced Residential Loads

The share of electronics-based residential load is expected to rise as devices such as variable frequency drives (VFDs), electric vehicle chargers, and inverter-based distributed energy resources (DERs), e.g., photovoltaic (PV) systems become more common. These loads may introduce significant harmonics into power networks that need to be closely studied in order to perform accurate load modeling and forecasting. However, it can be difficult to obtain harmonic-rich voltage and current data - necessary for identifying accurate load models - for residential electrical loads. Recognizing this need, we identify and model a set of electronics-based end-use loads and DERs in an electromagnetic transients program (EMTP) tool for a residence with a single- phase split-phase supply. Further, a procedure is developed to model harmonic interactions between end-use loads connected to the same non-ideal supply voltage in a residential setting. Finally, a harmonic-rich dataset produced via the proposed procedure is utilized to identify frequency coupling matrix (FCM) based load model. Numerical results demonstrate the accuracy of the model, and explore model identifiability with limited data points.

harmonics, power quality, load modeling↗
DOE OSTI2025

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

Automated medical encoding↗
DOE OSTI2023

Harmonized geospatial data to support infrastructure siting feasibility planning for energy system transitions

Climate change, energy system transitions, and socioeconomic change are compounding influences affecting the growth of electricity demand. While energy efficiency initiatives and distributed resources can address a significant amount of this demand, the United States will likely still need new utility-scale generation resources. The energy sector uses capacity expansion planning models to determine the aggregate need for new generation, but these models are typically at the state or regional scale and are not equipped to address the wide range of location- and technology-specific issues that are increasingly a factor in power plant siting. To help address these challenges, we have developed the Geospatial Raster Input Data for Capacity Expansion Regional Feasibility (GRIDCERF) data package, a high-resolution product to evaluate siting suitability for renewable and non-renewable power plants in the conterminous United States. GRIDCERF offers 264 suitability layers for use with 56 power plant technologies in a harmonized format that can be easily ingested by geospatially-enabled modeling software allowing for customization to robustly address science objectives when evaluating varying future conditions.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

Impacts of Experimentally Obtained Harmonic Spectrums of Residential Appliances on Distribution Feeder

Owing to the increased use of power electronic based appliances and energy-efficient home equipment such as modern lighting loads, the percentage of nonlinear loads has increased in the distribution system, which can impact system performance, loss, and stability. Thus a comprehensive knowledge of their power quality and harmonic analysis is essential to improve the load models, voltage stability assessment, determination of possible interactions at harmonic frequencies, protection planning, and the effect of system impedance. This paper focuses on harmonic load flow analysis for multiple residential load types to determine the expected impacts on an modeled distribution secondary. For this study, the household appliances include lighting, power electronic, resistive, and motor loads with a nominal supply voltage of 120 V single-phase or 240 V split phase at a nominal frequency of 60 Hz. The harmonic spectrums, as obtained from the experimental evaluation of real loads, are used to inform the harmonic load flow for a detailed secondary network, including the distribution service transformer extracted from a real distribution feeder's model. Additionally, the impact of voltage harmonics on a three-phase test load is presented with experimental results. The harmonic data of the transformer terminal voltage, as obtained from the former load flow study, are scaled to generate the source voltage for the three-phase configuration of the grid simulator to make the test setup very similar to a typical secondary design in the U.S.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗
DOE OSTI2021

Accelerated Assessment of Critical Infrastructure in Aiding Recovery Efforts During Natural and Human-made Disaster

Relief and recovery from disasters (both natural and human-made) require a coordinated approach across several federal and state government agencies. In order to achieve optimal resource allocation and deployment of first responders, accurate and timely assessment of the impact and extent of destruction are the cornerstones to any recovery effort. Ideally, this knowledge should be gathered and shared within the first 0-24 hours (termed as "Acute Phase" by the U.S. CDC guideline) for informed decision-making. But achieving this poses significant challenges for the data collection and data harmonization processes, particularly when voluminous data are being generated from diverse and distributed sources during the disaster responses. To this end, this work developed a scalable and efficient workflow to dynamically collect and harmonize crowd-sourced geographic multi-modal data, and then assess critical infrastructure (CI) damaged during disaster events. We demonstrate the application of our framework with two real-world experiences in addressing post-disaster recovery efforts - for the Bahamas (Natural - due to Hurricane Dorian, 2019) and Beirut (Human-made - due to explosion caused by the ammonium nitrate stored in a warehouse, 2020). We have illustrated that a coordinated effort is needed for planning as well as for execution to achieve informed decision making.

Thakur, Gautam Malviya↗
DOE OSTI2025

Computational tools and data integration to accelerate vaccine development: challenges, opportunities, and future directions

The development of effective vaccines is crucial for combating current and emerging pathogens. Despite significant advances in the field of vaccine development there remain numerous challenges including the lack of standardized data reporting and curation practices, making it difficult to determine correlates of protection from experimental and clinical studies. Significant gaps in data and knowledge integration can hinder vaccine development which relies on a comprehensive understanding of the interplay between pathogens and the host immune system. In this review, we explore the current landscape of vaccine development, highlighting the computational challenges, limitations, and opportunities associated with integrating diverse data types for leveraging artificial intelligence (AI) and machine learning (ML) techniques in vaccine design. We discuss the role of natural language processing, semantic integration, and causal inference in extracting valuable insights from published literature and unstructured data sources, as well as the computational modeling of immune responses. Furthermore, we highlight specific challenges associated with uncertainty quantification in vaccine development and emphasize the importance of establishing standardized data formats and ontologies to facilitate the integration and analysis of heterogeneous data. Through data harmonization and integration, the development of safe and effective vaccines can be accelerated to improve public health outcomes. Looking to the future, we highlight the need for collaborative efforts among researchers, data scientists, and public health experts to realize the full potential of AI-assisted vaccine design and streamline the vaccine development process.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

LungMAP Portal Ecosystem: Systems-level Exploration of the Lung

An improved understanding of the human lung necessitates advanced systems models informed by an ever-increasing repertoire of molecular omics, cellular, imaging, and pathological datasets. To centralize and standardize information across broad lung research efforts we expanded the LungMAP.net website into a new gateway portal. This portal connects a broad spectrum of research networks, bulk and single-cell multi-omics data and a diverse collection of image data that span mammalian lung development, and disease. The data are standardized across species and technologies using harmonized data and metadata models that leverage recent advances including those from the Human Cell Atlas, diverse ontologies, and the LungMAP CellCards initiative. To cultivate future discoveries, we have aggregated a diverse collection of single-cell atlases for multiple species (human, rhesus, mouse), to enable consistent queries across technologies, cohorts, age, disease, and drug treatment. These atlases are provided as independent and integrated queryable datasets, with an emphasis on dynamic visualization, figure generation, re-analysis, cell-type curation, and automated reference-based classification of user-provided single-cell genomics datasets (Azimuth). Finally, as this resource grows, we intend to increase the breadth of available interactive interfaces, supported data types, data portals and datasets from LungMAP and external research efforts.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Estimating building occupancy: a machine learning system for day, night, and episodic events

Building occupancy research increasingly emphasizes understanding the social and physical dynamics of how people occupy space. Opportunities in the open source domain including social media, Volunteered Geographic Information, crowdsourcing, and sensor data have proliferated, resulting in the exploration of building occupancy dynamics at varying spatiotemporal scales. At Oak Ridge National Laboratory, research into building occupancies through the development of a global learning framework that accommodates exploitation of open source authoritative sources, including governmental census and surveys, journal articles, real estate databases, and more, to report national and subnational building occupancies across the world continues through the Population Density Tables (PDT) project. This probabilistic learning system accommodates expert knowledge, experience, and open-source data to capture local, socioeconomic, and cultural information about human activity. It does so through a systematic process of data harmonization techniques in the development of observation models for over 50 building types to dynamically update baseline estimates and report probabilistic diurnal and episodic building occupancy estimates. This discussion will explore how PDT is implemented at scale and expanded based on the development of observation model classes and will explain how to interpret and spatially apply the reported probability occupancy estimates and uncertainty.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

At the interfaces of the hydrologic sciences: Connecting water, elements, ecosystems, and people through the major contributions of Dr. Emily Bernhardt

Here, in this paper, we describe the major contributions of Professor Emily Bernhardt to the hydrologic sciences. Dr. Bernhardt’s work addresses how carbon, nutrient, and contaminant dynamics respond to a wide range of environmental perturbations that alter hydrologic dynamics within and connectivity among ecosystems. Her research leverages intensive and extensive field sampling, experimental manipulations, macroscale data harmonization and exploration, and continental to global-scale synthesis activities to uncover key drivers and patterns of the impacts human perturbations have on water and elemental cycles. Dr. Bernhardt’s research program is defined by her ability to ask questions and use approaches that explicitly consider connectivity and interfaces in a variety of ways. Here, we highlight significant contributions from Dr. Bernhardt’s work, organized by connectivity, interfaces, and interactions among and across (1) elemental cycles, (2) ecosystems, (3) watersheds, (4) scales, and (5) disciplines. We conclude with a section on Dr. Bernhardt’s impact on the hydrologic sciences and beyond through her exceptional dedication to mentorship, engagement, and service.

58 GEOSCIENCES↗
DOE OSTI2023

The state of food systems worldwide in the countdown to 2030

This Analysis presents a recently developed food system indicator framework and holistic monitoring architecture to track food system transformation towards global development, health and sustainability goals. Five themes are considered: (1) diets, nutrition and health; (2) environment, natural resources and production; (3) livelihoods, poverty and equity; (4) governance; and (5) resilience. Each theme is divided into three to five indicator domains, and indicators were selected to reflect each domain through a consultative process. In total, 50 indicators were selected, with at least one indicator available for every domain. Harmonized data of these 50 indicators provide a baseline assessment of the world’s food systems. We show that every country can claim positive outcomes in some parts of food systems, but none are among the highest ranked across all domains. Furthermore, some indicators are independent of national income, and each highlights a specific aspiration for healthy, sustainable and just food systems. The Food Systems Countdown Initiative will track food systems annually to 2030, amending the framework as new indicators or better data emerge.

Schneider, Kate R.↗
DOE OSTI2020

Retrieval of full angular- and energy-dependent complex transition dipoles in the molecular frame from laser-induced high-order harmonic signals with aligned molecules

High-order harmonic signals generated in molecules are the consequence of coherent summation of complex laser-induced transition dipoles $\textit{d}(θ, ω)$ with each fixed-in-space molecule; here $\theta$ is the angle of the molecular axis with respect to the laser polarization axis and $\omega$ is the harmonic energy. In the so-called rotational coherent spectroscopy, it is proposed to extract the fixed-in-space $\textit{d}(θ, ω)$ in the molecular frame by measuring harmonics generated by a probing laser from the rotational molecular wave packets that have been prepared by a prior pump laser. By varying the time delay between the two lasers, methods have been utilized to extract the $\theta$ dependence of both the amplitude and phase of each individual harmonic, but the relative phase between harmonics cannot be retrieved. Here we report that this limitation can be removed. It requires the additional measurement of harmonic spectra versus the pump-probe angles at one fixed time delay. The two-dimensional input harmonic data (time-delay and pump-probe angle) are then used to retrieve the full complex transition dipole $\textit{d}(θ, ω)$ using a retrieval method based on machine learning algorithms. Finally, we demonstrate this method on N 2 and CO 2 molecules.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2024

Priorities, opportunities, and challenges for integrating microorganisms into Earth system models for climate change prediction

ABSTRACT Climate change jeopardizes human health, global biodiversity, and sustainability of the biosphere. To make reliable predictions about climate change, scientists use Earth system models (ESMs) that integrate physical, chemical, and biological processes occurring on land, the oceans, and the atmosphere. Although critical for catalyzing coupled biogeochemical processes, microorganisms have traditionally been left out of ESMs. Here, we generate a “top 10” list of priorities, opportunities, and challenges for the explicit integration of microorganisms into ESMs. We discuss the need for coarse-graining microbial information into functionally relevant categories, as well as the capacity for microorganisms to rapidly evolve in response to climate-change drivers. Microbiologists are uniquely positioned to collect novel and valuable information necessary for next-generation ESMs, but this requires data harmonization and transdisciplinary collaboration to effectively guide adaptation strategies and mitigation policy.

Microbiology↗