Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data diversity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

SBIR Phase II Final Report - Multi-Protocol Energy Management Gateway for Home-Area Networks

Significant amounts of electricity, natural gas, and heating oil are wasted by homeowners due to inefficient operation and inadequate maintenance of heating, ventilation, and air conditioning (HVAC) equipment. Coincident’s work under this award reduces energy waste, saves consumers money, and reduces carbon emissions. It does so in three ways: First, Coincident’s approach replaces the traditional thermostat with a wireless network of sensors and controllers that measure temperature, humidity and occupancy in multiple rooms in the house. The “Internet of Things” is a technology trend holding the promise of ubiquitous inexpensive sensors. The reality, however, is that energy and HVAC monitoring and management is a patchwork of incompatible protocols and expensive proprietary technologies. Coincident’s multi-protocol architecture, developed in part under this award tackles this problem and brings low cost interoperable sensor and control devices to market. Second, the Coincident system eliminates hard-to-program and rigid thermostat schedules and instead provides automatic operation of heating and cooling by combining individual temperature and comfort preferences with energy-saving targets, real-time utility use information, weather data, and room utilization patterns. Energy efficiency technology must be appealing to consumers otherwise it will not be used. The Coincident user interface has engaging features such as remote control from any smart phone or web browser and per-room performance breakdowns. Expected energy savings resulting from more efficient operation of heating and air conditioning equipment are in the range of 10-20%. Third, the Coincident system provides heating and air-conditioning contractors with fine-grained performance data for every residence they support (subject to customer privacy controls). This data is integrated from diverse networks within the residence and includes HVAC performance and fuel use data. This information allows the partner to validate energy savings and identify potential system faults (whether from installation problems or maintenance issues). When combined with professional installation as part of high-efficiency HVAC upgrade, energy savings levels of 20-30% can be achieved. Economic feasibility of energy efficiency technology is one of the key challenges addressed in this award. The Coincident system is engineered to be delivered at a disruptive price point, making the system financially feasible for new and retrofit homes of all types and sizes. The Coincident system is intended to be sold through the HVAC professional—the industry most capable of improving HVAC efficiency. Providing HVAC contractors with detailed home performance data motivates them to sell the product, provides them with maintenance and upgrade revenue opportunities, and therefore delivers customer savings and environmental benefits. Having demonstrated technical and financial feasibility, Coincident has won additional grants and awards, participated in pilot projects, started partnership discussions with several HVAC equipment vendors, and has lined up several large channel partners ready to participate in large pilot rollouts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

EVs@Scale Next-Gen Profiles - Fleet Utilization 2024

As part of the U.S. Department of Energy’s EVs@Scale initiative, the Next-Gen Profiles (NGP) project provides a comprehensive, data-driven analysis of electric vehicle (EV) and electric vehicle supply equipment (EVSE) operations across real-world fleet deployments. This paper presents findings from the NGP’s Fleet Utilization study, which investigates operational behavior and asset usage across seventeen EV fleets and two EVSE fleets, encompassing a wide range of vehicle types and use cases. Data collected from diverse sources—varying in format and temporal resolution—are first reformatted into a unified structure. From this harmonized dataset, a suite of rigorously defined performance metrics is calculated at an hourly cadence, enabling consistent cross-comparison of charging, routing, and other key operational behaviors. Amid rapidly increasing EV adoption and growing demands for energy-efficient fleet operations, the analysis reveals clear utilization trends—including diurnal and weekly activity cycles, differences in short versus long charging session dependencies, and route-specific energy usage patterns. These findings highlight the need for tailored infrastructure strategies and the deployment of advanced energy management systems, such as Distributed Energy Resource Management Systems (DERMS) and Site Energy Management Systems (SEMS), which can optimize charging schedules and mitigate peak loads. By leveraging anonymized, harmonized datasets and standardized metrics, this study offers critical insights into fleet behavior and performance, providing a foundation to improve operational efficiency, reduce costs, and enable the scalable deployment of electrified transportation.

Wells, Landon↗
DOE OSTI2025

An MLCommons Scientific Benchmarks Ontology

Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transformative applications of machine learning to critical scientific use-cases more fragmented and less clear in pathways to impact. This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort that extends the MLCommons ecosystem to cover physics, chemistry, materials science, biology, climate science, and more. Building on prior initiatives such as XAI-BENCH, FastML Science Benchmarks, PDEBench, and the SciMLBench framework, our effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy of scientific, application, and system-level benchmarks. New benchmarks can be added through an open submission workflow coordinated by the MLCommons Science Working Group and evaluated against a six-category rating rubric that promotes and identifies high-quality benchmarks, enabling stakeholders to select benchmarks that meet their specific needs. The architecture is extensible, supporting future scientific and AI/ML motifs, and we discuss methods for identifying emerging computing patterns for unique scientific workloads. The MLCommons Science Benchmarks Ontology provides a standardized, scalable foundation for reproducible, cross-domain benchmarking in scientific machine learning. A companion webpage for this work has also been developed as the effort evolves: https://mlcommons-science.github.io/benchmark/

Hawks, Ben [Fermilab] (ORCID:0000000157000288)↗
NASA NTRS1986

Atmospheric climate data: Problems and promises

The explosive growth in the quantity and diversity of weather and climate data, the growing handicap that the distinction between weather and climate in NOAA imposes on the efficient management and use of data is discussed. Also discussed is the uncertainty induced by the lack of clear commitment and consistent policies regarding federal roles and responsibilities in operating and maintaining the national weather and climate data system.

Source record↗
DOE OSTI2020

ATLAS: a Snakemake workflow for assembly, annotation, and genomic binning of metagenome sequence data

Background: Metagenomics and metatranscriptomics studies provide valuable insight into the composition and function of microbial populations from diverse environments, however the data processing pipelines that rely on mapping reads to gene catalogs or genome databases for cultured strains yield results that underrepresent the genes and functional potential of uncultured microbes. Recent improvements in sequence assembly methods have eased the reliance on genome databases, thereby allowing the recovery of genomes from uncultured microbes. However, configuring these tools, linking them with advanced binning and annotation tools, and maintaining provenance of the processing continues to be challenging for researchers. Results: Here we present ATLAS, a software package for customizable data processing from raw sequence reads to functional and taxonomic annotations using state-of-the-art tools to assemble, annotate, quantify, and bin metagenome and metatranscriptome data. Genome-centric resolution and abundance estimates are provided for each sample in a dataset. ATLAS is written in Python and the workflow implemented in Snakemake; it operates in a Linux environment, and is compatible with Python 3.5+ and Anaconda 3+ versions. The source code for ATLAS is freely available, distributed under a BSD-3 license. Conclusions: ATLAS provides a user-friendly, modular and customizable Snakemake workflow for metagenome and metatranscriptome data processing; it is easily installable with conda and maintained as open-source on GitHub at https://github.com/metagenome-atlas/atlas.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A Causal Approach to Integrate Component Health Data into System Reliability Models

Two of the challenges of current plant reliability approaches are the ability to integrate plant health data, and to support decision making. Condition based data and diagnostic/prognostic information are in fact not considered into plant reliability models to inform system engineers on the most critical components. Currently, the propagation of quantitative health data from the component to the system level is a challenge given the diverse nature/structure of the data. On the other hand, plant reliability methods (which are typically based on fault-trees or reliability block diagrams) can effectively propagate data from the component to the system level, but values of failure rates or failure probabilities are an approximated integral representation of the past industry-wide operational experience, and it neglects the present component health status (e.g., diagnostic and condition-based data) and health projection (when available from prognostic data). Our first claim is that system reliability models should propagate health information from the component to the system/plant level in order to provide a quantitative snapshot of system/plant health and identify the most critical components. Our second claim is that component health should be informed solely by that specific component current and historical performance data and should not be an approximated integral representation of the past industry-wide operational experience. This paper is directly supporting these two claims by proposing a different approach to perform reliability modeling which relies on available component diagnostic, prognostic and condition-based data to measure component health, and it propagates this information through fault tree models. The propagation of health data from the component to the system level is performed not in terms of probability, but in terms of margins where margin is defined as the “distance” between the present actual status and an undesired event (e.g., failure or unacceptable performance). Through a cause-effect lens, while classical reliability models target the effect associated to a component performance, a margin-based approach focuses on the cause of an undesired component performance (i.e., component health). Hence, thinking of reliability in terms of margins implies decision making based on causal reasoning. We will show how fault tree models can be solved using a margin language and how this process can effectively assist system engineers to identify the most critical components.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Development of physics-consistent conditional diffusion model to overcome data scarcity in critical heat flux

Deep generative modeling provides a powerful pathway to overcome data scarcity in energy-related applications where experimental data are often limited. By learning the underlying probability distribution of the training dataset, deep generative models, such as the diffusion model, can generate high-fidelity synthetic samples that statistically resemble the training data. Such synthetic data generation can significantly enrich the size and diversity of the available training data, and more importantly, improve the robustness of downstream machine learning models in predictive tasks. The objective of this paper is to investigate the effectiveness of diffusion models for overcoming data scarcity in nuclear energy applications. By leveraging a public dataset on critical heat flux which covers a wide range of commercial nuclear reactor operational conditions, we developed a diffusion model that can generate an arbitrary amount of synthetic samples. Since a vanilla diffusion model can only generate samples randomly, we also developed a conditional diffusion model capable of generating targeted critical heat flux data under user-specified thermal-hydraulic conditions. The performance of the diffusion model was evaluated based on its ability to capture empirical feature distributions and pair-wise correlations, as well as to maintain physical consistency. The results showed that both the diffusion model and conditional diffusion model can successfully generate realistic and physics-consistent critical heat flux data. Furthermore, uncertainty quantification results demonstrate that the conditional diffusion model is highly effective in augmenting critical heat flux data while maintaining acceptable levels of uncertainty.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS2016

Virtual Collections: An Earth Science Data Curation Service

The role of Earth science data centers has traditionally been to maintain central archives that serve openly available Earth observation data. However, in order to ensure data are as useful as possible to a diverse user community, Earth science data centers must move beyond simply serving as an archive to offering innovative data services to user communities. A virtual collection, the end product of a curation activity that searches, selects, and synthesizes diffuse data and information resources around a specific topic or event, is a data curation service that improves the discoverability, accessibility, and usability of Earth science data and also supports the needs of unanticipated users. Virtual collections minimize the amount of the time and effort needed to begin research by maximizing certainty of reward and by providing a trustworthy source of data for unanticipated users. This presentation will define a virtual collection in the context of an Earth science data center and will highlight a virtual collection case study created at the Global Hydrology Resource Center data center.

Data↗
NASA NTRSDate not supplied

pyQuARC: Preparing for Full Release

Metadata holds the contextual information about data and is the underlying structure for many data search portals. High quality metadata optimizes search results, allowing users to quickly retrieve the data they need. With the abundant volume and diversity of Earth observation datasets, data discovery and metadata quality are critical for end users. The Common Metadata Repository (CMR), for example, currently hosts metadata for over 9,000 Earth observation data products archived across 12 NASA Distributed Active Archive Centers (DAACs). The Analysis and Review of CMR (ARC) Team, located at Marshall Space Flight Center, assesses the completeness, correctness, and consistency of these metadata records to ensure they are accessible, usable, and discoverable. In 2021, ARC began developing pyQuARC, an open source library for Earth Observation Metadata Quality Assessment to automate this effort. The tool uses ARC’s existing metadata quality framework to provide prioritized recommendations for metadata improvement. During initial testing, pyQuARC automatically identified 58% of metadata findings when compared with a sample of manually reviewed records. Using the results from initial testing, this presentation will focus on recent advancements and improvements of the tool as the ARC team prepares for pyQuARC’s full release. It will also demonstrate pyQuARC's enrichment value, not only for the ARC team, but the broader EOSDIS metadata community as well.

Essence Raphael↗
NASA NTRS2017

Reusing JPSS Ground System Components to Process Aura Ozone Monitoring Instrument Science Products

New Earth observation instruments are planned to enable advancements in Earth science research over the next decade. Diversity of Earth observing instruments and their observing platforms will continue to increase as new instrument technologies emerge and are deployed as part of National programs such as Joint Polar Satellite System (JPSS), Geostationary Operational Environmental Satellite system (GOES), Landsat as well as the potential for many CubeSat and aircraft missions. The practical use and value of these observational data often extends well beyond their original purpose. The practicing community needs intuitive and standardized tools to enable quick unfettered development of tailored products for specific applications and decision support systems. However, the associated data processing system can take years to develop and requires inherent knowledge and the ability to integrate increasingly diverse data types from multiple sources. This paper describes the adaptation of a large-scale data processing system built for supporting JPSS algorithm calibration and validation (CalVal) node to a simplified science data system for rapid application. The new configurable data system reuses scalable JAVA technologies built for the JPSS Government Resource for Algorithm Verification, Independent Test, and Evaluation (GRAVITE) system to run within a laptop environment and support product generation and data processing of AURA Ozone Monitoring Instrument (OMI) science products. Of particular interest are the root requirements necessary for integrating experimental algorithms and Hierarchical Data Format (HDF) data access libraries into a science data production system. This study demonstrates the ability to reuse existing Ground System technologies to support future missions with minimal changes.

Science Data Systems↗
DOE OSTI2024

Data driven discovery and quantification of hyperspectral leaf reflectance phenotypes across a maize diversity panel

Abstract Estimates of plant traits derived from hyperspectral reflectance data have the potential to efficiently substitute for traits, which are time or labor intensive to manually score. Typical workflows for estimating plant traits from hyperspectral reflectance data employ supervised classification models that can require substantial ground truth datasets for training. We explore the potential of an unsupervised approach, autoencoders, to extract meaningful traits from plant hyperspectral reflectance data using measurements of the reflectance of 2151 individual wavelengths of light from the leaves of maize ( Zea mays ) plants harvested from 1658 field plots in a replicated field trial. A subset of autoencoder‐derived variables exhibited significant repeatability, indicating that a substantial proportion of the total variance in these variables was explained by difference between maize genotypes, while other autoencoder variables appear to capture variation resulting from changes in leaf reflectance between different batches of data collection. Several of the repeatable latent variables were significantly correlated with other traits scored from the same maize field experiment, including one autoencoder‐derived latent variable (LV8) that predicted plant chlorophyll content modestly better than a supervised model trained on the same data. In at least one case, genome‐wide association study hits for variation in autoencoder‐derived variables were proximal to genes with known or plausible links to leaf phenotypes expected to alter hyperspectral reflectance. In aggregate, these results suggest that an unsupervised, autoencoder‐based approach can identify meaningful and genetically controlled variation in high‐dimensional, high‐throughput phenotyping data and link identified variables back to known plant traits of interest.

Tross, Michael C.↗
DOE OSTI2025

A Scalable Multi-Modal Framework for High-Fidelity Distributed Human Mobility Simulations

The development of data-driven models for human mobility in urban settings requires access to substantial and diverse real-world data. However, existing historical data often presents challenges such as limited volume, variety, and veracity, as well as missing data and privacy preservation concerns. Also, urban mobility modeling is inherently time-variant, complex, and multi-modal, encompassing everything from individual walking and running to private road travel and large-scale public transportation. These challenges call for innovative solutions to overcome data limitations and compute needs to model mobility behaviors accurately. To address these challenges, we propose a distributed, co-simulation-based architecture DURMOSim that integrates real-world data with scalable, high-fidelity simulations, demonstrating distributed co-simulation feasibility with existing mobility models. DURMOSim underpins a modular integration that would enable using any available mobility simulators for greater extensibility and scalability in performing various urban scenarios. In this paper, we present the design, implementation, and performance evaluation of DURMOSim, highlighting its capability to model population-scale mobility patterns. Our initial results show its ability to dynamically synchronize multiple simulation models at runtime with negligible computational overhead. We believe DURMOSim could be a robust tool for advancing urban mobility research and intelligent transportation systems.

Yoginath, Srikanth [ORNL] (ORCID:0000000184236050)↗
NASA NTRS2011

Improvements in Space Geodesy Data Discovery at the CDDIS

The Crustal Dynamics Data Information System (CDDIS) supports data archiving and distribution activities for the space geodesy and geodynamics community. The main objectives of the system are to store space geodesy and geodynamics related data products in a central data bank. to maintain information about the archival of these data, and to disseminate these data and information in a timely manner to a global scientific research community. The archive consists of GNSS, laser ranging, VLBI, and DORIS data sets and products derived from these data. The CDDIS is one of NASA's Earth Observing System Data and Information System (EOSDIS) distributed data centers; EOSDIS data centers serve a diverse user community and arc tasked to provide facilities to search and access science data and products. Several activities are currently under development at the CDDIS to aid users in data discovery, both within the current community and beyond. The CDDIS is cooperating in the development of Geodetic Seamless Archive Centers (GSAC) with colleagues at UNAVCO and SIO. TIle activity will provide web services to facilitate data discovery within and across participating archives. In addition, the CDDIS is currently implementing modifications to the metadata extracted from incoming data and product files pushed to its archive. These enhancements will permit information about COOlS archive holdings to be made available through other data portals such as Earth Observing System (EOS) Clearinghouse (ECHO) and integration into the Global Geodetic Observing System (GGOS) portal.

Noll, C.↗
NASA NTRS2012

An Update on the CDDIS

The Crustal Dynamics Data Inforn1ation System (CoorS) supports data archiving and distribution activities for the space geodesy and geodynamics community. The main objectives of the system are to store space geodesy and geodynamics related data products in a central data bank, to maintain infom1ation about the archival of these data, and to disseminate these data and information in a timely mam1er to a global scientific research community. The archive consists of GNSS, laser ranging, VLBI, and OORIS data sets and products derived from these data. The coors is one of NASA's Earth Observing System Oata and Infom1ation System (EOSorS) distributed data centers; EOSOIS data centers serve a diverse user community and are tasked to provide facilities to search and access science data and products. The coors data system and its archive have become increasingly important to many national and international science communities, in pal1icular several of the operational services within the International Association of Geodesy (lAG) and its project the Global Geodetic Observing System (GGOS), including the International OORIS Service (IDS), the International GNSS Service (IGS), the International Laser Ranging Service (ILRS), the International VLBI Service for Geodesy and Astrometry (IVS), and the International Earth Rotation Service (IERS). The coors has recently expanded its archive to supp011 the IGS Multi-GNSS Experiment (MGEX). The archive now contains daily and hourly 3D-second and subhourly I-second data from an additional 35+ stations in RINEX V3 fOm1at. The coors will soon install an Ntrip broadcast relay to support the activities of the IGS Real-Time Pilot Project (RTPP) and the future Real-Time IGS Service. The coors has also developed a new web-based application to aid users in data discovery, both within the current community and beyond. To enable this data discovery application, the CDDIS is currently implementing modifications to the metadata extracted from incoming data and product files pushed to its archive. This poster will include background information about the system and its user communities, archive contents and updates, enhancements for data discovery, new system architecture, and future plans.

Noll, Carey↗
DOE OSTI2019

Data Mining Scientific Literature Demonstrates Use of Biological and Medical Data Across Scientific Disciplines

The Protein Data Bank (PDB) was established as the 1st open access digital data resource in biology and medicine. Today, the PDB archive contains over 140,000 atomic‐level biomolecular structures determined by crystallography, NMR spectroscopy, and 3D electron microscopy. About 20 new structures are deposited daily. PDB data is used in basic and applied research, patent applications, discovery of lifesaving drugs, product innovation, and education. About 80% of all PDB structures deposited have a “primary citation,” which is the first paper to describe the molecule and its structure and function. In turn, these primary citations have contributed to more than 1 million publications. An initial study by Clarivate Analytics on the Nucleic Acids Research publication demonstrated that PDB data drive high‐impact research in diverse scientific fields. To build on this study, we used Python programming to perform several data mining studies to understand the impact of these structures in fundamental biology, biomedicine, and energy in the scientific literature. The scientific literature database, Web of Science, tracks scientific article metrics such as the number of times cited, subject category, year published, funding source, and keywords. Web of Science data for PDB primary citations was compiled as used as the data set. Two main investigations were performed. In the first, we examined the number of citations per article (data overall and by individual subject categories) to develop “top 10 cited structures” lists and analyzed for trends. In the second investigation, we extended our data mining analysis to consider for each main subject category, the papers with the most citations and the structures they cite, and the most frequently occuring keywords. We have observed different aspects of biological data utility in various subject areas, at times in ways unanticipated from the structures' primary citation. This research demonstrates the value of biological data from the PDB by showing the diverse ways in which the data are being used. Support or Funding Information RCSB PDB is funded by a grant (DBI‐1338415) from the National Science Foundation, the National Institutes of Health, and the US Department of Energy. This abstract is from the Experimental Biology 2019 Meeting. There is no full text article associated with this abstract published in The FASEB Journal .

Verdiguel, Natalie↗
DOE OSTI2025

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗
DOE OSTI2023

IMG/PR: a database of plasmids from genomes and metagenomes with rich annotations and metadata

Plasmids are mobile genetic elements found in many clades of Archaea and Bacteria. They drive horizontal gene transfer, impacting ecological and evolutionary processes within microbial communities, and hold substantial importance in human health and biotechnology. To support plasmid research and provide scientists with data of an unprecedented diversity of plasmid sequences, we introduce the IMG/PR database, a new resource encompassing 699 973 plasmid sequences derived from genomes, metagenomes and metatranscriptomes. IMG/PR is the first database to provide data of plasmid that were systematically identified from diverse microbiome samples. IMG/PR plasmids are associated with rich metadata that includes geographical and ecosystem information, host taxonomy, similarity to other plasmids, functional annotation, presence of genes involved in conjugation and antibiotic resistance. The database offers diverse methods for exploring its extensive plasmid collection, enabling users to navigate plasmids through metadata-centric queries, plasmid comparisons and BLAST searches. The web interface for IMG/PR is accessible at https://img.jgi.doe.gov/pr. Plasmid metadata and sequences can be downloaded from https://genome.jgi.doe.gov/portal/IMG_PR.

59 BASIC BIOLOGICAL SCIENCES↗