Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Coordinating an operational data distribution network for CMIP6 data

The distribution of data contributed to the Coupled Model Intercomparison Project Phase 6 (CMIP6) is via the Earth System Grid Federation (ESGF). The ESGF is a network of internationally distributed sites that together work as a federated data archive. Data records from climate modelling institutes are published to the ESGF and then shared around the world. It is anticipated that CMIP6 will produce approximately 20 PB of data to be published and distributed via the ESGF. In addition to this large volume of data a number of value-added CMIP6 services are required to interact with the ESGF; for example the citation and errata services both interact with the ESGF but are not a core part of its infrastructure. With a number of interacting services and a large volume of data anticipated for CMIP6, the CMIP Data Node Operations Team (CDNOT) was formed. The CDNOT coordinated and implemented a series of CMIP6 preparation data challenges to test all the interacting components in the ESGF CMIP6 software ecosystem. This ensured that when CMIP6 data were released they could be reliably distributed.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Graph Databases for Large-Scale Healthcare Systems: A Framework for Efficient Data Management and Data Services

Designing a database system for both efficient data management and data services has been one of the enduring challenges in the healthcare domain. In many healthcare systems, data services and data management are often viewed as two orthogonal tasks; data services refer to retrieval and analytic queries such as search, joins, statistical data extraction, and simple data mining algorithms, while data management refers to building error-tolerant and non-redundant database systems. The gap between service and management has resulted in rigid database systems and schemas that do not support effective analytics. We compose a rich graph structure from an abstracted healthcare RDBMS to illustrate how we can fill this gap in practice. We show how a healthcare graph can be automatically constructed from a normalized relational database using the proposed 3NF Equivalent Graph (3EG) transformation.We discuss a set of real world graph queries such as finding self-referrals, shared providers, and collaborative filtering, and evaluate their performance over a relational database and its 3EG-transformed graph. Experimental results show that the graph representation serves as multiple de-normalized tables, thus reducing complexity in a database and enhancing data accessibility of users. Based on this finding, we propose an ensemble framework of databases for healthcare applications.

Park, Yubin↗
DOE OSTI2016

Analyzing How We Do Analysis and Consume Data, Results from the SciDAC-Data Project

One of the main goals of the Dept. of Energy funded SciDAC-Data project is to analyze the more than 410,000 high energy physics datasets that have been collected, generated and defined over the past two decades by experiments using the Fermilab storage facilities. These datasets have been used as the input to over 5.6 million recorded analysis projects, for which detailed analytics have been gathered. The analytics and meta information for these datasets and analysis projects are being combined with knowledge of their part of the HEP analysis chains for major experiments to understand how modern computing and data delivery is being used. We present the first results of this project, which examine in detail how the CDF, D0, NOvA, MINERvA and MicroBooNE experiments have organized, classified and consumed petascale datasets to produce their physics results. The results include analysis of the correlations in dataset/file overlap, data usage patterns, data popularity, dataset dependency and temporary dataset consumption. The results provide critical insight into how workflows and data delivery schemes can be combined with different caching strategies to more efficiently perform the work required to mine these large HEP data volumes and to understand the physics analysis requirements for the next generation of HEP computing facilities. In particular we present a detailed analysis of the NOvA data organization and consumption model corresponding to their first and second oscillation results (2014-2016) and the first look at the analysis of the Tevatron Run II experiments. We present statistical distributions for the characterization of these data and data driven models describing their consumption

Ding, P.↗
DOE OSTI2024

Implementation of a realistic artificial data generator for crash data generation

In this paper, a framework is outlined to generate realistic artificial data (RAD) as a tool for comparing different models developed for safety analysis. The primary focus of transportation safety analysis is on identifying and quantifying the influence of factors contributing to traffic crash occurrence and its consequences. The current framework of comparing model structures using only observed data has limitations. With observed data, it is not possible to know how well the models mimic the true relationship between the dependent and independent variables. Further, real datasets do not allow researchers to evaluate the model performance for different levels of complexity of the dataset. RAD offers an innovative framework to address these limitations. Hence, we propose a RAD generation framework embedded with heterogeneous causal structures that generates crash data by considering crash occurrence as a trip level event impacted by trip level factors, demographics, roadway and vehicle attributes. Within our RAD generator we employ three specific modules: (a) disaggregate trip information generation, (b) crash data generation and (c) crash data aggregation. For disaggregate trip information generation, we employ a daily activity-travel realization for an urban region generated from an established activity-based model for the Chicago region. We use this data of more than 2 million daily trips to generate a subset of trips with crash data. For trips with crashes crash location, crash type, driver/vehicle characteristics, and crash severity. The daily RAD generation process is repeated for generating crash records at yearly or multi-year resolution. In conclusion, the crash databases generated can be employed to compare frequency models, severity models, crash type and various other dimensions by facility type - possibly establishing a universal benchmarking system for alternative model frameworks in safety literature.

42 ENGINEERING↗
DOE OSTI2021

Data-driven building energy modeling with feature selection and active learning for data predictive control

Three gaps impede the development of cost-effective and accurate data-driven building energy modeling/models (DBEM) for energy forecasting and predictive control strategies. Gap 1: data bias is common in building operation data, but this topic is hardly studied in DBEM; Gap 2: high data dimensionality is common in DBEM, but a systematic and scalable methodology is lacking to solve the problem; Gap 3: the interactions between data bias and high dimensionality have not been systematically studied for DBEM and predictive control in buildings. In this work, to address the three gaps mentioned above, we develop a framework that integrates active learning and feature selection for DBEM used for whole building data predictive control (or DPC, which is a branch of model predictive control). The framework provides a systematic methodology and automatic workflow that starts with raw data from building automation systems to the establishment of data-driven energy models for DPC controllers. The developed strategies and framework are evaluated in a virtual testbed based on EnergyPlus and BCVTB. Improved performance and reduced computational complexity are observed from the DBEM built with the developed framework, as well as the DPC controller based on that DBEM, indicating the effectiveness of the developed framework.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

Challenges and solutions for the analysis of in situ, in crystallo micro-spectrophotometric data

The particular challenge of the analysis of optical absorption and Raman spectroscopic data measured from protein crystals and how the SLS-APE software toolbox supports scientists in dealing with such data is described. Combining macromolecular crystallography with in crystallo micro-spectrophotometry yields valuable complementary information on the sample, including the redox states of metal cofactors, the identification of bound ligands and the onset and strength of undesired photochemistry, also known as radiation damage. However, the analysis and processing of the resulting data differs significantly from the approaches used for solution spectrophotometric data. The varying size and shape of the sample, together with the suboptimal sample environment, the lack of proper reference signals and the general influence of the X-ray beam on the sample have to be considered and carefully corrected for. In the present article, how to characterize and treat these sample-dependent artefacts in a reproducible manner is discussed and the SLS-APEin situ, in crystallo optical spectroscopy data-analysis toolbox is demonstrated.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2019

Feasibility Analysis of Taxi Fleet Electrification Using 4.9 Million Miles of Real-World Driving Data: Preprint

Ride hailing activity is rapidly increasing, largely due to the growth of transportation network companies such as Uber and Lyft. However, traditional taxi companies continue to represent an important mobility option for travelers. Columbus Yellow Cab, a taxi company in Columbus, Ohio, offers traditional line-of-sight hailing as well as digital hailing through a mobile app. Data from Columbus Yellow Cab was provided to the National Renewable Energy Laboratory to analyze the potential for taxi electrification. Columbus Yellow Cab data contained information describing both global positioning system trajectories and taxi meter information. The data spanned a period of 13 months, containing approximately 70 million global system positioning system points, 840 thousand trips, and 170 unique vehicles. A variety of scenarios were evaluated using Columbus Yellow Cab data and the Electric Vehicle Infrastructure Projection Tool (EVI-Pro) to understand challenges and opportunities associated with operating an electrified taxi fleet. Two main factors-access to home charging and vehicle specifications-are shown to be major variables affecting successful electric fleet operation. The analysis indicates that 95.7% of taxi travel days can be successfully completed by a 250-mile-range electric vehicle assuming access to overnight and public charging infrastructure. However, when no overnight access is available to fleet vehicles, only 39.9% of taxi travel days are possible with 250-mile range electric vehicles. An additional scenario, reducing the vehicle range from 250 miles to 100 miles (while controlling for infrastructure access and permitting overnight charging) resulted in only 34.4% of taxi travel days being completed.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2018

The Geothermal Data Repository: Five Years of Open Geothermal Data, Benefits to the Community: Preprint

In the five years since its inception, the Department of Energy's (DOE) Geothermal Data Repository (GDR) has grown from the simple idea of storing public data in a centralized location to a valuable tool at the center of the DOE open data movement where it is providing a tangible benefit to the geothermal scientific community. Throughout this time, the GDR project team has been working closely with the community to refine the data submission process, improve the quality of submitted data, and embrace modern proper data management strategies to maximize the value and utility of submitted data. This paper explores some of the motivations behind various improvements to the GDR over the last 5 years, changes in data submission trends, and the ways in which these improvements have helped to drive research, fuel innovation, and accelerate the adoption of geothermal technologies.

15 GEOTHERMAL ENERGY↗
DOE OSTI2017

The Geothermal Data Repository: Five Years of Open Geothermal Data, Benefits to the Community

In the five years since its inception, the Department of Energy's (DOE) Geothermal Data Repository (GDR) has grown from the simple idea of storing public data in a centralized location to a valuable tool at the center of the DOE open data movement where it is providing a tangible benefit to the geothermal scientific community. Throughout this time, the GDR project team has been working closely with the community to refine the data submission process, improve the quality of submitted data, and embrace modern proper data management strategies to maximize the value and utility of submitted data. This paper explores some of the motivations behind various improvements to the GDR over the last 5 years, changes in data submission trends, and the ways in which these improvements have helped to drive research, fuel innovation, and accelerate the adoption of geothermal technologies.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Livewire Data Platform-A Solution for Energy Efficient Mobility Systems (EEMS) Data Sharing

This presentation is an overview of progress made on the Livewire Data Platform since June 2022. Livewire is a publicly accessible platform for sharing energy efficiency and mobility research data funded by DOE's Vehicle Technologies Office. Core services and platform capabilities of Livewire include: Free, secure data storage; Access management that allows data owners to control who sees their data; Data collection and preservation; Quality characterization; Detailed access and download metrics; and Increased visibility of projects and data. Anyone can create an account and access data at https://livewire.energy.gov/.

energy efficiency↗
DOE OSTI2016

An architecture for consolidating multidimensional time-series data onto a common coordinate grid

Consolidating measurement data for use by data models or in inter-comparison studies frequently requires transforming the data onto a common grid. Standard methods for interpolating multidimensional data are often not appropriate for data with non-homogenous dimensionality, and are hard to implement in a consistent manner for different datastreams. These challenges are increased when dealing with the automated procedures necessary for use with continuous, operational datastreams. In this paper we introduce a method of applying a series of one-dimensional transformations to merge data onto a common grid, examine the challenges of ensuring consistent application of data consolidation methods, present a framework for addressing those challenges, and describe the implementation of such a framework for the Atmospheric Radiation Measurement (ARM) program.

coordinate transformation↗
DOE OSTI2026

Transportation Secure Data Center: Frequently Asked Questions for Data Owners/Contributors

The Transportation Secure Data Center is a centralized repository for detailed transportation data from travel and transit surveys and studies conducted across the nation. It makes vital transportation data broadly available to users while preserving the privacy of survey participants. Hundreds of datasets from surveys and studies of household travel and transit passenger travel are archived in the TSDC, including surveys and studies conducted by state departments of transportation, metropolitan planning organizations, transit agencies, cities, and other public agencies. Detailed data from travel surveys and studies are extremely valuable for research purposes. However, the fine-grained information they contain could potentially be misused to identify individual travelers, so access to these data should only be granted with safeguards in place to protect participant privacy. The TSDC was created to address this challenge and to relieve public agencies from the burden of archiving their data and responding to data requests.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2019

Learning concave-convex profiles of data transport over dedicated connections

Dedicated data transport infrastructures are increasingly being deployed to support distributed big-data and high-performance computing scenarios. These infrastructures employ data transfer nodes that use sophisticated software stacks to support network transport among sites, which often house distributed file and storage systems. Throughput measurements collected over such infrastructures for a range of round trip times (RTTs) reflect the underlying complex end-to-end connections, and have revealed dichotomous throughput profiles as functions of RTT. In particular, concave regions of throughput profiles at lower RTTs indicate near-optimal performance, and convex regions at higher RTTs indicate bottlenecks due to factors such as buffer or credit limits. We present a machine learning method that explicitly infers these concave and convex regions and transitions between them using sigmoid functions. We also provide distribution-free confidence estimates for the generalization error of these concave-convex profile estimates. Throughput profiles for data transfers over 10 Gbps connections with 0–366 ms RTT provide important performance insights, including the near optimality of transfers performed with the XDD tool between XFS filesystems, and the performance limits of wide-area Lustre extensions using LNet routers. A direct application of generic machine learning packages does not adequately highlight these critical performance regions or provide as precise confidence estimates.

Rao, Nageswara↗
DOE OSTI2022

Opportunities and data requirements for data-driven prognostics and health management in liquid hydrogen storage systems

During the past decade, Prognostics and Health Management (PHM) has become an important set of tools in various areas of industry and academic reliability engineering. PHM consists of a variety of mathematical and computational methods used to support data-driven decision-making to increase the safety, availability, and reliability of complex engineering systems. In particular, PHM can provide crucial insight into reliability and safety design improvements for developing technologies where historical performance and failure data are limited. This is the case of hydrogen fueling and storage technologies. This work presents a high-level approach for designing data-driven PHM applications for bulk liquid hydrogen (LH 2 ) storage systems for hydrogen fueling stations. This paper addresses core aspects of the design, development, and implementation of data-driven PHM applications that can improve the reliability assessment of hydrogen components. The analysis focuses on the relationship between data availability and diagnostic/prognostic capabilities; potential challenges; and integration schemes for current risk mitigation measures. We identify potential condition-monitoring data sources for key components in an LH 2 storage system, including storage tanks, piping, and pumps. Further, we determine that the short-term goals for the implementation of data-driven models in PHM frameworks in hydrogen systems should focus on developing adequate data collection and analysis strategies, as well as exploring the effect on reliability, safety, and regulations for hydrogen systems.

08 HYDROGEN↗
DOE OSTI2018

Nuclear Data Sheets for A=171

Experimental data pertaining to all nuclei with mass number A=171 (Tb, Dy, Ho, Er, Tm, Yb, Lu, Hf, Ta, W, Re, Os, Ir, Pt, Au, Hg) have been evaluated. Level schemes from both radioactive decay and reaction studies are presented, along with associated tables of experimental data and adopted properties for levels and γ rays. This evaluation for A=171, which considers data received by 30 June 2018, supersedes the 2002 evaluation (with a literature cutoff date of August 2002) by Coral M. Baglin, published in Nuclear Data Sheets 96, 399 (2002) and the subsequent evaluation of the ENSDF file for 171Pt by C. Baglin (literature cutoff date of 15 April 2010).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Data for A Hybrid Biophysical-Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81–0.94) and H (R2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

AI/ML↗
DOE OSTI2015

Distributed Acoustic Sensing (DAS) Data for Periodic Hydraulic Tests: Hydraulic Data

Hydraulic responses from periodic hydraulic tests conducted at the Mirror Lake Fractured Rock Research Site, during the summer of 2015. These hydraulic responses were measured also using distributed acoustic sensing (DAS) which is cataloged in a different submission under this grant number. The tests are explained in detail in Matthew Cole's MS Thesis which is cataloged here. The injection and drawdown data and the codes used to analyze the data. Sinusoidal Data is a Matlab data file containing a data table for each period-length test. Within each table is a column labeled: time (seconds since beginning of pumping), Inj_m3pm (formation injection in cubic meters per minute), and head for each observation well (meters). The three Matlab script files (*.m) were used to analyze hydraulic responses from the data file above. High-Pass Sinusoid is a routine for filtering the data, computing the FFT, and extracting phase and amplitude values. Borestore is a routine which contains the borehole storage analytic solution and compares modeled amplitude and phase from this solution to computed amplitude and phase from the data. Patsearch Borestore is a routine containing the built-in pattern search optimization method. This minimizes the total error between modeled and actual amplitude and phase in Borestore. Comments within the script files contain more specific instructions for their use.

15 GEOTHERMAL ENERGY↗