Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Management, Analysis, and Visualization of Experimental and Observational Data -- The Convergence of Data and Computing

Scientific user facilities---particle accelerators, telescopes, colliders, supercomputers, light sources, sequencing facilities, and more---operated by the U.S. Department of Energy (DOE) Office of Science (SC) generate ever increasing volumes of data at unprecedented rates from experiments, observations, and simulations. At the same time there is a growing community of experimentalists that require real-time data analysis feedback, to enable them to steer their complex experimental instruments to optimized scientific outcomes and new discoveries. Recent efforts in DOE-SC have focused on articulating the data-centric challenges and opportunities facing these science communities. Key challenges include difficulties coping with data size, rate, and complexity in the context of both real-time and post-experiment data analysis and interpretation. Solutions will require algorithmic and mathematical advances, as well as hardware and software infrastructures that adequately support data-intensive scientific workloads. This paper presents the summary findings of a workshop held by DOE-SC in September 2015, convened to identify the major challenges and the research that is needed to meet those challenges.

Bethel, E. Wes↗
DOE OSTI2017

Improving Scientific Data Analysis Through Multi-touch Enabled Interactive Data Visualization with Applications to Neutron Science

Scientific data analysis is an inherently exploratory process that requires flexible human interaction with data visualization and anal- ysis tools. In this paper, we describe challenges in neutron scattering data analysis and present our approach for allowing more effective human interaction. In addition to multi-touch enabled data visualiza- tion techniques, we describe color scale enhancements, layouts, and visual analytics approaches for 3D volume slicing. Since challenges in neutron science are echoed in most scientific fields, we generalize the project objective and present a case for bridging the gap between the latest advances in interactive data visualization and practical scientific scenarios where new critical knowledge is buried in large volumes of complex data.

Steed, Chad↗
DOE OSTI2015

Adaptive Framework for Classification and Novel Class Detection over Evolving Data Streams with Limited Labeled Data.

Most approaches to classifying evolving data streams either divide the stream of data into fixed-size chunks or use gradual forgetting to address the problems of infinite length and concept drift. Finding the fixed size of the chunks or choosing a forgetting rate without prior knowledge about time-scale of change is not a trivial task. As a result, these approaches suffer from a trade-off between performance and sensitivity. To address this problem, we present a framework which uses change detection techniques on the classifier performance to determine chunk boundaries dynamically. Though this framework exhibits good performance, it is heavily dependent on the availability of true labels of data instances. However, labeled data instances are scarce in realistic settings and not readily available. Therefore, we present a second framework which is unsupervised in nature, and exploits change detection on classifier confidence values to determine chunk boundaries dynamically. In this way, it avoids the use of labeled data while still addressing the problems of infinite length and concept drift. Moreover, both of our proposed frameworks address the concept evolution problem by detecting outliers having similar values for the attributes. We provide theoretical proof that our change detection method works better than other state-of-the-art approaches in this particular scenario. Results from experiments on various benchmark and synthetic data sets also show the efficiency of our proposed frameworks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Renewable Energy Data and Decision Support for Kazakhstan - RE Data Explorer in Action

In 2016, with support from USAID Central Asia and the U.S. Department of State Leadership Compact, and in collaboration with the Government of Kazakhstan and other Central Asian governments, NREL launched the RE Data Explorer for Central Asia, including world-class wind and solar resource data for Kazakhstan and surrounding countries. The RE Data Explorer provides an easy- to-use online platform for users to view, analyze, and download high-quality datasets for informed decision-making. The RE Data Explorer is part of a broader platform called the RE Explorer that provides expertly curated materials and knowledge resources to enable data-driven decisions. This case study documents how the RE Data Explorer was used to support renewable energy decision-making in Kazakhstan.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Redirecting data to improve page locality in a scalable data fabric

A data processing system includes a host processor, a local memory coupled to the host processor, a plurality of remote memory media, and a scalable data fabric coupled to the host processor and to the plurality of remote memory media. The scalable data fabric includes a filter for storing information indicating a location of data that is stored by the data processing system. The host processor includes a hardware sequencer coupled to the filter for selectively moving data stored by the filter to the local memory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗
DOE OSTI2018

Lossy Data Compression Effects on Wall-bounded Turbulence: Bounds on Data Reduction

Postprocessing and storage of large data sets represent one of the main computational bottlenecks in computational fluid dynamics. We assume that the accuracy necessary for computation is higher than needed for postprocessing. Therefore, in the current work we assess thresholds for data reduction as required by the most common data analysis tools used in the study of fluid flow phenomena, specifically wall-bounded turbulence. These thresholds are imposed a priori by the user in L 2 -norm, and we assess a set of parameters to identify the minimum accuracy requirements. The method considered in the present work is the discrete Legendre transform (DLT), which we evaluate in the computation of turbulence statistics, spectral analysis and resilience for cases highly-sensitive to the initial conditions. Maximum acceptable compression ratios of the original data have been found to be around 97%, depending on the application purpose. Furthermore, the new method outperforms downsampling, as well as the previously explored data truncation method based on discrete Chebyshev transform (DCT).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Nuclear Data Adjustment for Nonlinear Applications in the OECD/NEA WPNCS SG14 Benchmark—A Bayesian Inverse UQ-Based Approach for Data Assimilation

The Organisation for Economic Co-operation and Development Working Party on Nuclear Criticality Safety has proposed a benchmark exercise to assess the performance of current nuclear data adjustment techniques applied to nonlinear applications and experiments with low correlation to applications. This work introduces Bayesian inverse uncertainty quantification (IUQ) employing scientific machine learning surrogate models as a method for nuclear data adjustments in this benchmark, and compares IUQ to the more traditional methods of generalized linear least squares (GLLS) and Monte Carlo Bayes (MOCABA). Posterior predictions from IUQ showed agreement with GLLS and MOCABA for linear applications. Here, when comparing GLLS, MOCABA, and IUQ posterior predictions to computed model responses using adjusted parameters, we observe that the GLLS predictions failed to replicate the computed response distributions for nonlinear applications, while MOCABA showed near agreement, and IUQ used the computed model responses directly. We also discuss observations on why experiments with low correlation to applications can be informative to nuclear data adjustments and identify some properties useful in selecting experiments for inclusion in nuclear data adjustment. Performance in this benchmark indicates potential for Bayesian IUQ in nuclear data adjustments.

Bayesian calibration↗
DOE OSTI2020

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Enabling Data Exchange and Data Integration with the Common Information Model: An Introduction for Power Systems Engineers and Application Developers

The Common Information Model (CIM) is an open-source information model that is used to model an electrical network and the various equipment used on the network. CIM is widely used for data exchange of bulk transmission power systems and is finding increasing use for distribution systems. Use of a non-proprietary information model (such as CIM) that has been agreed upon and adopted by numerous utilities, vendors, and researchers allows significant reduction in the effort and cost of data integration. Likewise, adoption of open data platforms built around the CIM increases available functionalities for managing and optimizing the smart grid of the future. This report is intended as an introduction to CIM for utility engineers, power systems researchers, and application developers, providing a broad view of the CIM and how particular profiles can be adapted for various use cases. Unlike most other CIM introduction documents and the International Electrotechnical Commission (IEC) standards (which are mostly targeted to an audience of data scientists, enterprise database managers, and platform developers), this report is intended for users of traditional power systems analysis software and other readers without any prior experience with canonical information models, data profiles, or UML modeling.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Livewire Data Platform - A Solution for Energy Efficient Mobility Systems (EEMS) Data Sharing

The Livewire Data Platform is a data storage and exchange platform. It provides: a platform, allowing data sharing and discovery by linking disparate data sources to a single website via application programming interfaces (APIs); a community, building partnerships and collaboration rather than competition; a system, allowing shared data to increase in size and complexity as EEMS evolves.

47 OTHER INSTRUMENTATION↗
DOE OSTI2017

Denton, Texas: Using Transportation Data to Reduce Fuel Consumption (City Energy: From Data to Decisions)

This fact sheet "Denton, Texas: Using Transportation Data to Reduce Fuel Consumption" explains how the City of Denton used data from the U.S. Department of Energy's Cities Leading through Energy Analysis and Planning (Cities-LEAP) and the State and Local Energy Data (SLED) programs to inform its city energy planning. It is one of ten fact sheets in the "City Energy: From Data to Decisions" series.

Cities leading through energy analysis and plannin↗
DOE OSTI2017

Lafayette, Colorado: Using Energy Data for Electric Vehicle Infrastructure Planning (City Energy: From Data to Decisions)

This fact sheet "Lafayette, Colorado: Using Energy Data for Electric Vehicle Infrastructure Planning" explains how the City of Lafayette used data from the U.S. Department of Energy's Cities Leading through Energy Analysis and Planning (Cities-LEAP) and the State and Local Energy Data (SLED) programs to inform its city energy planning. It is one of ten fact sheets in the "City Energy: From Data to Decisions" series.

Alternative↗
DOE OSTI2017

Moab, Utah: Using Energy Data to Target Carbon Reductions from Building Energy Efficiency (City Energy: From Data to Decisions)

This fact sheet "Moab, Utah: Using Energy Data to Target Carbon Reductions from Building Energy Efficiency" explains how the City of Moab used data from the U.S. Department of Energy's Cities Leading through Energy Analysis and Planning (Cities-LEAP) and the State and Local Energy Data (SLED) programs to inform its city energy planning. It is one of ten fact sheets in the "City Energy: From Data to Decisions" series.

100% renewable electricity↗
DOE OSTI2017

South Lake Tahoe, California: Using Energy Data to Partner on Building Energy Efficiency Actions (City Energy: From Data to Decisions)

This fact sheet "South Lake Tahoe, California: Using Energy Data to Partner on Building Energy Efficiency Actions" explains how the City of South Lake Tahoe used data from the U.S. Department of Energy's Cities Leading through Energy Analysis and Planning (Cities-LEAP) and the State and Local Energy Data (SLED) programs to inform its city energy planning. It is one of ten fact sheets in the "City Energy: From Data to Decisions" series.

building energy efficiency actions↗
DOE OSTI2017

Columbia, Missouri: Using Energy Data to Reduce Emissions and Achieve Low-Income Household Energy Savings (City Energy: From Data to Decisions)

This fact sheet "Columbia, Missouri: Using Energy Data to Reduce Emissions and Achieve Low-Income Household Energy Savings" explains how the City of Columbia used data from the U.S. Department of Energy's Cities Leading through Energy Analysis and Planning (Cities-LEAP) and the State and Local Energy Data (SLED) programs to inform its city energy planning. It is one of ten fact sheets in the "City Energy: From Data to Decisions" series.

Cities leading through energy analysis and plannin↗
DOE OSTI2013

Applications of Ensemble-based Data Assimilation Techniques for Aquifer Characterization using Tracer Data at Hanford 300 Area

Subsurface aquifer characterization often involves high parameter dimensionality and requires tremendous computational resources if employing a full Bayesian approach. Ensemble-based data assimilation techniques, including filtering and smoothing, are computationally efficient alternatives. Despite the increasing number of applications of ensemble-based methods in assimilating flow and transport related data for subsurface aquifer charaterization, most are limited to either synthetic studies or two-dimensional problems. In this study, we applied ensemble-based techniques for assimilating field tracer experimental data obtained from the Integrated Field Research Challenge (IFRC) site at the Hanford 300 Area. The forward problem was simulated using the massively-parallel three-dimensional flow and transport code PFLOTRAN to effectively deal with the highly transient flow boundary conditions at the site and to meet the computational demands of ensemble-based methods. This study demonstrates the effectiveness of ensemble-based methods for characterizing a heterogeneous aquifer by sequentially assimilating multiple types of data. The necessity of employing high performance computing is shown to enable increasingly mechanistic non-linear forward simulations to be performed within the data assimilation framework for a complex system with reasonable turnaround time.

data assimilation, ensemble kalman filter, inverse↗