Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Technical descriptions of the experimental dynamical downscaling simulations over North America by the CAM–MPAS variable-resolution model

Abstract. Comprehensive assessment of climate datasets is important for communicating model projections and associated uncertainties to stakeholders. Uncertainties can arise not only from assumptions and biases within the model but also from external factors such as computational constraint and data processing. To understand sources of uncertainties in global variable-resolution (VR) dynamical downscaling, we produced a regional climate dataset using the Model for Prediction Across Scales (MPAS; dynamical core version 4.0) coupled to the Community Atmosphere Model (CAM; version 5.4), which we refer to as CAM–MPAS hereafter. This document provides technical details of the model configuration, simulations, computational requirements, post-processing, and data archive of the experimental CAM–MPAS downscaling data. The CAM–MPAS model is configured with VR meshes featuring higher resolutions over North America as well as quasi-uniform-resolution meshes across the globe. The dataset includes multiple uniform- (240 and 120 km) and variable-resolution (50–200, 25–100, and 12–46 km) simulations for both the present-day (1990–2010) and future (2080–2100) periods, closely following the protocol of the North American Coordinated Regional Climate Downscaling Experiment. A deviation from the protocol is the pseudo-warming experiment for the future period, using the ocean boundary conditions produced by adding the sea surface temperature and sea-ice changes from the low-resolution version of the Max Planck Institute Earth System Model (MPI-ESM-LR) in the Coupled Model Intercomparison Project Phase 5 to the present-day ocean state from a reanalysis product. Some unique aspects of global VR models are evaluated to provide background knowledge to data users and to explore good practices for modelers who use VR models for regional downscaling. In the coarse-resolution domain, strong resolution sensitivity of the hydrological cycles exists over the tropics but does not appear to affect the midlatitude circulations in the Northern Hemisphere, including the downscaling target of North America. The pseudo-warming experiment leads to similar responses of large-scale circulations to the imposed radiative and boundary forcings in the CAM–MPAS and MPI-ESM-LR models, but their climatological states in the historical period differ over various regions, including North America. Such differences are carried to the future period, suggesting the importance of the base state climatology. Within the refined domain, precipitation statistics improve with higher resolutions, and such statistical inference is verified to be negligibly influenced by horizontal remapping during post-processing. Limited (≈50 % slower) throughput of the current code is found on a recent many-core/wide-vector high-performance computing system, which limits the lengths of the 12–46 km simulations and indirectly affects sampling uncertainty. Our experience shows that global and technical aspects of the VR downscaling framework require further investigations to reduce uncertainties for regional climate projection.

58 GEOSCIENCES↗
DOE OSTI2015

A new protocol to accurately determine microtubule lattice seam location

Microtubules (MTs) are cylindrical polymers of αβ-tubulin that display pseudo-helical symmetry due to the presence of a lattice seam of heterologous lateral contacts. The structural similarity between α- and β-tubulin makes it difficult to computationally distinguish them in the noisy cryo-EM images, unless a marker protein for the tubulin dimer, such as kinesin motor domain, is present. We have developed a new data processing protocol that can accurately determine αβ-tubulin register and seam location for MT segments. Our strategy can handle difficult situations, where the marker protein is relatively small or the decoration of marker protein is sparse. Using this new seam-search protocol, combined with movie processing for data from a direct electron detection camera, we were able to determine the cryo-EM structures of MT at 3.5. Å resolution in different functional states. The successful distinction of α- and β-tubulin allowed us to visualize the nucleotide state at the E-site and the configuration of lateral contacts at the seam.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Data Quality Assessment Process for Real-Time Data-Driven Traffic Microsimulation of Smart Corridor

Smart corridor digital twins are often created for the development and evaluation of emerging intelligent transportation systems and Connected and Autonomous Vehicle (CAV) technologies. However, limited guidance exists for data quality assessment for digital twin development. To address this, this paper discusses the data quality assessment utilized to develop data-driven real-time microscopic simulation models, i.e., digital twins, for two separate smart corridors: the North Avenue Smart Corridor in Atlanta, GA, and the Martin Luther King Smart Corridor in Chattanooga, Tennessee. This paper provides a summary of the author’s investigations of data requirements and data characteristics for the given smart corridor digital twin development efforts. With a focus on data, this summary includes a description of the data investigation process, key data issues observed, and strategies to address observed issues. Discussion is provided to help expand the lessons from these studies to other digital twin development efforts.

Saroj, Abhilasha [ORNL] (ORCID:0000000191178063)↗
DOE OSTI2023

A generalized approach to x-ray data modeling for high-energy-density plasma experiments

Accurate understanding of x-ray diagnostics is crucial for both interpreting high-energy-density experiments and testing simulations through quantitative comparisons. X-ray diagnostic models are complex. Past treatments of individual x-ray diagnostics on a case-by-case basis have hindered universal diagnostic understanding. Here, in this study, we derive a general formula for modeling the absolute response of non-focusing x-ray diagnostics, such as x-ray imagers, one-dimensional space-resolved spectrometers, and x-ray power diagnostics. The present model is useful for both data modeling and data processing. It naturally accounts for the x-ray crystal broadening. The new model verifies that standard approaches for a crystal response can be good approximations, but they can underestimate the total reflectivity and overestimate spectral resolving power by more than a factor of 2 in some cases near reflectivity edge features. We also find that a frequently used, simplified-crystal-response approximation for processing spectral data can introduce an absolute error of more than an order of magnitude and the relative spectral radiance error of a factor of 3. The present model is derived with straightforward geometric arguments. It is more general and is recommended for developing a unified picture and providing consistent treatment over multiple x-ray diagnostics. Such consistency is crucial for reliable multi-objective data analyses.

Nagayama, Taisuke↗
DOE OSTI2022

Optimizing the hit finding algorithm for liquid argon TPC neutrino detectors using parallel architectures

Neutrinos are particles that interact rarely, so identifying them requires large detectors which produce lots of data. Processing this data with the computing power available is becoming even more difficult as the detectors increase in size to reach their physics goals. Liquid argon time projection chamber (LArTPC) neutrino experiments are expected to grow in the next decade to have 100 times more wires than in currently operating experiments, and modernization of LArTPC reconstruction code, including parallelization both at data- and instruction-level, will help to mitigate this challenge. The LArTPC hit finding algorithm is used across multiple experiments through a common software framework. In this paper we discuss a parallel implementation of this algorithm. Using a standalone setup we find speedup factors of two times from vectorization and 30–100 times from multi-threading on Intel architectures. The new version has been incorporated back into the framework so that it can be used by experiments. On a serial execution, the integrated version is about 10 times faster than the previous one and, once parallelization is enabled, more speedups comparable to the standalone program are achieved.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2014

Operation of the Australian Store.Synchrotron for macromolecular crystallography

The Store.Synchrotron service, a fully functional, cloud computing-based solution to raw X-ray data archiving and dissemination at the Australian Synchrotron, is described. The Store.Synchrotron service, a fully functional, cloud computing-based solution to raw X-ray data archiving and dissemination at the Australian Synchrotron, is described. The service automatically receives and archives raw diffraction data, related metadata and preliminary results of automated data-processing workflows. Data are able to be shared with collaborators and opened to the public. In the nine months since its deployment in August 2013, the service has handled over 22.4 TB of raw data (∼1.7 million diffraction images). Several real examples from the Australian crystallographic community are described that illustrate the advantages of the approach, which include real-time online data access and fully redundant, secure storage. Discoveries in biological sciences increasingly require multidisciplinary approaches. With this in mind, Store.Synchrotron has been developed as a component within a greater service that can combine data from other instruments at the Australian Synchrotron, as well as instruments at the Australian neutron source ANSTO. It is therefore envisaged that this will serve as a model implementation of raw data archiving and dissemination within the structural biology research community.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2015

The NeXus data format

NeXus is an effort by an international group of scientists to define a common data exchange and archival format for neutron, X-ray and muon experiments. NeXus is built on top of the scientific data format HDF5 and adds domain-specific rules for organizing data within HDF5 files, in addition to a dictionary of well defined domain-specific field names. The NeXus data format has two purposes. First, it defines a format that can serve as a container for all relevant data associated with a beamline. This is a very important use case. Second, it defines standards in the form of application definitions for the exchange of data between applications. NeXus provides structures for raw experimental data as well as for processed data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Review of Visualization Methods for Cyber-Physical Security: Smart Grid Case Study

Cyber-Physical Systems (CPSs) are becoming increasingly complex and interconnected as they attempt to meet the demands of evolving society. As a result, monitoring and maintaining them becomes a more complex and demanding task for control system operators and cyber defenders. While the literature on visualization techniques in the context of cybersecurity is extensive, the same cannot be said for studies on visualization for the security of cyber-physical systems. This paper aims to fill that gap by: 1) defining the main features of a visualizations workflow for security visualizations in cyber-physical systems. The workflow includes the acquisition of cyber and physical data, processing of data, selection, and configuration of both visualization tools and end-user interactions. 2) Providing an overview of cyber-physical security visualization systems, with a focus on smart grids as a case study. Finally, we use the perspectives gained from this analysis to provide insights and directions for future research and design of cyber-physical visualization techniques.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2013

Trelliscope: A System for Detailed Visualization in Analysis of Large Complex Data

Visualization plays a critical role in the statistical model building and data analysis process. Data analysts, well-versed in statistical and machine learning methods, visualize data to hypothesize and validate models. These analysts need flexible, scalable visualization tools that are not decoupled from their analysis environment. In this paper we introduce Trelliscope, a visualization framework for statistical analysis of large complex data. Trelliscope extends Trellis, an effective visualization framework that divides data into subsets and applies a plotting method to each subset, arranging the results in rows and columns of panels. Trelliscope provides a way to create, arrange and interactively view panels for very large datasets, enabling flexible detailed visualization for data of any size. Scalability is achieved using distributed computing technologies coupled with . We discuss the underlying principles, design, and scalable architecture of Trelliscope, and illustrate its use on three analysis projects in the domains of proteomics, high energy physics, and power systems engineering.

Hafen, Ryan P.↗
DOE OSTI2014

Fast Demand Forecast of Electric Vehicle Charging Stations for Cell Phone Application

This paper describes the core cellphone application algorithm which has been implemented for the prediction of energy consumption at Electric Vehicle (EV) Charging Stations at UCLA. For this interactive user application, the total time of accessing database, processing the data and making the prediction, needs to be within a few seconds. We analyze four relatively fast Machine Learning based time series prediction algorithms for our prediction engine: Historical Average, kNearest Neighbor, Weighted k-Nearest Neighbor, and Lazy Learning. The Nearest Neighbor algorithm (k Nearest Neighbor with k=1) shows better performance and is selected to be the prediction algorithm implemented for the cellphone application. Two applications have been designed on top of the prediction algorithm: one predicts the expected available energy at the station and the other one predicts the expected charging finishing time. The total time, including accessing the database, data processing, and prediction is about one second for both applications.

Majidpour, Mostafa↗
DOE OSTI2018

Simulation of Space Experiments for Nuclear Planetology: Measurement of Relative Intensities of Lines of Gamma Ray Emitted upon Thermal-Neutron Capture by Nuclei

The results obtained by experimentally studying gamma rays emitted by samples prepared as analogs of planetary matter and irradiated with thermal neutrons are presented. The intensities of spectral lines of gamma rays emitted by such samples differing in chemical composition are compared. These results will be used in processing data on gamma-ray spectra of the Moon and Mercury from measurements performed onboard spacecrafts with the aim of studying the composition of the surface of these celestial bodies.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

HopPyBar

HopPyBar is a python program to import, analyze, and export split-Hopkinson pressure bar (SHPB, also known as Kolsky bar) data. Traditional analysis offers a black box approach, where input data is converted to analyzed output by performing a series of calculations without user involvement. This program serves as a developmental platform to "white box" the data analysis process. Data streams can be captured (in-situ) to enable advanced or unconventional analyses, statistics, and comparisons. Additionally, the program is geared towards the standardized forms of input and output used at LANL to streamline analysis, but the open nature of the program makes additional input/output schemes straightforward to add. General workflow will import SHPB data in one of a number of formats, identify relevant portions of data signals, and convert to stress-strain-strain rate to show material behavior as a function of dynamic testing.

Morrow, Benjamin↗
DOE OSTI2024

Utah FORGE: GES Well 16A(78)-32 and Well 16B(78)-32 Stimulation Seismic Event Catalogs

This dataset contains seismic event catalogs from the hydraulic stimulation of wells 16A(78)-32 and 16B(78)-32 at the Utah FORGE site in April 2024. The data was collected by Geo Energy Suisse (GES) using a variety of seismic monitoring technologies, including 3-component (3C) geophones and distributed acoustic sensing (DAS) systems. These technologies were deployed across several locations, including wells 16A, 16B, and Delano-1, with sensor arrays at multiple depths to capture microseismic activity during the stimulations. The catalogs provide both real-time and manually checked seismic event locations, with detailed parameters such as trigger conditions, velocity models, and data acquisition settings. The dataset includes information on the stimulation stages, event rates, and hydraulic injection conditions for each well, with a report detailing the data acquisition configuration and seismic event location methodologies. Users will need to reference the included report for a complete understanding of the sensor network, data processing techniques, and accuracy considerations.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

NEWTS EPA Gasification Dataset and Case Studies

Data from the US EPA Gasification Database containing gasification effluent data from selected U.S. coal power plants. Original data from: Nguyen, Dan-Tam, Eastern Research Group. Sep 29, 2015. Analytical Database for the Steam Electric Rulemaking - DCN SE05359. https://www.regulations.gov/document/EPA-HQ-OW-2009-0819-5640 Files include: Original data tab, processed data and charge balance information in "NEWTS_EPA_Gasification_Effluent_Dataset_all_tabs.xlsx" Individual .csv files for data in format for input into aqueous chemistry modeling software (OLI Systems and Geochemist's Workbench) Data input templates for modeling streams in OLI Systems and Geochemist's Workbench Case studies for selected solved streams modeled in OLI Systems and Geochemist's Workbench

gasification↗
DOE OSTI2017

Potential Futures for Information.

Information is one of the most powerful tools available today. All advances in technology may be used, as David Sarnoff said, for the benefit or harm of society. Information can be used to shape the future by free people, or used to control people by less than benevolent governments, as has been demonstrated since the mid - 1930s, and with growing frequency over the past 50 years. What promised to once set people free and fuel an industrial revolution that might improve the standard of living over most of the world, has also been used to manipulate and enslave entire populations. The future of information is tied to the future of technologies that support the collection of data, processing those data into information and knowledge, and distribution. Technologies supporting the future of information must include technologies that help protect the integrity of data and information, and help to guarantee its discoverability and appropriate availability -- often to the whole of society. This Page Intentionally Left Blank

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

Reconstruction for Liquid Argon TPC Neutrino Detectors Using Parallel Architectures [Slides]

Neutrinos are particles that interact rarely, so identifying them requires large detectors which produce lots of data. Processing this data with the computing power available is becoming more difficult as the detectors increase in size to reach their physics goals. In liquid argon time projection chambers (TPCs) the charged particles from neutrino interactions produce ionization electrons which drift in an electric field towards a series of collection wires, and the signal on the wires is used to reconstruct the interaction. The MicroBooNE detector currently collecting data at Fermilab has 8000 wires, and planned future experiments like DUNE will have 100 times more, which means that the time required to reconstruct an event will scale accordingly. Modernization of liquid argon TPC reconstruction code, including vectorization, parallelization and code portability to GPUs, will help to mitigate these challenges. The liquid argon TPC hit finding algorithm within the L arSoft framework used across multiple experiments has been vectorized and parallelized. This increases the speed of the algorithm on the order of ten times within a stand alone version on Intel architectures. This new version has been incorporated back into LarSoft so that it can be generally used. These methods will also be applied to other low level reconstruction of the wire signals such as the deconvolution. The applications and performance of this modernized liquid argon TPC wire reconstruction will be presented.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Navigation and Meteorological Data from Multiple Sensors on Airborne Platform (NAVMET-AIR) Value-Added Product Report

The Navigation and Meteorological Data from Multiple Sensors on Airborne Platform (NAVMET-AIR) value-added product encompasses the aafnaviwg data set in the ARM Data Center (ADC), known as the IWG file, named for the Inter-Agency Working Group for Airborne Data and Telemetry Systems (IWGADTS). Its purpose is to produce a suite of tools to promote standardization of instrument interface, data format, and data processing. This data set’s contents are navigational and meteorological state variables at 1 Hz. It also contains higher-order data flags created by ARM Aerial Facility (AAF) scientists to aid analysis such as periods when the aircraft is flying level, operating in maneuvers, or flying through cloud. Due to the nature of research flights, the payload of the aircraft consists of many duplicate and overlapping instruments to ensure, in case of instrument failure, there is a backup to record the data. Measurements from multiple instruments are consolidated into a single file. Each variable of this data set is carefully chosen from the onboard instrumentation and quality checked by AAF scientists to create this wholistic and most accurate data set for airborne research. This document is intended to be a hub towards the individual “read me” files produced for each campaign. For reference about what instrument was used for calculations on specific days, the read me files are located in the ARM IOP archive at the addresses listed, or access the IOP (intensive operational period) database at https://www.archive.arm.gov/.

54 ENVIRONMENTAL SCIENCES↗