Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Low-Power, Flexible Sensor Arrays with Solderless Board-to-Board Connectors for Monitoring Soil Deformation and Temperature

Landslides are a global and frequent natural hazard, affecting many communities and infrastructure networks. Technological solutions are needed for long-term, large-scale condition monitoring of infrastructure earthworks or natural slopes. However, current instruments for slope stability monitoring are often costly, require a complex installation process and/or data processing schemes, or have poor resolution. Wireless sensor networks comprising low-power, low-cost sensors have been shown to be a crucial part of landslide early warning systems. Here, we present the development of a novel sensing approach that uses linear arrays of three-axis accelerometers for monitoring changes in sensor inclination, and thus the surrounding soil’s deformation. By combining these deformation measurements with depth-resolved temperature measurements, we can link our data to subsurface thermal–hydrological regimes where relevant. In this research, we present a configuration of cascaded I2C sensors that (i) have ultra-low power consumption and (ii) enable an adjustable probe length. From an electromechanical perspective, we developed a novel board-to-board connection method that enables narrow, semi-flexible sensor arrays and a streamlined assembly process. The low-cost connection method relies on a specific FR4 printed circuit board design that allows board-to-board press fitting without using electromechanical components or solder connections. The sensor assembly is placed in a thin, semi-flexible tube (inner diameter 6.35 mm) that is filled with an epoxy compound. The resulting sensor probe is connected to an AA-battery-powered data logger with wireless connectivity. We characterize the system’s electromechanical properties and investigate the accuracy of deformation measurements. Our experiments, performed with probes up to 1.8 m long, demonstrate long-term connector stability, as well as probe mechanical flexibility. Furthermore, our accuracy analysis indicates that deformation measurements can be performed with a 0.390 mm resolution and a 95% confidence interval of ±0.73 mm per meter of probe length. This research shows the suitability of low-cost accelerometer arrays for distributed soil stability monitoring. In comparison with emerging low-cost measurements of surface displacement, our approach provides depth-resolved deformation, which can inform about shallow sliding surfaces.

deformation monitoring↗
NASA NTRS2017

Kepler Planet Detection Metrics: Pixel-Level Transit Injection Tests of Pipeline Detection Efficiency for Data Release 25

This document describes the results of the fourth pixel-level transit injection experiment, which was designed to measure the detection efficiency of both the Kepler pipeline (Jenkins 2002, 2010; Jenkins et al. 2017) and the Robovetter (Coughlin 2017). Previous transit injection experiments are described in Christiansen et al. (2013, 2015a,b, 2016).In order to calculate planet occurrence rates using a given Kepler planet catalogue, produced with a given version of the Kepler pipeline, we need to know the detection efficiency of that pipeline. This can be empirically determined by injecting a suite of simulated transit signals into the Kepler data, processing the data through the pipeline, and examining the distribution of successfully recovered transits. This document describes the results for the pixel-level transit injection experiment performed to accompany the final Q1-Q17 Data Release 25 (DR25) catalogue (Thompson et al. 2017)of the Kepler Objects of Interest. The catalogue was generated using the SOC pipeline version 9.3 and the DR25 Robovetter acting on the uniformly processed Q1-Q17 DR25 light curves (Thompson et al. 2016a) and assuming the Q1-Q17 DR25 Kepler stellar properties (Mathur et al. 2017).

Pixel-Level Transit Injection↗
NASA NTRS2015

Architectures Toward Reusable Science Data Systems

Science Data Systems (SDS) comprise an important class of data processing systems that support product generation from remote sensors and in-situ observations. These systems enable research into new science data products, replication of experiments and verification of results. NASA has been building systems for satellite data processing since the first Earth observing satellites launched and is continuing development of systems to support NASA science research and NOAAs Earth observing satellite operations. The basic data processing workflows and scenarios continue to be valid for remote sensor observations research as well as for the complex multi-instrument operational satellite data systems being built today. System functions such as ingest, product generation and distribution need to be configured and performed in a consistent and repeatable way with an emphasis on scalability. This paper will examine the key architectural elements of several NASA satellite data processing systems currently in operation and under development that make them suitable for scaling and reuse. Examples of architectural elements that have become attractive include virtual machine environments, standard data product formats, metadata content and file naming, workflow and job management frameworks, data acquisition, search, and distribution protocols. By highlighting key elements and implementation experience we expect to find architectures that will outlast their original application and be readily adaptable for new applications. Concepts and principles are explored that lead to sound guidance for SDS developers and strategists.

Data Processing↗
DOE OSTI2021

Tidal Energy Resource Characterization, Bottom Lander Measurements, Cook Inlet, AK, 2021

These datasets are from tidal resource characterization measurements collected on the Terrasond High Energy Oceanographic Mooring (THEOM) from 1 July 2021 to 30 August 2021 (60 days) in Cook Inlet, Alaska. The lander was deployed at 60.7207031 N, 151.4294998 W in ~50 m of water. The dataset contains raw and processed data from the following two instruments: 1. A Nortek Signature 500 kHz acoustic Doppler current profiler (ADCP). Data were recorded in 4 Hz in the beam coordinate system from all 5 beams. Processed data has been averaged into 5 minutes bins and converted to the East-North-Up (ENU) coordinate system. 2. A Nortek Vector acoustic Doppler velocimeter (ADV). Data were recorded at 8 Hz in the beam coordinate system. Processed data has been averaged into 5 minutes bins and converted to the Streamwise - Cross-stream - Vertical (Principal) coordinate system. Turbulence statistics were calculated from 5-minute bins, with an FFT length equal to the bin length, and saved in the processed dataset. Data was read and analyzed using the DOLfYN (version 1.0.2) python package and saved in MATLAB (.mat) and netCDF (.nc) file formats. Files containing analyzed data (".b1") were standardized using the TSDAT (version 0.4.2) python package. NetCDF files can be opened using DOLfYN (e.g., `dat = dolfyn.load(''*.nc")`) or the xarray python package (e.g. `dat = xarray.open_dataset("*.nc"). All distances are in meters (e.g., depth, range, etc), and all velocities in m/s. See the DOLfYN documentation linked in the submission, and/or the Nortek documentation for additional details.

16 TIDAL AND WAVE POWER↗
NASA NTRS1992

Onboard configuration and operation of the TDRSS onboard navigation system (TONS)

This paper presents an overview of onboard navigation via TDRSS for both the EP/EUVE (Explorer Platform/Extreme Ultraviolet Explorer) experiment and for future users of TONS. System goals and objectives are discussed, and the operations concept for the three stages of TONS development is presented. A description is given of the first stage of the TONS development, the experiment implementation on EP, including the hardware characteristics, signal acquisition process, onboard data acquisition, and ground data processing in the flight emulation environment of the TONS Ground Support System (TGSS). Evaluation of the experiment implementation is presented and improvements described in terms of the second development stage, a full TONS I system, which uses Doppler data derived from scheduled forward-link S-band services to provide onboard orbit and frequency determination. A method for onboard time maintenance commensurate with TONS I is also presented. TONS II, the third stage, will use Doppler and pseudorange data derived from a continuous, nonscheduled forward-link S-band beacon service to provide onboard orbit, time, and frequency determination.

Gramling, Cheryl J.↗
DOE OSTI2024

Portable Acceleration of CMS Computing Workflows with Coprocessors as a Service

Computing demands for large scientific experiments, such as the CMS experiment at the CERN LHC, will increase dramatically in the next decades. To complement the future performance increases of software running on central processing units (CPUs), explorations of coprocessor usage in data processing hold great potential and interest. Coprocessors are a class of computer processors that supplement CPUs, often improving the execution of certain functions due to architectural design choices. We explore the approach of Services for Optimized Network Inference on Coprocessors (SONIC) and study the deployment of this as-a-service approach in large-scale data processing. In the studies, we take a data processing workflow of the CMS experiment and run the main workflow on CPUs, while offloading several machine learning (ML) inference tasks onto either remote or local coprocessors, specifically graphics processing units (GPUs). With experiments performed at Google Cloud, the Purdue Tier-2 computing center, and combinations of the two, we demonstrate the acceleration of these ML algorithms individually on coprocessors and the corresponding throughput improvement for the entire workflow. This approach can be easily generalized to different types of coprocessors and deployed on local CPUs without decreasing the throughput performance. We emphasize that the SONIC approach enables high coprocessor usage and enables the portability to run workflows on different types of coprocessors.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2011

A Novel Automated Method for Analyzing Cylindrical Computed Tomography Data

A novel software method is presented that is applicable for analyzing cylindrical and partially cylindrical objects inspected using computed tomography. This method involves unwrapping and re-slicing data so that the CT data from the cylindrical object can be viewed as a series of 2-D sheets in the vertical direction in addition to volume rendering and normal plane views provided by traditional CT software. The method is based on interior and exterior surface edge detection and under proper conditions, is FULLY AUTOMATED and requires no input from the user except the correct voxel dimension from the CT scan. The software is available from NASA in 32- and 64-bit versions that can be applied to gigabyte-sized data sets, processing data either in random access memory or primarily on the computer hard drive. Please inquire with the presenting author if further interested. This software differentiates itself in total from other possible re-slicing software solutions due to complete automation and advanced processing and analysis capabilities.

Roth, D. J.↗
NASA NTRS2004

Processing of High Resolution, Multiparametric Radar Data for the Airborne Dual-Frequency Precipitation Radar APR-2

Following the successful Precipitation Radar (PR) of the Tropical Rainfall Measuring Mission, a new airborne, 14/35 GHz rain profiling radar, known as Airborne Precipitation Radar - 2 (APR-2), has been developed as a prototype for an advanced, dual-frequency spaceborne radar for a future spaceborne precipitation measurement mission. . This airborne instrument is capable of making simultaneous measurements of rainfall parameters, including co-pol and cross-pol rain reflectivities and vertical Doppler velocities, at 14 and 35 GHz. furthermore, it also features several advanced technologies for performance improvement, including real-time data processing, low-sidelobe dual-frequency pulse compression, and dual-frequency scanning antenna. Since August 2001, APR-2 has been deployed on the NASA P3 and DC8 aircrafts in four experiments including CAMEX-4 and the Wakasa Bay Experiment. Raw radar data are first processed to obtain reflectivity, LDR (linear depolarization ratio), and Doppler velocity measurements. The dataset is then processed iteratively to accurately estimate the true aircraft navigation parameters and to classify the surface return. These intermediate products are then used to refine reflectivity and LDR calibrations (by analyzing clear air ocean surface returns), and to correct Doppler measurements for the aircraft motion. Finally, the the melting layer of precipitation is detected and its boundaries and characteristics are identifIed at the APR-2 range resolution of 30m. The resulting 3D dataset will be used for validation of other airborne and spaceborne instruments, development of multiparametric rain/snow retrieval algorithms and melting layer characterization and statistics.

airborne radar↗
NASA NTRS1973

Significant techniques in the processing and interpretation of ERTS-1 data

The discipline oriented investigations underway at the Johnson Space Center (JSC) using ERTS-1 data provide an appropriate framework for the systematic evaluation of the various elements comprising a prototype multispectral data processing and analysis system. In particular such a system may be thought of as the integration of: (1) a preprocessing subsystem; (2) a spectral clustering subsystem, (3) a correlation and classification subsystem; (4) mensuration subsystem; and (5) an information management subsystem. Specific elements of this system are already operational at JSC. It is in the context of this system that technique development and application is being pursued at JSC. Aircraft, ERTS and EREP data will be utilized to refine the subsystem elements for each of the data acquisition systems or system combinations that are optimally suited for a specific Earth Resources application. The techniques reported are those that have been developed to date during the utilization of ERTS-1 data in this processing and analysis system.

Cousin, S. B.↗
NASA NTRS1983

Description of data base management activities

ARC's current and future data processing needs have been identified and defined. Requirements for improved data processing capabilities are listed: (1) Centralized control of system data files. Establish a DBA/data base administrator position with responsibility for management of the Centers numerous data bases; (2) Programmer tools to improve efficiency of performance; (3) Direct and timely access to information. Presently, the user submits query requests to the data processing department where they are prioritized with other queries and then batch processed using a report generator; (4) On line data entry. With the merger of the Dryden facility with ARC on line data entry, edit and updates have become mandatory for timely operation and reporting. A DBMS software package was purchased to meet the above requirements.

Source record↗
DOE OSTI2023

Technical descriptions of the experimental dynamical downscaling simulations over North America by the CAM–MPAS variable-resolution model

Abstract. Comprehensive assessment of climate datasets is important for communicating model projections and associated uncertainties to stakeholders. Uncertainties can arise not only from assumptions and biases within the model but also from external factors such as computational constraint and data processing. To understand sources of uncertainties in global variable-resolution (VR) dynamical downscaling, we produced a regional climate dataset using the Model for Prediction Across Scales (MPAS; dynamical core version 4.0) coupled to the Community Atmosphere Model (CAM; version 5.4), which we refer to as CAM–MPAS hereafter. This document provides technical details of the model configuration, simulations, computational requirements, post-processing, and data archive of the experimental CAM–MPAS downscaling data. The CAM–MPAS model is configured with VR meshes featuring higher resolutions over North America as well as quasi-uniform-resolution meshes across the globe. The dataset includes multiple uniform- (240 and 120 km) and variable-resolution (50–200, 25–100, and 12–46 km) simulations for both the present-day (1990–2010) and future (2080–2100) periods, closely following the protocol of the North American Coordinated Regional Climate Downscaling Experiment. A deviation from the protocol is the pseudo-warming experiment for the future period, using the ocean boundary conditions produced by adding the sea surface temperature and sea-ice changes from the low-resolution version of the Max Planck Institute Earth System Model (MPI-ESM-LR) in the Coupled Model Intercomparison Project Phase 5 to the present-day ocean state from a reanalysis product. Some unique aspects of global VR models are evaluated to provide background knowledge to data users and to explore good practices for modelers who use VR models for regional downscaling. In the coarse-resolution domain, strong resolution sensitivity of the hydrological cycles exists over the tropics but does not appear to affect the midlatitude circulations in the Northern Hemisphere, including the downscaling target of North America. The pseudo-warming experiment leads to similar responses of large-scale circulations to the imposed radiative and boundary forcings in the CAM–MPAS and MPI-ESM-LR models, but their climatological states in the historical period differ over various regions, including North America. Such differences are carried to the future period, suggesting the importance of the base state climatology. Within the refined domain, precipitation statistics improve with higher resolutions, and such statistical inference is verified to be negligibly influenced by horizontal remapping during post-processing. Limited (≈50 % slower) throughput of the current code is found on a recent many-core/wide-vector high-performance computing system, which limits the lengths of the 12–46 km simulations and indirectly affects sampling uncertainty. Our experience shows that global and technical aspects of the VR downscaling framework require further investigations to reduce uncertainties for regional climate projection.

58 GEOSCIENCES↗
DOE OSTI2025

Data Quality Assessment Process for Real-Time Data-Driven Traffic Microsimulation of Smart Corridor

Smart corridor digital twins are often created for the development and evaluation of emerging intelligent transportation systems and Connected and Autonomous Vehicle (CAV) technologies. However, limited guidance exists for data quality assessment for digital twin development. To address this, this paper discusses the data quality assessment utilized to develop data-driven real-time microscopic simulation models, i.e., digital twins, for two separate smart corridors: the North Avenue Smart Corridor in Atlanta, GA, and the Martin Luther King Smart Corridor in Chattanooga, Tennessee. This paper provides a summary of the author’s investigations of data requirements and data characteristics for the given smart corridor digital twin development efforts. With a focus on data, this summary includes a description of the data investigation process, key data issues observed, and strategies to address observed issues. Discussion is provided to help expand the lessons from these studies to other digital twin development efforts.

Saroj, Abhilasha [ORNL] (ORCID:0000000191178063)↗
DOE OSTI2023

A generalized approach to x-ray data modeling for high-energy-density plasma experiments

Accurate understanding of x-ray diagnostics is crucial for both interpreting high-energy-density experiments and testing simulations through quantitative comparisons. X-ray diagnostic models are complex. Past treatments of individual x-ray diagnostics on a case-by-case basis have hindered universal diagnostic understanding. Here, in this study, we derive a general formula for modeling the absolute response of non-focusing x-ray diagnostics, such as x-ray imagers, one-dimensional space-resolved spectrometers, and x-ray power diagnostics. The present model is useful for both data modeling and data processing. It naturally accounts for the x-ray crystal broadening. The new model verifies that standard approaches for a crystal response can be good approximations, but they can underestimate the total reflectivity and overestimate spectral resolving power by more than a factor of 2 in some cases near reflectivity edge features. We also find that a frequently used, simplified-crystal-response approximation for processing spectral data can introduce an absolute error of more than an order of magnitude and the relative spectral radiance error of a factor of 3. The present model is derived with straightforward geometric arguments. It is more general and is recommended for developing a unified picture and providing consistent treatment over multiple x-ray diagnostics. Such consistency is crucial for reliable multi-objective data analyses.

Nagayama, Taisuke↗
DOE OSTI2022

Optimizing the hit finding algorithm for liquid argon TPC neutrino detectors using parallel architectures

Neutrinos are particles that interact rarely, so identifying them requires large detectors which produce lots of data. Processing this data with the computing power available is becoming even more difficult as the detectors increase in size to reach their physics goals. Liquid argon time projection chamber (LArTPC) neutrino experiments are expected to grow in the next decade to have 100 times more wires than in currently operating experiments, and modernization of LArTPC reconstruction code, including parallelization both at data- and instruction-level, will help to mitigate this challenge. The LArTPC hit finding algorithm is used across multiple experiments through a common software framework. In this paper we discuss a parallel implementation of this algorithm. Using a standalone setup we find speedup factors of two times from vectorization and 30–100 times from multi-threading on Intel architectures. The new version has been incorporated back into the framework so that it can be used by experiments. On a serial execution, the integrated version is about 10 times faster than the previous one and, once parallelization is enabled, more speedups comparable to the standalone program are achieved.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

A Review of Visualization Methods for Cyber-Physical Security: Smart Grid Case Study

Cyber-Physical Systems (CPSs) are becoming increasingly complex and interconnected as they attempt to meet the demands of evolving society. As a result, monitoring and maintaining them becomes a more complex and demanding task for control system operators and cyber defenders. While the literature on visualization techniques in the context of cybersecurity is extensive, the same cannot be said for studies on visualization for the security of cyber-physical systems. This paper aims to fill that gap by: 1) defining the main features of a visualizations workflow for security visualizations in cyber-physical systems. The workflow includes the acquisition of cyber and physical data, processing of data, selection, and configuration of both visualization tools and end-user interactions. 2) Providing an overview of cyber-physical security visualization systems, with a focus on smart grids as a case study. Finally, we use the perspectives gained from this analysis to provide insights and directions for future research and design of cyber-physical visualization techniques.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

HopPyBar

HopPyBar is a python program to import, analyze, and export split-Hopkinson pressure bar (SHPB, also known as Kolsky bar) data. Traditional analysis offers a black box approach, where input data is converted to analyzed output by performing a series of calculations without user involvement. This program serves as a developmental platform to "white box" the data analysis process. Data streams can be captured (in-situ) to enable advanced or unconventional analyses, statistics, and comparisons. Additionally, the program is geared towards the standardized forms of input and output used at LANL to streamline analysis, but the open nature of the program makes additional input/output schemes straightforward to add. General workflow will import SHPB data in one of a number of formats, identify relevant portions of data signals, and convert to stress-strain-strain rate to show material behavior as a function of dynamic testing.

Morrow, Benjamin↗