Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Raw Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
NASA NTRS2010

SensorWeb 3G: Extending On-Orbit Sensor Capabilities to Enable Near Realtime User Configurability

This research effort prototypes an implementation of a standard interface, Web Coverage Processing Service (WCPS), which is an Open Geospatial Consortium(OGC) standard, to enable users to define, test, upload and execute algorithms for on-orbit sensor systems. The user is able to customize on-orbit data products that result from raw data streaming from an instrument. This extends the SensorWeb 2.0 concept that was developed under a previous Advanced Information System Technology (AIST) effort in which web services wrap sensors and a standardized Extensible Markup Language (XML) based scripting workflow language orchestrates processing steps across multiple domains. SensorWeb 3G extends the concept by providing the user controls into the flight software modules associated with on-orbit sensor and thus provides a degree of flexibility which does not presently exist. The successful demonstrations to date will be presented, which includes a realistic HyspIRI decadal mission testbed. Furthermore, benchmarks that were run will also be presented along with future demonstration and benchmark tests planned. Finally, we conclude with implications for the future and how this concept dovetails into efforts to develop "cloud computing" methods and standards.

Mandl, Daniel↗
NASA NTRS2004

MER SPICE Interface

MER SPICE Interface is a software module for use in conjunction with the Mars Exploration Rover (MER) mission and the SPICE software system of the Navigation and Ancillary Information Facility (NAIF) at NASA's Jet Propulsion Laboratory. (SPICE is used to acquire, record, and disseminate engineering, navigational, and other ancillary data describing circumstances under which data were acquired by spaceborne scientific instruments.) Given a Spacecraft Clock value, MER SPICE Interface extracts MER-specific data from SPICE kernels (essentially, raw data files) and calculates values for Planet Day Number, Local Solar Longitude, Local Solar Elevation, Local Solar Azimuth, and Local Solar Time (UTC). MER SPICE Interface was adapted from a subroutine, denoted m98SpiceIF written by Payam Zamani, that was intended to calculate SPICE values for the Mars Polar Lander. The main difference between MER SPICE Interface and m98SpiceIf is that MER SPICE Interface does not explicitly call CHRONOS, a time-conversion program that is part of a library of utility subprograms within SPICE. Instead, MER SPICE Interface mimics some portions of the CHRONOS code, the advantage being that it executes much faster and can efficiently be called from a pipeline of events in a parallel processing environment.

Sayfi, Elias↗
NASA NTRS2014

Robust Informatics Infrastructure Required For ICME: Combining Virtual and Experimental Data

With the increased emphasis on reducing the cost and time to market of new materials, the need for robust automated materials information management system(s) enabling sophisticated data mining tools is increasing, as evidenced by the emphasis on Integrated Computational Materials Engineering (ICME) and the recent establishment of the Materials Genome Initiative (MGI). This need is also fueled by the demands for higher efficiency in material testing; consistency, quality and traceability of data; product design; engineering analysis; as well as control of access to proprietary or sensitive information. Further, the use of increasingly sophisticated nonlinear, anisotropic and or multi-scale models requires both the processing of large volumes of test data and complex materials data necessary to establish processing-microstructure-property-performance relationships. Fortunately, material information management systems have kept pace with the growing user demands and evolved to enable: (i) the capture of both point wise data and full spectra of raw data curves, (ii) data management functions such as access, version, and quality controls;(iii) a wide range of data import, export and analysis capabilities; (iv) data pedigree traceability mechanisms; (v) data searching, reporting and viewing tools; and (vi) access to the information via a wide range of interfaces. This paper discusses key principles for the development of a robust materials information management system to enable the connections at various length scales to be made between experimental data and corresponding multiscale modeling toolsets to enable ICME. In particular, NASA Glenn's efforts towards establishing such a database for capturing constitutive modeling behavior for both monolithic and composites materials

Mutli-scale models↗
NASA NTRSDate not supplied

Preservation of Provenance and Context to Ensure Future Understandability of Airborne Earth Observations and Derived Data Products

Open-source science goes beyond making data from scientific projects (e.g., on-orbit/satellite missions, airborne and field investigations, and other data producing activities) openly available after they are generated, but involves and open sharing of information throughout the project lifecycle. Preservation of the data and associated information required for understanding and reusing the data well after the scientific projects is a contributor to open-source science as well. Considering the high investment in the on-orbit/satellite missions, we had developed a document titled “NASA Earth Science Data Preservation Content Specification (PCS)” in 2011. This document has been used as a requirement for recent on-orbit/satellite missions by NASA. Recently it became clear that the specifications should be applied to other scientific projects as well. Therefore, the document was revised to cover other types of projects, and a Preservation Content Implementation Guidance (PCIG) document was also developed. The revised PCS, and the PCIG, were published in 2022. The purpose of this presentation is to highlight the contents of these documents as they apply to suborbital/airborne investigations. The PCS calls for content preservation in eight general categories - Measuring Instrument/Platform Description, Instrument and Science Data Products and Metadata, Science Raw Data, Product and Algorithm Documentation, Instrument Calibration, Science Algorithm Software, Science Data Product Algorithm Inputs, Science Data Product Validation, and Science Data Access and Analysis Tools. While all these categories apply to various types of projects, a few clarifying sentences have been added to the descriptions of contents in each of the categories to show which categories are especially important to airborne and field investigations and where some contents are not applicable (or difficult to obtain). The PCIG document provides some general guidance applicable to all types of projects and specific guidance in a separate section for airborne and field investigations. This section calls out typical artifacts produced during such investigations that can meet the spirit of the various PCS categories.

remote sensing↗
DOE OSTI2021

DUNE Software and Computing Challenges

The DUNE experiment will begin running in the late 2020's. The goals of the experiment include 1) studying neutrino oscillations using a beam of neutrinos from Fermilab in Illinois to the Sanford Underground Research Facility, 2) studying astrophysical neutrino sources and rare processes and 3) understanding the physics of neutrino interactions in matter. The DUNE Far Detector, consisting of four 17 kt LArTPC modules, will produce "events" rang-ing in size from 6 GBs to more than 100 TBs, posing unique challenges for DUNE software and computing. The data processing algorithms, particularly for raw data, drive the requirements for the future DUNE software framework documented here.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Integration of Rucio in Belle II

The Belle II experiment, which started taking physics data in April 2019, will multiply the volume of data currently stored on its nearly 30 storage elements worldwide by one order of magnitude to reach about 340 PB of data (raw and Monte Carlo simulation data) by the end of operations. To tackle this massive increase and to manage the data even after the end of the data taking, it was decided to move the Distributed Data Management software from a homegrown piece of software to a widely used Data Management solution in HEP and beyond : Rucio. This contribution describes the work done to integrate Rucio with Belle II distributed computing infrastructure as well as the migration strategy that was successfully performed to ensure a smooth transition.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Quality Assurance and Quality Control (QA/QC) of Meteorological Time Series Data for Billy Barr, East River, Colorado USA

A comprehensive Quality Assurance (QA) and Quality Control (QC) statistical framework consists of three major phases: Phase 1—Preliminary raw data sets exploration, including time formatting and combining datasets of different lengths and different time intervals; Phase 2—QA of the datasets, including detecting and flagging of duplicates, outliers, and extreme values; and Phase 3—the development of time series of a desired frequency, imputation of missing values, visualization and a final statistical summary. The time series data collected at the Billy Barr meteorological station (East River Watershed, Colorado) were analyzed. The developed statistical framework is suitable for both real-time and post-data-collection QA/QC analysis of meteorological datasets.The files that are in this data package include one excel file, converted to CSV format (Billy_Barr_raw_qaqc.csv) that contains the raw meteorological data, i.e., input data used for the QA/QC analysis. The second CSV file (Billy_Barr_1hr.csv) is the QA/QC and flagged meteorological data, i.e., output data from the QA/QC analysis. The last file (QAQC_Billy_Barr_2021-03-22.R) is a script written in R that implements the QA/QC and flagging process. The purpose of the CSV data files included in this package is to provide input and output files implemented in the R script.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Tidal Resource Data from Sequim Bay Inlet, WA, August 2020

Data from a Nortek Signature1000 deployed on a lander for 14 days in Aug 2020 in the entrance to Sequim Bay, WA. Raw data were processed using the DOLfYN python package and standardized using the ME Data Pipeline python package, tsdat version 0.2.12. Processed data were partitioned into 24 hour increments and saved in the NETCDF file format.

16 TIDAL AND WAVE POWER↗
NASA NTRS1983

User's guide for the Solar Backscattered Ultraviolet (SBUV) and the Total Ozone Mapping Spectrometer (TOMS) RUT-S and RUT-T data sets: October 31, 1978 to November 1, 1980

Raw data from the Solar Backscattered Ultrviolet/Total Ozone Mapping Spectrometer (SBUV/TOMS) Nimbus 7 operation are available on computer tape. These data are contained on two separate sets of RUTs (Raw Units Tapes) for SBUV and TOMS, labelled RUT-S and RUT-T respectively. The RUT-S and RUT-T tapes contain uncalibrated radiance and irradiance data, housekeeping data, wavelength and electronic calibration data, instrument field-of-view location and solar ephemeris information. These tapes also contain colocated cloud, terrain pressure and snow/ice thickness data, each derived from an independent source. The "RUT User's Guide" describes the SBUV and TOMS experiments, the instrument calibration and performance, operating schedules, and data coverage, and provides an assessment of RUT-S and -T data quality. It also provides detailed information on the data available on the computer tapes.

Fleig, A. J.↗
NASA NTRSDate not supplied

Developing Deep Learning Models for System Remaining Useful Life Predictions: Application to Aircraft Engines

Prognostics and health management (PHM) is an important part of ensuring reliable operations of complex safety- critical systems. System-level remaining useful life (RUL) estimation is a much more complex problem than making estimations at the component level, and system-level RUL methodologies remain sparse in the literature. Model-based approaches have traditionally worked in the past for components such as capacitors, MOSFETs, batteries, or hard-drives (to name a few examples), but developing high fidelity dynamics models of cyber physical systems that can be used to study the effects of multiple degrading components in the system remains a challenging task. Some initial work on model-based System RUL predictions was demonstrated in Khorasgani, et al [1], but, to generalize the system-level prognostics problem, we have to resort to pure data driven and hybrid approaches. In this work, we propose an end-to-end data- driven framework for developing deep learning models to predict remaining useful life of cyber physical systems operating under unknown faulty conditions. The raw data is organized with a data schema that improves the model development process and down stream data analysis tasks. Due to the unknown faulty conditions, the raw sensor data is transformed into signals that expose the underlying degradation processes, which are then used for model development. Bayesian Optimization is used to tune the model parameters prior to training and validation. We show that this approach results in accurate predictions within 3 cycles to end of life (EOL). We demonstrate the effectiveness of our approach by applying it to the N-CMAPSS turbofan engine dataset recently released by NASA, which includes high fidelity degradation modeling, real world operating conditions, and a large set of fault operating modes.

Prognostics↗
DOE OSTI2021

Robotic Understanding of Spatial Relationships Using Neural-Logic Learning

Understanding spatial relations of objects is critical in many robotic applications such as grasping, manipulation, and obstacle avoidance. Humans can simply reason object's spatial relations from a glimpse of a scene based on prior knowledge of spatial constraints. The proposed method enables a robot to comprehend spatial relationships among objects from RGB-D data. This paper proposed a neural-logic learning framework to learn and reason spatial relations from raw data by following logic rules on spatial constraints. The neural-logic network consists of three blocks: grounding block, spatial logic block, and inference block. The grounding block extracts high-level features from the raw sensory data. The spatial logic blocks can predicate fundamental spatial relations by training a neural network with spatial constraints. The inference block can infer complex spatial relations based on the predicated fundamental spatial relations. Simulations and robotic experiments evaluated the performance of the proposed method.

Wang, Dali↗
DOE OSTI2025

EGS Collab Experiment 2: Continuous Active Source Seismic Monitoring (CASSM)

The dataset contains continuous active-source seismic monitoring (CASSM) data collected during EGS Collab Experiment 2, conducted from February to September 2022 at the Sanford Underground Research Facility in Lead, South Dakota. This experiment aimed to investigate enhanced geothermal systems through high-pressure fluid injections at depths of 1200-1500 meters. The seismic monitoring system included 16 three-component piezoelectric accelerometers and 24 hydrophones installed in boreholes around the injection zones, recording signals from piezoelectric seismic sources. Data were acquired using both continuous and triggered recording systems, with sampling rates of up to 100 kHz. The raw data are organized by timestamps and stored in .dat format, with accompanying log files. Calibration certificates for selected accelerometers are provided to aid in correcting sensor responses, though users are advised to consider possible effects of enclosures and installation on sensor performance. Users are strongly advised to consult the accompanying report, which outlines the experimental setup, data acquisition, sensor specifications, and recording systems.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Untargeted metabolite data from a root surface in a rhizobox

Raw data is provided from samples analyzed using separate reverse phase chromatographic methods on a high performance liquid chromatograph with mass spectrometry. These porewater samples were collected from a microdialysis which generated samples along the surface of a growing A. sative root (all_hc). This data was used to answer questions connecting rhizosphere metabolite (putatively identified metabolites, hc_putative_norm) changes over time (root growth) with changes in the surrounding rhizosphere biogeochemistry (DOC, redox, pH). Rhizosphere biogeochemistry values are provided in the hc_putative_norm file as averages over their respective range of time that they were collected at. The hc_putative_norm file also contains all normalized values over only the intensity values collected for putatively identified metabolites.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1984

Revised radiometric calibration technique for LANDSAT-4 Thematic Mapper data by the Canada Centre for Remote Sensing

Observations of raw image data, raw radiometric calibration data, and background measurements extracted from the raw data streams on high density tape reveal major shortcomings in a technique proposed by the Canadian Center for Remote Sensing in 1982 for the radiometric correction of TM data. Results are presented which correlate measurements of the DC background with variations in both image data background and calibration samples. The effect on both raw data and data corrected using the earlier proposed technique is explained and the correction required for these factors as a function of individual scan line number for each detector is described. How the revised technique can be incorporated into an operational environment is demonstrated.

Murphy, J.↗
DOE OSTI2024

EGS Collab Experiment 2: Microseismic Monitoring

This dataset contains continuous seismic waveform data recorded during stimulation and thermal circulation tests for the Enhanced Geothermal Systems (EGS) Collab Experiment #2, conducted from February to September 2022 at the Sanford Underground Research Facility in Lead, South Dakota. This experiment aimed to study and validate models of geothermal systems by injecting high-pressure fluids into rock formations 1200-1500 meters below the surface, inducing microseismic events. The seismic monitoring system included 16 three-component accelerometers and a 24-channel hydrophone array, installed in boreholes surrounding the test area. Data were recorded at high sampling rates using a continuous waveform recording system to monitor seismic activity in real time. The dataset contains the raw data stored in binary format, with files named based on timestamps, and includes calibration certificates for some sensors to facilitate corrections to real units. Users are strongly advised to consult the accompanying detailed report, which outlines the experimental setup, sensor specifications, installation procedures, and data processing methods. The report also describes important nuances, such as the hardware filters on hydrophones, sensor calibration details, and the naming conventions for the recorded data. Proper use of this dataset may require familiarity with seismic data analysis tools, such as the Obspy Python package, and an understanding of the SEED naming conventions used for channel identification.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Data transfer for STAR grid jobs

The Solenoidal Tracker at RHIC (STAR) is a multipurpose experiment at the Relativistic Heavy Ion Collider (RHIC) with the primary goal to study the formation and properties of the quark-gluon plasma. STAR is an international collaboration of member institutions and laboratories from around the world. Yearly data-taking period produces PBytes of raw data collected by the experiment. STAR primarily uses its dedicated facility at BNL to process this data, but has routinely leveraged distributed systems, both high throughput (HTC) and high performance (HPC) computing clusters, to significantly augment the processing capacity available to the experiment. The ability to automate the efficient transfer of large data sets on reliable, scalable, and secure infrastructure is critical for any large-scale distributed processing campaign. For more than a decade, STAR computing has relied upon GridFTP with its x509-based authentication to build such data transfer systems and integrate them into its larger production workflow. The end of support by the community for both GridFTP and the x509 standard requires STAR to investigate other approaches to meet its distributed processing needs. In this study we investigate two multi-purpose data distribution systems, Globus.org and XRootD, as alternatives to GridFTP. We compare both their performance and the ease by which each service is integrated into the type of secure and automated data transfer systems STAR has previously built using GridFTP. The presented approach and study may be applicable to other distributed data processing use cases beyond STAR.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Stem hydraulic conductivity and vulnerability to cavitation for 26 tree species in Panama

Stem hydraulic conductivity and vulnerability to cavitation were measured for 26 tree species located in Panama. The data were generated to better understand the ecology of the focal tree species. Complementary NGEE-Tropics datasets for these species include sap flow, leaf-level gas exchange, and leaf water potential. Stem samples were collected from distal branches of canopy trees, brought to the Smithsonian Tropical Research Institute laboratory in Gamboa, Panama, and allowed to dry to various water potentials before measurements. For each species, a Weibull function was fit to the 90% quantile of the relationship between stem area specific hydraulic conductivity (Ks) and stem water potential. From these functions, maximum Ks and vulnerability parameters were derived. The data files in the package include raw data, derived parameters, and the R script used for analysis.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗