Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Raw Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Ambient Mass Spectrometry Rapid Analysis Methods for Trace Inorganics

The ability to rapidly detect and characterize the chemistry of trace quantities of inorganic species has multiple potential applications, including nuclear safeguards, nuclear forensics, and environmental monitoring. Some mass spectrometric methods currently exist, but they are costly and time-intensive, often destroying chemical information in the process of ionization. Ambient mass spectrometry is a relatively new analytical method in which samples are introduced to the mass spectrometer from atmospheric pressure, rather than from high vacuum. Ambient MS can operate with gentler ionization methods such as electrospray ionization (ESI) and paper-spray ionization (PSI), reducing fragmentation and preserving the chemical information of the analyte. This may be useful for determining the intended purpose of material in a sample. Strontium, part of nuclear fallout, is analyzed here to test these new analytical methods for viability. Electrospray ionization mass spectrometry functions by spraying the sample as a solution through a needle charged to a few kV. A nebulizing gas flows out along with the sample to create the spray before droplets enter a drying chamber with heated drying gas to remove the solvent molecules. From there bare analyte ions enter the mass spectrometer. This method has the advantage of relatively fast sample preparation and stable ionization. More stable ionization produces higher-quality data. Paper-spray ionization mass spectrometry operates by applying sample to a paper triangle. The paper triangle is then electrified in a metal clamp to a few kV before spray solvent is applied. The spray solvent transports analyte in the sample to the tip, where it is ionized and enters the mass spectrometer. One advantage of this method is reduced sample preparation - an analyst can even swipe a surface and cut a triangle directly from the swipe. The data analysis serves to rapidly process tens of samples all at once. The mass spectrometer outputs a file containing a few hundred spectra for each sample. The pipeline processes each one to identify peaks of interest. A full palette of data analysis metrics are generated, including plots of the raw data and plots of calculated values such as isotopic ratios. The pipeline filters the raw data to remove noise and background peaks, as well as peaks representing detector saturation. A heat map is generated to show the overall spectrum in mass-time- intensity space. Post-processing, we obtained the desired calibration curves, demonstrating that both ESI-MS and PSI-MS are viable methods for quantifying trace amounts of inorganics while retaining their chemical speciation information. ESI-MS demonstrated a linear response region of 100 ppb to 20 ppm, with a limit-of-detection of around 100 ppb. At top left is an example spectrum with the {sup 88}SrNO{sub 3} peak identified to show the retention of chemical speciation. PSI-MS showed a linear response region of around 1 ppb to 1 ppm, with a limit-of-detection of near 1 ppb. PSI-MS produced more erratic results than ESI-MS, likely due to the variability in ionization resulting from manual positioning of the paper triangle near the MS inlet. Future efforts will focus on improving the repeatability of PSI-MS as well as analyzing other species of interest such as cerium and uranium. Improvements will also be made to create more reliable data extraction and intelligent data filtering.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Compression of a Data Stream by Selection among a Set of Compression Tools

A stream of raw data is compressed prior to transmissio in a communication channel by a system which includes modules for choosing a current segment of the raw data stream for processing and defining a set of operators for representing data segments by a mathematical operation and parameters thereof. The system performs a competitive evaluation of different tools comprising different combinations of one or more of the operators and the parameters threrof with respect to the current data segment in order to determine relative abilities among the different tools to reduce the number of bits required to represent the current data segment. The system then selects a tool and a set of parameters thereof found in the competitive evaluation to have a superior ability relative to others of the different tools to reduce a number of bits required to represent the current data segment.

Arlid, Bertelrud↗

SEASAT synthetic-aperture radar data user's manual

The SEASAT Synthetic-Aperture Radar (SAR) system, the data processors, the extent of the image data set, and the means by which a user obtains this data are described and the data quality is evaluated. The user is alerted to some potential problems with the existing volume of SEASAT SAR image data, and allows him to modify his use of that data accordingly. Secondly, the manual focuses on the ultimate focuses on the ultimate capabilities of the raw data set and evaluates the potential of this data for processing into accurately located, amplitude-calibrated imagery of high resolution. This allows the user to decide whether his needs require special-purpose data processing of the SAR raw data.

Pravdo, S. H.↗

Optimizing a Small RNAseq Analysis Pipeline for NASA GeneLab Using Open-Source Tools and Libraries

Small RNA sequencing (small RNAseq) is a powerful tool for studying the regulation of gene expression in various organisms. Small RNAseq has been leveraged in space biology research to study how expression of small RNAs, e.g. micro RNAs (miRNAs), small interfering RNAs (siRNAs), and piwi-interacting RNAs (piRNAs), change upon exposure to the space environment. NASA GeneLab currently hosts small RNAseq raw data derived from space-relevant experiments on the Open Science Data Repository (OSDR). To maximize the accessibility of these data to the scientific community, in addition to hosting raw data, which is only interpretable by bioinformaticians, GeneLab plans to process all small RNAseq datasets and make those processed data available to the scientific community via the OSDR. In this study, we present the development of the GeneLab standardized pipeline for processing small RNAseq datasets. Using human, plant, and synthetic small RNAseq datasets, we interrogate various open-source software and publicly available databases to evaluate their accuracy and reproducibility in each step of the pipeline. For quality control and adapter detection and trimming, we evaluated TrimGalore!, FASTX, SeqKit, and DNApi methods to optimize alignment to reference genomes. We compared BWA, Bowtie, and Bowtie2 to determine the optimal alignment tool. For each alignment tool we also assessed various reference databases, including Ensembl reference genomes and different types of small RNA reference databases, including genome, hairpin, and miRNA references from the miRbase and MirGeneDB databases. To quantify the aligned data, we compared SAMtools, HTSeq, and RSEM for counting alignment events from each alignment tool used. Finally, we evaluated various tools, including DESeq2 and EdgeR, for data normalization and subsequent differential expression analysis. We will present the results from our comparative analyses for each pipeline step and propose a consensus pipeline for processing small RNAseq data derived from various organisms exposed to the space environment.

SmallRNAseq, NASA GeneLab, quality control, adapte↗

Laser Disdrometer / Standardized Data

This dataset contains raw data from the laser disdrometer at Nantucket. The disdrometer measures droplet size distribution, hydrometer type, and precipitation rate.

17 WIND ENERGY↗

RHOD Site - Surface Meteorological Station / Processed Data

This dataset contains raw data from the WFIP3 RHOD site supplementary meteorological sensors: T/RH and barometer; 1-sec average. The supplementary meteorological suite was added to PNNL Surface Flux Station to provide independent measurements of air temperature, relative humidity, and atmospheric pressure.

17 WIND ENERGY↗

Intelligent Systems Technologies and Utilization of Earth Observation Data

The addition of raw data and derived geophysical parameters from several Earth observing satellites over the last decade to the data held by NASA data centers has created a data rich environment for the Earth science research and applications communities. The data products are being distributed to a large and diverse community of users. Due to advances in computational hardware, networks and communications, information management and software technologies, significant progress has been made in the last decade in archiving and providing data to users. However, to realize the full potential of the growing data archives, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications. Sponsored by NASA s Intelligent Systems Project within the Computing, Information and Communication Technology (CICT) Program, a conceptual architecture study has been conducted to examine ideas to improve data utilization through the addition of intelligence into the archives in the context of an overall knowledge building system (KBS). Potential Intelligent Archive concepts include: 1) Mining archived data holdings to improve metadata to facilitate data access and usability; 2) Building intelligence about transformations on data, information, knowledge, and accompanying services; 3) Recognizing the value of results, indexing and formatting them for easy access; 4) Interacting as a cooperative node in a web of distributed systems to perform knowledge building; and 5) Being aware of other nodes in the KBS, participating in open systems interfaces and protocols for virtualization, and achieving collaborative interoperability.

Ramapriyan, H. K.↗

Methodology to Define Delivery Accuracy Under Current Day ATC Operations

In order to enable arrival management concepts and solutions in a NextGen environment, ground- based sequencing and scheduling functions have been developed to support metering operations in the National Airspace System. These sequencing and scheduling algorithms as well as tools are designed to aid air traffic controllers in developing an overall arrival strategy. The ground systems being developed will support the management of aircraft to their Scheduled Times of Arrival (STAs) at flow-constrained meter points. This paper presents a methodology for determining the undelayed delivery accuracy for current day air traffic control operations. This new method analyzes the undelayed delivery accuracy at meter points in order to understand changes of desired flow rates as well as enabling definition of metrics that will allow near-future ground automation tools to successfully achieve desired separation at the meter points. This enables aircraft to meet their STAs while performing high precision arrivals. The research presents a possible implementation that would allow delivery performance of current tools to be estimated and delivery accuracy requirements for future tools to be defined, which allows analysis of Estimated Time of Arrival (ETA) accuracy for Time-Based Flow Management (TBFM) and the FAA's Traffic Management Advisor (TMA). TMA is a deployed system that generates scheduled time-of-arrival constraints for en- route air traffic controllers in the US. This new method of automated analysis provides a repeatable evaluation of the delay metrics for current day traffic, new releases of TMA, implementation of different tools, and across different airspace environments. This method utilizes a wide set of data from the Operational TMA-TBFM Repository (OTTR) system, which processes raw data collected by the FAA from operational TMA systems at all ARTCCs in the nation. The OTTR system generates daily reports concerning ATC status, intent and actions. Due to its availability, ease of use, and vast collection of data across several airspaces it was determined that the OTTR data set would be the best method to utilize moving forward with this analysis. The particular variables needed for further analysis were determined along with the necessary OTTR reports, by working closely with the repository team additional analysis reports were developed that provided key ETA and STA information at the freeze horizon. One major benefit of the OTTR data is that using the correct reports the data across several airports could be analyzed over large periods of time. The OTTR data processes the TBFM data daily and is stored in various formats across several airspaces. This allowed us to develop our own parsing methods and raw data processing that would not rely on other computationally expensive tools that perform more in depth analysis of similar sets of data. The majority of this work consisted of the development of the ability to filter flights to create a subset of flights that could be considered undelayed, which is defined as a flight at the freeze horizon with an ETA and STA difference that was minimal or close to zero. This was a broad method that allowed the consideration of a large data set which consisted of all the traffic across a two month period in 2013, the hottest and coldest months, arriving into four airports: George Bush Intercontinental, Denver International, Los Angeles International, and Phoenix Sky Harbor.

delivery accuracy↗

Machine Learning-Based Extreme Data Reduction for Prompt Supernova Pointing at DUNE

One of the goals of the Deep Underground Neutrino Experiment (DUNE) is to use the massive underground liquid argon time projection chamber (LArTPC) detectors at its far site for multimessenger astronomy (MMA), in the detection of neutrinos from core-collapse supernovae (SNe). Its current baseline trigger strategy detects activity in the detector that is consistent with supernova (SN) neutrinos and saves the raw data for further offline analysis but provides no prompt pointing information crucial for optical follow-ups by other observatories. This approach is based on the assumption that prompt pointing determination using raw data is computationally prohibitive. In this article, we demonstrate a proof-of-concept based on applying extreme data reduction on the buffered SN data in the DUNE data acquisition (DAQ) system’s front-end computers using a machine learning (ML) workflow. This reduces the data by ~5 orders of magnitude, allowing a full track reconstruction to be carried out quickly on a single server. The total time to perform the ML-based data reduction and the full track reconstruction is less than the time to transfer the SN data back to Fermilab or a high-performance computing (HPC) center. This shows that prompt processing of raw SN data is possible and, in fact, trivial once the data have been reduced to reject radiological backgrounds, paving the way to a high-quality SN pointing trigger that is based on fully reconstructed data instead of trigger primitives (TPs).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Real-Time XFEL Data Analysis at SLAC and NERSC: a Trial Run of Nascent Exascale Experimental Data Analysis

X-ray scattering experiments using Free Electron Lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID-19 viral proteins). XFEL experiments are a challenge to computing in two ways: i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; ii) data collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min -- sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS-NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Furthermore, completing data analysis with 10 mins is a first for XFEL experiments and an important milestone if we are to keep up with data collection trends.

Blaschke, Johannes P.↗

Data for Photosynthetic responses to temperature across the tropics: a meta-analytic approach

This data package contains raw data, data output, and code of photosynthetic temperature response curves from tropical forests. Data are used in "Photosynthetic responses to temperature across the tropics: a meta-analytic approach". The research investigates how photosynthetic optimum temperatures and shapes of photosynthetic temperature response curves varies across tropical forest climates and growth conditions. Growth climate variables include mean annual temperature, max and max temperature, diurnal temperature range, and aridity index. Meta-analysis combines 18 datasets with representation from Africa, Oceana, North American, and South America. Growth conditions analyzed considers deciduousness, successional status, light conditions, and whether plants are grown in situ or ex situ. All files, except for the raw datafile (Tropical_MetaAnalysis_Master_3.8.22_Edit.csv), have been processed using R code, which is provided.

54 ENVIRONMENTAL SCIENCES↗

Towards an Introspective Dynamic Model of Globally Distributed Computing Infrastructures

Large-scale scientific collaborations like ATLAS, Belle II, CMS, DUNE, and others involve hundreds of research institutes and thousands of researchers spread across the globe. These experiments generate petabytes of data, with volumes soon expected to reach exabytes. Consequently, there is a growing need for computation, including structured data processing from raw data to consumer-ready derived data, extensive Monte Carlo simulation campaigns, and a wide range of end-user analysis. To manage these computational and storage demands, centralized workflow and data management systems are implemented. However, decisions regarding data placement and payload allocation are often made disjointly and via heuristic means. A significant obstacle in adopting more effective heuristic or AI-driven solutions is the absence of a quick and reliable introspective dynamic model to evaluate and refine alternative approaches. In this study, we aim to develop such an interactive system using real-world data. By examining job execution records from the PanDA workflow management system, we have pinpointed key performance indicators such as queuing time, error rate, and the extent of remote data access. The dataset includes five months of activity. Additionally, we are creating a generative AI model to simulate time series of payloads, which incorporate visible features like category, event count, and submitting group, as well as hidden features like the total computational load—derived from existing PanDA records and computing site capabilities. These hidden features, which are not visible to job allocators, whether heuristic or AI-driven, influence factors such as queuing times and data movement.

kilic, Ozgur Ozan [Brookhaven National Laboratory ↗

Data for "Relative Reactivity and Bioavailability of Mercury Sorbed to or Coprecipitated with Aged Iron Sulfides"

The potential for inorganic mercury (Hg) to be converted to methylmercury depends, in part, on the chemical form of Hg and its bioavailability to anaerobic microorganisms that can methylate Hg. In anaerobic settings, Hg can be associated with sulfide phases, including ferrous iron sulfide (FeS), which can sorb or coprecipitated with Hg. The objective of this study was to determine if the aging state of FeS alters the Hg coordination environment as well as the reactivity and bioavailability of sorbed and coprecipitated Hg species. FeS particles were synthesized with and without Hg2+ and aged in anaerobic conditions for multiple time frames spanning from 1 hour to 1 month. For FeS particles synthesized without Hg, Hg2+ was subsequently sorbed to the FeS for 1 day. Analysis of Hg speciation of these materials by X-ray absorption near edge spectroscopy revealed a predominance of 4-coodinate Hg-S species in the sorbed Hg-FeS solids and a mixture of 2- and 4-coordinate Hg-S in the coprecipitated Hg-FeS. The leaching potential of the Hg was assessed by exposing the particles to a solution of dissolved glutathione (a thiolate-based Hg chelator). As expected, the sorbed Hg-FeS released more soluble Hg compared to the co-precipitated Hg-FeS. However, when these particles were exposed to Desulfovibrio desulfuricans ND132 (a known Hg methylator), more Hg was methylated from the co-precipitated Hg-FeS than the sorbed Hg-FeS, consistent with expectations from the Hg-S coordination state and inconsistent with the selective leaching results. Overall, these results suggest that the bioavailability of particulate Hg cannot be easily discerned by leaching potential into bulk solution. Rather, bioavailability entails more subtle interactions at particle-cell interfaces and perhaps correlates with the local Hg-S coordination state in the particles. This data package contains data shown in the referenced publication, including all figures. Each figure is available in the respective *.csv file and a summary of all files are also included in the the xlsx file. Additional formatted data include X-ray diffraction data of the FeS solids are also available in a HPF format (Panalytical software), X-ray absorption spectroscopy data as *.prj files (Athena), and X-ray fluorescence data in its raw data form.

54 ENVIRONMENTAL SCIENCES↗

Design concepts for an on-board coherent optical image processor

On-board spacecraft image data processing systems for transmitting processed data rather than raw data are discussed. A brief history of the development of the optical data processing techniques is presented along with the conceptual design of a coherent optical system with a noncoherent image input.

Husain-Abidi, A. S.↗

SEC sensor parametric test and evaluation system

This system provides the necessary automated hardware required to carry out, in conjunction with the existing 70 mm SEC television camera, the sensor evaluation tests which are described in detail. The Parametric Test Set (PTS) was completed and is used in a semiautomatic data acquisition and control mode to test the development of the 70 mm SEC sensor, WX 32193. Data analysis of raw data is performed on the Princeton IBM 360-91 computer.

Source record↗

Evaluation of RDBMS packages for use in astronomy

Tabular data sets arise in many areas of astronomical data analysis, from raw data (such as photon event lists) to final results (such as source catalogs). The Starlink catalog access and reporting package, SCAR, was originally developed to handle IRAS data and it has been the principal relational DBMS in the Starlink software collection for several years. But SCAR has many limitations and is VMS-specific, while Starlink is in transition from VMS to Unix. Rather than attempt a major re-write of SCAR for Unix, it seemed more sensible to see whether any existing database packages are suitable for general astronomical use. The authors first drew up a list of desirable properties for such a system and then used these criteria to evaluate a number of packages, both free ones and those commercially available. It is already clear that most commercial DBMS packages are not very well suited to the requirements; for example, most cannot carry out efficiently even fairly basic operations such as joining two catalogs on an approximate match of celestial positions. This paper reports the results of the evaluation exercise and notes the problems in using a standard DBMS package to process scientific data. In parallel with this the authors have started to develop a simple database engine that can handle tabular data in a range of common formats including simple direct-access files (such as SCAR and Exosat DBMS tables) and FITS tables (both ASCII and binary).

Page, C. G.↗

Harnessing the Risk-Related Data Supply Chain: An Information Architecture Approach to Enriching Human System Research and Operations Knowledge

NASA's Human Research Program (HRP) and Space Life Sciences Directorate (SLSD), not unlike many NASA organizations today, struggle with the inherent inefficiencies caused by dependencies on heterogeneous data systems and silos of data and information spread across decentralized discipline domains. The capture of operational and research-based data/information (both in-flight and ground-based) in disparate IT systems impedes the extent to which that data/information can be efficiently and securely shared, analyzed, and enriched into knowledge that directly and more rapidly supports HRP's research-focused human system risk mitigation efforts and SLSD s operationally oriented risk management efforts. As a result, an integrated effort is underway to more fully understand and document how specific sets of risk-related data/information are generated and used and in what IT systems that data/information currently resides. By mapping the risk-related data flow from raw data to useable information and knowledge (think of it as the data supply chain), HRP and SLSD are building an information architecture plan to leverage their existing, shared IT infrastructure. In addition, it is important to create a centralized structured tool to represent risks including attributes such as likelihood, consequence, contributing factors, and the evidence supporting the information in all these fields. Representing the risks in this way enables reasoning about the risks, e.g. revisiting a risk assessment when a mitigation strategy is unavailable, updating a risk assessment when new information becomes available, etc. Such a system also provides a concise way to communicate the risks both within the organization as well as with collaborators. Understanding and, hence, harnessing the human system risk-related data supply chain enhances both organizations' abilities to securely collect, integrate, and share data assets that improve human system research and operations.

Buquo, Lynn↗