Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Work flow of signal processing data of ground penetrating radar case of rigid pavement measurements

The signal processing of Ground Penetrating Radar (GPR) requires a certain work flow to obtain good results. Even though the Ground Penetrating Radar data looks similar with seismic reflection data, but the GPR data has particular signatures that the seismic reflection data does not have. This is something to do with coupling between antennae and the ground surface. Because of this, the GPR data should be treated differently from the seismic signal data processing work flow. Even though most of the processing steps still follow the same work flow of seismic reflection data such as: filtering, predictive deconvolution etc. This paper presents the work flow of GPR processing data on rigid pavement measurements. The processing steps start from raw data, de-Wow process, remove DC and continue with the standard process to get rid of noises i.e. filtering process. Some radargram particular features of rigid pavement along with pile foundations are presented.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

LoVoCCS. I. Survey Introduction, Data Processing Pipeline, and Early Science Results

We present the Local Volume Complete Cluster Survey (LoVoCCS; we pronounce it as "low-vox" or "law-vox," with stress on the second syllable), an NSF's National Optical-Infrared Astronomy Research Laboratory survey program that uses the Dark Energy Camera to map the dark matter distribution and galaxy population in 107 nearby (0.03 < z < 0.12) X-ray luminous ([0.1–2.4 keV] L X500 > 10 44 erg s –1 ) galaxy clusters that are not obscured by the Milky Way. The survey will reach Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) Year 1–2 depth (for galaxies r = 24.5, i = 24.0, signal-to-noise ratio (S/N) > 20; u = 24.7, g = 25.3, z = 23.8, S/N > 10) and conclude in ~2023 (coincident with the beginning of LSST science operations), and will serve as a zeroth-year template for LSST transient studies. We process the data using the LSST Science Pipelines that include state-of-the-art algorithms and analyze the results using our own pipelines, and therefore the catalogs and analysis tools will be compatible with the LSST. We demonstrate the use and performance of our pipeline using three X-ray luminous and observation-time complete LoVoCCS clusters: A3911, A3921, and A85. A3911 and A3921 have not been well studied previously by weak lensing, and we obtain similar lensing analysis results for A85 to previous studies.

79 ASTRONOMY AND ASTROPHYSICS↗

NJOY21: A Successor to the NJOY Nuclear Data Processing System

NJOY21 is an initiative at Los Alamos National Laboratory (LANL) to develop software to succeed the NJOY nuclear data processing system. Specifically, the aim of the project is to provide: - a stand-alone executable for nuclear data processing in the vein of the original NJOY and, - a hierarchy of reusable loosely-coupled, object-oriented component libraries. The name NJOY21 was chosen to denote a commitment to provide software suitable for the 21. Century; integrating modern development practices and tools from the inception of the project. This summary will expand upon the current goals and design of the project. Before going onto discuss NJOY21, it is worthwhile to briefly establish the historical context and motivating factors. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Planck 2015 results: II. Low Frequency Instrument data processings

In this paper, we present an updated description of the Planck Low Frequency Instrument (LFI) data processing pipeline, associated with the 2015 data release. We point out the places where our results and methods have remained unchanged since the 2013 paper and we highlight the changes made for the 2015 release, describing the products (especially timelines) and the ways in which they were obtained. We demonstrate that the pipeline is self-consistent (principally based on simulations) and report all null tests. For the first time, we present LFI maps in Stokes Q and U polarization. Finally, we refer to other related papers where more detailed descriptions of the LFI data processing pipeline may be found if needed.

79 ASTRONOMY AND ASTROPHYSICS↗

The Spectroscopic Data Processing Pipeline for the Dark Energy Spectroscopic Instrument

Abstract We describe the spectroscopic data processing pipeline of the Dark Energy Spectroscopic Instrument (DESI), which is conducting a redshift survey of about 40 million galaxies and quasars using a purpose-built instrument on the 4 m Mayall Telescope at Kitt Peak National Observatory. The main goal of DESI is to measure with unprecedented precision the expansion history of the universe with the baryon acoustic oscillation technique and the growth rate of structure with redshift space distortions. Ten spectrographs with three cameras each disperse the light from 5000 fibers onto 30 CCDs, covering the near-UV to near-infrared (3600–9800 Å) with a spectral resolution ranging from 2000 to 5000. The DESI data pipeline generates wavelength- and flux-calibrated spectra of all the targets, along with spectroscopic classifications and redshift measurements. Fully processed data from each night are typically available to the DESI collaboration the following morning. We give details about the pipeline’s algorithms, and provide performance results on the stability of the optics, the quality of the sky background subtraction, and the precision and accuracy of the instrumental calibration. This pipeline has been used to process the DESI Survey Validation data set, and has exceeded the project’s requirements for redshift performance, with high efficiency and a purity greater than 99% for all target classes.

79 ASTRONOMY AND ASTROPHYSICS↗

A Parallelised ROOT for Future HEP Data Processing

In the coming years, HEP data processing will need to exploit parallelism on present and future hardware resources to sustain the bandwidth requirements. As one of the cornerstones of the HEP software ecosystem, ROOT embraced an ambitious parallelisation plan which delivered compelling results. In this contribution the strategy is characterised as well as its evolution in the medium term. The units of the ROOT framework are discussed where task and data parallelism have been introduced, with runtime and scaling measurements. We will give an overview of concurrent operations in ROOT, for instance in the areas of I/O (reading and writing of data), fitting / minimization, and data analysis. This paper introduces the programming model and use cases for explicit and implicit parallelism, where the former is explicit in user code and the latter is implicitly managed by ROOT internally.

97 MATHEMATICS AND COMPUTING↗

User input verification and test driven development in the NJOY21 nuclear data processing code

Before physically-meaningful data can be used in nuclear simulation codes, the data must be interpreted and manipulated by a nuclear data processing code so as to extract the relevant quantities (e.g. cross sections and angular distributions). Perhaps the most popular and widely-trusted of these processing codes is NJOY, which has been developed and improved over the course of 10 major releases since its creation at Los Alamos National Laboratory in the mid-1970’s. The current phase of NJOY development is the creation of NJOY21, which will be a vast improvement from its predecessor, NJOY2016. Designed to be fast, intuitive, accessible, and capable of handling both established and modern formats of nuclear data, NJOY21 will address many issues that many NJOY users face, while remaining functional for those who prefer the existing format. Although early in its development, NJOY21 is quickly providing input validation to check user input. By providing rapid and helpful responses to users while writing input files, NJOY21 will prove to be more intuitive and easy to use than any of its predecessors. Furthermore, during its development, NJOY21 is subject to regular testing, such that its test coverage must strictly increase with the addition of any production code. This thorough testing will allow developers and NJOY users to establish confidence in NJOY21 as it gains functionality. This document serves as a discussion regarding the current state input checking and testing practices of NJOY21.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Scientific Data Processing Framework for Time Series NetCDF Data

ARM Data Integrator (ADI) is a framework to streamline the development of scientific algorithms that analyze time-series NetCDF data, and to improve the content and consistency of the output data products produced by these algorithms. ADI achieves these goals by automating the process of retrieving and preparing data for analysis, supporting the definition of output data products through a graphical interface, and providing a modular, flexible software development architecture. The input data, preprocessing, and output data specifications are defined through a graphical interface and stored in a database. ADI also includes a workflow for data integration, a library of software modules to support the workflow, and a source code generator that produces C, IDL and Python templates. Data preparation support includes automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog. The variable and attribute definitions of the existing data products can be used to build new output data products. In addition, the rules that make up the ARM archive’s data standards are laid on top of the view of the new data product providing the user with a visual cue indicating where their output violates an archive standard. The necessary configurations are stored in a database that is accessed by the ADI libraries. This paper discusses the ADI framework, its supporting components, and how ADI can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Atmospheric Science↗

Processes, data structures, and apparatuses for representing knowledge

Processes, data structures, and apparatuses to represent knowledge are disclosed. The processes can comprise labeling elements in a knowledge signature according to concepts in an ontology and populating the elements with confidence values. The data structures can comprise knowledge signatures stored on computer-readable media. The knowledge signatures comprise a matrix structure having elements labeled according to concepts in an ontology, wherein the value of the element represents a confidence that the concept is present in an information space. The apparatus can comprise a knowledge representation unit having at least one ontology stored on a computer-readable medium, at least one data-receiving device, and a processor configured to generate knowledge signatures by comparing datasets obtained by the data-receiving devices to the ontologies.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

RE-INTEGRATE EMT Simulation Tool: Input Data Processing Layer for Bulk Power System

This paper introduces an advanced input data processing layer for EMT simulations of large-scale bulk power systems. The paper proposes two versions of the RE-INTEGRATE EMT simulation tool, RE-INTEGRATE Gen-0 and RE-INTEGRATE Gen-1, which are developed to enhance simulation generalizability, scalability, and accuracy. The framework leverages a generic class design for components to incorporate linear equations, which are generated by discretizing the Differential-Algebraic Equations (DAEs) that represent the dynamics of the components. In addition, the framework employs a parsing algorithm that parses a power system’s raw and dyr files to generate a connectivity graph which is then traversed to form the overall system’s dynamics. The proposed input data processing layer is used to simulate the IEEE 39-bus test system. The obtained results demonstrate the framework’s capability to achieve simulation scalability and accuracy. Further, the results indicate that EMT simulations performed using the proposed automations can effectively handle complex grid configurations.

Mishra, Rahul [ORNL] (ORCID:0000000328205932)↗

NANT Site - ASSIST / Processed Data

This dataset contains processed summary data from the ASSIST operated by NOAA Physical Science Laboratory on Nantucket Island for WFIP3.

17 WIND ENERGY↗

Lidar / Processed Data

This dataset contains processed, standardized data from the UND scanning Doppler lidar at WFIP3's BARG site, consisting of range- and time-resolved measurements of radial velocity, attenuated backscatter, intensity, and spectral width. We note that the beam azimuth angles are NOT referenced to true north.

17 WIND ENERGY↗

CACO Site - ANL Scanning Doppler Lidar / Processed Data

This dataset contains processed, standardized data from the ANL scanning Doppler lidar, consisting of range- and time-resolved measurements of radial velocity, attenuated backscatter, intensity, and spectral width. We note that the beam azimuth angles are NOT referenced to true north.

17 WIND ENERGY↗

Lidar / Processed Data

This dataset contains processed, standardized data from the UND scanning Doppler lidar at WFIP3's BARG site, consisting of range- and time-resolved measurements of radial velocity, attenuated backscatter, intensity, and spectral width. We note that the beam azimuth angles are NOT referenced to true north.

17 WIND ENERGY↗

Radar / Processed Data

This dataset contains processed raw data from the UND 94 GHz W-band Cloud Doppler Radar.

17 WIND ENERGY↗

Big Data Processing for Power Grid Event Detection

In this paper we present the application of big data processing for the development of machine learning(ML) models to detect relevant events in power gridoperations. This is based on almost 20TB of phasormeasurement unit data corresponding to up to two years of operation of three grid interconnections which provide power to most of the United States. A significant aspect of the work consists in having all data processing performed on a single standard GPU server, from pre-processing to ML model training and testing. We describe the data and computational infrastructure, challenges faced and methods used in dataprocessing, main findings and results. The ML approach employed for best utilization of the big data is also discussed, including sample results.

Paes Leao, Bruno↗

DEIMoS GUI: An Open-Source User Interface for a High-Dimensional Mass Spectrometry Data Processing Tool

In this paper, we report the creation of a graphical user interface (GUI) for the Data Extraction for Integrated Multidimensional Spectrometry (DEIMoS) tool. DEIMoS is a Python package to process data from high-dimensional mass spectrometry measurements. It is divided into several modules, each representing a data processing step, such as peak detection, alignment, and tandem mass spectra extraction and deconvolution. The inputs for and outputs from DEIMoS can include millions of N-dimensional data points, which can be challenging to visualize in a way that is interactive, informative, and responsive. Here, we used the HoloViz Python data stack, including DataShader and Param, to create an interactive visualization of mass spectrometry data. We believe the GUI will increase the accessibility of DEIMoS, and the visualization methods could be useful for other open-source mass spectrometry tools.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗