Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

AWAKEN Site A1 - NREL Scanning Lidar (Halo XR+ #235) / Processed Data

These data include raw scanning Doppler lidar measurements from the deployment of the NREL HALO XR+ (s/n 235) at the A1 site. The processed measurements include uncalibrated beam azimuth angles, radial velocity, backscatter, signal to noise ratio per each line of sight, and range-gate.

17 WIND ENERGY↗

Arrangements for communicating and processing data in a computing system

Systems and methods for reducing data movement in a computer system. The systems and methods use information or knowledge about the structure of an algorithm, operations to be executed at a receiving processing unit, variables or subsets or groups of variables in a distributed algorithm, or other forms of contextual information, for reducing the number of bits transmitted from at least one transmitting processing unit to at least one receiving processing unit or storage device.

Gonzalez, Juan Guillermo↗

GUC CSU X-Band Precipitation Radar Plan Position Indicator data processed with Corrected Moments in Antenna Coordinates

For the Surface Atmosphere Integrated Field Laboratory (SAIL) campaign, Corrected Moments in Antenna Coordinates (CMAC) is a set of algorithms and code that does corrections to CSU X-Precipitation Radar PPI data. Additionally, CMAC adds calculations of derived snowfall rates to the original data in order to provide precipitation estimates for the campaign.

54 ENVIRONMENTAL SCIENCES↗

Initial operation and data processing on a system for real-time evaluation of Thomson scattering signals on the Large Helical Device

A scalable system for real-time analysis of electron temperature and density based on signals from the Thomson scattering diagnostic, initially developed for and installed on the NSTX-U experiment, was recently adapted for the Large Helical Device and operated for the first time during plasma discharges. During its initial operation run, it routinely recorded and processed signals for four spatial points at the laser repetition rate of 30 Hz, well within the system’s rated capability for 60 Hz. We present examples of data collected from this initial run and describe subsequent adaptations to the analysis code to improve the fidelity of the temperature calculations.

47 OTHER INSTRUMENTATION↗

Initial operation and data processing on a system for real-time evaluation of Thomson scattering signals on the Large Helical Device

A scalable system for real-time analysis of electron temperature and density based on signals from the Thomson scattering diagnostic, initially developed for and installed on the NSTX-U experiment, was recently adapted for the Large Helical Device (LHD) and operated for the first time during plasma discharges. During its initial operation run, it routinely recorded and processed signals for four spatial points at the laser repetition rate of 30 Hz, well within the system's rated capability for 60 Hz. We present examples of data collected from this initial run and describe subsequent adaptations to the analysis code to improve the fidelity of the temperature calculations.

Thomson scattering, real-time, LHD, plasma diagnos↗

The real-time data processing and acquisition system for Project 8 Phase II

In Phase II of the Project 8 neutrino mass experiment, electrons from the decays of tritium or 83m Kr are detected via their ≈26 GHz cyclotron radiation while contained within a circular waveguide. The signal from a given electron is characterized as a brief chirp, lasting ≲10 ms and changing in frequency by ≲1 MHz/ms. To detect these signals, the Project 8 collaboration developed a Data Acquisition (DAQ) system tailored to the signal properties. The DAQ is responsible for simultaneously selecting up to three 100 MHz-wide frequency windows to study, detect, and trigger on likely signals from different electron kinetic energies, and for writing the relevant data to disk. Finally, we describe the Phase II DAQ system in detail and address how the system was used for data-taking operations.

Data acquisition↗

EXD HME MicroCT Data Acquisition, Processing and Data Request Overview

This document is a short summary of the steps required for MicroCT evaluation of a specimen. This includes data acquisition through image analysis, for the EXD HME program. Expected outputs for each stage are provided. Data shall be shipped to LLNL as described herein.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Rock Valley Accelerated Weight Drop Seismic Data Processing and Picking of P-wave and S-wave Arrival Times

Rock Valley, in the southern end of the Nevada National Security Site, hosts a fault system that was responsible for a shallow (< 3 km below surface ) magnitude 3.7 earthquake in May 1993. In order to better understand this system, seismic properties of the shallow subsurface need to be better constrained. In April and May of 2021, accelerated weight drop (AWD) active-source seismic data were recorded in order to measure P- and S-wave travel-times for the area. This report describes the processing and phase picking of the recorded seismic waveforms. In total, we picked 7,982 P-wave arrivals at offsets up to ~2500 m, and 4,369 S-wave arrivals at offsets up to ~2200 m. These travel-time picks can be inverted for shallow P-wave and S-wave velocity structure in future studies.

58 GEOSCIENCES↗

EdgeAI: Machine learning via direct attached accelerator for streaming data processing at high shot rate x-ray free-electron lasers

We present a case for low batch-size inference with the potential for adaptive training of a lean encoder model. We do so in the context of a paradigmatic example of machine learning as applied in data acquisition at high data velocity scientific user facilities such as the Linac Coherent Light Source-II x-ray Free-Electron Laser. We discuss how a low-latency inference model operating at the data acquisition edge can capitalize on the naturally stochastic nature of such sources. We simulate the method of attosecond angular streaking to produce representative results whereby simulated input data reproduce high-resolution ground truth probability distributions. By minimizing the mean-squared error between the decoded output of the latent representation and the ground truth distributions, we ensure that the encoding layers and resulting latent representation maintains full fidelity for any downstream task, be it classification or regression. We present throughput results for data-parallel inference of various batch sizes, some with throughput exceeding 100 k images per second. We also show in situ training below 10 s per epoch for the full encoder–decoder model as would be relevant for streaming and adaptive real-time data production at our nation’s scientific light sources.

97 MATHEMATICS AND COMPUTING↗

Planck 2015 results: VIII. High Frequency Instrument data processing: Calibration and maps

This paper describes the processing applied to the cleaned, time-ordered information obtained from the Planck High Frequency Instrument (HFI) with the aim of producing photometrically calibrated maps in temperature and (for the first time) in polarization. The data from the entire 2.5-year HFI mission include almost five full-sky surveys. HFI observes the sky over a broad range of frequencies, from 100 to 857 GHz. To obtain the best accuracy on the calibration over such a large range, two different photometric calibration schemes have been used. The 545 and 857 GHz data are calibrated using models of planetary atmospheric emission. The lower frequencies (from 100 to 353 GHz) are calibrated using the time-variable cosmological microwave background dipole, which we call the orbital dipole. This source of calibration only depends on the satellite velocity with respect to the solar system. Using a CMB temperature of T CMB = 2.7255 ± 0.0006 K, it permits an independent measurement of the amplitude of the CMB solar dipole (3364.3 ± 1.5 μK), which is approximatively 1σ higher than the WMAP measurement with a direction that is consistent between the two experiments. We describe the pipeline used to produce the maps of intensity and linear polarization from the HFI timelines, and the scheme used to set the zero level of the maps a posteriori. We also summarize the noise characteristics of the HFI maps in the 2015 Planck data release and present some null tests to assess their quality. Lastly, we discuss the major systematic effects and in particular the leakage induced by flux mismatch between the detectors that leads to spurious polarization signal.

79 ASTRONOMY AND ASTROPHYSICS↗

JANA2 Framework for Event Based and Triggerless Data Processing

Development of the second generation JANA2 multi-threaded event processing framework is ongoing through an LDRD initiative grant at Jefferson Lab. The framework is designed to take full advantage of all cores on modern many-core compute nodes. JANA2 efficiently handles both traditional hardware triggered event data and streaming data in online triggerless environments. Development is being done in conjunction with the Electron Ion Collider development. Anticipated to be the next large scale Nuclear Physics facility constructed. The core framework is written in modern C++ but includes an integrated Python interface. The status of development and summary of the more interesting features are presented.

Lawrence, David↗

Simulating the Impacts of Disturbances on Forest Carbon Cycling in North America: Processes, Data, Models, and Challenges

Disturbances disrupt the forest structures and alter forest resources, substrate availability, or the physical environment. Understanding disturbances and their impacts is critical for a better quantification of North American carbon dynamics. We reviewed the status and major challenges in simulating the impacts of disturbances on forest carbon cycling in North America from the perspectives of process understanding, disturbance characterization, and modeling, and found some major issues. First, we do not have adequate continentally-consistent disturbance databases to support the NACP missions. Community effort, in collaborating with the modeling community, should be designated to systematically define international standards for disturbance databases with explicit inclusion of uncertainty measurements, adequate spatial and temporal resolutions, and explicit and quantifiable linkages with model simulations. Second, there is a need to develop effective and comprehensive process-based procedures or algorithms that can be used to quantify the immediate and long-term impacts of disturbances on forest succession, soils, microclimate, and cycles of carbon, water, and nutrients. These algorithms should be evaluated not only on the carbon cycle but also on the simulations of vegetation succession, alterations of soil and microclimate, and water and nutrients dynamics because adequate simulations of carbon dynamics rely on the correct quantification of the changes of soil and vegetation conditions. Third, the scaling challenges have rarely been addressed at the continental scale. We do not understand which processes and properties at a given temporal or spatial scales are critical while others can be simplified when we try to balance the North American carbon budget. Previous and current model simulations at the continental scale did not incorporate a full suite of disturbances information (because they do not exist as described above), and therefore provided little insight on the relative importance of various disturbances at the continent and regional scales on the carbon cycle. Fourth, our capability of simulating the occurrences and severity of disturbances under climate change and management futures are very limited. Fifth, uncertainty exists in every step of the effort to simulating the impacts of disturbances on carbon dynamics. Procedures are needed to quantify the uncertainty of model inputs or data layers, model parameters, and model structures, and their impacts on model simulations. Working together, modelers and data layer developers can identify the most uncertain areas and develop working hypotheses to reduce the uncertainty.

54 ENVIRONMENTAL SCIENCES↗

Continuing Development of the Nuclear Data Processing Code AMPX [Poster]

The ENDF/B-VIII.1 evaluation library has seen a great growth in the thermal neutron scattering sub-library. The SCALE code system has traditionally approached CE transport by assuming that the CE library on disk represented the fully expanded cumulative probability distributions, conditional on exiting angle and marginal on exiting energy. While this is a complete description of the data, it comes at the potential cost of large amounts of on-disk storage. This approach was strained by several TSL files in ENDF/B-VIII.1, such as graphite, which contained data for a large number of Bragg edges. In the fully expanded probability distributions, this was found to be a disproportionately large fraction of the SCALE CE library.

GNDS↗

Data Federation Challenges in Remote Near-Real-Time Fusion Experiment Data Processing

Fusion energy experiments and simulations provide critical information needed to plan future fusion reactors. As next-generation devices like ITER move toward long-pulse experiments, analyses, including AI and ML, should be performed in a wide range of time and computing constraints, from near-real-time constraints, between-shot analysis, and to campaign-wide long-term analysis. However, the data volume, velocity, and variety make it extremely challenging for analyses using only local computational resources. Researchers need the ability to compose and execute workflows spanning edge resources to large-scale high-performance computing facilities.We present Delta, a system to address data analysis challenges, including AI/ML, in fusion science, by leveraging the ADIOS I/O library and middleware, to support executing science workflows over the wide area network for near-real-time streaming. We discuss the data federation challenges in performing remote workflows, focusing on on-going research work in (1) managing, reducing, and streaming data to minimize I/O and data movement overheads, (2) decompressing and reorganizing data for analysis, and (3) executing workflows for automated data analysis. We introduce examples for deep-learning based data analysis for the fusion domain and demonstrate how we use Delta to construct end-to-end workflows for a fusion device in Korea, connecting a remote DOE facility in the USA. The capability demonstrated by this project is the basis for improving the state of the art for near-real-time data federation amongst remote facilities.

Choi, Jong Youl↗

The real-time data processing and acquisition system for Project 8 Phase II

In Phase II of the Project 8 neutrino mass experiment, electrons from the decays of tritium or 83 m Kr are detected via their ≈ 26GHz cyclotron radiation while contained within a circular waveguide. The signal from a given electron is characterized as a brief chirp, lasting ≲ 10ms and changing in frequency by ≲ 1MHz/ms. To detect these signals, the Project 8 collaboration developed a Data Acquisition (DAQ) system tailored to the signal properties. The DAQ is responsible for simultaneously selecting up to three 100MHz-wide frequency windows to study, detect, and trigger on likely signals from different electron kinetic energies, and for writing the relevant data to disk. We describe the Phase II DAQ system in detail and address how the system was used for data-taking operations.

Ashtari Esfahani, A. [Washington U., Seattle; Shar↗

Planck 2015 results: VII. High Frequency Instrument data processing: Time-ordered information and beams

The Planck High Frequency Instrument (HFI) has observed the full sky at six frequencies (100, 143, 217, 353, 545, and 857 GHz) in intensity and at four frequencies in linear polarization (100, 143, 217, and 353 GHz). In order to obtain sky maps, the time-ordered information (TOI) containing the detector and pointing samples must be processed and the angular response must be assessed. The full mission TOI is included in the Planck 2015 release. This study describes the HFI TOI and beam processing for the 2015 release. HFI calibration and map making are described in a companion paper. The main pipeline has been modified since the last release (2013 nominal mission in intensity only), by including a correction for the nonlinearity of the warm readout and by improving the model of the bolometer time response. The beam processing is an essential tool that derives the angular response used in all the Planck science papers and we report an improvement in the effective beam window function uncertainty of more than a factor of 10 relative to the2013 release. Noise correlations introduced by pipeline filtering function are assessed using dedicated simulations. Finally, angular cross-power spectra using data sets that are decorrelated in time are immune to the main systematic effects.

79 ASTRONOMY AND ASTROPHYSICS↗