Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73

BeyondPlanck: V. Minimal ADC Corrections for Planck LFI

We describe the correction procedure for Analog-to-Digital Converter (ADC) differential non-linearities (DNL) adopted in the Bayesian end-to-end BEYONDPLANCK analysis framework. This method is nearly identical to that developed for the official Planck Low Frequency Instrument (LFI) Data Processing Center (DPC) analysis, and relies on the binned rms noise profile of each detector data stream. However, rather than building the correction profile directly from the raw rms profile, we first fit a Gaussian to each significant ADC-induced rms decrement, and then derive the corresponding correction model from this smooth model. The main advantage of this approach is that only samples which are significantly affected by ADC DNLs are corrected, as opposed to the DPC approach in which the correction is applied to all samples, filtering out signals not associated with ADC DNLs. The new corrections are only applied to data for which there is a clear detection of the non-linearities, and for which they perform at least comparably with the DPC corrections. Out of a total of 88 LFI data streams (sky and reference load for each of the 44 detectors) we apply the new minimal ADC corrections in 25 cases, and maintain the DPC corrections in 8 cases. All these corrections are applied to 44 or 70 GHz channels, while, as in previous analyses, none of the 30 GHz ADCs show significant evidence of non-linearity. By comparing the BEYONDPLANCK and DPC ADC correction methods, we estimate that the residual ADC uncertainty is about two orders of magnitude below the total noise of both the 44 and 70 GHz channels, and their impact on current cosmological parameter estimation is small. However, we also show that non-idealities in the ADC corrections can generate sharp stripes in the final frequency maps, and these could be important for future joint analyses with the Planck High Frequency Instrument (HFI), Wilkinson Microwave Anisotropy Probe (WMAP), or other datasets. We therefore conclude that, although the existing corrections are adequate for LFI-based cosmological parameter analysis, further work on LFI ADC corrections is still warranted.

79 ASTRONOMY AND ASTROPHYSICS↗

Advances in utilizing event based data structures for neutron scattering experiments

This article strives to expand on existing work to demonstrate advancements in data processing made available using event mode measurements. Most spallation neutron sources in the world have data acquisition systems that provide event recording. The new science that is enabled by utilizing event mode has only begun to be explored. In the past, these studies were difficult to perform because histograms forced dealing with either large chunks of time or a large number of files. With event based data collection, data can be explored and rebinned long after the measurement has completed. This article will review some of the principles of event data and how the method opens up new possibilities for in situ measurements, highlighting techniques that can be used to explore changes in the data. We also demonstrate the statistical basis for determining data quality and address the challenge of determining how long to measure mid-measurement. Lastly, we demonstrate a model independent method of grouping data via hierarchical clustering methods that can be used to improve calibration, reduction, and data exploration.

74 ATOMIC AND MOLECULAR PHYSICS↗

CMS Data transfer operations after the first years of LHC collisions

CMS experiment utilizes distributed computing infrastructure and its performance heavily depends on the fast and smooth distribution of data between different CMS sites. Data must be transferred from the Tier-0 (CERN) to the Tier-1s for processing, storing and archiving, and time and good quality are vital to avoid overflowing CERN storage buffers. At the same time, processed data has to be distributed from Tier-1 sites to all Tier-2 sites for physics analysis while Monte Carlo simulations sent back to Tier-1 sites for further archival. At the core of all transferring machinery is PhEDEx (Physics Experiment Data Export) data transfer system. It is very important to ensure reliable operation of the system, and the operational tasks comprise monitoring and debugging all transfer issues. Based on transfer quality information Site Readiness tool is used to create plans for resources utilization in the future. We review the operational procedures created to enforce reliable data delivery to CMS distributed sites all over the world. Additionally, we need to keep data and meta-data consistent at all sites and both on disk and on tape. In this presentation, we describe the principles and actions taken to keep data consistent on sites storage systems and central CMS Data Replication Database (TMDB/DBS) while ensuring fast and reliable data samples delivery of hundreds of terabytes to the entire CMS physics community.

Kaselis, Rapolas↗

Review of Enabling Technologies to Facilitate Secure Compute Customization

High performance computing environments are often used for a wide variety of workloads ranging from simulation, data transformation and analysis, and complex workflows to name just a few. These systems may process data for a variety of users, often requiring strong separation between job allocations. There are many challenges to establishing these secure enclaves within the shared infrastructure of high-performance computing (HPC) environments. The isolation mechanisms in the system software are the basic building blocks for enabling secure compute enclaves. There are a variety of approaches and the focus of this report is to review the different virtualization technologies that facilitate the creation of secure compute enclaves. The report reviews current operating system (OS) protection mechanisms and modern virtualization technologies to better understand the performance/isolation properties. We also examine the feasibility of running ``virtualized'' computing resources as non-privileged users, and providing controlled administrative permissions for standard users running within a virtualized context. Our examination includes technologies such as Linux containers (LXC [32], Docker [15]) and full virtualization (KVM [26], Xen [5]). We categorize these different approaches to virtualization into two broad groups: OS-level virtualization and system-level virtualization. The OS-level virtualization uses containers to allow a single OS kernel to be partitioned to create Virtual Environments (VE), e.g., LXC. The resources within the host's kernel are only virtualized in the sense of separate namespaces. In contrast, system-level virtualization uses hypervisors to manage multiple OS kernels and virtualize the physical resources (hardware) to create Virtual Machines (VM), e.g., Xen, KVM. This terminology of VE and VM, detailed in Section 2, is used throughout the report to distinguish between the two different approaches to providing virtualized execution environments. As part of our technology review we analyzed several current virtualization solutions to assess their vulnerabilities. This included a review of common vulnerabilities and exposures (CVEs) for Xen, KVM, LXC and Docker to gauge their susceptibility to different attacks. The complete details are provided in Section 5 on page 33. Based on this review we concluded that system-level virtualization solutions have many more vulnerabilities than OS level virtualization solutions. As such, security mechanisms like sVirt (Section 3.3) should be considered when using system-level virtualization solutions in order to protect the host against exploits. The majority of vulnerabilities related to KVM, LXC, and Docker are in specific regions of the system. Therefore, future "zero day attacks" are likely to be in the same regions, which suggests that protecting these areas can simplify the protection of the host and maintain the isolation between users. The evaluations of virtualization technologies done thus far are discussed in Section 4. This includes experiments with 'user' namespaces in VEs, which provides the ability to isolate user privileges and allow a user to run with different UIDs within the container while mapping them to non-privileged UIDs in the host. We have identified Linux namespaces as a promising mechanism to isolate shared resources, while maintaining good performance. In Section 4.1 we describe our tests with LXC as a non-root user and leveraging namespaces to control UID/GID mappings and support controlled sharing of parallel file-systems. We highlight several of these namespace capabilities in Section 6.2.3. The other evaluations that were performed during this initial phase of work provide baseline performance data for comparing VEs and VMs to purely native execution. In Section 4.2 we performed tests using the High-Performance Computing Conjugate Gradient (HPCCG) benchmark to establish baseline performance for a scientific application when run on the Native (host) machine in contrast with execution under Docker and KVM. Our tests verified prior studies showing roughly 2-4% overheads in application execution time & MFlops when running in hypervisor-base environments (VMs) as compared to near native performance with VEs. For more details, see Figures 4.5 (page 28), 4.6 (page 28), and 4.7 (page 29). Additionally, in Section 4.3 we include network measurements for TCP bandwidth performance over the 10GigE interface in our testbed. The Native and Docker based tests achieved >= ~9Gbits/sec, while the KVM configuration only achieved 2.5Gbits/sec (Table 4.6 on page 32). This may be a configuration issue with our KVM installation, and is a point for further testing as we refine the network settings in the testbed. The initial network tests were done using a bridged networking configuration. The report outline is as follows: - Section 1 introduces the report and clarifies the scope of the proj...

Aderholdt, Ferrol↗

Unlimited Release of non-proprietary experimental data from the Scaled Wind Farm Technology (SWiFT) Facility

Each year Wind Energy Technologies Dept. 08821 submits a memo through the Sandia National Labs Review and Approval (R&A) system to facilitate the release of the Scaled Wind Farm Technology (SWiFT) Facility raw logged data. This release of data explicitly does not cover specialized instruments, or guest researcher instruments (i.e. SpiDAR, SpinnerLidar), nor processed data.

17 WIND ENERGY↗

Watchmen 3.0.0 Users Guide: Revision 6

Watchmen is a research software application developed by Pacific Northwest National Laboratory (PNNL) that incorporates the scientific and operational expertise for reviewing data from treaty monitoring radionuclide stations. The radionuclide stations are part of the International Monitoring System (IMS), a worldwide network to monitor for nuclear explosions. The data the IMS produces are critical to determine if a radionuclide release event is from a nuclear explosion. Stations deliver their measurements and system status to the International Data Centre (IDC), which forwards it via email to all subscribers. Watchmen is capable of processing data from several radioxenon station types.

97 MATHEMATICS AND COMPUTING↗

Watchmen 3.1.0 Users Guide: Revision 7

Watchmen is a research software application developed by Pacific Northwest National Laboratory (PNNL) that incorporates the scientific and operational expertise for reviewing data from treaty monitoring radionuclide stations. The radionuclide stations are part of the International Monitoring System (IMS), a worldwide network to monitor for nuclear explosions. The data the IMS produces are critical to determine if a radionuclide release event is from a nuclear explosion. Stations deliver their measurements and system status to the International Data Centre (IDC), which forwards it via email to all subscribers. Watchmen is capable of processing data from several radioxenon station types.

97 MATHEMATICS AND COMPUTING↗

Watchmen v3.2.0 User Guide: Revision 8

Watchmen is a research software application developed by Pacific Northwest National Laboratory (PNNL) that incorporates the scientific and operational expertise for reviewing data from treaty monitoring radionuclide stations. The radionuclide stations are part of the International Monitoring System (IMS), a worldwide network to monitor for nuclear explosions. The data the IMS produces are critical to determine if a radionuclide release event is from a nuclear explosion. Stations deliver their measurements and system status to the International Data Centre (IDC), which forwards it via email to all subscribers. Watchmen is capable of processing data from several radioxenon station types.

97 MATHEMATICS AND COMPUTING↗

Watchmen 3.3.0 User Guide: Revision 9

Watchmen is a research software application developed by Pacific Northwest National Laboratory (PNNL) that incorporates the scientific and operational expertise for reviewing data from treaty monitoring radionuclide stations. The radionuclide stations are part of the International Monitoring System (IMS), a worldwide network to monitor for nuclear explosions. The data the IMS produces are critical to determine if a radionuclide release event is from a nuclear explosion. Stations deliver their measurements and system status to the International Data Centre (IDC), which forwards it via email to all subscribers. Watchmen is capable of processing data from several radioxenon station types.

97 MATHEMATICS AND COMPUTING↗

Data for "Which plant traits increase soil carbon sequestration? Empirical evidence from a long-term poplar genetic diversity trial"

This archive contains all data and code used by the following publication: Field, J. L., Sloan, B. P., Craig, M. E., Calloway, P., Ottinger, S. L., Mead, T., Abramoff, R. Z., Venegas, M. P., Chhetri, H. B., Haiby, K., Kalluri, U. C., Muchero, W., Schadt, C. W., & Mayes, M. A. (2025). Which plant traits increase soil carbon sequestration? Empirical evidence from a long-term poplar genetic diversity trial (p. 2025.02.17.638464). bioRxiv. https://doi.org/10.1101/2025.02.17.638464 Our analysis combined several soil and root data sets collected by Oak Ridge National Laboratory (ORNL) researchers/collaborators from the Clatskanie Poplar Common Garden in Clatskanie, OR by from 2009-2024. The raw data data files are located */02-data/01-raw/* which we harmonized using the codes in */01-codes/01-harmonize-clatskanie-data-pub.qmd*. The final processed data set used in the paper is found at */02-data/02-processed/clatskanie-c-fit-data.csv* and its columns are described in the table below.

Sloan, Brandon [ORNL] (ORCID:0000000316304271)↗

The South Pole Telescope AGN Monitoring Campaign: First Release of SPTpol Bright AGN Light Curves

The South Pole Telescope (SPT) collaboration has recently embarked upon a campaign to monitor the brightness of a sample of active galactic nuclei (AGN), both in real time and in archival SPT data. The original design of the SPT was optimized for observations of the cosmic microwave background (CMB) at arc-minute and larger angular scales, and it has been used for this purpose for nearly twenty years, using three generations of CMB cameras. Recently it has been recognized that data from CMB experiments have the potential to be used for AGN monitoring. In this paper, we present the first public release of data from a full sample of SPT-monitored AGN, comprising 158 AGN light curves and associated data from the SPTpol camera, which was operational from 2012-2016. These light curves were created using observations from the SPTpol 500 deg$^{2}$ survey, in which the instrument was used to scan a 500 deg$^2$ patch of the sky several times per day with detectors sensitive to radiation in bands centered at 90 and 150 GHz. We provide a comprehensive description of the observations, the data processing methods, and the resulting light curve catalog. As an example of analyses that these data enable, we searched for a correlation between variability and spectral index, and we looked for ``bluer-when-brighter'' trends in the sample. Our analysis finds $> 10 σ$ correlation between fractional intrinsic variance and mean spectral index in the sample, but no significant evidence for bluer-when-brighter trends. The datasets from this study can be accessed through the SPT Treasury Record of AGN With Historical Activity and Time-Series or STRAWHAT catalog. This initial data release includes SPTpol light curves at 90 and 150 GHz, focusing on total intensity. In later updates, SPTpol polarization data and new observations from the SPT-3G instrument at 90, 150, and 220 GHz will be included.

Hood, J.C., II [Chicago U., KICP; Chicago U., Astr↗

Structure Determination of An Anti-HIV-1 Fab 447-52d-Peptide Complex From An Expitaxially Twinned Data Set

Although antibodies against the third variable loop (V3) of the HIV-1 viral envelope glycoprotein are among the first neutralizing antibodies to be detected in infected individuals, they are normally restricted in their specificity. X-ray crystallographic studies of V3-specific antibodies have contributed to a more thorough understanding of recognition of this epitope and of conserved features in the V3 loop that could potentially aid in the design of a multi-component vaccine. The human antibody 447-52D exhibits relatively broad neutralization of primary viral isolates compared with other V3-loop antibodies. A crystal structure of Fab 447-52D in complex with a V3 peptide (UG1033) was determined at 2.1 {angstrom} resolution. The structure was determined using an epitaxially twinned data set and in-house programs to detect and remove overlapping reflections. Although the processed data have lower than desired completeness and slightly higher than normal R values for the resolution, good-quality electron-density maps were obtained that enabled structure determination. The structure revealed an extended CDR H3 loop that forms a {beta}-sheet with the peptide, with the predominant contacts being main-chain hydrogen bonds. The V3 peptide and Fab show high structural homology with the previously reported structures of other Fab 447-52D complexes, reinforcing the idea that the V3 loop may adopt a small set of conserved structures, particularly around the crown of the {beta}-hairpin.

08 HYDROGEN↗

WE-AB-BRB-12: Nanoscintillator Fiber-Optic Detector System for Microbeam Radiation Therapy Dosimetry

Purpose: Microbeam Radiation Therapy (MRT) is an experimental radiation therapy that has demonstrated a higher therapeutic ratio than conventional radiation therapy in animal studies. There are several roadblocks in translating the promising treatment technology to clinical application, one of which is the lack of a real-time, high-resolution dosimeter. Current clinical radiation detectors have poor spatial resolution and, as such, are unsuitable for measuring microbeams with submillimeter-scale widths. Although GafChromic film has high spatial resolution, it lacks the real-time dosimetry capability necessary for MRT preclinical research and potential clinical use. In this work we have demonstrated the feasibility of using a nanoscintillator fiber-optic detector (nanoFOD) system for real-time MRT dosimetry. Methods: A microplanar beam array is generated using a x-ray research irradiator and a custom-made, microbeam-forming collimator. The newest generation nanoFOD has an effective size of 70 µm in the measurement direction and was calibrated against a kV ion chamber (RadCal Accu-Pro) in open field geometry. We have written a computer script that performs automatic data collection with immediate background subtraction. A computer-controlled detector positioning stage is used to precisely measure the microbeam peak dose and beam profile by translating the stage during data collection. We test the new generation nanoFOD system, with increased active scintillation volume, against the previous generation system. Both raw and processed data are time-stamped and recorded to enable future post-processing. Results: The real-time microbeam dosimetry system worked as expected. The new generation dosimeter has approximately double the active volume compared to the previous generation resulting in over 900% increase in signal. The active volume of the dosimeter still provided the spatial resolution that meets the Nyquist criterion for our microbeam widths. Conclusion: We have demonstrated that real-time dosimetry of MRT microbeams is feasible using a nanoscintillator fiber-optic detector with integrated positioning system.

60 APPLIED LIFE SCIENCES↗

AnalyzeThat: A Programmable Shared-Memory System for an Array of Processing-In-Memory Devices

Processing In Memory (PIM), the concept of integrating processing directly with memory, has been attracting a lot of attention since PIM can assist in overcoming the throughput limitation caused by data movement between CPU and memory. The challenge, however, is that it requires the programmers to have a deep understanding of the PIM architecture to maximize the benefits such as data locality and parallel thread execution on multiple PIM devices. In this study, we present AnalyzeThat, a programmable shared-memory system for parallel data processing with PIM devices. Thematic to AnalyzeThat is a rich PIM-Aware Data Structure (PADS), which is an encapsulation that integrally ties together the data, the analysis tasks and the runtime needed to interface with the PIM device array. The PADS abstraction provides (i) a key-value data container that allows programmers to easily store data on multiple PIMs, (ii) a suite of parallel operations with which users can easily implement data analysis applications, and (iii) a runtime, hidden to programmers, which provides the mechanisms needed to overlay both the data and the tasks on the PIM device array in an intelligent fashion, based on PIM-specific information collected from the hardware. We have developed a PIM emulation framework called AnalyzeThat. Our experimental evaluation with representative data analytics applications suggests that the proposed system can significantly reduce the PIM programming effort without losing its technology benefits.

97 MATHEMATICS AND COMPUTING↗

Automatic microseismic event picking via unsupervised machine learning

SUMMARY Effective and efficient arrival picking plays an important role in microseismic and earthquake data processing and imaging. Widely used short-term-average long-term-average ratio (STA/LTA) based arrival picking algorithms suffer from the sensitivity to moderate-to-strong random ambient noise. To make the state-of-the-art arrival picking approaches effective, microseismic data need to be first pre-processed, for example, removing sufficient amount of noise, and second analysed by arrival pickers. To conquer the noise issue in arrival picking for weak microseismic or earthquake event, I leverage the machine learning techniques to help recognizing seismic waveforms in microseismic or earthquake data. Because of the dependency of supervised machine learning algorithm on large volume of well-designed training data, I utilize an unsupervised machine learning algorithm to help cluster the time samples into two groups, that is, waveform points and non-waveform points. The fuzzy clustering algorithm has been demonstrated to be effective for such purpose. A group of synthetic, real microseismic and earthquake data sets with different levels of complexity show that the proposed method is much more robust than the state-of-the-art STA/LTA method in picking microseismic events, even in the case of moderately strong background noise.

Chen, Yangkang↗

Enrichment Meter Dataset from High-Resolution Gamma Spectroscopy Measurements of U3O8 Enrichment Standards and UF6 Cylinder Wall Equivalents

The Enrichment Meter Principle (EMP) is the basis for a commonly used standard test method for the non-destructive assay of 235 U enrichment in bulk compounds [1]. The technique involves determining the net count rate in the direct 186 keV peak using medium or high energy gamma-ray spectrometry in a fixed geometry. With suitable correction for wall attenuation, compound type, rate loss (live time), and peaked background (if significant), the atom fraction of 235 U may be obtained from the counting rate from a linear relationship through the origin. The widespread use of this method for field verification of enrichment [2,3] together with the fact that the response function rests on fundamental physics considerations (i.e., is not represented by a convenient but arbitrary form) makes it an interesting example of uncertainty quantification, one in which one can expect a valid measurement model can be applied. When applied using NaI(Tl) and region of interest analysis, the technique is susceptible to both interference error and bias [2-4]. When implemented using high-resolution gamma-ray spectroscopy, the spectrum interpretation is considerable simplified and more robust [5]. However, a practical challenge to studying the uncertainty budget of the EMP method (for example, to test linearity, extract wall corrections and so forth using modern methods) is the availability of quality experimental data that can be referenced and shared. To fill this gap, the research team undertook an experimental campaign [6]. A measurement campaign was conducted to produce high-resolution gamma spectroscopy enrichment meter data comparable to UF 6 cylinder measurements. The purpose of this report is to provide both an introduction to and quality assurance (QA) of the raw data produced. This report is intended for the analyst or researcher who uses the raw data. Unfortunately, the raw data (i.e., the spectra files) are too voluminous to include in this report but can be requested from Steven Croft of the Safeguards & Security Technology Group (scroft@ornl.gov 865-241-2834). The complete processed data are tabulated in Appendix A. The analysis techniques used to produce the QA data presented in this report [e.g., three regions-ofinterest (ROI) peak extraction and batch analysis processes] are not the most sophisticated techniques available; analysts are encouraged to reanalyze the raw data using more sensitive techniques and to improve upon the results presented here. With that being said, the analysis techniques used here are more than adequate to present and inspect the quality of the data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Quantitative acetylene measurements in laminar and turbulent flames using 1D Raman/Rayleigh scattering

Our research presents the extension of the established detection of seven major species in 1D Raman/Rayleigh measurements to incorporate acetylene (C 2 H 2 ) as an eighth species. Acetylene is an important soot precursor which is generated in hydrocarbon flames as an intermediate species following decomposition. It occurs over a broad temperature range in flames and has been identified as a potentially detectable species in 1D Raman/Rayleigh measurements. In this paper, we discuss the Raman spectral signature of acetylene, its temperature dependence, calibration procedures, and the interference with other Raman-active species, C 2 and broadband interferences, all of which are essential to quantify for accurate data processing. In this regard, Raman measurements in laminar and turbulent dimethyl ether flames were acquired. The data are spectrally analyzed and low-temperature calibrations of the Raman response are combined with an extrapolation model for high temperatures in order to place the acetylene signals on an absolute scale. Single-shot and conditional-mean values are presented as a function of mixture fraction for a laminar partially-premixed DME/air flame. Measurements from the laminar flame are compared to a 1D laminar flame calculation to assess the accuracy of the calibration and analysis procedures. Furthermore, detection limits, signal-to-noise, and signal-to-interference ratio of the C 2 H 2 measurement are discussed, and single-shot measurements of acetylene in a turbulent DME/air jet flame are demonstrated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Correlative montage parallel array cryo-tomography for in situ structural cell biology

Abstract Imaging large fields of view while preserving high-resolution structural information remains a challenge in low-dose cryo-electron tomography. Here we present robust tools for montage parallel array cryo-tomography (MPACT) tailored for vitrified specimens. The combination of correlative cryo-fluorescence microscopy, focused-ion-beam milling, substrate micropatterning, and MPACT supports studies that contextually define the three-dimensional architecture of cells. To further extend the flexibility of MPACT, tilt series may be processed in their entirety or as individual tiles suitable for sub-tomogram averaging, enabling efficient data processing and analysis.

59 BASIC BIOLOGICAL SCIENCES↗