Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

The CDMS II data acquisition system

The Data Acquisition System for the CDMS II dark matter experiment was designed and built when the experiment moved to its new underground installation at the Soudan Lab. The combination of remote operation and increased data load necessitated a completely new design. Elements of the original LabView system remained as stand-alone diagnostic programs, but the main data processing moved to a VME-based system with custom electronics for signal conditioning, trigger formation and buffering. The data rate was increased 100-fold and the automated cryogenic system was linked to the data acquisition. A modular server framework with associated user interfaces was implemented in Java to allow control and monitoring of the entire experiment remotely.

99 GENERAL AND MISCELLANEOUS↗

Monitoring system for high-tech equipment

Many organizations and automated data-processing systems are involved in analysis of the logistical support (ALS) of high-tech equipment. Therefore, the initial data and the results of analysis must be presented in standardized form. That calls for an integrated informational model describing all the relevant data elements, their attributes, and their relationships. In ALS, the instruments that monitor the operation of the high-tech equipment form an important component of the information system.

42 ENGINEERING↗

THE CDF ARCHIVE: HERSCHEL PACS AND SPIRE SPECTROSCOPIC DATA PIPELINE AND PRODUCTS FOR PROTOSTARS AND YOUNG STELLAR OBJECTS

We present the COPS-DIGIT-FOOSH (CDF) Herschel spectroscopy data product archive, and related ancillary data products, along with data fidelity assessments, and a user-created archive in collaboration with the Herschel-PACS and SPIRE ICC groups. Our products include datacubes, contour maps, automated line fitting results, and best 1D spectra products for all protostellar and disk sources observed with PACS in RangeScan mode for two observing programs: the DIGIT Open Time Key Program (KPOT-nevans-1 and SDP-nevans-1; PI: N. Evans), and the FOOSH Open Time Program (OT1-jgreen02-2; PI: J. Green). In addition, we provide our best SPIRE-FTS spectroscopic products for the COPS Open Time Program (OT2-jgreen02-6; PI: J. Green) and FOOSH sources. We include details of data processing, descriptions of output products, and tests of their reliability for user applications. We identify the parts of the data set to be used with caution. The resulting absolute flux calibration has improved in almost all cases. Compared to previous reductions, the resulting rotational temperatures and numbers of CO molecules have changed substantially in some sources. On average, however, the rotational temperatures have not changed substantially (<2%), but the number of warm (T{sub rot} ∼ 300 K) CO molecules has increased by about 18%.

79 ASTRONOMY AND ASTROPHYSICS↗

Pyrogenic Organic Matter Laboratory Experiment: Aerobic Respiration and Geochemistry from Variably Inundated Stream Sediments (v3)

This dataset supports a broader study examining the effects of variable inundation and pyrogenic organic matter on ecosystem respiration. The dataset provides data generated from a laboratory batch experiment investigating the interaction between variable inundation conditions (wet and dry sediment) and pyrogenic organic matter (burned and unburned treatments). The contents include time series dissolved oxygen, sediment geochemistry data, and field metadata (including qualitative information on instream and river corridor characteristics). This data package was originally published in November 2025. It was updated in April 2026 (v2; new and modified files) and May 2026 (v3; modified files). See the change history section in the readme for more details For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) file-level metadata; (2) data dictionary; (3) field metadata; (4) international generic sample number (IGSN) mapping file; (5) readme; (6) field protocol; (7) sample name metadata; (8) an environmental context picture for the dry and inundated sampling locations; and (9) a subfolder with sample data from the sediment incubation experiment. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC); (2) total nitrogen (TN); (3) gravimetric moisture; (4) partial pressure and production rates of carbon dioxide, methane, and nitrous oxide; (5) field wet sediment mass, dry sediment mass, water mass, and field wet sediment volume in incubation and sediment NPOC/TN vials; (6) methods codes; (7) respiration rates, pH, and temperature from after the incubation, raw time series dissolved oxygen and temperature, and a subfolder containing associated plots and scripts; (8) ions; (9) FTICR-MS methods; and (10) a subfolder of 12 Tesla (12T) FTICR-MS data. This folder contains the CoreMS processed data and three subfolders, one containing the .xml files, one containing the CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .html, .Rmd, .py, .cal, .json, or .jpg.

54 ENVIRONMENTAL SCIENCES↗

Improving Neutron Measurement Capabilities; Expanding the Limits of Correlated Neutron Counting

A number of technical and practical limitations exist within the neutron correlated counting techniques used in safeguards, especially within the algorithms that are used to process and analyze the detected neutron signals. A multi-laboratory effort is underway to develop new and improved analysis and data processing algorithms based on fundamental physics principles to extract additional or more accurate information about nuclear material bearing items.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NJOY2012 - Current Status and Future Plans

An overview of the NJOY Nuclear Data Processing Code System, including recent history, current status and near-term upgrade plans is presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

XRCT characterisation of Ti particles inside porous Al{sub 2}O{sub 3}

Computed X-ray tomography was used to characterise distribution and sphericity of Ti granules within highly porous (> 35%) Al{sub 2}O{sub 3} powder compacts, as they are key parameters for a successful infiltration by Fe-based alloys. Setting of reconstruction constraints, image editing as well as data processing are the most challenging parts of computed X-ray tomography and principal sources of errors that bias the generated data. Thus, corrective measures have to be applied and the reliability of generated data has to be proved with respect to statistical, stereological and volumetric aspects. Combining an adapted Interface Particle Treatment Algorithm with the Marching Cube Method, Equilibrium Random State Model, cluster splitting and conventional laser diffraction measurements a significant improvement of the three-dimensional reconstructed data was achieved. This study points out the need of the applied algorithms for the proof and improvement of generated data by computed X-ray tomography and gives a short survey of methods that can be applied.

36 MATERIALS SCIENCE↗

The GRISM Lens-Amplified Survey from Space (GLASS). I. Survey overview and first data release

We give an overview of the Grism Lens Amplified Survey from Space (GLASS), a large Hubble Space Telescope program aimed at obtaining grism spectroscopy of the fields of 10 massive clusters of galaxies at redshift z = 0.308–0.686, including the Hubble Frontier Fields (HFF). The Wide Field Camera 3 (WFC3) yields near-infrared spectra of the cluster cores covering the wavelength range 0.81–1.69 μm through grisms G102 and G141, while the Advanced Camera for Surveys in parallel mode provides G800L spectra of the infall regions of the clusters. The WFC3 spectra are taken at two almost orthogonal position angles in order to minimize the effects of confusion. After summarizing the scientific drivers of GLASS, we describe the sample selection as well as the observing strategy and data processing pipeline. We then utilize MACS J0717.5+3745, a HFF cluster and the first one observed by GLASS, to illustrate the data quality and the high-level data products. Each spectrum brighter than H{sub AB}=23 is visually inspected by at least two co-authors and a redshift is measured when sufficient information is present in the spectra. Furthermore, we conducted a thorough search for emission lines through all of the GLASS WFC3 spectra with the aim of measuring redshifts for sources with continuum fainter than H{sub AB}=23. We provide a catalog of 139 emission-line-based spectroscopic redshifts for extragalactic sources, including three new redshifts of multiple image systems (one probable, two tentative). In addition to the data itself, we also release software tools that are helpful to navigate the data.

79 ASTRONOMY AND ASTROPHYSICS↗

MicroED data collection and processing

The collection and processing of MicroED data are presented. MicroED, a method at the intersection of X-ray crystallography and electron cryo-microscopy, has rapidly progressed by exploiting advances in both fields and has already been successfully employed to determine the atomic structures of several proteins from sub-micron-sized, three-dimensional crystals. A major limiting factor in X-ray crystallography is the requirement for large and well ordered crystals. By permitting electron diffraction patterns to be collected from much smaller crystals, or even single well ordered domains of large crystals composed of several small mosaic blocks, MicroED has the potential to overcome the limiting size requirement and enable structural studies on difficult-to-crystallize samples. This communication details the steps for sample preparation, data collection and reduction necessary to obtain refined, high-resolution, three-dimensional models by MicroED, and presents some of its unique challenges.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Image segmentation for uranium isotopic analysis by SIMS: Combined adaptive thresholding and marker controlled watershed approach

A novel approach to particle identification and particle isotope ratio determination has been developed for nuclear safeguard applications. This particle search approach combines an adaptive thresholding algorithm and marker-controlled watershed segmentation (MCWS) transform, which improves the secondary ion mass spectrometry (SIMS) isotopic analysis of uranium containing particle populations for nuclear safeguards applications. The Niblack assisted MCWS approach (a.k.a. SEEKER) developed for this work has improved the identification of isotopically unique uranium particles under conditions that have historically presented significant challenges for SIMS image data processing techniques. Particles obtained from five NIST uranium certified reference materials (CRM U129A, U015, U150, U500 and U850) were successfully identified in regions of SIMS image data 1) where a high variability in image intensity existed, 2) where particles were touching or were in close proximity to one another and/or 3) where the magnitude of ion signal for a given region was count limited. Analysis of the isotopic distributions of uranium containing particles identified by SEEKER showed four distinct, accurately identified 235U enrichment distributions, corresponding to the NIST certified 235U/238U isotope ratios for CRM U129A/U015 (not statistically differentiated), U150, U500 and U850. Additionally, comparison of the minor uranium isotope (234U, 235U and 236U) atom percent values verified that, even in the absence of high precision isotope ratio measurements, SEEKER could be used to segment isotopically unique uranium particles from SIMS image data. Although demonstrated specifically for SIMS analysis of uranium containing particles for nuclear safeguards, SEEKER has application in addressing a broad set of image processing challenges.

adaptive thresholding↗

PROBING THE DARK AGES AT z ∼ 20: THE SCI-HI 21 cm ALL-SKY SPECTRUM EXPERIMENT

We present first results from the SCI-HI experiment, which we used to measure the all-sky-averaged 21 cm brightness temperature in the redshift range 14.8 < z < 22.7. The instrument consists of a single broadband sub-wavelength size antenna and a sampling system for real-time data processing and recording. Preliminary observations were completed in 2013 June at Isla Guadalupe, a Mexican biosphere reserve located in the Pacific Ocean. The data was cleaned to excise channels contaminated by radio frequency interference, and the system response was calibrated by comparing the measured brightness temperature to the Global Sky Model of the Galaxy and by independent measurement of Johnson noise from a calibration terminator. We present our results, discuss the cosmological implications, and describe plans for future work.

79 ASTRONOMY AND ASTROPHYSICS↗

Leveraging Pre-Built Catalogs and Object-Level Scheduling to Eliminate I/O Bottlenecks in HPC Environments

Modern High-Performance Computing (HPC) environments face mounting challenges due to the shift from large to small file datasets, along with an increasing number of users and parallelized applications. As HPC systems rely on Parallel File Systems (PFS), such as Lustre for data processing, performance bottlenecks stemming from Object Storage Target (OST) contention have become a significant concern. Existing solutions, such as LADS with its object-level scheduling approach, fall short in large-scale HPC environments due to their inability to effectively address metadata I/O bottlenecks and the growing number of I/O processes. This study highlights the pressing need for a comprehensive solution that tackles both OST contention and metadata I/O challenges in diverse HPC workloads. To address these challenges, we propose SwiftLoad, an object-level I/O scheduling framework that leverages a metadata catalog to enhance the performance and efficiency of parallel HPC utilities. The adoption of the metadata catalog mitigates the metadata I/O bottlenecks that commonly occur in HPC utilities, a challenge that is particularly pronounced in object-level I/O scheduling. SwiftLoad addresses OST contention and the uneven distribution of I/O processes across different OSTs through mathematical modeling and incorporates a Loader Configuration Module to regulate the number of I/O processes. Evaluated with two representative utilities—data deduplication profiling and data augmentation—SwiftLoad achieved performance improvements of up to 5.63x and 11.0x, respectively, on a production supercomputer.

HPC↗

Graphene milling dynamics during helium ion beam irradiation

We explore the potential of the Helium Ion Microscope (HIM) as a tool for direct-write patterning of graphene and describe the underlying processes of graphene milling with image data processing. Controlled helium ion irradiation of suspended graphene conducted while monitoring the mill in-situ revealed the localized formation of nanopores, their growth, and coalescence. We also explore the effects of defects on the milling dynamics, and show that pre-exposed membranes rupture by cracking and rapid crack propagation at the edges of the growing defects. The mechanism for the rupturing process is described by local defect formation by excessive irradiation of helium ions, dictated by the scanning direction of the beam. As a result, these findings enrich fundamental understanding of the graphene milling process with a helium ion beam that, is necessary for high-resolution and high throughput patterning of graphene with nanoscale precision.

36 MATERIALS SCIENCE↗

Modeling of time-lapse seismic monitoring using CO 2 leakage simulations for a model CO 2 storage site with realistic geology: Application in assessment of early leak-detection capabilities

Time-lapse surface seismic surveys have been widely used at carbon sequestration sites for site characterization, monitoring subsurface CO 2 plume migration, and detecting potential CO 2 leakage from a storage reservoir. Monitoring in the first permeable unit directly above the primary seal is important for early detection of CO 2 leakage. Forward modeling of time-lapse seismic data can be used to assess the utility of the seismic method for CO 2 leakage detection. We develop a workflow for forward modeling of time-lapse seismic data, including constructing seismic velocity models using flow simulation outputs, modeling of pre-stack and post-stack synthetic seismic data following seismic data processing sequence and analysis of processed synthetic time-lapse seismic data. We apply the forward modeling and analysis workflow to assessing the detectability and the earliest detection time of seismic monitoring using the hypothetical CO 2 leakage scenarios for a model geologic storage site with realistic geology. We then derive the detection thresholds using the simulated normalized root-mean-square (NRMS) differences for the no-leakage case at a range of signal-to-noise ratios, representing the background noise levels in seismic data. We then compare NRMS differences triggered by the CO 2 leakage to the detection thresholds at each time step to quantify the detectability and the earliest detection time of seismic monitoring. We analyze the effects of the acquisition parameters and elastic parameters on the produced synthetic seismic data and earliest detection time. Our modeling results indicate that high signal-to-noise ratio is needed to detect the CO 2 leakage at the model site. Minimizing the background noise in seismic data is crucial for improving the detectability of the seismic method. Increasing the shot density or increasing the dominant frequency of the source wavelet is likely to increase the possibility of the leakage detection and reduce the time needed for the detection. The elastic parameters used in the rock physics modeling have significant effects on the resultant seismic velocity models and synthetic seismic data, highlighting their critical roles in understanding and interpreting time-lapse seismic reflection data associated with CO 2 monitoring, verification and accounting activities.

58 GEOSCIENCES↗

iVirus: facilitating new insights in viral ecology with software and community data sets imbedded in a cyberinfrastructure

Microbes affect nutrient and energy transformations throughout the world’s ecosystems, yet they do so under viral constraints. In complex communities, viral metagenome (virome) sequencing is transforming our ability to quantify viral diversity and impacts. Although some bottlenecks, for example, few reference genomes and nonquantitative viromics, have been overcome, the void of centralized data sets and specialized tools now prevents viromics from being broadly applied to answer fundamental ecological questions. Here we present iVirus, a community resource that leverages the CyVerse cyberinfrastructure to provide access to viromic tools and data sets. The iVirus Data Commons contains both raw and processed data from 1866 samples and 73 projects derived from global ocean expeditions, as well as existing and legacy public repositories. Through the CyVerse Discovery Environment, users can interrogate these data sets using existing analytical tools (software applications known as ‘Apps’) for assembly, open reading frame prediction and annotation, as well as several new Apps specifically developed for analyzing viromes. Because Apps are web based and powered by CyVerse supercomputing resources, they enable scalable analyses for a broad user base. Finally, a use-case scenario documents how to apply these advances toward new data. This growing iVirus resource should help researchers utilize viromics as yet another tool to elucidate viral roles in nature.

59 BASIC BIOLOGICAL SCIENCES↗

Planck 2013 results. VII. HFI time response and beams

This paper characterizes the effective beams, the effective beam window functions and the associated errors for the Planck High Frequency Instrument (HFI) detectors. The effective beam is the angular response including the effect of the optics, detectors, data processing and the scan strategy. The window function is the representation of this beam in the harmonic domain which is required to recover an unbiased measurement of the cosmic microwave background angular power spectrum. The HFI is a scanning instrument and its effective beams are the convolution of: a) the optical response of the telescope and feeds; b) the processing of the time-ordered data and deconvolution of the bolometric and electronic transfer function; and c) the merging of several surveys to produce maps. The time response transfer functions are measured using observations of Jupiter and Saturn and by minimizing survey difference residuals. The scanning beam is the post-deconvolution angular response of the instrument, and is characterized with observations of Mars. The main beam solid angles are determined to better than 0.5% at each HFI frequency band. Observations of Jupiter and Saturn limit near sidelobes (within 5°) to about 0.1% of the total solid angle. Time response residuals remain as long tails in the scanning beams, but contribute less than 0.1% of the total solid angle. The bias and uncertainty in the beam products are estimated using ensembles of simulated planet observations that include the impact of instrumental noise and known systematic effects. Here, the correlation structure of these ensembles is well-described by five error eigenmodes that are sub-dominant to sample variance and instrumental noise in the harmonic domain. A suite of consistency tests provide confidence that the error model represents a sufficient description of the data. The total error in the effective beam window functions is below 1% at 100 GHz up to multipole &ell; ~ 1500, and below 0.5% at 143 and 217 GHz up to &ell; ~ 2000.

79 ASTRONOMY AND ASTROPHYSICS↗

Connected Traffic Signal Coordination Optimization Framework through Network-Wide Adaptive Linear Quadratic Regulator–Based Control Strategy

Traffic congestion in metropolitan areas causes several significant challenges, such as longer travel times, decreased productivity, increased fuel consumption and vehicle emissions, and even severe injuries during crashes. Traffic signal control is a management approach to reduce traffic congestion and allocate the appropriate right of way for safety and mobility efficiency, both in temporal and spatial domains. Here, this study proposes a network-wide adaptive signal control coordination optimization framework based on the linear quadratic regulator algorithm. The traffic flow conditions driven by signal control inputs are formulated based on their network-wide state-space representation. After modeling traffic control regulation constraints, an adaptive linear quadratic regulator algorithm is designed to maximize the network-wide total throughput under the current conditions. Optimal signal control split time durations for multiple intersections in the network are derived by solving the algebraic Riccati equation. Furthermore, the recursive least square parameter estimation method is employed to quantify dynamic traffic condition changes. To verify the effectiveness of this proposed signal control framework, both simulation and real-world experimental tests are conducted for multiple intersections in downtown Chattanooga, Tennessee, United States. In preparation for real-world experimental tests, pipelines for real-time data processing implementation and historical traffic flow data analysis are conducted. The test results demonstrate that the proposed control framework achieves a decrease in travel time by up to 19.4%, total time spent (TTS) by up to 11.9%, and relative queue balance (RQB) by up to 15.6%. The research findings indicate that the proposed signal control framework can be generalized to handle large scale signal control optimization network-wide.

97 MATHEMATICS AND COMPUTING↗

Online learning of quadratic manifolds from streaming data for nonlinear dimensionality reduction and nonlinear model reduction

Here, this work introduces an online greedy method for constructing quadratic manifolds from streaming data, designed to enable in situ analysis of numerical simulation data on the Petabyte scale. Unlike traditional batch methods, which require all data to be available upfront and take multiple passes over the data, the proposed online greedy method incrementally updates quadratic manifolds in one pass as data points are received, eliminating the need for expensive disk input/output operations as well as storing and loading data points once they have been processed. A range of numerical examples demonstrate that the online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.

97 MATHEMATICS AND COMPUTING↗