Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

Evaluation of emerging parallel optical link technology for high energy physics

Modern particle detectors utilize optical fiber links to deliver event data to upstream trigger and data processing systems. Future detector systems can benefit from the development of dense arrangements of high speed optical links emerging from industry advancements in transceiver technology. Supporting data transfers of up to 120 Gbps in each direction, optical engines permit assembly of the optical transceivers in close proximity to ASICs and FPGAs. Test results of some of these parallel components will be presented including the development of pluggable FPGA Mezzanine Cards equipped with optical engines to provide to collaborators on the Versatile Link Common Project for the HI-LHC at CERN.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Production Operations Management System (POMS) for Fermilab Experiments

The Production Operations Management System (POMS) software allows production teams and analysis groups across multiple Fermilab experiments to launch, modify and monitor large scale campaigns of related Monte Carlo or data processing jobs, and currently manages the majority of production computing of experiments at Fermilab.POMS provides a web service interface that enables automated job submission on distributed resources according to customers’ requests, as well as subsequent monitoring of those submissions as well as recovery of failed submissions, debugging and record keeping.POMS interfaces with existing HEP data access, processing, movement and monitoring tools at Fermilab, including Jobsub (for job submission), SAM (for data management), and Landscape (for monitoring), and in combination with them handles the creation of intermediate datasets of files for multistage campaigns.POMS visualizes these campaigns with an interactive GUI interface. An important feature of POMS is a one-to-one connection between the GUI campaign visualizer and a text based representation of the complete campaign configuration. An extensible library of template campaign configurations is available. The templates are user modifiable and map cleanly to and from the GUI description of the campaign.This paper will discuss the current usage, history, and some technical details of POMS, as well as planned future extensions, particularly interfacing with Rucio as well as SAM for file movement.

Mengel, Marc↗

A DEEP, WIDE-FIELD H{alpha} SURVEY OF NEARBY CLUSTERS OF GALAXIES: DATA

We present the results of a wide-field H{alpha} imaging survey of eight nearby (z = 0.02-0.03) Abell clusters. We have measured H{alpha} fluxes and equivalent widths for 465 galaxies, of which 360 are new detections. The survey was designed to obtain complete emission-line-selected inventories of star-forming galaxies in the inner regions of these clusters, extending to star formation rates below 0.1 M{sub Sun} yr{sup -1}. This paper describes the observations, data processing, and source identification procedures, and presents an H{alpha} and R-band catalog of detected cluster members and other candidates. Future papers in the series will use these data to study the completeness of spectroscopically based star formation surveys, and to quantify the effects of cluster environment on the present-day populations of star-forming galaxies. The data will also provide a valuable foundation for imaging surveys of redshifted H{alpha} emission in more distant clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

SUBTASK 1.6 – BASIN ELECTRIC CARBON STORAGE RESEARCH PROJECT: NOVEL MONITORING TECHNIQUES

The Energy & Environmental Research Center (EERC) conducted baseline activities associated with an applied research project at Basin Electric Power Cooperative’s (Basin’s) carbon capture and storage (CCS) site in Beulah, North Dakota, to establish novel carbon storage-monitoring techniques as commercial methods under Cooperative Agreement No. DE-FE0024233, Subtask 1.6. The following report summarizes the baseline activities performed and briefly describes the subsequent (operational monitoring) activities that have been proposed to the U.S. Department of Energy (DOE) as part of the overall project to develop and demonstrate novel monitoring techniques at North America’s largest permitted CCS operation. Dakota Gasification Company (DGC), a wholly owned subsidiary of Basin, owns and operates the Great Plains Synfuels Plant (GPSP) approximately 5 miles northwest of the town of Beulah, North Dakota (Figure 1). In 2023, DGC received approval from the North Dakota Industrial Commission (NDIC) to develop a storage facility on-site for injecting a stream of carbon dioxide (CO2) captured from GPSP. DGC will transport the captured CO2 stream with approximately 6.8 miles of transmission lines that extend north of GPSP and inject >1 million tonnes (MMt) of CO2 annually (>1 MMt/yr) over a 12-year period with up to six underground injection control (UIC) Class VI-compliant injection wells completed in the Broom Creek Formation, a predominantly sandstone reservoir and saline aquifer underlying GPSP. The Broom Creek Formation lies approximately 5900 feet (ft) below ground surface (bgs) at GPSP. The commercial scale (i.e., >1 MMt/yr) of DGC’s permitted carbon storage project is ideal for developing and testing the novel monitoring techniques included within Subtask 1.6. The goals of this project are to demonstrate 1) the cost-effectiveness of novel monitoring technologies included as part of this research, 2) technology capability for tracking the CO2 plume and/or associated pressure response in the subsurface and monitoring out-of-zone migration, and 3) compliance with UIC Class VI program requirements. The research activities proposed for the overall project include 1) design of an automated, integrated, modular (AIM) monitoring station; 2) time-lapse electromagnetic (EM) field surveys; 3) drone-based surveillance studies; 4) time-lapse monitoring with seismic methods; 5) advanced wellbore-monitoring methods; 6) deployment of an AIM monitoring network; 7) EM monitoring of CO2 with real-time data processing; 8) continued seasonal drone-based surveillance studies; 9) seismic monitoring with passive and active surveys; and 10) wellbore monitoring with nuclear magnetic resonance (NMR) for near-surface characterization. Completion of Activities 1.0–5.0 (baseline activities) are described in this report. Upon authorization of funding by DOE, the EERC will initiate Activities 6.0– 10.0 (operational monitoring activities). Current state-of-the-art (SOA) carbon storage-monitoring techniques require countless labor hours dedicated to the acquisition of data. Once data are gathered, these SOA techniques often rely on commercial facilities to process raw data from the field. However, it is anticipated that next-generation monitoring techniques, such as those being demonstrated, will lower acquisition footprints, be less operationally intensive, and improve data acquisition efficiencies. These new techniques are more conducive to the application of machine learning, artificial intelligence, and automation, thus providing a pathway for integration into active control systems, informing site operability, and improving the integration of data for future CCS projects across the United States. Additionally, reclaimed and active mining lands are present within the project site, creating a unique opportunity to demonstrate the effectiveness of remote sensing and surface-based geophysics monitoring techniques at similar project sites that may include disturbed, unconsolidated, or actively excavated near-surface environments. The efforts included in the overall project will produce necessary designs, learnings, and data acquired during the baseline and operational monitoring periods that are necessary for time-lapse demonstration and validation of the described monitoring techniques. In addition, it is anticipated that the monitoring technologies included in this study will be compliant with UIC Class VI requirements to enable the potential for implementation at other CCS sites across the United States.

42 ENGINEERING↗

Production Operations Management System

The Production Operations Management System (POMS) is a set of software tools which allows production teams and analysis groups across multiple Fermilab experiments to launch, modify and monitor large scale campaigns of related Monte Carlo or data processing jobs. POMS provides a web service interface that enables automated jobs submission on distributed resources according to customers’ requests and subsequent monitoring and recovery of failed submissions, debugging and record keeping. POMS interfaces with existing HEP data access, processing, movement and monitoring tools at Fermilab, including Jobsub, dCache, SAM, and FIFEmon, and in combination with them handles the creation and tracking of unique filenames for multistage campaigns. A flexible, interactive GUI interface exists as part of the system to visualize campaign stages, parallelization tracks and data flow. An important feature of POMS is a one-to-one connection between the GUI campaign visualizer and a text based representation of the complete campaign configuration. An extensible library of template campaign configurations is available. The templates are user modifiable and map cleanly to and from the GUI description of the campaign. Future releases of POMS will interface with Rucio for file movement.

97 MATHEMATICS AND COMPUTING↗

Applications of Emerging Parallel Optical Link Technology to High Energy Physics Experiments

Modern particle detectors depend upon optical fiber links to deliver event data to upstream trigger and data processing systems. Future detector systems can benefit from the development of dense arrangements of high speed optical links emerging from the telecommunications and storage area network market segments. These links support data transfers in each direction at rates up to 120 Gbps in packages that minimize or even eliminate edge connector requirements. Emerging products include a class of devices known as optical engines which permit assembly of the optical transceivers in close proximity to the electrical interfaces of ASICs and FPGAs which handle the data in parallel electrical format. Such assemblies will reduce required printed circuit board area and minimize electromagnetic interference and susceptibility. We will present test results of some of these parallel components and report on the development of pluggable FPGA Mezzanine Cards equipped with optical engines to provide to collaborators on the Versatile Link Common Project for the HI-LHC at CERN.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Modular Subsurface Sensors and Integrated Software for Advanced Subsurface Characterization and Monitoring using Unoccupied Vehicles

The advent and subsequent proliferation of autonomous airborne, waterborne, and groundbased vehicles (i.e., “drones”) promises to broadly transform the geosciences and associated industries, including fossil energy exploration and development, mineral resource exploration and development, water-resource management, and environmental remediation. For geophysical characterization and monitoring, the prospect of programming highly repeatable and low-cost drone missions for subsurface imaging will allow for deployments in hazardous and previously inaccessible areas. Coupled with autonomous workflows for data processing, management, and visualization, drone-based geophysical characterization and monitoring will enable unprecedented, real-time insight into diverse subsurface properties and processes of scientific and engineering importance. Toward this end, the objectives of this Lab Directed Research and Development (LDRD) project were to develop new (1) instrumentation for dronebased electromagnetic induction (EMI) geophysical imaging, including separated transmitter and receivers and associated electronics, (2) software for real-time data telemetry, processing, management, and visualization. Although EMI has been previously deployed using unoccupied aerial systems (UASs), these applications failed to capitalize on the game-changing capabilities of drone platforms. Whereas drone-based data acquisition allows for collection of rich, three-dimensional (3D) multi-offset/multi-angle configurations between transmitters and receivers, past efforts have relied on conventional instrumentation that was designed for ground-based data collection with the transmitter and a single receiver housed in the same unit; nor did these previous applications demonstrate real-time delivery of results to support rapid management decisions in the field. In this 1-year project, we (1) designed and constructed new lightweight independent transmitter and receiver antenna platforms that communicate with a laptop computer; (2) developed software to control data acquisition, manage/transfer data, and visualize data as its collected; and (3) demonstrated the operation of the new hardware and software systems in a ground-based field test. Our work entails major technological advances for EMI and established a foundation on which to build a new drone-based, real-time geophysical EMI imaging capability to support diverse challenges facing the nation.

47 OTHER INSTRUMENTATION↗

Fault Tolerance and Scaling in e-Science Cloud Applications: Observations from the Continuing Development of MODISAzure

It can be natural to believe that many of the traditional issues of scale have been eliminated or at least greatly reduced via cloud computing. That is, if one can create a seemingly wellfunctioning cloud application that operates correctly on small or moderate-sized problems, then the very nature of cloud programming abstractions means that the same application will run as well on potentially significantly larger problems. In this paper, we present our experiences taking MODISAzure, our satellite data processing system built on the Windows Azure cloud computing platform, from the proof-of-concept stage to a point of being able to run on significantly larger problem sizes (e.g., from national-scale data sizes to global-scale data sizes). To our knowledge, this is the longest-running eScience application on the nascent Windows Azure platform. We found that while many infrastructure-level issues were thankfully masked from us by the cloud infrastructure, it was valuable to design additional redundancy and fault-tolerance capabilities such as transparent idempotent task retry and logging to support debugging of user code encountering unanticipated data issues. Further, we found that using a commercial cloud means anticipating inconsistent performance and black-box behavior of virtualized compute instances, as well as leveraging changing platform capabilities over time. We believe that the experiences presented in this paper can help future eScience cloud application developers on Windows Azure and other commercial cloud providers.

97 MATHEMATICS AND COMPUTING↗

Invited Review Article: Pump-probe microscopy

Multiphoton microscopy has rapidly gained popularity in biomedical imaging and materials science because of its ability to provide three-dimensional images at high spatial and temporal resolution even in optically scattering environments. Currently the majority of commercial and home-built devices are based on two-photon fluorescence and harmonic generation contrast. These two contrast mechanisms are relatively easy to measure but can access only a limited range of endogenous targets. Recent developments in fast laser pulse generation, pulse shaping, and detection technology have made accessible a wide range of optical contrasts that utilize multiple pulses of different colors. Molecular excitation with multiple pulses offers a large number of adjustable parameters. For example, in two-pulse pump-probe microscopy, one can vary the wavelength of each excitation pulse, the detection wavelength, the timing between the excitation pulses, and the detection gating window after excitation. Such a large parameter space can provide much greater molecular specificity than existing single-color techniques and allow for structural and functional imaging without the need for exogenous dyes and labels, which might interfere with the system under study. In this review, we provide a tutorial overview, covering principles of pump-probe microscopy and experimental setup, challenges associated with signal detection and data processing, and an overview of applications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

AZIMUTHAL STRUCTURE OF THE Cyg X-2 X-RAY DUST HALO

There is little information concerning the azimuthal distribution of X-rays in dust-scattering halos. This paper describes a Chandra observation of the bright source Cyg X-2 designed specifically for this purpose. After measuring and subtracting Almost-Equal-To 10% instrument effects, we find the scattering halo to be rather uniform with possible fluctuations in the surface brightness of only 2%. Observations and data processing are discussed in detail. Some information about the dust distribution is derived.

79 ASTRONOMY AND ASTROPHYSICS↗

Exploring High-Dimensional Data Space: Identifying Optimal Process Conditions in Photovoltaics: Preprint

We demonstrate how advanced exploratory data analysis coupled to data-mining techniques can be used to scrutinize the high-dimensional data space of photovoltaics in the context of thin films of Al-doped ZnO (AZO), which are essential materials as a transparent conducting oxide (TCO) layer in CuInxGa1-xSe2 (CIGS) solar cells. AZO data space, wherein each sample is synthesized from a differentprocess history and assessed with various characterizations, is transformed, reorganized, and visualized in order to extract optimal process conditions. The data-analysis methods used include parallel coordinates, diffusion maps, and hierarchical agglomerative clustering algorithms combined with diffusion map embedding.

14 SOLAR ENERGY↗

High-count rate effects in event processing for XRISM/ Resolve X-ray microcalorimeter: I. Ground test

The spectroscopic performance of an X-ray microcalorimeter is compromised at high count rates. We utilize the Resolve X-ray microcalorimeter onboard the XRISM satellite to examine the effects observed during high-count rate measurements and propose modeling approaches to mitigate them. We specifically address the following instrumental effects that impact performance: CPU limit, pile-up, and untriggered electrical cross-talk. Experimental data at high count rates were acquired during ground testing using the flight model instrument and a calibration X-ray source. In the experiment, data processing not limited by the performance of the onboard CPU was run in parallel, which cannot be done in orbit. This makes it possible to access the data degradation caused by limited CPU performance. We use these data to develop models that allow for a more accurate estimation of the aforementioned effects. To illustrate the application of these models in observation planning, we present a simulated observation of GX 13+1. Understanding and addressing these issues is crucial to enhancing the reliability and precision of X-ray spectroscopy in situations characterized by elevated count rates.

47 OTHER INSTRUMENTATION↗

A method of building of decision trees based on data from wearable device during a rehabilitation of patients with tibia fractures

Nowadays technologies which are used in traumatology are a combination of mechanical, electronic, calculating and programming tools. Relevance of development of mobile applications for an expeditious data processing which are received from medical devices (in particular, wearable devices), and formulation of management decisions increases. Using of a mathematical method of building of decision trees for an assessment of a patient’s health condition using data from a wearable device considers in this article.

97 MATHEMATICS AND COMPUTING↗

Directional x-ray dark-field imaging of strongly ordered systems

Recently a novel grating based x-ray imaging approach called directional x-ray dark-field imaging was introduced. Directional x-ray dark-field imaging yields information about the local texture of structures smaller than the pixel size of the imaging system. In this work we extend the theoretical description and data processing schemes for directional dark-field imaging to strongly scattering systems, which could not be described previously. We develop a simple scattering model to account for these recent observations and subsequently demonstrate the model using experimental data. The experimental data includes directional dark-field images of polypropylene fibers and a human tooth slice.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Resonant diffraction gratings for spatial differentiation of optical beams

Diffraction of a two-dimensional optical beam from a resonant diffraction grating is considered. It is shown that at certain resonance parameters the diffraction grating allows for spatial differentiation and integration of the incident beam. The parameters of the diffraction grating for spatial differentiation of optical beams in the transmission geometry are calculated. It is shown that the differentiating diffraction grating allows the conversion of the two-dimensional beam into the two-dimensional Hermite – Gaussian mode. The presented results of numerical modelling are in good agreement with the proposed theoretical description. The use of the considered resonant diffraction gratings is promising for solving the problems of all-optical data processing. (laser applications and other topics in quantum electronics)

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Understanding the Hierarchy of Dense Subgraphs in Stationary and Temporally Varying Setting

Graphs are widely used to model relationships in a wide variety of domains such as sociology, bioinformatics, infrastructure, the WWW, to name a few. One of the key observations is that while real-world graphs are often globally sparse, they are locally dense. In other words, the average degree is often quite small (say at most 10 in a million vertex graph), but vertex neighborhoods are often dense. Finding dense subgraphs is a critical aspect of graph mining It has been used for finding communities and spam link farms in web graphs, graph visualization, real-time story identification, DNA motif detection in biological networks, finding correlated genes, epilepsy prediction, finding price value motifs in financial data, graph compression, distance query indexing, and increasing the throughput of social networking site servers. However, most standard formulations of this problem (like clique, quasi-clique, k-densest subgraph) are NP-hard. Furthermore, current dense subgraph finding algorithms usually optimize some objective, and only find a few such subgraphs without providing any structural relations, whereas the goal is rarely to find the "true optimum," but to identify many (if not all) dense substructures, understand their distribution in the graph, and ideally determine relationships among them. In this project, we first aim to devise algorithms and provide 3 implementations with nice visualizations to find the hierarchy between dense subgraphs, and then understand the structure of the hierarchy to gain more insight on the hidden patterns in real-world networks. Another important aspects in graph analysis is the temporal nature of networks. Networks evolve over time and in many applications data arrives at a high velocity, and thus it is important to design algorithms that can process data efficiently. We report three main results towards identifying dense structures in large evolving graphs. First, we will show how the hierarchical connectedness structure can be maintained efficiently, where connectedness is defined by increasing levels of connectivity strength. Next, we present dense structure can be identified in bipartite graphs without building projection graphs. And finally, we present a new method for peeling algorithms This new approach avoids sequential nature of peeling algorithms and is amenable to parallelization, which is crucial for processing high velocity data.

97 MATHEMATICS AND COMPUTING↗

Carbon dioxide, water vapor and methane soil efflux (soil respiration) in a Pinus palustris root exclusion in Georgetown, SC

This dataset contains processed data from a combination of survey flux chambers and long-term automated flux chambers. Soil flux measurements were conducted from June 2023 through December 2025 in a mature longleaf pine forest in Georgetown, SC. Soil respiration measurements were conducted approximately biweekly for two and a half years, before and after a root exclusion that took place on May 5, 2024. Processed, QAQC’d data for the treatment (root exclusion) and control (roots intact) before and after the root exclusion can be found in the file: 1_DATA_ESS_DOE_HR_RS_HB2_QAQC_Survey_Data_20260223.csv. Two multiday deployments were also conducted prior to the root exclusion using long-term automated chambers to continuously monitor greenhouse gas soil efflux. Processed, QAQC’d data for both long-term deployments can be found in the file: 2_DATA_ESS_DOE_HR_RS_HB2_QAQC_Longterm_Data_20260209.csv. Raw and working data files (.json, .81x, & .82z format) from LI-COR equipment are included for reference and can be accessed using SoilFluxPro software. CSV metadata files describe the raw data and modifications made using SoilFluxPro v5 and Matlab R2024b, as well as formatting and units for processed CSVs. Matlab code is included for reading in the processed CSVs, with sample figures comparing treatment and control. This research was performed as part of the project: “Improving models of stand and watershed carbon and water fluxes with more accurate representations of soil-plant-water dynamics in southern pine ecosystems”, which examines in part the effects hydraulic redistribution on soil efflux of carbon dioxide, water vapor and methane, as well as soil moisture and temperature in a southern pine ecosystem with sandy soils and high water table.

CARBON DIOXIDE FLUX↗

Measurement of fast-changing low velocities by photonic Doppler velocimetry

Despite the increasing popularity of photonic Doppler velocimetry (PDV) in shock wave experiments, its capability of capturing low particle velocities while changing rapidly is still questionable. The paper discusses the performance of short time Fourier transform (STFT) and continuous wavelet transform (CWT) in processing fringe signals of fast-changing low velocities measured by PDV. Two typical experiments are carried out to evaluate the performance. In the laser shock peening test, the CWT gives a better interpretation to the free surface velocity history, where the elastic precursor, main plastic wave, and elastic release wave can be clearly identified. The velocities of stress waves, Hugoniot elastic limit, and the amplitude of shock pressure induced by laser can be obtained from the measurement. In the Kolsky-bar based tests, both methods show validity of processing the longitudinal velocity signal of incident bar, whereas CWT improperly interprets the radial velocity of the shocked sample at the beginning period, indicating the sensitiveness of the CWT to the background noise. STFT is relatively robust in extracting waveforms of low signal-to-noise ratio. Data processing method greatly affects the temporal resolution and velocity resolution of a given fringe signal, usually CWT demonstrates a better local temporal resolution and velocity resolution, due to its adaptability to the local frequency, also due to the finer time-frequency product according to the uncertainty principle.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗