Results for “science data processing” — Page 98

DOE OSTI2012

CMS Workflow Execution using Intelligent Job Scheduling and Data Access Strategies

Complex scientific workflows can process large amounts of data using thousands of tasks. The turnaround times of these workflows are often affected by various latencies such as the resource discovery, scheduling and data access latencies for the individual workflow processes or actors. Minimizing these latencies will improve the overall execution time of a workflow and thus lead to a more efficient and robust processing environment. In this paper, we propose a pilot job based infrastructure that has intelligent data reuse and job execution strategies to minimize the scheduling, queuing, execution and data access latencies. The results have shown that significant improvements in the overall turnaround time of a workflow can be achieved with this approach. The proposed approach has been evaluated, first using the CMS Tier0 data processing workflow, and then simulating the workflows to evaluate its effectiveness in a controlled environment.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2016

SU-F-T-304: Complex Multi-PTV Treatment Evaluation Using a Remotely Processed 3D Gel Dosimeter

Purpose: A new 3D gel dosimeter (ClearView™, Modus Medical Systems) was investigated for use as a QA tool for stereotactic radiosurgery (SRS) plans exhibiting high dose gradients and spatially separated treatment targets. The unique feature of this gel dosimeter is the remote processing service provided by Modus Medical Systems. Methods: The gel dosimeters were filled in either 10 cm diameter or 15 cm diameter clear plastic jars. The jars were then shipped in ice-cooled containers to our department for irradiation. Clinical SRS plans for treatment of multiple metastases and plans with simulated concave structures were applied to a CT scan of the gel dosimeter. The gel was irradiated in treatment position using modulated arcs and then returned in the cooled container for processing. The 3D gel dose was compared to the DICOM-RT dose from the treatment plan to assess dosimetric and geometric agreement. Results: There was no discernible difference between the planned and measured dose for dose gradients as high as 10%/mm, which was the highest gradient we evaluated. Geometric agreement for distant metastases separated by 6 cm was within 1.5 mm. Among three identically irradiated gels using a plan intended for nine metastases, the 3%/3mm gamma passing rate was 84.5% with a range of 14.7%, measured over the entire volume of the dosimeter. Regions of larger gamma values correlated with geometric offsets between the planned and measured data. Conclusion: The gel dosimeter exhibits the dosimetric and geometric characteristics necessary for 3D evaluation of treatment plan deliverability. The range of observed gamma passing rates suggests a high sensitivity to geometric registration. With proper management of geometric registration between planned and measured data, this service should enable a radiation oncology department to use 3D dosimetry in end-to-end testing or patient plan delivery QA without the expense of an in-house processing system.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Microbes and Climate Change: a Research Prospectus for the Future

Climate change is the most serious challenge facing humanity. Microbes produce and consume three major greenhouse gases—carbon dioxide, methane, and nitrous oxide—and some microbes cause human, animal, and plant diseases that can be exacerbated by climate change. Hence, microbial research is needed to help ameliorate the warming trajectory and cascading effects resulting from heat, drought, and severe storms. We present a brief summary of what is known about microbial responses to climate change in three major ecosystems: terrestrial, ocean, and urban. We also offer suggestions for new research directions to reduce microbial greenhouse gases and mitigate the pathogenic impacts of microbes. These include performing more controlled studies on the climate impact on microbial processes, system interdependencies, and responses to human interventions, using microbes and their carbon and nitrogen transformations for useful stable products, improving microbial process data for climate models, and taking the One Health approach to study microbes and climate change.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

REDI – Readiness Engine for Data Integration

The Readiness Engine for Data Integration (REDI) is an open-source framework for automating, standardizing, and assessing the process of preparing scientific data for AI training. REDI implements a five-stage pipeline (ingest, preprocess, transform, structure, output) with per-stage provenance instrumentation via Flowcept, domain-aware transformation logic (PII anonymization, regridding, graph encoding, and more), and built-in readiness assessment and validation modes. REDI has been evaluated across climate, proteomics, materials science, and nuclear fusion datasets, demonstrating near-ideal parallel scaling to 100 nodes on OLCF's Frontier system. REDI is deployable as an agent-callable skill in coding environments such as Claude Code and OpenAI Codex, and is complemented by SetGo for FAIR compliance and catalog publication.

Brewer, Wesley [Oak Ridge National Laboratory (ORN↗

DOE OSTI2021

Data processing methods and data acquisition for samples larger than the field of view in parallel-beam tomography

Parallel-beam tomography systems at synchrotron facilities have limited field of view (FOV) determined by the available beam size and detector system coverage. Scanning the full size of samples bigger than the FOV requires various data acquisition schemes such as grid scan, 360-degree scan with offset center-of-rotation (COR), helical scan, or combinations of these schemes. Though straightforward to implement, these scanning techniques have not often been used due to the lack of software and methods to process such types of data in an easy and automated fashion. The ease of use and automation is critical at synchrotron facilities where using visual inspection in data processing steps such as image stitching, COR determination, or helical data conversion is impractical due to the large size of datasets. Here, we provide methods and their implementations in a Python package, named Algotom, for not only processing such data types but also with the highest quality possible. The efficiency and ease of use of these tools can help to extend applications of parallel-beam tomography systems.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Blodgett Forest CA Warming Experiment Soil Gas Well Data March 2014 to June 2017

These are soil gas data from the whole soil profile warming experiment at the University of California Blodgett Forest Research Station, in the Sierra Nevada foothills near Georgetown, CA at 1370 m above sea level. Mean annual precipitation is 1774 mm with most of it occurring from November through April and mean annual temperature is about 12.5°C . The experiment is in a thinned 80-year-old stand of mixed conifers including ponderosa pine (Pinus ponderosa), sugar pine (Pinus lambertiana), incense cedar (Calodefrus decurrens), white fir (Abies concolor), and douglas fir (Pseudotsuga menziesii). The soils are Holland series: fine-loamy, mixed, superactive, mesic Ultic Haploxeralfs of granitic origin with thick, >5 cm O horizons, minimal carbonates , and a pH that ranges from 5.6 to 6.5. The warming experiment warms the top meter of the soil profile to +4°C above ambient. These data include depth-resolved measurements of CO2 concentrations, 13CO2, apparent respiration quotients, soil temperature, and soil moisture (VWC). These data can be used to model CO2 production rates across the soil profile or to track how the sources and processes contributing to soil respiration change seasonally and with warming.

13C↗

DOE OSTI2015

Campaign 1.7 Pu Aging. Development of Time of Flight Secondary Ion Mass Spectroscopy

The first application of Time-of-Flight Secondary Ion Mass Spectroscopy (ToF-SIMS) to an aged plutonium surface has resulted in a rich set of surface chemistry data, as well as some unexpected results. FY15 was highlighted by not only the first mapping of hydrogen-containing features within the metal, but also a prove-in series of experiments using the system’s Sieverts Reaction Cell. These experiments involved successfully heating the sample to ~450 oC for nearly 24 hours while the sample was dosed several times with hydrogen, followed by an in situ ToF-SIMS analysis. During this year, the data allowed for better and more consistent identification of the myriad peaks that result from the SIMS sputter process. In collaboration with the AWE (U.K), the system was also fully aligned for sputter depth profiling for future experiments.

36 MATERIALS SCIENCE↗

DOE OSTI2021

How AI Predicts the Untrained and Unseen

Focus Area: Model predictability improvements (Primary); Data optimization (secondary); Data complexity insights (secondary). The Scientific Challenge: If we believe that a future under extreme conditions will look very differently from today, we can likely agree that ML/AI models trained on past and present datasets will not be adequate to make reliable predictions into the future. This is true for water cycling, as well as biogeochemistry and other Earth system components and behaviors. Additionally, ML/AI models are inherently non-physical. Despite the flourishing success of ML/AI in many applications, such as computer vision, natural language process, and gaming, even the most sophisticated AI models don’t understand the very basic physical laws. Therefore, a natural question is: Can we trust ML/AI based predictions of Earth system behaviors that are fundamentally driven by physical laws? So, are physics models with meticulous process representation a better choice? Not exactly. Physical models, when firmly rooted in first principles, work great at predicting behaviors of systems with a well-defined set of boundary conditions and variables. However, as a complex system, the number of parameters and the degree of complexity and dynamics in processes, coupling, and scale dependent emergent behaviors make the Earth system behaviors very challenging to predict with physical models composed of deterministic laws. In addition, due to the lack of fundamental understandings, physical models often implement empirical correlations derived from observations with biases from locality of data generation. Because correlation is not necessarily causation or comply with first principles, scaling of model predictions beyond locality is often invalid. Beyond the limitation of models, physics or AI, our knowledge of the Earth system is limited by the lack of observational technologies and resources. Insufficient data density, dimensionality and diversity only offer a sliced (or projected) view of the Earth system, e.g. Plato’s Cave analogy, limiting our capability to better understand and represent fundamental processes in models.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2016

Ka-Band Scanning ARM Cloud Radar (a0)

The Ka-Band Scanning ARM Cloud Radar (KASACR) records cloud properties. ARM's scanning cloud radars are dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (1.7 kW peak power) or an X-band (20 kW peak power). Beamwidths for Ka-bands paired with W-bands are roughly matched at 0.3 degrees. The X-band beamwidth is approximately 1 degree. Due to the narrow antenna beamwidth, ARM’s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the KASACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. Measurements collected with the KASACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, and linear depolarization ratio. KASACR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2012

Ka ARM Zenith Radar (a0)

The Ka-band ARM Zenith Radar (KAZR) remotely probes the extent and composition of clouds at millimeter wavelengths. The KAZR is a zenith-pointing Doppler radar that operates at a frequency of approximately 35 GHz. The main purpose of this radar is to determine the first three Doppler moments (reflectivity, vertical velocity, and spectral width) at a range resolution of approximately 30 meters from near-ground to nearly 20 kilometers in altitude. The KAZR replaces the millimeter-wavelength cloud radar (MMCR) and uses a new digital receiver that provides higher spatial and temporal resolution than the MMCR. In addition, spectral artifacts in the data are significantly reduced in the KAZR, allowing researchers to study cloud dynamics much more closely than with the MMCR. KAZR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Ka-Band Scanning ARM Cloud Radar (a1)

The Ka-Band Scanning ARM Cloud Radar (KASACR) records cloud properties. ARM's scanning cloud radars are dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (1.7 kW peak power) or an X-band (20 kW peak power). Beamwidths for Ka-bands paired with W-bands are roughly matched at 0.3 degrees. The X-band beamwidth is approximately 1 degree. Due to the narrow antenna beamwidth, ARM’s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the KASACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. Measurements collected with the KASACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, and linear depolarization ratio. KASACR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Ka-Band Scanning ARM Cloud Radar (a1)

The Ka-Band Scanning ARM Cloud Radar (KASACR) records cloud properties. ARM's scanning cloud radars are dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (1.7 kW peak power) or an X-band (20 kW peak power). Beamwidths for Ka-bands paired with W-bands are roughly matched at 0.3 degrees. The X-band beamwidth is approximately 1 degree. Due to the narrow antenna beamwidth, ARM’s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the KASACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. Measurements collected with the KASACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, and linear depolarization ratio. KASACR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2011

GROUNDWATER MONITORING REPORT GENERATION TOOLS - 12005

Compliance with National and State environmental regulations (e.g. Resource Conservation and Recovery Act (RCRA) and Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) aka SuperFund) requires Savannah River Site (SRS) to extensively collect and report groundwater monitoring data, with potential fines for missed reporting deadlines. Several utilities have been developed at SRS to facilitate production of the regulatory reports which include maps, data tables, charts and statistics. Components of each report are generated in accordance with complex sets of regulatory requirements specific to each site monitored. SRS developed a relational database to incorporate the detailed reporting rules with the groundwater data, and created a set of automation tools to interface with the information and generate the report components. These process improvements enhanced quality and consistency by centralizing the information, and have reduced manpower and production time through automated efficiencies.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2014

Accelerating Data Acquisition, Reduction, and Analysis at the Spallation Neutron Source

ORNL operates the world's brightest neutron source, the Spallation Neutron Source (SNS). Funded by the US DOE Office of Basic Energy Science, this national user facility hosts hundreds of scientists from around the world, providing a platform to enable break-through research in materials science, sustainable energy, and basic science. While the SNS provides scientists with advanced experimental instruments, the deluge of data generated from these instruments represents both a big data challenge and a big data opportunity. For example, instruments at the SNS can now generate multiple millions of neutron events per second providing unprecedented experiment fidelity but leaving the user with a dataset that cannot be processed and analyzed in a timely fashion using legacy techniques. To address this big data challenge, ORNL has developed a near real-time streaming data reduction and analysis infrastructure. The Accelerating Data Acquisition, Reduction, and Analysis (ADARA) system provides a live streaming data infrastructure based on a high-performance publish subscribe system, in situ data reduction, visualization, and analysis tools, and integration with a high-performance computing and data storage infrastructure. ADARA allows users of the SNS instruments to analyze their experiment as it is run and make changes to the experiment in real-time and visualize the results of these changes. In this paper we describe ADARA, provide a high-level architectural overview of the system, and present a set of use-cases and real-world demonstrations of the technology.

Campbell, Stuart I↗

DOE OSTI2015

Radiometric characterization process for locating radioactivity hold-up and measuring non-uniformly distributed sources - 15342

A novel approach to facility radiometric characterization for non-uniformly distributed radiation sources has been developed by ANTECH and is currently being employed at the Argonne National Laboratory. It is being applied to the measurement of a wide range of objects and the interiors and contents of laboratories and buildings. The integrated non destructive assay measurement process involves the use of a scanning spectroscopic gamma ray camera as well as high purity Germanium detectors with different collimators and hence differing detector fields of view to measure and quantify non-uniformly distributed sources of radioactivity in laboratory equipment, glove boxes, storage tanks, pipework and general laboratory objects and process components. The purpose of the radiometric characterization is to quantify radiation sources and radioactive contamination levels in facilities containing nuclear material and to ensure compliance with facility characterization requirements and for decontamination and decommissioning planning and execution. The integrated measurement process also involves automated data capture and processing software and a database with which both measurement instrument operators and subject matter expert analysts can interact. An appropriate measuring instrument (gamma camera, scintillation detector, high purity Germanium detector system, neutron detector or alpha-beta counter) records the measurement data and the measurement site is recorded in a photograph. All of the data relevant to the measurement (or set of measurements for a particular object) are recorded in the database including measured gamma ray spectra, equipment description and relevant calibration, measurement site photograph and measurement descriptive data. Once data analysis protocols are established for specific geometries and measurement situations they can be applied to the measured data in a partially automated manner, reducing analysis and transcription errors. In addition to measurements and data analysis the process also utilises radiometric modelling computer codes including MicroShield and Monte Carlo Neutrons Photons in order to perform model calculations where appropriate. In a typical measurement, the ANTECH RadSearch spectroscopic gamma camera is utilized to find areas of concentrated activity so that these areas can be measured and calculated separately from areas with diffuse contamination. The use of the gamma camera in scanning mode helps to ensure that the assay does not overestimate the detected radiation source, which is a frequent consequence of using standard far field geometry and attenuation analysis codes that assume uniformity of both density and radioactive source distribution. Once the hot spots are identified, high purity Germanium detector measurements are made using various collimators with wide or narrow fields of view as required. These measurements are then processed through a semi-automatic calculation process assuming an appropriate source configuration as identified by RadSearch. These source configurations include point sources, line sources or surface sources. The Generalized Geometry Hold-up algorithms are employed to determine an estimate of the activity of the various sources that are identified in a measurement. The ANTECH integrated measurement process provides an automated calculation procedure where the analyst can select the source type, geometry and appropriate input data, and then the software calculates the activity by radionuclide for all sources identified in the measurement. The measurement and analysis process is controlled under the ANTECH ISO9001: 2008 Quality Assurance System and managed by a site specific Quality Assurance Plan. Both Argonne National Laboratory and Department of Energy technical reviewers have vetted the Integrated Measurement and Analysis Process. The application of the measurement and analysis process is described and measurement examples are provided. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

Designing an Optimal Sensor Network via Minimizing Information Loss

Optimal experimental design is a classic topic in statistics, with many well-studied problems, applications, and solutions. The design problem we study is the placement of sensors to monitor spatiotemporal processes, explicitly accounting for the temporal dimension in our modeling and optimization. We observe that recent advancements in computational sciences often yield large datasets based on physics-based simulations, which are rarely leveraged in experimental design. We introduce a novel model-based sensor placement criterion, along with a highly-efficient optimization algorithm, which integrates physics-based simulations and Bayesian experimental design principles to identify sensor networks that “minimize information loss” from simulated data. Our technique relies on sparse variational inference and (separable) Gauss-Markov priors, and thus may adapt many techniques from Bayesian experimental design. We validate our method through a case study monitoring air temperature in Phoenix, Arizona, using state-of-the-art physics-based simulations. Our results show our framework to be superior to random or quasi-random sampling, particularly with a limited number of sensors. We conclude by discussing practical considerations and implications of our framework, including more complex modeling tools and real-world deployments.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2012

Glow-to-arc transition events in H{sub 2}-Ar direct current pulsed plasma: Automated measurement of current and voltage

The glow-to-arc transition phenomena (arcing) observed in plasma reactors used in materials processing was studied through the arcs characteristic current and voltage waveforms. In order to capture these arcs signals, a LABVIEW based automated instrumentation system (ARCVIEW) was developed, including the integration of an oscilloscope equipped with proper current and voltage probes. The system also allows capturing the process parameters at the arc occurrence moments, which were used to map the arcs events conditions. Experiments in H{sub 2}-Ar DC pulsed plasma returned signals data from 215 arcs events, which were analyzed through software routines. According to the results, an anti-arcing system should react in the time order of few microseconds to prevent most of the damage caused by the undesired arcing phenomena.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2015

Autoantibodies in dilated cardiomyopathy induce vascular endothelial growth factor expression in cardiomyocytes

Background: Autoantibodies have been identified as major predisposing factors for dilated cardiomyopathy (DCM). Patients with DCM show elevated serum levels of vascular endothelial growth factor (VEGF) whose source is unknown. Besides its well-investigated effects on angiogenesis, evidence is present that VEGF signaling is additionally involved in fibroblast proliferation and cardiomyocyte hypertrophy, hence in cardiac remodeling. Whether autoimmune effects in DCM impact cardiac VEGF signaling needs to be elucidated. Methods: Five DCM patients were treated by the immunoadsorption (IA) therapy on five consecutive days. The eluents from the IA columns were collected and prepared for cell culture. Cardiomyocytes from neonatal rats (NRCM) were incubated with increasing DCM-immunoglobulin-G (IgG) concentrations for 48 h. Polyclonal IgG (Venimmun N), which was used to restore IgG plasma levels in DCM patients after the IA therapy was additionally used for control cell culture purposes. Results: Elevated serum levels of VEGF decreased significantly after IA (Serum VEGF (ng/ml); DCM pre-IA: 45 ± 9.1 vs. DCM post–IA: 29 ± 6.7; P < 0.05). In cell culture, pretreatment of NRCM by DCM-IgG induced VEGF expression in a time and dose dependent manner. Biologically active VEGF that was secreted by NRCM significantly increased BNP mRNA levels in control cardiomyocytes and induced cell-proliferation of cultured cardiac fibroblast (Fibroblast proliferation; NRCM medium/HC-IgG: 1 ± 0.0 vs. NRCM medium/DCM-IgG 100 ng/ml: 5.6 ± 0.9; P < 0.05). Conclusion: The present study extends the knowledge about the possible link between autoimmune signaling in DCM and VEGF induction. Whether this observation plays a considerable role in cardiac remodeling during DCM development needs to be further elucidated. - Highlights: • Mechanisms of remodeling in dilated cardiomyopathy (DCM) are not fully understood. • Autoantibodies have been identified as major predisposing factors for DCM. • DCM patients show high serum levels of VEGF. • Recent data indicate that VEGF is involved in cardiac remodeling processes. • Whether autoimmune processes in DCM are involved in VEGF signaling are unclear.

60 APPLIED LIFE SCIENCES↗