Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Long-term hydrological, biogeochemical, and climatological data from Walker Branch Watershed, East Tennessee, USA

In 1967, the original Walker Branch Watershed (WBW) project was established to study elemental cycling and mass balances in a relatively unimpacted watershed. Over the next 50+ years, findings from additional experimental studies and long-term observations on WBW advanced understanding of catchment hydrology, biogeochemistry, and ecology and established WBW as a seminal site for catchment science. The 97.5-ha WBW is located in East Tennessee, USA, on the U.S. Department of Energy's Oak Ridge Reservation. Vegetation on the watershed is characteristic of an eastern deciduous, second-growth forest. The watershed is divided into two subcatchments: the West Fork (38.4 ha) and the East Fork (59.1 ha). Headwater streams draining these subcatchments are fed by multiple springs, and thus flow is perennial. Stream water is high in base cations due to weathering of dolomite bedrock and nutrient concentrations are low. Long-term observations of climate, hydrology, and biogeochemistry include daily (1969–2014) and 15-min (1994–2014) stream discharge and annual runoff (1969–2014); hourly, daily, and annual rainfall (1969–2012); daily climate and soil temperature (1993–2010); and weekly stream water chemistry (1989–2013). These long-term datasets are publicly available on the WBW website (https://walkerbranch.ornl.gov/long-term-data/). While collection of these data has ceased, related long-term measurements continue through the National Ecological Observatory Network (NEON), where WBW is the core terrestrial and aquatic site in the Appalachian and Cumberland Plateau region (NEON's Domain 7) of the United States. These long-term datasets have been and will continue to be important in evaluating the influence of climatic and environmental drivers on catchment processes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Reproducibility of protein x-ray diffuse scattering and potential utility for modeling atomic displacement parameters

Protein structure and dynamics can be probed using x-ray crystallography. Whereas the Bragg peaks are only sensitive to the average unit-cell electron density, the signal between the Bragg peaks—diffuse scattering—is sensitive to spatial correlations in electron-density variations. Although diffuse scattering contains valuable information about protein dynamics, the diffuse signal is more difficult to isolate from the background compared to the Bragg signal, and the reproducibility of diffuse signal is not yet well understood. We present a systematic study of the reproducibility of diffuse scattering from isocyanide hydratase in three different protein forms. Both replicate diffuse datasets and datasets obtained from different mutants were similar in pairwise comparisons (Pearson correlation coefficient ≥0.8). The data were processed in a manner inspired by previously published methods using custom software with modular design, enabling us to perform an analysis of various data processing choices to determine how to obtain the highest quality data as assessed using unbiased measures of symmetry and reproducibility. The diffuse data were then used to characterize atomic mobility using a liquid-like motions (LLM) model. This characterization was able to discriminate between distinct anisotropic atomic displacement parameter (ADP) models arising from different anisotropic scaling choices that agreed comparably with the Bragg data. Our results emphasize the importance of data reproducibility as a model-free measure of diffuse data quality, illustrate the ability of LLM analysis of diffuse scattering to select among alternative ADP models, and offer insights into the design of successful diffuse scattering experiments.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Big PanDa Workflow Management on Titan for High Energy and Nuclear Physics and for Future Extreme Scale Scientific Application

Over a three year period, from 2016-2019, this project demonstrated the scientific benefits of integrating the Titan supercomputer at Oak Ridge Leadership Computing Facility into traditional high throughput grid based distributed computing systems managed by PanDA, the workflow management system used for the execution of all distributed computing applications by the ATLAS experiment at the Large Hadron Collider. PanDA manages millions of batch jobs daily at hundreds of clusters worldwide on request by thousands of physicist users, and processes more than an exabyte of data annually using grid middleware. High levels of operational use of Titan was sustained by PanDA in order to meet the physics goals of ATLAS. The success of this project led to the use of other supercomputers worldwide by ATLAS, and to the adoption of PanDA by other experiments and other scientists. Multiple innovative operational and computer science research goals were achieved supporting the use of supercomputers for scientific domains with large scale distributed data and distributed processing needs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Toward a Multimodal-Deep Learning Retrieval System for Monitoring Nuclear Proliferation Activities

As the amount of available data increases, the human ability to locate, process, and analyze it is strained and eventually overwhelmed. To address this challenge for nonproliferation analysts, the authors have been creating a large-scale multimodal retrieval system to help analysts triage and search open source science, technology, and news data for indicators of nuclear proliferation capabilities and activities. The system relies on a set of deep neural networks (DNNs) trained to evaluate conceptual similarities across data modalities, such as text, image, and video. These DNNs can be used to search and prioritize data, according to a nuclear fuel cycle (NFC) process template, that are conceptually closest to the seed query items regardless of data modality. The DNNs have been trained to map conceptually related words, sentences, and images to nearby points in a multimodal feature space, enabling intra- and intermodal retrieval via nearest-neighbor calculations to seed query points. The authors evaluate the system’s ability to retrieve NFC-related data that have been purposely hidden in collections of unrelated background data. They demonstrate quantitative and qualitative results for text-to-image, image-to-image, and image-to-video retrievals. This paper discusses data challenges confronting nonproliferation analysts, describes the DNNs designed to retrieve multimodal data that are proximal in the semantic space, and demonstrates their effectiveness by applying the DNNs to nonproliferation-specific multimodal data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Building near-real-time processing pipelines with the spark-MPI platform

Advances in detectors and computational technologies provide new opportunities for applied research and the fundamental sciences. Concurrently, dramatic increases in the three V's (Volume, Velocity, and Variety) of experimental data and the scale of computational tasks produced the demand for new real-time processing systems at experimental facilities. Recently, this demand was addressed by the Spark-MPI approach connecting the Spark data-intensive platform with the MPI high-performance framework. In contrast with existing data management and analytics systems, Spark introduced a new middleware based on resilient distributed datasets (RDDs), which decoupled various data sources from high-level processing algorithms. The RDD middleware significantly advanced the scope of data-intensive applications, spreading from SQL queries to machine learning to graph processing. Spark-MPI further extended the Spark ecosystem with the MPI applications using the Process Management Interface. The paper explores this integrated platform within the context of online ptychographic and tomographic reconstruction pipelines.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Characterizing Canopy Openness in Open Forests: Spherical Densiometer and Canopy Photography Are Equivalent but Less Sensitive than Direct Measurements of Solar Radiation

In forest ecosystems, canopy openness affects understory light availability, plant growth, and tree species recruitment, thus shaping future forest composition, structure, and functional diversity. Foresters must correctly and quickly measure canopy openness to meet their management objectives. To help guide the selection of an appropriate method for measuring canopy openness, we compared three common techniques that vary in cost, complexity, and time required for measurements and data processing: smartphone-based hemispherical photography, spherical densiometer measurements, and direct measurements of solar radiation (using AccuPAR ceptometer). We measured canopy openness using these three methods on 28 permanent forest health monitoring plots in pine-oak forests of the Central Pine Barrens of Long Island in New York State. By analysis of variance and regression analyses, we found the three methods (particularly densiometer and hemispherical photographs) yielded broadly equivalent and strongly positively correlated descriptions of canopy openness. The direct measurements of solar radiation seemed to have a greater potential to detect subtle variation in forest understory light. Forest managers may sufficiently characterize canopy openness using quick and cheap methods (e.g., spherical densiometers) and avoid larger costs of devices for direct light measurements (e.g., ceptometers) and the larger data processing times of hemispherical photography.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Continued Data Acquisition Development

This task focused on improving techniques for integrating data acquisition of secondary particles correlated in time with detected cosmic-ray muons. Scintillation detectors with Pulse Shape Discrimination (PSD) capability show the most promise as a detector technology based on work in FY13. Typically PSD parameters are determined prior to an experiment and the results are based on these parameters. By saving data in list mode, including the fully digitized waveform, any experiment can effectively be replayed to adjust PSD and other parameters for the best data capture. List mode requires time synchronization of two independent data acquisitions (DAQ) systems: the muon tracker and the particle detector system. Techniques to synchronize these systems were studied. Two basic techniques were identified: real time mode and sequential mode. Real time mode is the preferred system but has proven to be a significant challenge since two FPGA systems with different clocking parameters must be synchronized. Sequential processing is expected to work with virtually any DAQ but requires more post processing to extract the data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

SAIL-Net CloudPuck CCN Data

SAIL-Net is a DOE funded project in the East River Watershed near Crested Butte, Colorado with the goal of advancing our understanding of aerosol-cloud interactions in complex, mountainous regions. Through the deployment of a network of six low cost microphysics nodes in Fall 2021 in the same domain at the SAIL campaign, SAIL-Net provides data on aerosol size distributions, cloud condensation nuclei (CCN), and ice nucleations particles (INP). This network enables the investigation of small-scale variations in complex terrain. This specific dataset provides the cleaned data recorded from the CloudPuck, an in house instrument made by Handix Scientific that counts CCN concentrations. The CloudPuck was deployed at the sites for the summer/fall of 2022 before winter conditions were too harsh to maintain the instrument. For more information on the the CloudPuck or how the raw data are processed, see the read me.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Interdisciplinary Research Collaborative Trains Students to See Through Turbulent Systems

Despite the availability of interdisciplinary academic training programs, the practice of environmental science is often hampered by a lack of convergence across diverse disciplines. This gap is particularly salient in settings characterized by complex environmental issues, such as multiple-use coastal ecosystems. In response, we developed and implemented a training, research, and communication framework to provide undergraduates with an authentic operative experience working at the interface of interdisciplinary science and public decision-making within a case study of marine renewable energy. In our program, students gained hands-on experience with the scientific process and learned how to make information relevant, useful, and accessible to diverse stakeholder groups. Application of this framework demonstrates that the process of integrating data from biological (visual and acoustic monitoring of fish and marine mammals), physical (hydrodynamics), and social (local ecological knowledge) sciences can provide a more complete understanding of complex and turbulent ecosystems for better informed decision-making. We offer several recommendations to facilitate the adaptation and implementation of our interdisciplinary framework to diverse research contexts, with a focus on interdisciplinary training for the next generation of marine scientists.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Data Transferability and Collection Consistency in Marine Renewable Energy

Concerns about the potential effects of tidal turbines and wave energy devices on the marine environment continue to slow siting and permitting/consenting of single devices and arrays worldwide. While research studies and early results from post-installation monitoring over the past decade have informed interactions between marine renewable energy (MRE) devices, marine animals, and habitats, regulators still demonstrate considerable reluctance to accelerate the permitting/consenting process for devices and arrays. Furthermore, the MRE industry is struggling with the high costs of baseline assessments and post-installation monitoring, as well as long timelines for obtaining permits, which leads to uncertainty and risk related to project financing. Regulators require assessment and monitoring information to allow them to carry out the necessary analyses to describe, permit/consent, and manage the environmental risks associated with new MRE technologies and new uses of ocean space. One way to reduce risks to the industry and the environment and to allow for acceleration of the permitting/consenting process could be to transfer learning, analyses, and data sets from one country to another, among projects, and across jurisdictional boundaries. In addition, data are collected around early-stage MRE devices using many different methods, instruments, and measurement scales. If similar parameters and accessible methods of data collection were used for baseline assessments and post-installation monitoring around all early-stage devices and MRE developments, the results would be more readily comparable. This comparability would lead to a decrease in scientific uncertainty and support a common understanding of the risk of MRE devices to the marine environment. This in turn would facilitate more efficient and shorter permitting/consenting processes, which would decrease the financial risk for MRE project development. As a means of addressing the concept of transferring data (information, learning, analyses, and data sets) among projects and collecting data consistently, Annex IV has developed a data transferability process that has been socialized with the MRE community, which includes regulators, industry, developers, consultants, and researchers. The data transferability process consists of five components: 1. A Data Transferability Framework brings together data sets in an organized fashion, compares the applicability of each data set for use on other projects, and guides the process of data transfer 2. A Data Collection Consistency Table provides preferred measurement methods or processes, reporting units, and the most common methods of analysis or interpretation and use of data 3. A Monitoring Data Sets Discoverability Matrix allows a practitioner to discover data sets based on the approach presented in the Framework 4. Best Management Practices (BMPs) include five BMPs related to data transferability and collection consistency 5. An Implementation Plan presents an approach for implementing and applying the data transferability process. This report documents the background and development of the data transferability process and associated components and summarizes the next steps needed to successfully implement and apply the data transferability process.

16 TIDAL AND WAVE POWER↗
DOE OSTI2021

PNNL Soil Microbiome SFA

PNNL's Soil Microbiome Science Focus Area (SFA) is focused on understanding the basic biology underpinning how interactions among various soil microbial community members, across trophic levels, lead to the emergence of community functions. Moisture, in particular, drives microbial interactions and influences everything from cell function to substrate fate within soils. The group predicts this results in repeatable, predictable phenotypes. The sum of these phenotypes comprises the “soil metaphenome”. Understanding how the soil metaphenome shifts in response to moisture will provide a basis for modeling and predicting these shifts in reaction network responses. PNNL’s Soil Microbiome SFA project repository on DataHub allows for exploring and downloading integrated experimental omics data, experimental metadata, pre- and post-processed data files, and other associated materials directly related to experimental project publication data. Visit the PNNL Soil Microbiome Science Focus Area Program homepage for more information.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

BisQue for 3D Materials Science in the Cloud: Microstructure–Property Linkages

Accelerating the design and development of new advanced materials is one of the priorities in modern materials science. These efforts are critically dependent on the development of comprehensive materials cyber-infrastructures which enable effcient data storage, management, sharing, and collaboration as well as integration of computational tools that help establish processing{structure{property relationships. In this contribution, we present implementation of such computational tools into a cloud-based platform called BisQue (Kvilekval et al., 2010). We first describe the current state of BisQue as an open-source platform for multidisciplinary research in the cloud and its potential for 3D materials science. We then demonstrate how new computational tools, primarily aimed at processing{structure{property relationships, can be implemented into the system. Speci cally, in this work, we develop a module for BisQue that enables microstructure-sensitive predictions of effective yield strength of two-phase materials. Towards this end, we present an implementation of a computationally effcient data-driven model into the BisQue platform. The new module is made available online and can be used from a web browser without any special software and with minimal computational requirements on the user end. The capabilities of the module for rapid property screening are demonstrated in case studies with two different methodologies based on datasets containing 3D microstructure information from (i) synthetic generation and (ii) sampling large 3D volumes obtained in experiments.

36 MATERIALS SCIENCE↗
DOE OSTI2014

Experimental comparison of grating- and propagation-based hard X-ray phase tomography of soft tissue

When imaging soft tissues with hard X-rays, phase contrast is often preferred over conventional attenuation contrast due its superior sensitivity. However, it is unclear which of the numerous phase tomography methods yields the optimized results at given experimental conditions. Therefore, we quantitatively compared the three phase tomography methods implemented at the beamline ID19 of the European Synchrotron Radiation Facility: X-ray grating interferometry (XGI), and propagation-based phase tomography, i.e., single-distance phase retrieval (SDPR) and holotomography (HT), using cancerous tissue from a mouse model and an entire heart of a rat. We show that for both specimens, the spatial resolution derived from the characteristic morphological features is about a factor of two better for HT and SDPR compared to XGI, whereas the XGI data generally exhibit much better contrast-to-noise ratios for the anatomical features. Moreover, XGI excels in fidelity of the density measurements, and is also more robust against low-frequency artifacts than HT, but it might suffer from phase-wrapping artifacts. Thus, we can regard the three phase tomography methods discussed as complementary. The application will decide which spatial and density resolutions are desired, for the imaging task and dose requirements, and, in addition, the applicant must choose between the complexity of the experimental setup and the one of data processing.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Ptychography at all wavelengths

Ptychography is a computational imaging technique that operates across multiple wavelength regimes, from electron (picometres) to X-ray (~0.1 nm), extreme ultraviolet (~10 nm) and visible light (micrometres). By reconstructing both amplitude and phase from diffraction patterns, ptychography enables high-resolution, quantitative imaging without conventional limitations imposed by lens-based optics. Ptychography has enabled advances across a range of scales: achieving deep-sub-angstrom resolution with electron microscopy, becoming an indispensable tool at X-ray synchrotron facilities worldwide and overcoming the trade-offs between resolution and field-of-view in optical imaging. This Primer provides a unified treatment of ptychography across these wavelength regimes. First, we discuss theoretical foundations, reconstruction algorithms, experimental considerations and wavelength-specific challenges. We then give examples of raw and processed data from various configurations and wavelengths. Next, we highlight key applications of ptychography in life sciences, materials science and industry. We also discuss data standards, open-source software implementations and best practices for ensuring reproducibility across different wavelength regimes. Finally, we consider limitations and future opportunities for ptychography. Together with accompanying datasets and code implementations, this Primer aims to serve newcomers and experienced practitioners in the field, facilitating broader adoption of ptychography across different disciplines.

47 OTHER INSTRUMENTATION↗
DOE OSTI2016

Recording Vavilov – Cherenkov radiation in a linear accelerator using a picosecond streak camera

Using a picosecond image converter camera with a linear sweep (PS-1/S1 streak camera developed at GPI RAS, Moscow), we have measured temporal parameters of Vavilov – Cherenkov radiation pulses. The radiation was generated by relativistic electrons passing through a quartz cone mounted on the axis of a vacuum chamber of a linear accelerator, which is a part of the VEPP-5 injection complex at the Budker Institute of Nuclear Physics, Siberian Branch of the Russian Academy of Sciences (BINP SB RAS, Novosibirsk). The data obtained in these experiments provide an insight into the processes of formation of electron bunches and their ‘quality’ in a linear accelerator prior to injection of electrons into the accumulator-cooler. A conclusion is made regarding the advisability of streak camera application in tuning the linear accelerators for optimisation of electron bunch parameters. (Vavilov – Cherenkov radiation)

43 PARTICLE ACCELERATORS↗
DOE OSTI2017

The Hyper Suprime-Cam software pipeline

Here in this article, we describe the optical imaging data processing pipeline developed for the Subaru Telescope’s Hyper Suprime-Cam (HSC) instrument. The HSC Pipeline builds on the prototype pipeline being developed by the Large Synoptic Survey Telescope’s Data Management system, adding customizations for HSC, large-scale processing capabilities, and novel algorithms that have since been reincorporated into the LSST codebase. While designed primarily to reduce HSC Subaru Strategic Program (SSP) data, it is also the recommended pipeline for reducing general-observer HSC data. The HSC pipeline includes high-level processing steps that generate coadded images and science-ready catalogs as well as low-level detrending and image characterizations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Customs: Inspect and Report Python Imports (customs) v1.0.0

We have developed a simple, minimally invasive monitoring framework that enables us to capture key metrics from Python user processes. This is part of a larger effort to collect data across our entire data-intensive science workload at NERSC. The approach for Python is inspired by a similar framework developed for Blue Waters at NCSA. Both frameworks leverage standard Python features, sitecustomize and atexit, to capture Python imports of interest and other job data. The customs package decomposes the information capture into separate inspection and reporting interfaces.

Thomas, Rollin↗