Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Standardized phylogenetic and molecular evolutionary analysis applied to species across the microbial tree of life

There is growing interest in reconstructing phylogenies from the copious amounts of genome sequencing projects that target related viral, bacterial or eukaryotic organisms. To facilitate the construction of standardized and robust phylogenies for disparate types of projects, we have developed a complete bioinformatic workflow, with a web-based component to perform phylogenetic and molecular evolutionary (PhaME) analysis from sequencing reads, draft assemblies or completed genomes of closely related organisms. Furthermore, the ability to incorporate raw data, including some metagenomic samples containing a target organism (e.g. from clinical samples with suspected infectious agents), shows promise for the rapid phylogenetic characterization of organisms within complex samples without the need for prior assembly.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Multimodal X-ray nano-spectromicroscopy analysis of chemically heterogeneous systems

Abstract Understanding the nanoscale chemical speciation of heterogeneous systems in their native environment is critical for several disciplines such as life and environmental sciences, biogeochemistry, and materials science. Synchrotron-based X-ray spectromicroscopy tools are widely used to understand the chemistry and morphology of complex material systems owing to their high penetration depth and sensitivity. The multidimensional (4D+) structure of spectromicroscopy data poses visualization and data-reduction challenges. This paper reports the strategies for the visualization and analysis of spectromicroscopy data. We created a new graphical user interface and data analysis platform named XMIDAS (X-ray multimodal image data analysis software) to visualize spectromicroscopy data from both image and spectrum representations. The interactive data analysis toolkit combined conventional analysis methods with well-established machine learning classification algorithms (e.g. nonnegative matrix factorization) for data reduction. The data visualization and analysis methodologies were then defined and optimized using a model particle aggregate with known chemical composition. Nanoprobe-based X-ray fluorescence (nano-XRF) and X-ray absorption near edge structure (nano-XANES) spectromicroscopy techniques were used to probe elemental and chemical state information of the aggregate sample. We illustrated the complete chemical speciation methodology of the model particle by using XMIDAS. Next, we demonstrated the application of this approach in detecting and characterizing nanoparticles associated with alveolar macrophages. Our multimodal approach combining nano-XRF, nano-XANES, and differential phase-contrast imaging efficiently visualizes the chemistry of localized nanostructure with the morphology. We believe that the optimized data-reduction strategies and tool development will facilitate the analysis of complex biological and environmental samples using X-ray spectromicroscopy techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Mass and galaxy distributions of four massive galaxy clusters from Dark Energy Survey Science Verification data

We measure the weak lensing masses and galaxy distributions of four massive galaxy clusters observed during the Science Verification phase of the Dark Energy Survey (DES). This pathfinder study is meant to (1) validate the Dark Energy Camera (DECam) imager for the task of measuring weak lensing shapes, and (2) utilize DECam's large field of view to map out the clusters and their environments over 90 arcmin. We conduct a series of rigorous tests on astrometry, photometry, image quality, point spread function (PSF) modelling, and shear measurement accuracy to single out flaws in the data and also to identify the optimal data processing steps and parameters. We find Science Verification data from DECam to be suitable for the lensing analysis described in this paper. The PSF is generally well behaved, but the modelling is rendered difficult by a flux-dependent PSF width and ellipticity. We employ photometric redshifts to distinguish between foreground and background galaxies, and a red-sequence cluster finder to provide cluster richness estimates and cluster-galaxy distributions. By fitting Navarro-Frenk-White profiles to the clusters in this study, we determine weak lensing masses that are in agreement with previous work. For Abell 3261, we provide the first estimates of redshift, weak lensing mass, and richness. In addition, the cluster-galaxy distributions indicate the presence of filamentary structures attached to 1E 0657-56 and RXC J2248.7-4431, stretching out as far as 1. (approximately 20 Mpc), showcasing the potential of DECam and DES for detailed studies of degree-scale features on the sky.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Protein abundances can distinguish between naturally-occurring and laboratory strains of Yersinia pestis , the causative agent of plague

Adaptive processes in bacterial species can occur rapidly in laboratory culture, leading to genetic divergence between naturally occurring and laboratory-adapted strains. Differentiating wild and closely-related laboratory strains is clearly important for biodefense and bioforensics; however, DNA sequence data alone has thus far not provided a clear signature, perhaps due to lack of understanding of how diverse genome changes lead to adapted phenotypes. Protein abundance profiles from mass spectrometry-based proteomics analyses are a molecular measure of phenotype. Proteomics data contains sufficient information that powerful statistical methods can uncover signatures that distinguish wild strains of Yersinia pestis from laboratory-adapted strains.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

ExaFEL: extreme-scale real-time data processing for X-ray free electron laser science

ExaFEL is an HPC-capable X-ray Free Electron Laser (XFEL) data analysis software suite for both Serial Femtosecond Crystallography (SFX) and Single Particle Imaging (SPI) developed in collaboration with the Linac Coherent Lightsource (LCLS), Lawrence Berkeley National Laboratory (LBNL) and Los Alamos National Laboratory. ExaFEL supports real-time data analysis via a cross-facility workflow spanning LCLS and HPC centers such as NERSC and OLCF. Our work therefore constitutes initial path-finding for the US Department of Energy's (DOE) Integrated Research Infrastructure (IRI) program. We present the ExaFEL team's 7 years of experience in developing real-time XFEL data analysis software for the DOE's exascale supercomputers. We present our experiences and lessons learned with the Perlmutter and Frontier supercomputers. Furthermore we outline essential data center services (and the implications for institutional policy) required for real-time data analysis. Finally we summarize our software and performance engineering approaches and our experiences with NERSC's Perlmutter and OLCF's Frontier systems. This work is intended to be a practical blueprint for similar efforts in integrating exascale compute resources into other cross-facility workflows.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Application of Data Science and Engineering

Metal additive manufacturing (AM) processes exhibit significant variability in the quality and properties of components that are produced. This variability has prevented the widespread adoption of AM in industry. The need for more advanced and descriptive process monitoring, part qualification, and process control has led to an increasing number of sensors on machines and subsequent data to analyze. Increasingly, data science principles are being leveraged in each of these domains in order to process this data and better understand the causes of variability and the corresponding quality inconsistencies that occur in additive manufacturing.

Halsey, William↗
DOE OSTI2020

Data-driven materials research enabled by natural language processing and information extraction

Given the emergence of data science and machine learning throughout all aspects of society, but particularly in the scientific domain, there is increased importance placed on obtaining data. Data in materials science are particularly heterogeneous, based on the significant range in materials classes that are explored and the variety of materials properties that are of interest. This leads to data that range many orders of magnitude, and these data may manifest as numerical text or image-based information, which requires quantitative interpretation. The ability to automatically consume and codify the scientific literature across domains - enabled by techniques adapted from the field of natural language processing - therefore has immense potential to unlock and generate the rich datasets necessary for data science and machine learning. This review focuses on the progress and practices of natural language processing and text mining of materials science literature and highlights opportunities for extracting additional information beyond text contained in figures and tables in articles. Here, we discuss and provide examples for several reasons for the pursuit of natural language processing for materials, including data compilation, hypothesis development, and understanding the trends within and across fields. Current and emerging natural language processing methods along with their applications to materials science are detailed. We, then, discuss natural language processing and data challenges within the materials science domain where future directions may prove valuable.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Hyper Suprime-Cam Legacy Archive

Abstract We present the launch of the Hyper Suprime-Cam Legacy Archive (HSCLA), a public archive of processed, science-ready data from the Hyper Suprime-Cam (HSC). HSC is an optical wide-field imager installed at the prime focus of the Subaru Telescope and has been in operation since 2014. While ∼1/3 of the total observing time of HSC has been used for the Subaru Strategic Program (SSP), the remainder of the time is used for Principal Investigator (PI)-based programs. We have processed the data from these PI-based programs and make the processed, high-quality data available to the community through HSCLA. The current version of HSCLA includes data taken in the first year of science operation, 2014. We provide both individual and coadd images as well as photometric catalogs. The photometric catalog from the coadd is loaded to the database, which offers a fast access to the large catalog. There are other online tools such as an image browser and an image cutout tool and they will be useful for science analyses. The coadd images reach 24–27th magnitudes at 5σ for point sources and cover approximately 580 square degrees in at least one filter with 150 million objects in total. We perform extensive quality assurance tests and verify that the photometric and astrometric quality of the data is good enough for most scientific explorations. However, the data are not without problems and users are referred to the list of known issues before exploiting the data for science. All the data and documentations can be found at the data release site, 〈https://hscla.mtk.nao.ac.jp/〉.

Tanaka, Masayuki↗
DOE OSTI2022

An Overview in the Development of a Multi-Sensor Data Science System for Monitoring a Solvent Extraction Process

To enhance nuclear nonproliferation stewardship, Idaho National Laboratory is building the Beartooth nuclear fuel cycle processing test bed. The Beartooth test bed will allow researchers the opportunity to study nuclear fuel processing operations including the use of centrifugal contactors in solvent extraction processes. The test bed is designed to support data collection and machine learning to monitor process operations. As part of this project, researchers will study data collected from a host of sensors that have not been typically used to monitor solvent extraction processes such as vibration, acoustic, colormetric, and thermal measurement data. The goal of this research is to employ machine learning and data analytics to study the confluence of signals collected from both traditionally and non-traditionally used sensors to provide operator process awareness and discover process anomalies. An overview of planned sensors and experiments that will focus on signal discovery will be presented.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

An Overview in the Development of a Multi-Sensor Data Science System for Monitoring a Solvent Extraction Process

To enhance nuclear nonproliferation stewardship, Idaho National Laboratory is building the Beartooth nuclear fuel cycle processing test bed. The Beartooth test bed will allow researchers the opportunity to study nuclear fuel processing operations including the use of centrifugal contactors in solvent extraction processes. The test bed is designed to support data collection and machine learning to monitor process operations. As part of this project, researchers will study data collected from a host of sensors that have not been typically used to monitor solvent extraction processes such as vibration, acoustic, colormetric, and thermal measurement data. The goal of this research is to employ machine learning and data analytics to study the confluence of signals collected from both traditionally and non-traditionally used sensors to provide operator process awareness and discover process anomalies. An overview of planned sensors and experiments that will focus on signal discovery will be presented.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Development of a Multi Sensor Data Science System for Monitoring a Solvent Extraction Process

Idaho National Laboratory is designing and constructing a solvent extraction test bed, Beartooth, as part of a nuclear fuel cycle stewardship initiative. The test bed will provide facilities and equipment for testing novel extraction processes and give early career scientists opportunities to gain skills in performing separations chemistry. The test bed is being uniquely designed to enable machine learning capabilities for the characterization of chemical process operations. One of the intentions of incorporating machine learning methods is to provide process operators with enhanced situational awareness for the optimization of separations techniques and possible detection of a diversion event. As a result, the objective of this Laboratory Directed Research and Development project is to develop a multi-sensor data collections system and implement machine learning techniques on signals from an existing centrifugal contactor cascade to identify equipment usage and process events. This presentation will provide an overview of sensors used and experiments conducted to date.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Preliminary Results of a Multi-Sensor Data Science System for Monitoring a Solvent Extraction Process

Idaho National Laboratory is building a test bed to allow researchers the opportunity to study nuclear fuel processing operations. This includes studying the solvent extraction process and the use of centrifugal contactors. The goal of this project is to develop a system that utilizes non-traditional measurement sources such as vibration, acoustics, current, color, flow, and temperature in conjunction with data-based, machine learning techniques that will allow for signal discovery. This multi-sensor data supports the development of safeguards by design, provides operator process awareness, and aids in the discovery of process anomalies. This paper highlights some of the preliminary results from initial data collection campaigns and shares some of the lessons learned.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

Data Challenges in Multi-Sensor Data Science System for Monitoring a Solvent Extraction Process

Idaho National Laboratory (INL) is maintaining and gaining knowledge into the nuclear fuel cycle by building a test bed to allow researchers the opportunity to study nuclear fuel processing operations. This includes studying solvent extraction processes that use centrifugal contactors. As part of INL’s mission, the goal of this project is to develop a system that utilizes non-traditional measurement sources such as vibration, acoustics, current, light, flow, and temperature in conjunction with data-based, machine learning techniques that will allow for signal discovery. This multisensory data can support the development of safeguards by design, provide operator process awareness, and discover process anomalies. This poster will highlight some of the data collection and analytics challenges for the multi-sensor system as well as the mitigation strategies to build a robust system. Additionally, some preliminary data from the first testing campaign will be shown to help illustrate the data needs of the system.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2024

Multi-Attribute Subset Selection enables prediction of representative phenotypes across microbial populations

The interpretation of complex biological datasets requires the identification of representative variables that describe the data without critical information loss. This is particularly important in the analysis of large phenotypic datasets (phenomics). Here we introduce Multi-Attribute Subset Selection (MASS), an algorithm which separates a matrix of phenotypes (e.g., yield across microbial species and environmental conditions) into predictor and response sets of conditions. Using mixed integer linear programming, MASS expresses the response conditions as a linear combination of the predictor conditions, while simultaneously searching for the optimally descriptive set of predictors. We apply the algorithm to three microbial datasets and identify environmental conditions that predict phenotypes under other conditions, providing biologically interpretable axes for strain discrimination. MASS could be used to reduce the number of experiments needed to identify species or to map their metabolic capabilities. The generality of the algorithm allows addressing subset selection problems in areas beyond biology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

The LCLStream Ecosystem for Multi-Institutional Dataset Exploration

We describe a new end-to-end experimental data streaming framework designed from the ground up to support new types of applications – AI training, extremely high-rate X-ray time-of-flight analysis, crystal structure determination with distributed processing, and custom data science applications and visualizers yet to be created. Throughout, we use design choices merging cloud microservices with traditional HPC batch execution models for security and flexibility. This project makes a unique contribution to the DOE Integrated Research Infrastructure (IRI) landscape. By creating a flexible, API-driven data request service, we address a significant need for high-speed data streaming sources for the X-ray science data analysis community. With the combination of data request API, mutual authentication web security framework, job queue system, high-rate data buffer, and complementary nature to facility infrastructure, the LCLStreamer framework has prototyped and implemented several new paradigms critical for future generation experiments.

Rogers, David [ORNL] (ORCID:0000000251871768)↗