Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

ARM Aerosol Measurement Science Group 2019 Strategic Planning Workshop Report

This report summarizes the results of a U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility workshop held in November 2019 to advance a science-based strategy for ARM’s aerosol measurement program. This was the second such workshop since the Aerosol Measurement Science Group (AMSG) was chartered in 2015 to enhance coordination of ARM observations of aerosols and atmospheric trace gases with the needs of ARM users. The results presented here reflect the AMSG’s focus in recent years on science-based strategies that will contribute to the increased use of ARM data to fulfill its mission of improving process representations and predictability in climate models. Sessions held during the workshop range from interfacing with models through aerosol sampling strategies to calibration protocols and data products. The strategies set forth here were also developed to be directly relevant to ARM’s updated Decadal Vision. The AMSG workshops have been designed to recommend actions that will enable ARM to evolve and continue to meet its mission. To that end, the AMSG will develop an actionable plan from the recommendations outlined here. Some are well defined and can reasonably be accomplished in the short term. Others are less definite or of a larger scope that calls for a longer-term implementation. Further discussion will be required to develop and prioritize actionable items related to such areas. Task teams comprising the appropriate expertise and perspective from the AMSG and other members of the community will be formed to achieve this outcome. Some particular topics are recognized as high priority, so plans are underway to develop task teams and to hold follow-on discussions to address them. Four areas currently being considered for short, focused discussion are 1) aerosol measurements on the North Slope of Alaska, 2) improving data usability for modeling, 3) strategies for advancing remote sensing, vertical profiling, and distributed measurements of aerosols, and 4) aerosol sampling strategies at existing ARM sites to provide intensive modes of operation to promote data usage for process and modeling studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

ARM Aerosol Measurement Science Group 2019 Strategic Planning Workshop Report

This report summarizes the results of a U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility workshop held in November 2019 to advance a science-based strategy for ARM’s aerosol measurement program. This was the second such workshop since the Aerosol Measurement Science Group (AMSG) was chartered in 2015 to enhance coordination of ARM observations of aerosols and atmospheric trace gases with the needs of ARM users. The results presented here reflect the AMSG’s focus in recent years on science-based strategies that will contribute to the increased use of ARM data to fulfill its mission of improving process representations and predictability in climate models. Sessions held during the workshop range from interfacing with models through aerosol sampling strategies to calibration protocols and data products. The AMSG workshops have been designed to recommend actions that will enable ARM to evolve and continue to meet its mission. To that end, the AMSG will develop an actionable plan from the recommendations outlined here. Some are well defined and can reasonably be accomplished in the short term. Others are less definite or of a larger scope that calls for a longer-term implementation. Further discussion will be required to develop and prioritize actionable items related to such areas. Task teams comprising the appropriate expertise and perspective from the AMSG and other members of the community will be formed to achieve this outcome. Some particular topics are recognized as high priority, so plans are underway to develop task teams and to hold follow-on discussions to address them. Four areas currently being considered for short, focused discussion are 1) aerosol measurements on the North Slope of Alaska, 2) improving data usability for modeling, 3) strategies for advancing remote sensing, vertical profiling, and distributed measurements of aerosols, and 4) aerosol sampling strategies at existing ARM sites to provide intensive modes of operation to promote data usage for process and modeling studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Pattern-matching indexing of Laue and monochromatic serial crystallography data for applications in materials science

Serial crystallography data can be challenging to index, as each frame is processed individually, rather than being processed as a whole like in conventional X-ray single-crystal crystallography. An algorithm has been developed to index still diffraction patterns arising from small-unit-cell samples. The algorithm is based on the matching of reciprocal-lattice vector pairs, as developed for Laue microdiffraction data indexing, combined with three-dimensional pattern matching using a nearest-neighbors approach. As a result, large-bandpass data (e.g. 5–24 keV energy range) and monochromatic data can be processed, the main requirement being prior knowledge of the unit cell. Furthermore, angles calculated in the vicinity of a few theoretical and experimental reciprocal-lattice vectors are compared, and only vectors with the highest number of common angles are selected as candidates to obtain the orientation matrix. Global matching on the entire pattern is then checked. Four indexing options are available, two for the ranking of the theoretical reciprocal-lattice vectors and two for reducing the number of possible candidates. Furthermore, the algorithm has been used to index several data sets collected under different experimental conditions on a series of model samples. Additionally, knowing the crystallographic structure of the sample and using this information to rank the theoretical reflections based on the structure factors helps the indexing of large-bandpass data for the largest-unit-cell samples. For small-bandpass data, shortening the candidate list to determine the orientation matrix should be based on matching pairs of reciprocal-lattice vectors instead of triplet matching.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Compactly‐Supported Nonstationary Kernels for Computing Exact Gaussian Processes on Big Data

The Gaussian process (GP) is a widely used method for analyzing large-scale data sets, including spatio-temporal measurements of nonlinear processes that are now commonplace in the environmental sciences. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility, and exact methods for inference that prevent application to data sets with more than about 10,000 points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space–time prediction based on more than 1 million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

Gaussian processes↗
DOE OSTI2019

High-Resolution Mass Spectrometry Quantification: Impact of Differences in Data Processing of Centroid and Continuum Data

High-resolution mass spectrometry (HRMS) in full scan mode acquires all ions present in the sample of interest offering a lot of qualitative information. This, in combination with the improved performance of the new generation HRMS systems, triggers more (bio) analysts to switch from triple quad MS systems to HRMS for quantitative analysis. Quantitative processing of HRMS data is performed based on narrow mass extraction windows rather than on nominal mass product ion chromatograms (SRM or MRM). Optimal processing of HRMS data requires different considerations and software tools and can have an impact on data processing and final results. The selection of centroid versus continuum/profile data for processing, selection of the optimal narrow mass extraction window, using theoretical versus measured accurate mass for the extraction of the ion chromatograms as well as differences in calculations and data handling residing in the different vendor software packages are tackled in the presented manuscript. These differences are illustrated on HRMS data acquired for the same plasma samples on three different platforms, i.e., a Sciex QToF, a Waters QToF, and a Thermo Orbitrap system, and processed in four different software packages, i.e., Sciex Analyst® TF, Waters Masslynx, Waters Unifi, and Thermo Xcalibur. The impact of these differences on quantitative HRMS performance was evaluated on calibration curves of eight small molecule compounds in plasma using four different ways of processing. Simple guidelines are provided for the selection of an optimal mass extraction window for continuum and centroided data. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2017

A Full Mesh ATCA-based General Purpose Data Processing Board (Pulsar II)

The Pulsar II is a custom ATCA full mesh enabled FPGA-based processor board which has been designed with the goal of creating a scalable architecture abundant in flexible, non-blocking, high bandwidth interconnections. The design has been motivated by silicon-based tracking trigger needs for LHC experiments. In this technical memo we describe the Pulsar II hardware and its performance, such as the performance test results with full mesh backplanes from different vendors, how the backplane is used for the development of low-latency time-multiplexed data transfer schemes and how the inter-shelf and intra-shelf synchronization works.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2017

Scaling up to address data science challenges

Statistics and Data Science provide a variety of perspectives and technical approaches for exploring and understanding Big Data. Partnerships between scientists from different fields such as statistics, machine learning, computer science, and applied mathematics can lead to innovative approaches for addressing problems involving increasingly large amounts of data in a rigorous and effective manner that takes advantage of advances in computing. Here, this article will explore various challenges in Data Science and will highlight statistical approaches that can facilitate analysis of large-scale data including sampling and data reduction methods, techniques for effective analysis and visualization of large-scale simulations, and algorithms and procedures for efficient processing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Directional gamma sensing from covariance processing of inter-detector Compton crosstalk energy asymmetries

Energy asymmetry of inter-detector crosstalk from Compton scattering can be exploited to infer the direction to a gamma source. A covariance approach extracts the correlated crosstalk from data streams to estimate matched signals from Compton gammas split over two detectors. On a covariance map the signal appears as an asymmetric cross diagonal band with axes intercepts at the full photo-peak energy of the original gamma. The asymmetry of the crosstalk band can be processed to determine the direction to the radiation source. The technique does not require detector shadowing, masking, or coded apertures, thus sensitivity is not sacrificed to obtain the directional information. An angular precision of better than 1° of arc is possible, and processing of data streams can be done in real time with very modest computing hardware.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2015

SU-E-T-11: A Cloud Based CT and LINAC QA Data Management System

Purpose: The current status quo of QA data management consists of a mixture of paper-based forms and spreadsheets for recording the results of daily, monthly, and yearly QA tests for both CT scanners and LINACs. Unfortunately, such systems suffer from a host of problems as, (1) records can be easily lost or destroyed, (2) data is difficult to access — one must physically hunt down records, (3) poor or no means of historical data analysis, and (4) no remote monitoring of machine performance off-site. To address these issues, a cloud based QA data management system was developed and implemented. Methods: A responsive tablet interface that optimizes clinic workflow with an easy-to-navigate interface accessible from any web browser was implemented in HTML/javascript/CSS to allow user mobility when entering QA data. Automated image QA was performed using a phantom QA kit developed in Python that is applicable to any phantom and is currently being used with the Gammex ACR, Las Vegas, Leeds, and Catphan phantoms for performing automated CT, MV, kV, and CBCT QAs, respectively. A Python based resource management system was used to distribute and manage intensive CPU tasks such as QA phantom image analysis or LaTeX-to-PDF QA report generation to independent process threads or different servers such that website performance is not affected. Results: To date the cloud QA system has performed approximately 185 QA procedures. Approximately 200 QA parameters are being actively tracked by the system on a monthly basis. Electronic access to historical QA parameter information was successful in proactively identifying a Linac CBCT scanner’s performance degradation. Conclusion: A fully comprehensive cloud based QA data management system was successfully implemented for the first time. Potential machine performance issues were proactively identified that would have been otherwise missed by a paper or spreadsheet based QA system.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2014

ARM Data Integrator

The Atmospheric Radiation Measurement (ARM) Data Integrator (ADI) streamlines the development of scientific algorithms and analysis of time-series NetCDF data, and improves the content and consistency of the output data products produced by these algorithms. The framework automates the process of retrieving and preparing data for analysis, and allows users to design output data products through a graphical interface. It also provides a modular, flexible software development architecture that scientists can use to generate C, Python, and IDL source code templates that embed the pre and post processing logic allowing the scientist to focus on only their science. The input data, preprocessing, and output data specifications of algorithms are defined through a graphical interface and stored in a database. ADI implements workflow for data integration and supports user access to data through a library of software modules. Data preprocess capabilities supported include automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog and allows developers to use existing data product to build new data products. Views of the output data products include an overlay of how the design meets ARM archive's data standards providing the user with a visual cue indicating where their output violates an archive standard. The ADI libraries access the information provided through the GUI via a Postgres database. The ADI framework and its supporting components can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Source record↗
DOE OSTI2014

ARM Data Integrator

The Atmospheric Radiation Measurement (ARM) Data Integrator (ADI) streamlines the development of scientific algorithms and analysis of time-series NetCDF data, and improves the content and consistency of the output data products produced by these algorithms. The framework automates the process of retrieving and preparing data for analysis, and allows users to design output data products through a graphical interface. It also provides a modular, flexible software development architecture that scientists can use to generate C, Python, and IDL source code templates that embed the pre and post processing logic allowing the scientist to focus on only their science. The input data, preprocessing, and output data specifications of algorithms are defined through a graphical interface and stored in a database. ADI implements workflow for data integration and supports user access to data through a library of software modules. Data preprocess capabilities supported include automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog and allows developers to use existing data product to build new data products. Views of the output data products include an overlay of how the design meets ARM archive's data standards providing the user with a visual cue indicating where their output violates an archive standard. The ADI libraries access the information provided through the GUI via a Postgres database. The ADI framework and its supporting components can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Source record↗
DOE OSTI2021

NOAA National Centers for Environmental Information Fisheries Acoustics Archive Network Deep Dive

EPOC uses the Deep Dive process to discuss and analyze current and planned science use cases and anticipated data output of a particular use case, site, or project to help inform the strategic planning of a campus or regional networking environment. This includes understanding future needs related to network operations, network capacity upgrades, and other technological service investments. A Deep Dive comprehensively surveys major research stakeholders’ plans and processes in order to investigate data management requirements over the next 5–10 years. Deep Dives help ensure that key stakeholders have a common understanding of the issues and the actions that a campus or regional network may need to undertake to offer solutions. The EPOC team leads the effort and relies on collaboration with the hosting site or network, and other affiliated entities that participate in the process. EPOC organizes, convenes, executes, and shares the outcomes of the review with all stakeholders. Between May 2021 and August 2021, staff members from the Engagement and Performance Operations Center (EPOC) met with researchers and staff from the National Oceanic and Atmospheric Administration (NOAA)'s N-Wave (the Enterprise network that supports the NOAA mission) and National Centers for Environmental Information (NCEI)'s Fisheries Acoustics Archive for the purpose of a recording a Deep Dive into research drivers. The goal of these meetings was to help characterize the requirements for the research use case, and to enable cyberinfrastructure support staff to better understand the needs of the researchers they support.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Removal of spurious data in Bragg coherent diffraction imaging: an algorithm for automated data preprocessing

Bragg coherent diffraction imaging (BCDI) provides a powerful tool for obtaining high-resolution structural information from nanocrystalline materials. Here a BCDI sample consisting of a large number of randomly oriented nanoscale crystals is considered. Ideally, only one crystal is oriented to produce a Bragg peak on the detector. However, diffraction from other crystals often produces additional signals on the detector. Before the measured diffraction patterns can be processed into structural images, scientists routinely need to manually identify and remove the `alien' intensities from sources other than the intended crystal. With the development of modern high-coherence storage rings, such as the upgraded Advanced Photon Source (APS), the already slow process of manual preprocessing will be untenable for the large volumes of data that will be produced. An automated method of identifying and deleting alien intensities is proposed. This method exploits the fact that BCDI of a perfect crystal produces diffraction data with inversion symmetry around the Bragg peak. This approach uses the machine learning clustering method DBSCAN to distinguish between diffraction from multiple sources, and then calculates cluster size and inversion symmetry to assess whether clusters of intensity belong to desired data or alien signals. This approach can dramatically reduce the amount of time spent manually processing data, allowing BCDI data processing capabilities to keep pace with the technological advances of fourth-generation synchrotron light sources.

36 MATERIALS SCIENCE↗
DOE OSTI2021

University of Central Florida Campus-Wide Deep Dive

EPOC uses the Deep Dive process to discuss and analyze current and planned science use cases and anticipated data output of a particular use case, site, or project to help inform the strategic planning of a campus or regional networking environment. This includes understanding future needs related to network operations, network capacity upgrades, and other technological service investments. A Deep Dive comprehensively surveys major research stakeholders’ plans and processes in order to investigate data management requirements over the next 5–10 years. Deep Dives help ensure that key stakeholders have a common understanding of the issues and the actions that a campus or regional network may need to undertake to offer solutions. The EPOC team leads the effort and relies on collaboration with the hosting site or network, and other affiliated entities that participate in the process. EPOC organizes, convenes, executes, and shares the outcomes of the review with all stakeholders. Between December 2020 and August 2021, staff members from the Engagement and Performance Operations Center (EPOC) met with researchers and staff at University of Central Florida (UCF) for the purpose of a Campus-Wide Deep Dive into research drivers. The goal of this meeting was to help characterize the requirements for five campus research use cases and to enable cyberinfrastructure support staff to better understand the needs of the researchers they support.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Accelerating manufacturing for biomass conversion via integrated process and bench digitalization: a perspective

We present a perspective for accelerating biomass manufacturing via digitalization. We summarize the challenges for manufacturing and identify areas where digitalization can help. A profound potential in using lignocellulosic biomass and renewable feedstocks, in general, is to produce new molecules and products with unmatched properties that have no analog in traditional refineries. Discovering such performance-advantaged molecules and the paths and processes to make them rapidly and systematically can transform manufacturing practices. Furthermore, we discuss retrosynthetic approaches, text mining, natural language processing, and modern machine learning methods to enable digitalization. Laboratory and multiscale computation automation via active learning are crucial to complement existing literature and expedite discovery and valuable data collection without a human in the loop. Such data can help process simulation and optimization select the most promising processes and molecules according to economic, environmental, and societal metrics. We propose the close integration between bench and process scale models and data to exploit the low dimensionality of the data and transform the manufacturing for renewable feedstocks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Advanced Interactive 3D Visualization Tool for Customizable Analyses of Tomography Datasets in Material Science

Current methods for visualizing and analyzing 3D tomography datasets in materials science often lack the interactivity and depth required for detailed structural insights. This limitation restricts a researchers' ability to accurately interpret complex data, which is critical for advancing material innovations and understanding structural properties. To address this issue, we have developed a novel, web-based interactive 3D visualization and analysis tool from the Trame framework that offers customizable features to enhance data interpretability. The tool allows users to adjust parameters such as visible range, slice planes, data rotation, and layering, providing a more detailed and dynamic view of complex structures. Its user-friendly web interface increases the accessibility and ease of use for both novice and experienced researchers, to visualize large volumetric datasets. The tool supports a diverse range of data formats, making it versatile for various research applications. Unique capabilities include real-time data manipulation, automated feature detection, context-sensitive feedback, and real-time volume calculations and distributions per sliced region or layer, alongside the ability to quickly generate high-quality screenshots and videos for presentations and reports. These advancements offer a comprehensive solution for enhanced 3D data exploration, significantly improving the analysis process and communication of results in materials science.

36 - MATERIALS SCIENCE↗
DOE OSTI2024

Building a framework to genetically characterize “feather spots” and understand demographic impacts of solar energy sites on migratory bird populations

The lack of data on the impact of utility-scale solar facilities on avian species and populations adds to the cost of siting and operation. As much as 32 percent of the avian biological material (feathers and carcasses) recovered from solar facilities remain unidentified, because they often take the form of “feather spots”. Feather spots are remains of impacted animals that can be separated into two broad categories: 1) those remains that may be visually identified to a species, or 2) those that cannot be visually identified to a species due to degradation from the environment and/or scavenger activity (listed as “unknown”). Even when feather spots can be identified to species, they cannot be visually assigned to particular breeding populations. In some cases, it is unknown whether multiple feather spots represent single or multiple individuals. This project’s objectives were to: 1. Use a developed, genetic-based technique to identify and determine the species, population of origin, and number of individuals found in feather spots recovered from solar facilities. 2. Implement collected data and resulting analyses to develop a publicly accessible web-based decision-making tool that can be used by the solar industry, regulators and other stakeholders to inform siting, mitigation, and conservation management efforts. 3. Establish a not-for-profit fee-for-service center at UCLA to ensure collection and identification of feather spots continue after the project period of performance. During the Project Period, we proposed to establish a pipeline for collecting, transporting, and storing of avian biological material collected at solar facilities and the collection and identification of feather spots to species and individual. We proposed the development of a genetic-based framework that would recover viable DNA from feather spots, amplify this DNA (i.e., make millions of copies of the original DNA), and use it to match the resulting sequences to a national database of known species of birds. The result would be the identification of feathers spots that were previously unidentified, and the incorporation of these samples into a larger database that included all samples recovered from solar facilities. The resulting report (below) details the result of this work and its alignment with proposed activities. We proposed the use of the data collected to assess the comparative risk to specific species or populations of species from solar facilities. For some species, we have already identified genomic markers of specific breeding populations and developed “genoscapes,” maps of unique genetic variation across the full breeding range of a species. We used these (previously and newly developed) genoscapes to probabilistically link a feather spot to the specific breeding populations from which it originated (assignment probabilities range from 75%-100% depending on species and population groups). For those species without genoscapes, we developed a vulnerability and susceptibility estimate that determines the relative local and regional risk to populations that are in geographic proximity to solar facilities, using citizen science data (Breeding Bird Survey (BBS) and eBird). These two feather spot processing pipelines (see Figure 1 below) provide quantitative estimates as to the numbers of individuals from a given population of origin that are affected by solar facilities, and ultimately can reduce costs to the consumer by reducing the industry costs associated with mitigation and siting strategies for future solar energy development.

14 SOLAR ENERGY↗