Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Temperature Anomalies from the AIRS Product in Giovanni for the Climate Community

The Atmospheric Infrared Sounder (AIRS) mission began with the launch of Aqua in 2002. Over 15 years of AIRS products have been used by the climate research and application communities. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), in collaboration with NASA Sounder Team at JPL, provides processing, archiving, and distribution services for NASA sounders: the present Aqua AIRS mission and the succeeding Suomi National Polar-Orbiting Partnership (SNPP) Cross-track Infrared Sounder (CrIS) mission. We generated a Multi-year Monthly Mean and Anomaly product using 14 years of AIRS standard monthly product. The product includes Air Temperature at the Surface and Surface Skin Temperature, both in Ascending/Daytime and Descending/Nighttime mode. The temperature variables and their anomalies are deployed to Giovanni, a Web-based application developed by the GES DISC. Giovanni provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science remote sensing data without having to download the data. It is also a powerful tool that stakeholders can use for decision support in planning and preparing for increased climate variability. In this presentation, we demonstrate the functions in Giovanni with use cases employing AIRS Multi-year Monthly Mean and Anomaly variables.

Giovann↗
NASA NTRS2019

Understanding Machine Learning in Earth Science: A Natural Language Processing Approach

Machine learning (ML) is being increasingly utilized in Earth science research. Benefits of ML include efficiency, reduction of human error, and ability to extract hidden patterns within data. However, the mutual lack of each other’s domain knowledge by ML and Earth science stands as a barrier to timely and effective implementation. Earth science, in particular, faces challenges in generating sample data, compared to those of traditional ML problems such as face recognition or stock predictions, where data is abundant and not lacking in ground truth, which is necessary for labeling. Earth science data are more varying in formats, such as HDF5 and image resolutions, and are not standardized across instruments, even within a given Earth science discipline. Previous studies have been done to outline the specific challenges that Earth science faces with ML, while others have focused on using existing publications to mine information efficiently. Other resources such as Scikit-Learn have developed decision trees for choosing appropriate machine learning algorithms, but application within Earth science subjects becomes much more complex. For the current study, we propose a methodology and tool that aids in implementation of ML in Earth science using natural language processing (NLP). Our work comprises three main parts: (1) analyzing existing publications related to ML and Earth science, using natural language processing: (2) extracting from the publications information on ML models subjects in Earth Science: and (3) visualizing the extracted relationships as a network graph. The resulting network graph should aid the Earth science communities in applying optimal ML algorithms and guiding data preparation through visualization of similar studies. The network graph and analysis of document similarity will be the basis of our next step, which is to develop a decision tree for selecting optimal machine learning methodologies for specified Earth science applications.

Zheng, Laura↗
DOE OSTI2025

Constrained GAN-Generated X-Ray CT Data For Self-Supervised And Foundation-Model Segmentation Of Concrete Microstructures

Three-dimensional characterization of materials using X-ray computed tomography (XCT) is challenging due to the complexity of internal structures, noise, and variations in resolution. Traditional computer vision models often struggle to accurately segment these images, particularly in domain-specific applications like materials science. While supervised deep learning approaches have been developed to address the limitations of conventional algorithms, they typically require large amounts of labeled training data and often fail to generalize across different datasets. Self-supervised, few-and zero-shot learning methods have gained prominence in natural image processing and segmentation tasks, but their application to scientific imaging remains limited due to the unique structural complexity, noise, and textural artifacts present in materials science data. In this work, we investigate how domain adaptation, leveraging physics-based and GAN-generated synthetic data, impacts segmentation performance. We introduce a modified Contrastive Unpaired Translation (CUT) model designed to generate realistic labeled data, which can be used for training, pre-training, and fine-tuning segmentation models for real XCT microstructure data. We evaluate the performance of two segmentation approaches: a self-supervised network (SSL-ALPNet) and a foundation model (Segment Anything Model), assessing their improvements when pre-trained and/or fine-tuned on the synthesized data. Our results demonstrate that leveraging synthetic data significantly enhances segmentation performance, particularly in challenging materials science applications.

Ziabari, Amir [ORNL] (ORCID:000000034776457X)↗
NASA NTRS2016

Stewardship of NASA's Earth Science Data and Ensuring Long-Term Active Archives

Program, NASA has followed an open data policy, with non-discriminatory access to data with no period of exclusive access. NASA has well-established processes for assigning and or accepting datasets into one of 12 Distributed Active Archive Centers (DAACs) that are parts of EOSDIS. EOSDIS has been evolving through several information technology cycles, adapting to hardware and software changes in the commercial sector. NASA is responsible for maintaining Earth science data as long as users are interested in using them for research and applications, which is well beyond the life of the data gathering missions. For science data to remain useful over long periods of time, steps must be taken to preserve: (1) Data bits with no corruption, (2) Discoverability and access, (3) Readability, (4) Understandability, (5) Usability' and (6). Reproducibility of results. NASAs Earth Science data and Information System (ESDIS) Project, along with the 12 EOSDIS Distributed Active Archive Centers (DAACs), has made significant progress in each of these areas over the last decade, and continues to evolve its active archive capabilities. Particular attention is being paid in recent years to ensure that the datasets are published in an easily accessible and citable manner through a unified metadata model, a common metadata repository (CMR), a coherent view through the earthdata.gov website, and assignment of Digital Object Identifiers (DOI) with well-designed landing product information pages.

Data Management↗
DOE OSTI2023

A framework for forming thermoset polymer networks during laser powder bed fusion additive manufacturing

It is commonly reported that the limited availability of material types is a major barrier to widespread adoption for polymer laser powder bed fusion (PBF/L) additive manufacturing (AM). A deeper understanding of the physico-chemical relationships between AM process stimuli and polymer responsive behavior is needed to make data-driven decisions for PBF/L specific material development and formulation. Many researchers have cumulatively built the current state of PBF/L “science-based manufacturing” framework where an initial guess for process parameter values can be determined from measured polymer properties and the first-principles of physics. However, published literature has focused on developing this framework for thermoplastic processing of polymers omitting many specific concerns unique to processing thermosetting materials. The authors consider in the present work the implications of many specific aspects unique to thermosetting polymers during PBF/L manufacturing. These aspects are contextually discussed according to the five sub-functions of PBF/L and are described mathematically where appropriate alongside example data from thermosetting materials commercially available in powder form. Further, this expansion of the prevailing material development framework to include thermosetting polymers paves the way for the unique performance properties offered by that class to be manifest through AM’s geometric freedom.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Exascale Virtualized and Programmable Distributed Cyber Resource Control: Final Scientific Technical Report

Extreme-science drives the need for distributed exascale processing and communications that are carefully, yet flexibly, managed. Exponential growth of data for scientific simulations, experimental data, collaborative data analyses, remote visualization and GRID computing requirements of scientists in fields as diverse as high energy physics, climate change, genomics, fusion, synchrotron radiation, material science, medicine, and other scientific disciplines cannot be accommodated by simply applying existing transport protocols to faster pipes. Further, scientific challenges today demand diverse research teams, heightening the need for and increasing the complexity of collaboration. To address these issues within the network layer and physical layer, we have performed a number of research activities surrounding effective allocation and management of elastic optical network (EON) resources, particularly focusing on FlexGrid transponders. FlexGrid transponders support the opportunity to build Layer-1 connections at a wide range of bandwidths and to reconfigure them rapidly. The new flexibility supports complex new ways of using the physical layer that must be carefully managed and hidden from the scientist end-users. FlexGrid networks utilize flexible (or elastic) spectral bandwidths for each data link without using fixed wavelength grids. The flexibility in spectrum allocation brings many appealing features to network operations. Current networks are designed for the worst case impairments in transmission performance and the assigned spectrum is over-provisioned. In contrast, the FlexGrid networks can operate with the highest spectral efficiency and minimum bandwidth for the given traffic demand while meeting the minimum quality of transmission (QoT) requirement. Two primary focuses of our research are: (1) resource and spectrum allocation (RSA) for IP traffic over EONs, and (2) RSA for cross-domain optical networks. Previous work concentrates primarily on large file transfers within a single domain. Adding support for IP traffic changes the nature of the RSA problem: instead of choosing to accept or deny each request for network support, IP traffic is inherently elastic and thus lends itself to a bandwidth maximization formulation. We developed a number of algorithms that could be easily deployed within existing and new FlexGrid networks, leading to networks that better support scientific collaboration. Cross-domain RSA research is essential to support large-scale FlexGrid networks, since configuration information is generally not shared or coordinated across domains. The results presented here are in their early stages. They are technically feasible and practical, but still require coordination among organizations and equipment owners and a higher-layer framework for managing network requests.

42 ENGINEERING↗
NASA NTRS2016

Progress Towards AIRS Science Team Version-7 at SRT

The AIRS Science Team Version-6 retrieval algorithm is currently producing level-3 Climate Data Records (CDRs) from AIRS that have been proven useful to scientists in understanding climate processes. CDRs are gridded level-3 products which include all cases passing AIRS Climate QC. SRT has made significant further improvements to AIRS Version-6. At the last Science Team Meeting, we described results using SRT AIRS Version-6.22. SRT Version-6.22 is now an official build at JPL called 6.2.4. Version-6.22 results are significantly improved compared to Version-6, especially with regard to water vapor and ozone profiles. We have adapted AIRS Version-6.22 to run with CrIS/ATMS, at the Sounder SIPS which processed CrIS/ATMS data for August 2014. JPL AIRS Version-6.22 uses the Version-6 AIRS tuning coefficients. AIRS Version-6.22 has at least two limitations which must be improved before finalization of Version-7: Version-6.22 total O3 has spurious high values in the presence of Saharan dust over the ocean; and Version-6.22 retrieved upper stratospheric temperatures are very poor in polar winter. SRT Version-6.28 addresses the first concern. John Blaisdell ran the analog of AIRS Version-6.28 in his own sandbox at JPL for the 14th and 15th of every month in 2014 and all of July and October for 2014. AIRS Version-6.28a is hot off the presses and addresses the second concern.

CrIS↗
NASA NTRS2010

Flagging and Correction of Pattern Noise in the Kepler Focal Plane Array

In order for Kepler to achieve its required less than 20 PPM photometric precision for magnitude 12 and brighter stars, instrument-induced variations in the CCD readout bias pattern (our "2D black image"), which are either fixed or slowly varying in time, must be identified and the corresponding pixels either corrected or removed from further data processing. The two principle sources of these readout bias variations are crosstalk between the 84 science CCDs and the 4 fine guidance sensor (FGS) CCDs and a high frequency amplifier oscillation on less than 40% of the CCD readout channels. The crosstalk produces a synchronous pattern in the 2D black image with time-variation observed in less than 10% of individual pixel bias histories. We will describe a method of removing the crosstalk signal using continuously-collected data from masked and over-clocked image regions (our "collateral data"), and occasionally-collected full-frame images and reverse-clocked readout signals. We use this same set to detect regions affected by the oscillating amplifiers. The oscillations manifest as time-varying moir pattern and rolling bands in the affected channels. Because this effect reduces the performance in only a small fraction of the array at any given time, we have developed an approach for flagging suspect data. The flags will provide the necessary means to resolve any potential ambiguity between instrument-induced variations and real photometric variations in a target time series. We will also evaluate the effectiveness of these techniques using flight data from background and selected target pixels.

Kolodziejczak, Jeffery J.↗
NASA NTRS2002

Onboard Autonomous Rock Shape Analysis for Mars Rovers

In the quest to maximize the scientific return of future robotic missions, it is imperative that our rovers be capable of determining the importance of the science they collect so that they may prioritize the acquisition and relay of that data. As an important step in this process, we present an automated technique to allow a rover to classify the shape and other geologic characteristics of rocks from two-dimensional photographic images and three-dimensional stereographically produced data. Experiments were conducted in the Matlab environment using images returned by JPL's Mars Pathfinder mission. Our method begins by first segmenting the rocks from the background using a combination of image intensity and height data. Various metrics are then used to classify the region's sphericity, roundness, and other geometric properties. Preliminary experiments to determine the most useful metrics were conducted by characterizing the two-dimensional rock shape while the three-dimensional shape was later studied with metrics derived from these two-dimensional techniques. Seven measures were developed and implemented. The performance of each measure was characterized by analyzing imagesfrom the Pathfinder mission and ranking the rocks according to the measured properties. Combined, the measures would provide a tool by which an automated rover could discover a greater amount of information about the data it collects, leading to a more productive mission.

image segmentation↗
NASA NTRS2018

Use of Open Networks and Delay-Tolerant Protocol to Decrease WAN Latency of EOS near Real-Time Data

Since 1999, NASA's Earth Observing System Data Operations System (EDOS) project at Goddard Space Flight Center (GSFC) has provided high-rate data capture, level zero processing, and product distribution services for a majority of NASA's EOS (Earth Observing System) high-rate missions, including Terra, Aqua, Aura, ICESat, EO-1, SMAP, and OCO-2. EDOS high-rate science and engineering (150-300 Mbps) data-driven capture systems are deployed at 7 worldwide ground stations which are connected via both private (closed) and public (open) wide area networks (WANs) to the centralized EDOS Level Zero Processing Facility (LZPF) located at GSFC, where the data is processed and Level 0 products are distributed to users worldwide. All data transferred over the open networks to GSFC traverse an IPSec tunnel, providing the same level of security as a VPN connection. EDOS produces both time-based and near real-time products (session-based). Near real-time data products are produced from a single ground station contact; time-based products are produced from multiple ground station contacts. EDOS is the primary supplier of EOS Level 0 data to the NASA near real-time user community known as the Land, Atmosphere Near real-time Capability for EOS (LANCE). For the past few years, EDOS has streamlined its systems to reduce WAN latency for near real-time data delivery, including implementing Quality of Service (QoS), expanding closed network bandwidth, adding open network connections with more bandwidth, and implementing a delay-tolerant protocol to mitigate long round-trip times to remote ground stations.

Delay-Tolerant Protocol↗
DOE OSTI2023

Data Archive and Portal (DAP) Platform for Solid Phase Processing Technologies

The scale and speed of data generated by modern scientific experiments have constantly challenged the research community to store, curate, manage and optimally use it to drive scientific discoveries. In this work, we have developed a data archive and portal (DAP) platform including analytics capabilities to collect, curate, and manage data and metadata stream for solid phase processing (SPP) techniques. We successfully hosted around ~347K files of data related to processing parameters, microscopic images, and spectroscopic data related to solid phase processing. The DAP platform for SPP will establish an enduring capability to support machine learning and grow collaboration at the intersection of materials science and data science.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

OpenNEX: An open collaboration platform for the earth science community

Satellite data from the past several decades provide the most consistent record of land-surface processes that form the basis for scientific assessments of the impacts of climate variations and changes on the environment and human social-economical activities. During this time, scientific research on the characterization and assessment of environmental changes had tended to focus on large-scale land-surface changes with significant social-economic impacts. Increasingly, attention is shifting toward changes that occur more locally and that most directly relate to the everyday life of the majority of the population. In addition, there has been needs to develop management and policy decision support systems that are based on local environmental information. Almost at the same time, the advancement in sensor technology has allowed us to collect an unprecedented volume of environmental data. These data must be curated and analyzed to extract useful information for research and decision support purposes. Established in 2013 and funded by NASA, the Open NASA Earth eXchange (OpenNEX; https://opennex.org/ , Jia et al., 2019 ) project partnered with Amazon Web Services (AWS) to make available a large amount of Earth observing data, modeling results, and analysis tools on the AWS. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of NASA data assets and software tools. To encourage the public's engagement in this project, NASA ran virtual workshops and prize competitions. The virtual workshops provided online lectures and tutorials about how prominent scientists used the data in their research and the tutorials gave examples how to use the tools to interrogate the data in the Amazon cloud. Finally, the prize competitions allowed much wider participation in the OpenNEX project and enable testing the non-traditional projects and out-of-box ideas. OpenNEX has continued to evolve and mature. Here, we highlight new features and functionalities available to the community.

Jian Zhang↗
DOE OSTI2016

SU-F-J-54: Towards Real-Time Volumetric Imaging Using the Treatment Beam and KV Beam

Purpose: Existing real-time imaging uses dual (orthogonal) kV beam fluoroscopies and may result in significant amount of extra radiation to patients, especially for prolonged treatment cases. In addition, kV projections only provide 2D information, which is insufficient for in vivo dose reconstruction. We propose real-time volumetric imaging using prior knowledge of pre-treatment 4D images and real-time 2D transit data of treatment beam and kV beam. Methods: The pre-treatment multi-snapshot volumetric images are used to simulate 2D projections of both the treatment beam and kV beam, respectively, for each treatment field defined by the control point. During radiation delivery, the transit signals acquired by the electronic portal image device (EPID) are processed for every projection and compared with pre-calculation by cross-correlation for phase matching and thus 3D snapshot identification or real-time volumetric imaging. The data processing involves taking logarithmic ratios of EPID signals with respect to the air scan to reduce modeling uncertainties in head scatter fluence and EPID response. Simulated 2D projections are also used to pre-calculate confidence levels in phase matching. Treatment beam projections that have a low confidence level either in pre-calculation or real-time acquisition will trigger kV beams so that complementary information can be exploited. In case both the treatment beam and kV beam return low confidence in phase matching, a predicted phase based on linear regression will be generated. Results: Simulation studies indicated treatment beams provide sufficient confidence in phase matching for most cases. At times of low confidence from treatment beams, kV imaging provides sufficient confidence in phase matching due to its complementary configuration. Conclusion: The proposed real-time volumetric imaging utilizes the treatment beam and triggers kV beams for complementary information when the treatment beam along does not provide sufficient confidence for phase matching. This strategy minimizes the use of extra radiation to patients. This project is partially supported by a Varian MRA grant.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1981

MAGSAT science investigations

A preliminary magnetization model for the U.S. was produced from MAGSAT data. The double grid processing method resulted in a 100 km source spacing. While some spurious features are present in the map, much of the detail appears to be real. The determination of the optional source spacing for inversion of MAGSAT delta B data is in progress. A fine attitude vector data set was obtained for comparison with the theoretical vector field associated with the magnetization model.

Source record↗
NASA NTRS1988

Use of MCIDAS as an earth science information systems tool

The application of the man computer interactive data access system (MCIDAS) to information processing is examined. The computer systems that interface with the MCIDAS are discussed. Consideration is given to the computer networking of MCIDAS, data base archival, and the collection and distribution of real-time special sensor microwave/imager data.

Goodman, H. Michael↗
NASA NTRS1991

NASA information sciences and human factors program

The Data Systems Program consists of research and technology devoted to controlling, processing, storing, manipulating, and analyzing space-derived data. The objectives of the program are to provide the technology advancements needed to enable affordable utilization of space-derived data, to increase substantially the capability for future missions of on-board processing and recording and to provide high-speed, high-volume computational systems that are anticipated for missions such as the evolutionary Space Station and Earth Observing System.

Source record↗
DOE OSTI2016

Citizen Science: Potential New Opportunity for Consent-Based Siting Processes - 16608

Citizen science is a global phenomenon, as millions of volunteers are collecting, analyzing, and sharing data that contribute to scientific knowledge. Recognizing its potential, the U.S. 2013 Open Government National Action Plan encouraged Federal agencies to harness American ingenuity toward helping address a broad range of scientific and societal challenges, specifically invoking crowd-sourcing and citizen science. The October 2015 Plan reinforced this charge, calling on agencies to increase public participation through both means. Thus, citizen science opportunities are actively being sought across a wide range of Federal programs. Meanwhile, all nations with commercial nuclear reactors face the challenge of managing the spent nuclear fuel and high-level radioactive waste produced. Consent-based siting of facilities to manage these materials is considered an essential aspect of this shared challenge. A variety of other agency programs have illustrated the role citizens can play in strengthening engagement, empowerment, and scientific research. With these same themes important to planning for nuclear waste management, citizen science could represent a potential new opportunity for this national effort. Online resources were investigated to understand the nature and range of citizen science projects and seek insights for possible new applications. Results show that volunteers are contributing to research in areas from archaeology to zoology, and extending from the ocean floor to outer space. A large number of the collaborations involve characterizing local environmental conditions and monitoring changes over time. This paper highlights insights from several projects that illustrate the contributions citizen scientists can make to advance scientific knowledge and assist national and international programs. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2024

Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time

The Vera C. Rubin Observatory is preparing to execute the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently the final phase of construction is under way in the Chilean Andes, with the Observatory’s ten-year science mission scheduled to begin in 2025. Rubin’s 8.4-meter telescope will nightly scan the southern hemisphere collecting imagery in the wavelength range 320–1050 nm covering the entire observable sky every 4 nights using a 3.2 gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties. In this article we present an overview of the system currently being constructed to perform data distribution as well as the annual campaigns which reprocess the entire image dataset collected since the beginning of the survey. These processing campaigns will utilize computing and storage resources provided by three Rubin data facilities (one in the US and two in Europe). Each year a Data Release will be produced and disseminated to science collaborations for use in studies comprising four main science pillars: probing dark matter and dark energy, taking inventory of solar system objects, exploring the transient optical sky and mapping the Milky Way. Also presented is the method by which we leverage some of the common tools and best practices used for management of large-scale distributed data processing projects in the high energy physics and astronomy communities. We also demonstrate how these tools and practices are utilized within the Rubin project in order to overcome the specific challenges faced by the Observatory.

79 ASTRONOMY AND ASTROPHYSICS↗