Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “preprocessed data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Design and Analysis of Convolutional Neural Network for RF Signal Modulation Classification for In-Orbit Deployment

To effectively transmit data to and from satellites requires a complex and robust RF communication system. Commonly, several different types of signal modulations may be required to maximize satellite efficiency depending on a variety of unexpected channel impairments. We propose a neural network algorithm capable of learning these RF signal modulations using a supervised learning technique designed for low power, high-efficiency in-orbit deployment. The work presented demonstrates a convolutional neural network (CNN) capable of learning and recognizing a set of modulation schemes commonly used to transmit RF information. We are capable of recognizing the modulation scheme from the I and Q data channels directly, with no preprocessing or data conversion required other than breaking the incoming signal into a set of uniform normalized samples. We perform a network design and size analysis, showing that reasonably high accuracy can be obtained using networks with a relatively low number of trainable parameters. Given that a user of a system such as this may wish to receive a signal using a modulation scheme that the network has not previously learned, we demonstrate that transfer learning can learn new modulation schemes by retraining only the fully connected layers in the CNN. Thus, this type of network would excel in outer space deployment using high-efficiency transfer learning hardware. Modulation recognition can be performed through rapid feedforward computation, and the CNN training process is significantly simplified when learning new modulations is required.

CNN↗

A Knowledge Graph Approach to Analyze Systems and Assets Health

Nuclear power plants collect large amounts of equipment reliability data elements that contain information on the statuses of component, assets, and systems. All these data elements precisely record asset and system performance and health throughout the lifecycle of those assets and systems. However, several challenges have proved to be roadblocks to this process. While some of these challenges are technical in nature (i.e., data are often distributed over several physical servers or databases), others are conceptual in nature (i.e., data elements come in different formats, numeric or textual), and measured values have different scales (e.g., vibration spectra and oil temperature). This paper directly focuses on the integration of numeric and textual data elements in order to assist plant system engineers in analyzing equipment reliability data. This task begins with preprocessing the data by extracting knowledge from textual data via natural language processing methods and quantifying system, asset, and component health based on numeric data. We then employed model-based system engineering (MBSE) models of systems and assets to identify their architecture and functional (i.e., cause and effect) relations. Data elements were then associated with a single MBSE graph element, based on their nature. This bonding of MBSE models and data elements constitutes a first-of-its-kind knowledge graph of a nuclear power plants system, with data elements being organized in a structured manner that enables system engineers to identify cause-effect trends in data elements and carry out appropriate actions in response.

97 - MATHEMATICS AND COMPUTING↗

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research

Earth observation satellites and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose significant challenges for processing and analysis. In this paper we discuss our experiences in developing and using a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our workflow can be seamlessly integrated and orchestrated across research facilities managed by different federal agencies, thus allowing users to extract new scientific insights from climate datasets. The experimental results indicate that the multi-facility workflow significantly reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Notably, our workflow processes 12,000 high-resolution satellite images in just 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems. Such high throughput is essential for dynamic tokenization and sharding of petascale satellite data for distributed AI model training and inferencing at scale across thousands of GPUs.

Kurihana, Takuya [ORNL] (ORCID:0000000156698565)↗

Data management for earth observations

The management of NASA earth observation data is discussed. User requirements are identified, as well as means to facilitate data acquisition. It is shown that LANDSAT data can be preprocessed to condense data into a more accessible format, thus reducing data acquisition costs.

Mcdonough, G. F.↗

Exploring and Analyzing Climate Variations Online by Using NASA MERRA-2 Data at GES DISC

NASA Giovanni (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) (http:giovanni.sci.gsfc.nasa.govgiovanni) is a web-based data visualization and analysis system developed by the Goddard Earth Sciences Data and Information Services Center (GES DISC). Current data analysis functions include Lat-Lon map, time series, scatter plot, correlation map, difference, cross-section, vertical profile, and animation etc. The system enables basic statistical analysis and comparisons of multiple variables. This web-based tool facilitates data discovery, exploration and analysis of large amount of global and regional remote sensing and model data sets from a number of NASA data centers. Long term global assimilated atmospheric, land, and ocean data have been integrated into the system that enables quick exploration and analysis of climate data without downloading, preprocessing, and learning data. Example data include climate reanalysis data from NASA Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) which provides data beginning in 1980 to present; land data from NASA Global Land Data Assimilation System (GLDAS), which assimilates data from 1948 to 2012; as well as ocean biological data from NASA Ocean Biogeochemical Model (NOBM), which provides data from 1998 to 2012. This presentation, using surface air temperature, precipitation, ozone, and aerosol, etc. from MERRA-2, demonstrates climate variation analysis with Giovanni at selected regions.

knowledge base↗

Analysis Ready Data in Analytics Optimized Data Stores for Analysis of Big Earth Data in the Cloud

Cloud computing offers the possibility of making the analysis of Big Data approachable for a wider community due to affordable access to computing power, an ecosystem of usable tools for parallel processing, and migration of many large datasets to archives in the cloud, allowing data-proximal computing. Generally, data analysis acceleration in the cloud comes from running multiple nodes in a split-combine-apply strategy. Data systems such as the Earth Observing System Data and Information System are in a position to "pre-split" the data by storing them in a data store that is optimized for data parallel computing, i.e., an Analytics-Optimized Data Store (AODS). A variety of approaches to AODS are possible, from highly scalable databases to scalable filesystems to data formats optimized for cloud access (e.g., zarr and cloud-optimized datasets), with the optimal choice dependent on both the types of analysis and the geospatial structure of the data. A key question is how much preprocessing of the data to do, both before splitting and as the first part of the apply step. Again, the geospatial structure of the data and the analysis type influence the decision, with the added complexity of the user type. Trans-disciplinary users who are not well-versed in the nuances of quality-filtering and georeferencing of remote sensing orbit/swath/scene data tend to ask for more highly processed data, relying on the data provider to make sensible decisions on preprocessing parameters. (This accounts for the popularity of "Level 3" gridded data, despite the lower spatial resolution it provides.) In this case, data can be preprocessed before the split, resulting in higher performance in the rest of the "apply" step, which can be transformative for use cases such as interactive data exploration at scale. Discipline researchers who are experienced with remote sensing data often prefer more flexibility in customizing the preprocessing data into Analysis Ready Data, resulting in more need for on-the-fly preprocessing.

Lynnes, Christopher↗

A sensor for control of arterials and networks

The sensor for control of arterials and networks (SCAN) uses imaging technology and processing technology to jointly provide real-time quantification. The SCAN concept consists of a television camera mounted on a pole to obtain images of the traffic, and a microprocessor to process the image data into traffic parameters. The current activities focus on the development of SCAN for surveillance of arterials or urban highways. If these efforts are successful, an attempt will be made to extend the concept to network applications. The key software developed for the SCAN is a vehicle detection and tracking algorithm which reduces the TV image data to vehicle descriptions and trajectories. These preprocessed trajectory data can be transmitted over a phone line or can be easily reduced to a wide range of traffic parameters. A SCAN breadboard has been implemented and installed in a van which enables remote field tests and evaluations. The SCAN breadboard and its operation, evaluation, and potential applications are described.

Hilbert, E. E.↗

Preprocessing: Geocoding of AVIRIS data using navigation, engineering, DEM, and radar tracking system data

Remotely sensed data have geometric characteristics and representation which depend on the type of the acquisition system used. To correlate such data over large regions with other real world representation tools like conventional maps or Geographic Information System (GIS) for verification purposes, or for further treatment within different data sets, a coregistration has to be performed. In addition to the geometric characteristics of the sensor there are two other dominating factors which affect the geometry: the stability of the platform and the topography. There are two basic approaches for a geometric correction on a pixel-by-pixel basis: (1) A parametric approach using the location of the airplane and inertial navigation system data to simulate the observation geometry; and (2) a non-parametric approach using tie points or ground control points. It is well known that the non-parametric approach is not reliable enough for the unstable flight conditions of airborne systems, and is not satisfying in areas with significant topography, e.g. mountains and hills. The present work describes a parametric preprocessing procedure which corrects effects of flight line and attitude variation as well as topographic influences and is described in more detail by Meyer.

Meyer, Peter↗

Automated preprocessing of spaceborne SAR data

An efficient algorithm has been developed for estimation of the echo phase delay in spaceborne synthetic aperture radar (SAR) data. This algorithm utilizes the spacecraft ephemeris data and the radar echo data to produce estimates of two parameters: (1) the centroid of the Doppler frequency spectrum f(d) and (2) the Doppler frequency rate. Results are presented from tests conducted with Seasat SAR data. The test data indicates that estimation accuracies of 3 Hz for f(d) and 0.3 Hz/sec for the Doppler frequency rate are attainable. The clutterlock and autofocus techniques used for estimation of f(d) and the Doppler frequency rate, respectively are discussed and the algorithm developed for optimal implementation of these techniques is presented.

Curlander, J. C.↗

Some current uses of array processors for preprocessing of remote sensing data

The preparation of remotely sensed data sets into a form useful to the analyst is a significant computational task, involving the processing of spacecraft data (e.g., orbit, attitude, temperatures, etc.), decommutation of the video telemetry stream, radiometric correction and geometric correction. Many of these processes are extremely well suited for implementation on attached array processors. Currently, at Goddard Space Flight Center a number of computer systems provide such capability for earth observations or are under development as test beds for future ground segment support. Six such systems will be discussed.

Fischel, D.↗

Pre-Processor for Compression of Multispectral Image Data

A computer program that preprocesses multispectral image data has been developed to provide the Mars Exploration Rover (MER) mission with a means of exploiting the additional correlation present in such data without appreciably increasing the complexity of compressing the data.

Klimesh, Matthew↗

An investigation for the development of an integrated optical data preprocessor

The successful fabrication and demonstration of an integrated optical circuit designed to perform a parallel processing operation by utilizing holographic subtraction to simultaneously compare N analog signal voltages with N predetermined reference voltages is summarized. The device alleviates transmission, storage and processing loads of satellite data systems by performing, at the sensor site, some preprocessing of data taken by remote sensors. Major accomplishments in the fabrication of integrated optics components include: (1) fabrication of the first LiNbO3 waveguide geodesic lens; (2) development of techniques for polishing TIR mirrors on LiNbO3 waveguides; (3) fabrication of high efficiency metal-over-photoresist gratings for waveguide beam splitters; (4) demonstration of high S/N holographic subtraction using waveguide holograms; and (5) development of alignment techniques for fabrication of integrated optics circuits. Important developments made in integrated optics are the discovery and suggested use of holographic self-subtraction in LiNbO3, development of a mathematical description of the operating modes of the preprocessor, and the development of theories for diffraction efficiency and beam quality of two dimensional beam defined gratings.

Verber, C. M.↗

Real-time infrared spectroscopy coupled with blind source separation for nuclear waste process monitoring

On-line infrared absorbance spectroscopy enables rapid measurement of solution-phase molecular species. Many spectra-to-concentration models exist for spectral data, with some models able to handle overlapping spectral bands and nonlinearities. However, model accuracy is limited by the quality of training data used in model fitting. The process spectra of nuclear waste simulants at the Savannah River Site display incongruity between training and process spectra; the glycolate spectral signature in the training data does not match the glycolate signature in Savannah River National Laboratory process data. A novel blind source separation algorithm is proposed that preprocesses spectral data so that process spectra more closely resemble training spectra, thereby improving model quantification accuracy when unexpected sources of variation appear in process spectra. The novel blind source separation preprocessing algorithm is shown to improve nitrate quantification from an R 2 of 0.934 to 0.988 and from 0.267 to 0.978 in two instances analyzing nuclear waste simulants from the Slurry Receipt Adjustment Tank and Slurry Mix Evaporator cycle at the Savannah River Site.

Crouse, Steven H.↗

Signature extension techniques applied to multispectral scanner data.

Review of a number of spectral radiance signature extension techniques based on the concept of preprocessing the data to reduce the effects due to atmospheric effects, scanner look angle, etc. One of the promising methods studied to date involves using a ratio preprocessing transformation wherein the signals generated in adjacent spectral bands are ratioed on a point-by-point basis prior to classification. This method is easily and efficiently implemented and tests to date have yielded excellent results. Signatures have been successfully extended over 100+ miles, four days, different times of day, and very different atmospheric conditions.

Nalepka, R. F.↗

ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

This dataset release corresponds to the work conducted in ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling, where large-scale AI methods were applied to improve climate and weather resolution. The collection integrates four widely used, publicly available datasets: ERA5, PRISM, DAYMET, and IMERG. To prepare the data for ORBIT-2 model training and evaluation, we applied a preprocessing pipeline that generates paired low-resolution and high-resolution samples, enabling supervised downscaling experiments. The transformation from coarse to fine scales was performed using bilinear regridding, consistent with the procedures described in WeatherBench2, a community benchmark for weather and climate AI models. This dataset supports the development and evaluation of foundation models designed for weather and climate downscaling at exascale. Additional details on methodology and applications can be found in Wang et al., ORBIT-2 (arXiv:2505.04802, 2025).

54 ENVIRONMENTAL SCIENCES↗