SEARCH · Engineering Papers

Results for “preprocessed data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

All sources NASA NTRS DOE OSTI

NASA NTRS1982

Evaluation of small area crop estimation techniques using LANDSAT- and ground-derived data

Studies completed in fiscal year 1981 in support of the clustering/classification and preprocessing activities of the Domestic Crops and Land Cover project. The theme throughout the study was the improvement of subanalysis district (usually county level) crop hectarage estimates, as reflected in the following three objectives: (1) to evaluate the current U.S. Department of Agriculture Statistical Reporting Service regression approach to crop area estimation as applied to the problem of obtaining subanalysis district estimates; (2) to develop and test alternative approaches to subanalysis district estimation; and (3) to develop and test preprocessing techniques for use in improving subanalysis district estimates.

Amis, M. L.↗

NASA NTRS2007

Generating a Long-Term Land Data Record from the AVHRR and MODIS Instruments

The goal of NASA's Land Long Term Iiata Record (LTDR) project is to produce a consistent long term data set from the AVHRR and MODIS instruments for land climate studies. The project will create daily surface reflectance and normalized difference vegetation index (NDVI) products at a resolution of 0.05 deg., which is identical to the Climate Modeling Grid (CMG) used for MODIS products from EOS Terra and Aqua. Higher order products such as burned area, land surface temperature, albedo, bidirectional reflectance distribution function (BRDF) correction, leaf area index (LAI), and fraction of photosyntheticalIy active radiation absorbed by vegetation (fPAR), will be created. The LTDR project will reprocess Global Area Coverage (GAC) data from AVHRR sensors onboard NOAA satellites by applying the preprocessing improvements identified in the AVHRR Pathfinder Il project and atmospheric and BRDF corrections used in MODIS processing. The preprocessing improvements include radiometric in-flight vicarious calibration for the visible and near infrared channels and inverse navigation to relate an Earth location to each sensor instantaneous field of view (IFOV). Atmospheric corrections for Rayleigh scattering, ozone, and water vapor are undertaken, with aerosol correction being implemented. The LTDR also produces a surface reflectance product for channel 3 (3.75 micrometers). Quality assessment (QA) is an integral part of the LTDR production system, which is monitoring temporal trands in the AVHRR products using time-series approaches developed for MODIS land product quality assessment. The land surface reflectance products have been evaluated at AERONET sites. The AVHRR data record from LTDR is also being compared to products from the PAL (Pathfinder AVHRR Land) and GIMMS (Global Inventory Modeling and Mapping Studies) systems to assess the relative merits of this reprocessing vis-a-vis these existing data products. The LTDR products and associated information can be found at http://ltdr.nascom.nasa.gov/ltdr/ltdr.html.

Pedelty, Jeffrey↗

DOE OSTI2021

Preprocessing for Unintended Conducted Emissions Classification with ResNet

Characterization of Unintended Conducted Emissions (UCE) from electronic devices is important when diagnosing electromagnetic interference, performing nonintrusive load monitoring (NILM) of power systems, and monitoring electronic device health, among other applications. Prior work has demonstrated that UCE analysis can serve as a diagnostic tool for energy efficiency investigations and detailed load analysis. While explaining the feature selection of deep networks with certainty is often not fully comprehensive, or in other applications, quite lacking, additional tools/methods for further corroboration and confirmation can help further the understanding of the researcher. This is true especially in the subject application of the study in this paper. Often the focus of such efforts is the selected features themselves, and there is not as much understanding gained about the noise in the collected data. If selected feature and noise characteristics are known, it can be used to further shape the design of the deep network or associated preprocessing. This is additionally difficult when the available data are limited, as in the case which the authors investigated in this study. Here, the authors present a novel work (which is a proposed complementary portion of the overall solution to the deep network classification explainability problem for this application) by applying a systematic progression of preprocessing and a deep neural network (ResNet architecture) to classify UCE data obtained via current transformers. By using a methodical application of preprocessing techniques prior to a deep classifier, hypotheses can be produced concerning what features the deep network deems important relative to what it perceives as noise. For instance, it is hypothesized in this particular study as a result of execution of the proposed method and periodic inspection of the classifier output that the UCE spectral features are relatively close to each other or to the interferers, as systematically reducing the beta parameter of the Kaiser window produced progressively better classification performance, but only to a point, as going below the Beta of eight produced decreased classifier performance, as well as the hypothesis that further spectral feature resolution was not as important to the classifier as rejection of the leakage from a spectrally distant interference. This can be very important in unpredictable low-FNR applications, where knowing the difference between features and noise is difficult. As a side-benefit, much was learned regarding the best preprocessing to use with the selected deep network for the UCE collected from these low power consumer devices obtained via current transformers. Baseline rectangular windowed FFT preprocessing provided a 62% classification increase versus using raw samples. After performing a more optimal preprocessing, more than 90% classification accuracy was achieved across 18 low-power consumer devices for scenarios in which the in-band features-to-noise ratio (FNR) was very poor.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2026

Adaptively coupled phase retrieval in multi-peak Bragg coherent diffraction imaging

Recent advances in Bragg coherent diffraction imaging (BCDI) experimental techniques permit routine measurement of multiple Bragg peaks from a single crystalline grain. The resulting images contain the full lattice distortion vector field which can be differentiated to provide lattice strain and rotation. With the advent of fourth-generation synchrotron light sources, such multi-peak datasets are produced at high rates, facilitating the need for rapid phase retrieval of the multiple peaks and subsequent image analysis. Here we describe and demonstrate a new implementation of a coupled phase retrieval technique for multi-peak BCDI which simultaneously treats each Bragg peak of the dataset and produces a three-dimensional image of the crystal's morphology and lattice distortion field. In addition, this method uses the redundant information contained in the various Bragg diffraction patterns to detect and suppress spurious signal appearing on the detector in a subset of the measurements. Compared with manual data editing, adaptive coupling produces a more consistent phase profile in reciprocal space and sharper surfaces in direct space, with no significant difference in computational cost. These improvements reduce the need for manual preprocessing and enable robust high-throughput analysis of multi-peak BCDI data, supporting near-real-time strain microscopy at modern synchrotron facilities.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Common Column Identification for Table Similarity Detection in Electrified Transportation Data Lakes

Electrified transportation often requires researchers and operators to interact with datasets from a wide range of sources and disciplines, such as transportation, power systems, public health, policies, and regulations. These datasets vary in quality and format, making it difficult to understand, preprocess, and identify key columns representing real-world entities or values for indexing and joining, which can negatively impact downstream analysis and operation. Existing solutions are limited, requiring extensive manual customization or data expertise to utilize. In this article, we propose a multi-layered approach to automatically identify key columns to expedite preprocessing and aid in analysis of electrified transportation data. Our method leverages a dynamic ontology to identify common fields and an information theory-based strategy for edge cases that are difficult to generalize. Evaluations on a number of datasets from data.gov and kaggle.com show improved performance of our methods over several baseline techniques, and our ablation analyses illustrate the efficacy of individual components of our method. Our case studies also demonstrate that our methods have the potential to improve analysis of electrified transportation data and aid in automatic integration of such datasets.

33 ADVANCED PROPULSION SYSTEMS↗

NASA NTRS1981

KONFIG and REKONFIG: Two interactive preprocessing to the Navy/NASA Engine Program (NNEP)

The NNEP is a computer program that is currently being used to simulate the thermodynamic cycle performance of almost all types of turbine engines by many government, industry, and university personnel. The NNEP uses arrays of input data to set up the engine simulation and component matching method as well as to describe the characteristics of the components. A preprocessing program (KONFIG) is described in which the user at a terminal on a time shared computer can interactively prepare the arrays of data required. It is intended to make it easier for the occasional or new user to operate NNEP. Another preprocessing program (REKONFIG) in which the user can modify the component specifications of a previously configured NNEP dataset is also described. It is intended to aid in preparing data for parametric studies and/or studies of similar engines such a mixed flow turbofans, turboshafts, etc.

Fishbach, L. H.↗

DOE OSTI2026

A Centralized AI Lakehouse Framework for Brain Tumor MRI Classification and Segmentation, University KPI Forecasting, and Water Potability Prediction

In many university and healthcare projects, models are built for very different data types such as tables, institutional time series, and medical images, but they are deployed as separate applications. In this work, that separation made testing and maintenance difficult because each module had its own pipeline and runtime requirements. This paper presents an integrated AI lakehouse-style implementation that runs three model pipelines inside one containerized backend. For medical imaging, we used MRI datasets from IEEE DataPort: a four-class classification set with 7012 images (5708 train/1304 test) and a segmentation set with 3063 image–mask pairs. The classification model (ResNet50 transfer learning) is evaluated using a proper train–validation–test protocol across multiple splits (80/10/10, 70/10/20, 60/10/30, and 10/30/60), achieving a test accuracy of 99.00% under the standard 80/10/10 split. Additionally, a patient-level evaluation is conducted using an external glioma dataset to provide a more realistic assessment without data leakage. The segmentation model (DeepLabV3-ResNet50) achieved 83.09% validation mIoU and 88.79% Dice score. For university KPI forecasting, we used annual IPEDS and NSF HERD data from 2010 to 2023 for three universities (BSU, EOU, and UAB). To examine the effect of preprocessing on forecasting performance, two case studies are conducted. In the first case, linear interpolation is applied to generate semester-level data. In the second case, the original annual data is used directly without interpolation. Random Forest regression and ARIMA models are evaluated using MAE, RMSE, MAPE, and R 2 . The results showed that interpolation improved apparent forecasting performance due to smoothing, while evaluation on the original annual data provided a more realistic assessment of model behavior. To further validate the framework on a larger dataset, an additional case study is conducted using a student dropout dataset. For water potability, we trained and compared multiple tabular classifiers on a large dataset (1,048,575 samples). A Random Forest model (100 trees, max depth 10) achieved 85.86% test accuracy and high recall for unsafe samples (0.8447). All modules are served via FastAPI and deployed together using Docker, with workflow automation routing requests to the correct endpoint. System-level benchmarking indicates that the backend maintains stable throughput and latency under concurrent requests.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Particle Scale Impacts on Deconstruction Energy of Pine Residues

The goal of this Case Study was to quantify the impacts of variable moisture and ash on hammer mill throughput and energy consumption and on generation of fines that are not able to be fed to conversion, as compared to a status quo Base Case system. Also considered was convertible carbon content (minimum carbon specification) and maximum ash content and the delivered feedstock cost impacts of not being able to feed residue not meeting both specifications to the conversion reactor. Laboratory data on the impacts of input particle size and moisture content on the exit particle size were received from FCIC Subtask 5.2: Preprocessing, High Temperature Conversion Preprocessing from their single particle impact population balance modeling study (Tiasha Bhattacharjee, INL). Additional throughput and energy consumption data were obtained from FCIC Subtask 5.2 (Jordan Klinger, INL) for the same grinder with a 6 mm screen in place. These data were utilized to develop the necessary response surface equations to perform throughput analysis using discrete event simulation. Because the ash contents in the separated fines had not been analyzed in the laboratory at the time of the model runs, we chose to assume that the ash distributed proportionally with total mass into the overs and unders in the disk screen following grinding.

energy consumption↗

DOE OSTI2023

0BGRaman: Graph Network based Simulator for Forecasting Molecular Polarizability

This report presents the work performed under the GRaman project, sponsored by the PCSD LDRD Seed program. The project aimed at accelerating ab initio molecular dynamics simulation using Graph Networks. The Graph Network framework is a ML framework that has been successfully employed to simulate the dynamics of several physical systems: including water splashing in a container and flags moving with the wind. In this effort, we performed a data collection campaign for 3 different molecules of interest. We have built tools for preprocessing the trajectories obtained by simulating Raman Spectroscopy with NWChem and translating them into a suitable format for training. We have developed a training algorithm to train the Graph Network based simulators based on our data and developed a simulator that produces trajectories in the same NWChem format. While the tool has improved with each iteration of development and subsequent experiments, the current state of the tool does not allow to directly incorporate the technology within the NWChem framework because the trajectories produced by the tool are not yet accurate enough. However, the technology has proved to have good potential and it is certainly worth further research and development.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1972

Smoothing of functions of range and range rate measurements from earth orbiting satellites

It is shown that for satellites in circular Earth orbits with altitudes of 500 kilometers to 1500 kilometers, and for satellites in elliptical orbits with an approximate 4000 kilometer height of perigee, a high degree least squares polynomial (e.g., a 9th or 10th degree in some cases) is required to smooth both range and range rate data for purposes of input to orbit determination programs. In order to circumvent this problem, functions of range and range rate are smoothed with lower degree least squares polynomials (e.g., 3rd and 4th degree) and it is shown that under the above geometric constraints the standard deviation of fit can be reduced to levels commensurate with typical S-band tracking system resolution which is 1 to 2 meters in range and 0.005 meters/second in range rate for a 1 per second data rate. Also shown are the effects of Gaussian random noise, biases, and periodic noise. This analysis includes numerous examples applied to the 44 point data smoothing interval currently used in much of the operational preprocessing at the Goddard Space Flight Center.

Grenchik, T. J.↗

NASA NTRS1975

A versatile data acquisition system for Goldstone

A data acquisition system set up at Goldstone was described. It is capable of recording a wide variety of parameters from widely separated geographical locations with varying integration times and recording speeds. It is fast, accurate, and modular in form so that it can be readily expanded. The accuracies of the system are discussed, and examples of data recorded with the system's capability of operation with preprocessing are given.

Reid, M. S.↗

NASA NTRS1976

A methodology for small scale rural land use mapping in semi-arid developing countries using orbital imagery. Part 5: Experimental and operational techniques of mapping land use

The author has identified the following significant results. Scope of the preprocessing techniques was restricted to standard material from the EROS Data Center accompanied by some enlarging procedures and the use of the diazo process. Investigation has shown that the most appropriate sampling strategy for this study is the stratified random technique. A viable sampling procedure, together with a method for determining minimum number of sample points in order to test results of any interpretation are presented.

Vangenderen, J. L.↗

NASA NTRS1981

Concepts for on-board satellite image registration. Volume 2: IAS prototype performance evaluation standard definition

Problems encountered in testing onboard signal processing hardware designed to achieve radiometric and geometric correction of satellite imaging data are considered. These include obtaining representative image and ancillary data for simulation and the transfer and storage of a large quantity of image data at very high speed. The high resolution, high speed preprocessing of LANDSAT-D imagery is considered.

Daluge, D. R.↗

NASA NTRS1993

Design and construction of a prototype ACTS propagation terminal

The launch schedule for the Advanced Communication Technology Satellite (ACTS) spacecraft did not leave sufficient time for completion of the prototype ACTS Propagation Terminals (APT) prior to initiation of the APT production phase. In fact, the approach used was to construct and test all subassemblies of the terminal with special emphasis on the technically challenging portions. These include the RF front end that uses a state-of-the-art down converter which integrates a low noise amplifier, mixer, post amplifier, filter, and local oscillator port frequency doubler into a single small package. In addition, a new digital receiver that uses the latest DSP technology was developed. Both of these subassemblies were thoroughly tested. The highest risk technology in the APT program was the digital receiver. Several candidate algorithms and DSP chips were investigated early on, primarily under JPL sponsorship. A receiver was constructed based on Texas Instruments chip. The final prototype digital receiver was one based on an Analog Devices chip. The design and test results are documented in a report prepared for this grant. A Primary Design Review (PDR) was conducted 30 May 1991, and a Critical Design Review was held 7 Jul. 1992. Final complete documentation of the APT's will appear in the form of three reports: a hardware description report, a report on the data collection code (ACTS VIEW), and a report on the preprocessing code.

Stutzman, Warren↗

NASA NTRS1993

Spacecraft contamination investigation by direct simulation Monte Carlo - Contamination on UARS/HALOE

A three-dimensional version of the direct simulation Monte Carlo method is adapted to study the contamination environment surrounding a highly detailed model of the Upper Atmosphere Research Satellite, with the ultimate goal of estimating the cumulative level of molecular deposits on critical optical surfaces of the Halogen Occultation Experiment. Emphasis is placed on describing the development of preprocessing codes that simplify the incorporation of geometry and outgassing data into the main run code. A novel use of parallel processing software allows use of multiple processors to solve different regions of the overall flowfield. Initial results relating to contaminant cloud structure, cloud composition, and statistics of simulated molecules impinging on the Halogen Occultation telescope's aperture plane are presented, along with data related to code performance.

Rault, Didier F. G.↗

DOE OSTI2023

Impact of Anatomical Fractionation of Corn Stover on Hammer Mill Throughput and Energy Consumption

The goal of this Case Study was to quantify the impacts of variable moisture and ash on hammer mill throughput and energy consumption and on loss of very wet stover that causes failures in the first stage grinder and that are not able to be fed to conversion, as compared to a status quo Base Case system. Also considered was convertible carbohydrate content (minimum total carbohydrate specification), maximum ash content and the delivered feedstock cost impacts of not being able to feed stover that did not meet the total carbohydrate specification to the conversion reactor. Laboratory data on the impacts of moisture content and tissue fraction on throughput and energy consumption in a stage 2 hammer mill were received from FCIC Subtask 5.1: Preprocessing, Corn Stover Preprocessing (Neal Yancey and Sergio Hernandez, INL). Additional air classifier throughput, energy consumption and separation efficiency data were obtained from FCIC Subtask 5.1 (Neal Yancey, INL) for the new air classifier, which has three exit streams (lights, middle and heavies). These data were utilized to develop the necessary response surface equations to perform throughput analysis using discrete event simulation. Because the ash contents and particle sizes had not been analyzed in the laboratory at the time of the model runs, we assumed that the ash distributed proportionally with total mass into the lights and heavies in an air classifier having two exit streams (lights and heavies) and that the lights fraction from the air classifier was not removed.

ash content↗

DOE OSTI2023

Unsupervised classification for region of interest in X-ray ptychography

X-ray ptychography offers high-resolution imaging of large areas at a high computational cost due to the large volume of data provided. To address the cost issue, we propose a physics-informed unsupervised classification algorithm that is performed prior to reconstruction and removes data outside the region of interest (RoI) based on the multimodal features present in the diffraction patterns. The preprocessing time for the proposed method is inconsequential in contrast to the resource-intensive reconstruction process, leading to an impressive reduction in the data workload to a mere 20% of the initial dataset. This capability consequently reduces computational time dramatically while preserving reconstruction quality. Through further segmentation of the diffraction patterns, our proposed approach can also detect features that are smaller than beam size and correctly classify them as within the RoI.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1992

Compiler analysis for irregular problems in FORTRAN D

We developed a dataflow framework which provides a basis for rigorously defining strategies to make use of runtime preprocessing methods for distributed memory multiprocessors. In many programs, several loops access the same off-processor memory locations. Our runtime support gives us a mechanism for tracking and reusing copies of off-processor data. A key aspect of our compiler analysis strategy is to determine when it is safe to reuse copies of off-processor data. Another crucial function of the compiler analysis is to identify situations which allow runtime preprocessing overheads to be amortized. This dataflow analysis will make it possible to effectively use the results of interprocedural analysis in our efforts to reduce interprocessor communication and the need for runtime preprocessing.

Vonhanxleden, Reinhard↗