Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “preprocessed data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Dataset: "Widespread Drought-driven Declines in Streamflows and Water quality in the Upper Colorado River Basin (1998-2022)"

This data package contains the associated data and scripts for Nagamoto, E., Ombadi, M., Ciulla, F. et al. Widespread drought-driven declines in streamflows and water quality in the Upper Colorado River Basin during 1998-2022. Commun Earth Environ 7, 734 (2026). https://doi.org/10.1038/s43247-026-03890-5. This purpose of this study was to investigate the impact of the 21st century drought on water quantity and quality at catchments throughout the Upper Colorado River Basin (UCRB). We used stream flow, water temperature, specific conductance, air temperature, precipitation, and catchment attribute data for over 200 sites in the UCRB, collected from the National Water Information System using Basin3D (Varadharajan, 2023), GAGESII (Falcone, 2010), and the Google Earth Engine. We identified years of severe drought between 1998 and 2022 using the Standardized Precipitation Evaporation Index (SPEI), then calculated the relative change percentage of the stream flow, water temperature, and specific conductance from drought versus non-drought years. We used the attribute information from GAGESII to investigate what physical traits of catchments are associated streamflow vulnerability (greater relative change) or resilience to drought. We used land cover data from the National Land Cover Database (USGS, 2024) to assess any changes to physical attributes that may not be represented in the static attributes information in GAGESII. To increase data availability, we modeled stream temperature using methods from Willard, 2023. While the study period is water years 1998 to 2022, the raw water quantity and quality data extends to 1950 and the meteorological data extends to 1980. The data and code can be downloaded via the UCRB_drought.zip. Within the zip, the files are organized as follows: - INPUTS: Contains all input data used in UCRB_Drought_Workflow.ipynb - OUTPUTS: Contains all intermediate data created from UCRB_Drought_Workflow.ipynb as well as final products including the calculated Standardized Evapotranspiration Index (SPEI) - climatic_variables: The code used to collect meteorologic data from Google Earth Engine - feature_importance: The code used for the catchment attributes analysis - preprocessing: Code used in UCRB_Drought_Workflow_Preprocessing.ipynb - pyeto: Code used in UCRB_Drought_Workflow_Preprocessing.ipynb - calculations: Code used in UCRB_Drought_Workflow_Impacts.ipynb - plotting: Code used in UCRB_Drought_Workflow_Impacts.ipynb - README.md - UCRB_Drought_Workflow_Preprocessing.ipynb: The code used to prep raw data for the analysis - UCRB_Drought_Workflow_Impact.ipynb: The code which uses the prepped raw data for analysis, and plots all figures - requirements_ucrb-drought_v2.yml: The requirements file to create a virtual environment and Jupyter Lab kernel to run the code The INPUTS folder is organized into the following major directories and sub-directories. The "RDC_WT_SC_RAW" folder contains raw data for streamflow, water temperature, and specific conductance in a ".h5" file. The "NLCD_RAW" folder contains ".csv" files with annual land cover percentages for counties within the UCRB. The "MET_RAW" folder contains a ".csv" file with monthly meteorological data (air temperature and precipitation) for the sites in the UCRB which was obtained from code in the climatic_variables folder. The "GAGESII" folder contains ".csv" files with physical catchment attribute variables for catchments across the country. The "WT_LSTM_data" folder contains ".csv" files with calculated WT (Willard, 2023) and the associated RMSEs. The "Upper_Colorado_River_Basin_Boundary" folder contains geographic data including a shapefile for plotting in the UCRB_Drought_Workflow.ipynb. The "RESERVOIRS_RAW" folder contains ".csv" files for each reservoir in the UCRB with daily reservoir storage. There are also two files in the INPUTS folder that have combined reservoir storage data and reservoir metadata. The OUTPUTS folder is organized into the following major directories and sub-directories. The "RDC_WT_SC_data" folder contains a folder "Water_year" with the associated cleaned data, metadata, and data availability information in ".csv" files, a folder "Median_Relchange" with the relative change comparing drought to non-drought years in ".csv" files, and a folder "Peak95_Min5_Relchange" that has ".csv" files for the relative change in peak (95th %) and minimum (5th %) variables. The "NLCD_data" folder contains the difference in land cover from the beginning to end of the study period and the percentage of the county that is within UCRB bounds can be found in Nagamoto et al (2025)). The "MET_data" folder contains separated monthly air temperature and precipitation data and the calculated PET in ".csv" files. The "SPEI_data" folder contains ".csv" files with calculated SPEI values (one restricted to the study period and the other with information from the entire MET data period). The "Paper_Tables" folder contains two ".csv" files containing site information and data availability and information about the GAGESII trait aggregated categories. The base directory includes the file “flmd.csv” for a list and description of all files and the file “dd.csv” for data dictionaries. Scripts for preprocessing, analysis, and figure generation are located in the associated GitHub repository found at [https://github.com/iNAIADS/drought-impacts/tree/develop/UCRB-drought]. UPDATE 1: Title and code file updated to match submitted manuscript 10-15-2025. UPDATE 2: Code and data files updated to match revised manuscript 3-4-2026. UPDATE 3: Code and data files updated to match revised manuscript 6-7-2026. ** NOTE: DD and FLMD have not been updated yet. UPDATE 4: Added associated Manuscript information and DD and FLMD have been updated. To cite this code, please use the following BibTeX: @misc{nagamoto2025drought, author = {Emily Nagamoto and Fabio Ciulla and Mohammad Ombadi and Jared Willard and Rosemary Carroll and Charuleka Varadharajan}, title = {Dataset: "Widespread Drought-driven Declines in Streamflows and Water quality in the Upper Colorado River Basin (1998-2022)"}, year = {2025}, doi = {10.15485/2551894}, publisher = {ESS-DIVE Repository}, url = {https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2551894} }

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1977

Digital exploitation of synthetic aperture radar

A digital processing and analysis scheme for use with digitized synthetic aperture radar data was developed. Using data from a four channel system, the imagery is preprocessed using specially designed software and then analyzed using preexisting facilities originally intended for use with MSS type data. Geometric and radiometric correction may be performed if desired, as well as classification analysis, Fast Fourier transform, filtering and level slice and display functions. The system provides low cost output in real time, permitting interactive imagery analysis. System information flow diagrams as well as sample output products are shown.

Wagner, H. L.↗
NASA NTRS1981

Potential utility of the thematic mapper for surface mine monitoring

One of many potential applications of the thematic mapper (TM) is surface mine monitoring. To assess this potential, data acquired by an aircraft multispectral scanner over Pennsylvania surface mines were preprocessed to simulate the anticipated spectral, spatial, and radiometric characteristics of TM data. False color imagery and thematic maps were derived from the simulated data and compared to imagery and maps derived from LANDSAT multispectral scanner subsystems data. On the basis of this comparison, TM data should definitely increase the detail and accuracy of remotely acquired surface mine information and may enable the remote determination of compliance with reclamation regulations.

Irons, J. R.↗
NASA NTRS1987

A computer code to calculate line by line atmospheric transmission spectra on a microcomputer

A computer program is described which calculates line by line atmospheric transmission spectra on a microcomputer. Radiance calculations are not included. The program is written in FORTRAN and could be modified to run on a microcomputer other than the one on which it was implemented, except that the plotting routine would have to be replaced. The program is based on the 1978 Air Force Geophysics Laboratory LASER routine, and uses absorption line data from the 1982 update of the AFGL Atmospheric Absorption Line Parameters Compilation; segments of needed line data are taken from the tape, preprocessed into the form used by the program and stored on floppy disks. The program calculates transmission spectra over a variety of paths and spanning an arbitrarily chosen wavelength or frequency range; the path may be chosen to be horizontal, vertical or slanted at any zenith angle, and may extend between any two altitudes between 0 and 15 km. Aerosol effects are included, presently based on the 1976 Shettle and Fenn models, plus some molecular continuum effects and Rayleigh scattering.

Safren, H. G.↗
NASA NTRS1991

Improved Computation of Dynamic Stresses

Method combining modified computational procedure with preprocessing and postprocessing program yields additional data and reduces time and cost of using DYNRE1 module of STARDYNE computer program to compute distributions of stresses in machinery under dynamic loads.

Ghosh, Tarun K.↗
NASA NTRS1992

Image masking using polygon fills and morphological transformations

Polygon-fill operations and morphological transformations are effective computational tools for the land-masking and coastline-correction preprocessing operations often applied to AVHRR data prior to oceanographic applications. These masking operations, in conjunction with cloud-screening techniques, can be used on such other oceanographically significant remote-sensing data as those of the Coastal Zone Color Scanner, GOES, and Landsat. The sensitivity of the methods to regional variations in atmospheric conditions and land-ocean temperature gradients is assessed for tropical, midlatitude, and high latitude regions.

Simpson, James J.↗
NASA NTRS2005

A least-squares parameter estimation algorithm for switched hammerstein systems with applications to the VOR

A "Multimode" or "switched" system is one that switches between various modes of operation. When a switch occurs from one mode to another, a discontinuity may result followed by a smooth evolution under the new regime. Characterizing the switching behavior of these systems is not well understood and, therefore, identification of multimode systems typically requires a preprocessing step to classify the observed data according to a mode of operation. A further consequence of the switched nature of these systems is that data available for parameter estimation of any subsystem may be inadequate. As such, identification and parameter estimation of multimode systems remains an unresolved problem. In this paper, we 1) show that the NARMAX model structure can be used to describe the impulsive-smooth behavior of switched systems, 2) propose a modified extended least squares (MELS) algorithm to estimate the coefficients of such models, and 3) demonstrate its applicability to simulated and real data from the Vestibulo-Ocular Reflex (VOR). The approach will also allow the identification of other nonlinear bio-systems, suspected of containing "hard" nonlinearities.

Evaluation Studies↗
NASA NTRSDate not supplied

Making NASA GES DISC Level 2 Data GIS Analysis Ready

There are many valuable data hosted by NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) for GIS applications in the areas of extreme weather events, climatic anomaly, and public health. However, using NASA Earth Science Data poses some challenges for GIS users. Many of these users are not experts in Earth Observation and have little knowledge about NASA's Earth science data. In the GIS community, GeoTiff is the most widely used raster format, whereas NASA's data is primarily in complex multidimensional netCDF and HDF formats. This complexity makes it difficult for GIS users, especially those who are unfamiliar with these formats. Although GIS software like ArcGIS has made progress in processing multidimensional netCDF data, certain issues still remain, particularly with level 2 data. In this study, we use TROPSpheric Monitoring instrument (TROPOMI) level 2 data as an example to demonstrate how to make such data GIS analysis ready. The process involves: 1. Creating a feature layer from the TROPOMI level 2 data. 2. Converting the feature layer to a gridded raster dataset. 3. Mosaicking gridded raster datasets into a raster dataset covering the entire desired extent. 4. Generating a symbology with a GIBS-specific style that aligns with the visual standards and requirements of GIBS. 5. Publishing image services. By performing these preprocessing and transformation steps, NASA level 2 data can be made compatible and ready for use within GIS software for various spatial analysis and visualization tasks.

Geographic Information System↗
NASA NTRS2007

Advanced Systems for Monitoring Underwater Sounds

The term "Passive Acoustic Monitoring System" (PAMS) describes a developmental sensing-and-data-acquisition system for recording underwater sounds. The sounds (more precisely, digitized and preprocessed versions from acoustic transducers) are subsequently analyzed by a combination of data processing and interpretation to identify and/or, in some cases, to locate the sources of those sounds. PAMS was originally designed to locate the sources such as fish of species that one knows or seeks to identify. The PAMS unit could also be used to locate other sources, for example, marine life, human divers, and/or vessels. The underlying principles of passive acoustic sensing and analyzing acoustic-signal data in conjunction with temperature and salinity data are not new and not unique to PAMS. Part of the uniqueness of the PAMS design is that it is the first deep-sea instrumentation design to provide a capability for studying soniferous marine animals (especially fish) over the wide depth range described below. The uniqueness of PAMS also lies partly in a synergistic combination of advanced sensing, packaging, and data-processing design features with features adapted from proven marine instrumentation systems. This combination affords a versatility that enables adaptation to a variety of undersea missions using a variety of sensors. The interpretation of acoustic data can include visual inspection of power-spectrum plots for identification of spectral signatures of known biological species or artificial sources. Alternatively or in addition, data analysis could include determination of relative times of arrival of signals at different acoustic sensors arrayed at known locations. From these times of arrival, locations of acoustic sources (and errors in those locations) can be estimated. Estimates of relative locations of sources and sensors can be refined through analysis of the attenuation of sound in the intervening water in combination with water-temperature and salinity data acquired by instrumentation systems other than PAMS. A PAMS is packaged as a battery-powered unit, mated with external sensors, that can operate in the ocean at any depth from 2 m to 1 km. A PAMS includes a pressure housing, a deep-sea battery, a hydrophone (which is one of the mating external sensors), and an external monitor and keyboard box. In addition to acoustic transducers, external sensors can include temperature probes and, potentially, underwater cameras. The pressure housing contains a computer that includes a hard drive, DC-to- DC power converters, a post-amplifier board, a sound card, and a universal serial bus (USB) 4-port hub.

Lane, Michael↗
DOE OSTI2023

Improving the Quality of Geothermal Data Through Data Standards and Pipelines Within the Geothermal Data Repository: Preprint

For machine learning outputs to be applicable to real world problems, high quality data are needed to ensure high quality results. With the more recent emphasis on machine learning in geothermal, there is an increasing need for greater focus on the quality of the data available for use in these projects. For example, Geothermal Operational Optimization Using Machine Learning (GOOML) utilized large quantities of geothermal power plant operational data to inform power plant operational configurations to maximize power generation. High quality datasets result from dependable sensors or devices collecting data, high frequency of measurements, sufficient data points, adequate metadata, reliable storage of data, and sufficient data curation. Another component that contributes to high quality data is reusability, which can be enhanced through data standardization. Data Standardization creates consistency in formatting and contents of like datasets, lessening preprocessing requirements and ensuring adequate information provided by a given dataset. The Geothermal Data Repository (GDR) aims to help improve data quality through automated data standardization for high-value datasets through the implementation of data pipelines alongside reliable and accessible long-term storage for datasets. As such, the GDR has decided to shift away from recommending the use of Excel-based content models and towards the implementation of automated data pipelines. This takes the burden of data standardization off the user and project team and will increase the availability of standardized geothermal data available through the GDR. A set of recommendations, or a data standard for each data type will exist with each data pipeline in order to advise data collection for maximum usability for future research. This paper serves to describe the GDR's proposed transition towards data standardization through automated data pipelines, to discuss the need for and value of such a shift, and to call for suggestions from the community regarding the most useful data standards and pipelines.

data↗
NASA NTRS1977

Blob - An unsupervised clustering approach to spatial preprocessing of MSS imagery

A basic concept of MSS data processing has been developed for use in agricultural inventories; namely, to introduce spatial coordinates of each pixel into the vector description of the pixel and to use this information along with the spectral channel values in a conventional unsupervised clustering of the scene. The result is to isolate spectrally homogeneous field-like patches (called 'blobs'). The spectral mean vector of a blob can be regarded as a defined feature and used in a conventional pattern recognition procedure. The benefits of use are: ease in locating training units in imagery; data compression of from 10 to 30 depending on the application; reduction of scanner noise and consequently potential improvements in classification/proportion estimation performances.

Kauth, R. J.↗
NASA NTRS1980

Digital preprocessing of SEASAT imagery

A model for radar image data is derived for use in developing enhancement techniques. This model describes the image data as the result of a multiplicative-convolved noise process. This information is then used to design a minimum mean square error (MMSE) filter. The resulting filter is implemented adaptively to change with local statistics. A radar image processing technique which provides the MMSE estimate inside homogeneous areas and tends to preserve edge structure is thus developed. This technique has been implemented and tested using digitally correlated SEASAT-A synthetic aperture radar (SAR) imagery.

Frost, V. S.↗
DOE OSTI2021

Inpainting radar missing data regions with deep learning

Abstract. Missing and low-quality data regions are a frequent problem for weather radars. They stem from a variety of sources: beam blockage, instrument failure, near-ground blind zones, and many others. Filling in missing data regions is often useful for estimating local atmospheric properties and the application of high-level data processing schemes without the need for preprocessing and error-handling steps – feature detection and tracking, for instance. Interpolation schemes are typically used for this task, though they tend to produce unrealistically spatially smoothed results that are not representative of the atmospheric turbulence and variability that are usually resolved by weather radars. Recently, generative adversarial networks (GANs) have achieved impressive results in the area of photo inpainting. Here, they are demonstrated as a tool for infilling radar missing data regions. These neural networks are capable of extending large-scale cloud and precipitation features that border missing data regions into the regions while hallucinating plausible small-scale variability. In other words, they can inpaint missing data with accurate large-scale features and plausible local small-scale features. This method is demonstrated on a scanning C-band and vertically pointing Ka-band radar that were deployed as part of the Cloud Aerosol and Complex Terrain Interactions (CACTI) field campaign. Three missing data scenarios are explored: infilling low-level blind zones and short outage periods for the Ka-band radar and infilling beam blockage areas for the C-band radar. Two deep-learning-based approaches are tested, a convolutional neural network (CNN) and a GAN that optimize pixel-level error or combined pixel-level error and adversarial loss respectively. Both deep-learning approaches significantly outperform traditional inpainting schemes under several pixel-level and perceptual quality metrics.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1983

Comparison of modelled and empirical atmospheric propagation data

The radiometric integrity of TM thermal infrared channel data was evaluated and monitored to develop improved radiometric preprocessing calibration techniques for removal of atmospheric effects. Modelled atmospheric transmittance and path radiance were compared with empirical values derived from aircraft underflight data. Aircraft thermal infrared imagery and calibration data were available on two dates as were corresponding atmospheric radiosonde data. The radiosonde data were used as input to the LOWTRAN 5A code which was modified to output atmospheric path radiance in addition to transmittance. The aircraft data were calibrated and used to generate analogous measurements. These data indicate that there is a tendancy for the LOWTRAN model to underestimate atmospheric path radiance and transmittance as compared to empirical data. A plot of transmittance versus altitude for both LOWTRAN and empirical data is presented.

Schott, J. R.↗
NASA NTRS1983

Comparison of modelled and empirical atmospheric propagation data

The radiometric integrity of TM thermal infrared channel data was evaluated and monitored to develop improved radiometric preprocessing calibration techniques for removal of atmospheric effects. Modelled atmospheric transmittance and path radiance were compared with empirical values derived from aircraft underflight data. Aircraft thermal infrared imagery and calibration data were available on two dates as were corresponding atmospheric radiosonde data. The radiosonde data were used as input to the LOWTRAN 5A code which was modified to output atmospheric path radiance in addition to transmittance. The aircraft data were calibrated and used to generate analogous measurements. These data indicate that there is a tendency for the LOWTRAN model to underestimate atmospheric path radiance and transmittance as compared to empirical data. A plot of transmittance versus altitude for both LOWTRAN and empirical data is presented.

Schott, J. R.↗
NASA NTRS1979

Some practical universal noiseless coding techniques

Some practical adaptive techniques for the efficient noiseless coding of a broad class of such data sources are developed and analyzed. Algorithms are designed for coding discrete memoryless sources which have a known symbol probability ordering but unknown probability values. A general applicability of these algorithms to solving practical problems is obtained because most real data sources can be simply transformed into this form by appropriate preprocessing. These algorithms have exhibited performance only slightly above all entropy values when applied to real data with stationary characteristics over the measurement span. Performance considerably under a measured average data entropy may be observed when data characteristics are changing over the measurement span.

Rice, R. F.↗
NASA NTRS1977

Feasibility investigation of integrated optics Fourier transform devices

The possibility of producing an integrated optics data processing device based upon Fourier transformations or other parallel processing techniques, and the ways in which such techniques may be used to upgrade the performance of present and projected NASA systems were investigated. Activities toward this goal include; (1) production of near-diffraction-limited geodesic lenses in glass waveguides; (2) development of grinding and polishing techniques for the production of geodesic lenses in LiNbO3 waveguides; (3) development of a characterization technique for waveguide lenses; and (4) development of a theory for corrected aspheric geodesic lenses. A holographic subtraction system was devised which should be capable of rapid on-board preprocessing of a large number of parallel data channels. The principle involved is validated in three demonstrations.

Verber, C. M.↗