Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Automatic Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
NASA NTRS1974

Automatic mapping of strip mine operations from spacecraft data

The author has identified the following significant results. Computer techniques were applied to process ERTS tapes acquired over coal mining operations in southeastern Ohio on 21 August 1972 and 3 September 1973. ERTS products obtained included geometrically-correct map overlays, at scales from 1:24,000 to 1:250,000, showing stripped earth, partially reclaimed earth, water, and natural vegetation. Computer-generated tables listing the area covered by each land-water category in square kilometers were also produced. By comparing these mapping products, the study demonstrates the capability of ERTS to monitor changes in the extent of stripping and reclamation. NASA C-130 photography acquired on 7 September 1973 when compared with the ERTS products generated from the 3 September 1973 tape established the categorization accuracy to be better than 90%. It is estimated that the stripping and reclamation maps and data were produced from the ERTS CCTs at a tenth of the cost of conventional techniques.

Rogers, R. H.↗
NASA NTRS1971

ADMAP (automatic data manipulation program)

Instructions are presented on the use of ADMAP, (automatic data manipulation program) an aerospace data manipulation computer program. The program was developed to aid in processing, reducing, plotting, and publishing electric propulsion trajectory data generated by the low thrust optimization program, HILTOP. The program has the option of generating SC4020 electric plots, and therefore requires the SC4020 routines to be available at excution time (even if not used). Several general routines are present, including a cubic spline interpolation routine, electric plotter dash line drawing routine, and single parameter and double parameter sorting routines. Many routines are tailored for the manipulation and plotting of electric propulsion data, including an automatic scale selection routine, an automatic curve labelling routine, and an automatic graph titling routine. Data are accepted from either punched cards or magnetic tape.

Mann, F. I.↗
NASA NTRS2017

GEONEX: Land Monitoring From a New Generation of Geostationary Satellite Sensors

The latest generation of geostationary satellites carry sensors such as ABI (Advanced Baseline Imager on GOES-16) and the AHI (Advanced Himawari Imager on Himawari) that closely mimic the spatial and spectral characteristics of Earth Observing System flagship MODIS for monitoring land surface conditions. More importantly they provide observations at 5-15 minute intervals. Such high frequency data offer exciting possibilities for producing robust estimates of land surface conditions by overcoming cloud cover, enabling studies of diurnally varying local-to-regional biosphere-atmosphere interactions, and operational decision-making in agriculture, forestry and disaster management. But the data come with challenges that need special attention. For instance, geostationary data feature changing sun angle at constant view for each pixel, which is reciprocal to sun-synchronous observations, and thus require careful adaptation of EOS algorithms. Our goal is to produce a set of land surface products from geostationary sensors by leveraging NASA's investments in EOS algorithms and in the data/compute facility NEX. The land surface variables of interest include atmospherically corrected surface reflectances, snow cover, vegetation indices and leaf area index (LAI)/fraction of photosynthetically absorbed radiation (FPAR), as well as land surface temperature and fires. In order to get ready to produce operational products over the US from GOES-16 starting 2018, we have utilized 18 months of data from Himawari AHI over Australia to test the production pipeline and the performance of various algorithms for our initial tests. The end-to-end processing pipeline consists of a suite of modules to (a) perform calibration and automatic georeference correction of the AHI L1b data, (b) adopt the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to produce surface spectral reflectances along with compositing schemes and QA, and (c) modify relevant EOS retrieval algorithms (e.g., LAI and FPAR, GPP, etc.) for subsequent science product generation. Initial evaluation of Himawari AHI products against standard MODIS products indicate general agreement, suggesting that data from geostationary sensors can augment low earth orbit (LEO) satellite observations.

geostationary↗
DOE OSTI2025

Identification of uranium oxidation states using oxygen K-edge scanning transmission X-ray microscopy

The field of nuclear forensics is growing in importance, and the increasing capabilities at synchrotron radiation light sources enable non-destructive characterization of oxide particles with better spatial, compositional, and oxidation state speciation resolution than ever before. Here, uranium oxide particles derived from multiple wet chemical processing methods were examined using a scanning transmission X-ray microscope (STXM), and a weakly-supervised method was developed to automatically analyze the collected data. Multiple uranium oxidation states were observed and quantified within and between samples, yielding information about differences between particles produced via the various processing routes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Image Deconvolution and Point-spread Function Reconstruction with STARRED: A Wavelet-based Two-channel Method Optimized for Light-curve Extraction

We present starred, a point-spread function (PSF) reconstruction, two-channel deconvolution, and light-curve extraction method designed for high-precision photometric measurements in imaging time series. An improved resolution of the data is targeted rather than an infinite one, thereby minimizing deconvolution artifacts. In addition, starred performs a joint deconvolution of all available data, accounting for epoch-to-epoch variations of the PSF and decomposing the resulting deconvolved image into a point source and an extended source channel. The output is a high-signal-to-noise-ratio, high-resolution frame combining all data and the photometry of all point sources in the field of view as a function of time. Of note, starred also provides exquisite PSF models for each data frame. We showcase three applications of starred in the context of the imminent LSST survey and of JWST imaging: (i) the extraction of supernovae light curves and the scene representation of their host galaxy; (ii) the extraction of lensed quasar light curves for time-delay cosmography; and (iii) the measurement of the spectral energy distribution of globular clusters in the "Sparkler," a galaxy at redshift z = 1.378 strongly lensed by the galaxy cluster SMACS J0723.3-7327. starred is implemented in jax, leveraging automatic differentiation and graphics processing unit acceleration. This enables the rapid processing of large time-domain data sets, positioning the method as a powerful tool for extracting light curves from the multitude of lensed or unlensed variable and transient objects in the Rubin-LSST data, even when blended with intervening objects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Learning from Correlated Events for Equipment Relation Inference in Buildings

Modern buildings produce thousands of data streams, and the ability to automatically infer the physical context of such data is the key to enabling building analytics at scale. As acquiring this contextual information is currently a time-consuming and error-prone manual process, in this study we make the first attempt at automatically inferring one important contextual aspect of the equipment in buildings — how each equipment is functionally connected with another. The main insight behind our solution is that functionally connected equipment is exposed to the same events in the physical world, creating correlated changes in the time series data of both equipment. Because events are of indeterminate length in time series, however, identifying them requires solving a non-polynomial combinatorial data segmentation problem. We present a solution that first extracts latent events from the sensory time series data, and then sifts out coincident events with a customized correlation procedure to identify the relationship between equipment.We evaluated our approach on data collected from over 1,000 pieces of equipment from 5 commercial buildings of various sizes located in different geographical regions in the US. Results show that this approach achieves 94.38% accuracy in relation inference, compared to 85.49% by the best baseline.

Relation inference, smart buildings, time series a↗
DOE OSTI2020

Automatic Classification of Biological Targets in a Tidal Channel Using a Multibeam Sonar

Multibeam sonars are widely used for environmental monitoring of fauna at marine renewable energy sites. However, they can rapidly accrue vast volumes of data, which poses a challenge for data processing. Here, using data from a deployment in a tidal channel with peak currents of 1–2 m s –1 , we demonstrate the data-reduction benefits of real-time automatic classification of targets detected and tracked in multibeam sonar data. First, we evaluate classification capabilities for three machine learning algorithms: random forests, support vector machines, and k-nearest neighbors. For each algorithm, a hill-climbing search optimizes a set of hand-engineered attributes that describe tracked targets. Here, the random forest algorithm is found to be most effective—in postprocessing, discriminating between biological and nonbiological targets with a recall rate of 0.97 and a precision of 0.60. In addition, 89% of biological targets are correctly classified as either seals, diving birds, fish schools, or small targets. Model dependence on the volume of training data is evaluated. Second, a real-time implementation of the model is shown to distinguish between biological targets and nonbiological targets with nearly the same performance as in postprocessing. From this, we make general recommendations for implementing real-time classification of biological targets in multibeam sonar data and the transferability of trained models.

16 TIDAL AND WAVE POWER↗
DOE OSTI2024

Learning thermodynamically constrained equations of state with uncertainty

Numerical simulations of high energy-density experiments require equation of state (EOS) models that relate a material’s thermodynamic state variables—specifically pressure, volume/density, energy, and temperature. EOS models are typically constructed using a semi-empirical parametric methodology, which assumes a physics-informed functional form with many tunable parameters calibrated using experimental/simulation data. Since there are inherent uncertainties in the calibration data (parametric uncertainty) and the assumed functional EOS form (model uncertainty), it is essential to perform uncertainty quantification (UQ) to improve confidence in EOS predictions. Model uncertainty is challenging for UQ studies since it requires exploring the space of all possible physically consistent functional forms. Thus, it is often neglected in favor of parametric uncertainty, which is easier to quantify without violating thermodynamic laws. This work presents a data-driven machine learning approach to constructing EOS models that naturally captures model uncertainty while satisfying the necessary thermodynamic consistency and stability constraints. We propose a novel framework based on physics-informed Gaussian process regression (GPR) that automatically captures total uncertainty in the EOS and can be jointly trained on both simulation and experimental data sources. A GPR model for the shock Hugoniot is derived, and its uncertainties are quantified using the proposed framework. We apply the proposed model to learn the EOS for the diamond solid state of carbon using both density functional theory data and experimental shock Hugoniot data to train the model and show that the prediction uncertainty is reduced by considering thermodynamic constraints.

Sharma, Himanshu (ORCID:000900050235718X)↗
DOE OSTI2018

Laboratory Studies of Particle Cementation and PV Module Soiling

Monitoring of photovoltaic (PV) systems can maintain efficient operations. However, extensive monitoring of large quantities of data can be a cumbersome process. The present work introduces a simple, inexpensive, yet effective data monitoring strategy for detecting faults and determining lost revenues automatically. This was achieved through the deployment of Raspberry Pi (RPI) device at a PV system's combiner box. The RPI was programmed to collect PV data through Modbus communications, and store the data locally in a MySQL database. Then, using a Gaussian Process Regression algorithm the RPI device was able to accurately estimate string level current, voltage, and power values. The device could also detect system faults using a Support Vector Novelty Detection algorithm. Finally, the RPI was programmed to output the potential lost revenue caused by the abnormal condition. The system analytics information was then displayed on a user interface. The interface could be accessed by operations personal to direct maintenance activity so that critical issues can be solved quickly.

14 SOLAR ENERGY↗
NASA NTRS1971

Automated geologic mapping using rock reflectances.

Investigation of the feasibility of preparing geologic maps automatically through computer processing of calibrated narrow-band visible and near IR reflectivity data collected with a 12-channel scanner. Five procedures were followed in the computer analysis of the radiances recorded as voltages on analog magnetic tape. Three recognition maps have been generated iteratively using a progressively more complex classification scheme. The overall accuracy of the first recognition map was 80%, but the discrimination of the limestone and dolomite was only 50-60%. All three maps are very accurate outcrop maps. The results demonstrate the feasibility of automated geologic mapping in this relatively simple setting.

Watson, R. D.↗
NASA NTRS1985

Measurement of wind near the surface

A survey is presented of instrumentation suitable for measurement of wind near the surface of the earth by using in-situ techniques and further restricted to sensors that are operational. In this case, a sensor is deemed to be operational if it is commercially available. There is no discussion here of the systems that might be used to acquire, process, display, and store the sensor data. It is assumed that some sort of automatic data logging equipment would be used. Without special requirements such as need for high frequency response, low power consumption, etc., this treatment must be quite general and provides little detail. Also, without special restrictions, emphasis must be placed on conventional sensors that provide the bulk of wind data today.

Brock, Fred V.↗
NASA NTRS2014

Adventures in Modern Time Series Analysis: From the Sun to the Crab Nebula and Beyond

With the generation of long, precise, and finely sampled time series the Age of Digital Astronomy is uncovering and elucidating energetic dynamical processes throughout the Universe. Fulfilling these opportunities requires data effective analysis techniques rapidly and automatically implementing advanced concepts. The Time Series Explorer, under development in collaboration with Tom Loredo, provides tools ranging from simple but optimal histograms to time and frequency domain analysis for arbitrary data modes with any time sampling. Much of this development owes its existence to Joe Bredekamp and the encouragement he provided over several decades. Sample results for solar chromospheric activity, gamma-ray activity in the Crab Nebula, active galactic nuclei and gamma-ray bursts will be displayed.

solar chromospheric activity↗
NASA NTRSDate not supplied

Evaluating Machine Learning Approaches to Plume Tracking

On July 15, 2022, the Hunga Tonga-Hunga Ha’apai (HTHH) submarine volcano erupted, propelling trace gasses and ash through the troposphere and up into the stratosphere. Previous studies manually tracked the aerosol and trace gas plumes over time across different positions in the southern hemisphere. Using imagery from NASA’s Earth Observing System, including MODIS aerosol products and OMI sulfur dioxide products, this research demonstrates how open-source machine learning (ML) models, like Meta’s Segment Anything Model (SAM), can perform automatic plume tracking following the Hunga Tonga eruption. This extensible methodology, and modular data processing and modeling pipeline, establishes a framework for systematically and rapidly studying natural disasters, including additional volcanic eruptions and large-scale wildfires. By combining advanced machine learning techniques, such as SAM’s zero-shot learning, with large volumes of NASA’s Earth Observation and remote sensing data, this work shows how AI and open science can accelerate research and generate actionable results, even for unprecedented events. The tools and technologies presented here can help translate earth science to action from NASA’s current and future Earth observing satellite missions, and assist researchers and stakeholders in understanding, mapping, and responding to natural disasters in a changing world.

machine learning↗
NASA NTRSDate not supplied

Tracking the Hunga Tonga-Hunga Ha’apai Eruption Stratospheric Aerosol and Trace Gas Plumes Using Machine Learning

On January 15, 2022, the Hunga Tonga-Hunga Ha’apai (hereafter, Hunga Tonga) submarine volcano had an explosive eruption that thrusted ash, gases, and water vapor through the troposphere into the stratosphere and mesosphere. Previous studies manually tracked the aerosol and trace gas plumes over time across different positions in the southern hemisphere. Using data retrieved from low earth orbiting satellite instruments (e.g., OMPS, OMI, and CALIPSO), this research demonstrates how open-source machine learning (ML) models, like Meta’s Segment Anything Model (SAM), with prompt engineering can perform automatic plume tracking following the Hunga Tonga eruption. This extensible methodology, and modular data processing and modeling pipeline using NASA Earthdata and Openscapes, establishes a framework for systematically and rapidly studying extreme events, including volcanic eruptions and large-scale wildfires. By combining advanced machine learning techniques, such as SAM’s zero-shot learning, with large volumes of remote sensing data, this work demonstrates how AI and open science can accelerate research and generate actionable results. The tools and technologies presented here can help translate earth science to action from NASA’s current and future Earth observing satellite missions (e.g., the Atmosphere Observing System (AOS)), and assist researchers and stakeholders in understanding, mapping, and responding to natural disasters and extreme events in a changing world.

David M. Giles↗
DOE OSTI2020

A Visual Analytics Framework for Reviewing Streaming Performance Data

Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.

Human-centered computing↗
NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗
NASA NTRS2018

JSC OCFO Cloud Integrated Budget Analytical Toolbox

The JSC OCFO analyst's job function relies heavily on analytics to provide every directorate customer with quality service in managing their respective budgets. This technology, configured on the Salesforce cloud platform, enhances the JSC OCFO customer user interface, provides state of the art data analytics and statistics, reduces IT costs, increases productivity, eliminates duplicate entry and errors, and utilizes machine learning and artificial intelligence to process day to day transactional tasks automatically without analyst manipulation. The unique configuration of the COTS technology allows for highly advanced analysis and reporting of government budget data. This provides the ability for complex budgets to be fully understood in detail by non-budget organizations and eliminates time consuming analysis on the OCFO analyst's behalf.

Robertson, Arden↗