Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Automatic Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package

Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.

58 GEOSCIENCES↗
NASA NTRS1971

A practical method of determining water current velocities and diffusion coefficients in coastal waters by remote sensing techniques

A simplified procedure is presented for determining water current velocities and diffusion coefficients. Dye drops which form dye patches in the receiving water are made from an aircraft. The changes in position and size of the patches are recorded from two flights over the area. The simplified data processing procedure requires only that the ground coordinates about the dye patches be determined at the time of each flight. With an automatic recording coordinatograph for measuring coordinates and a computer for processing the data, this technique provides a practical method of determining circulation patterns and mixing characteristics of large aquatic systems. This information is useful in assessing the environmental impact of waste water discharges and for industrial plant siting.

James, W. P.↗
NASA NTRS1991

Radar measurements of the orbital debris environment

The collection and preliminary processing of the first significant accumulation of orbital debris data collected by the Haystack radar is described. The data are collected in the 'beam park' mode of operation in which the radar stares in a fixed direction and debris randomly passes through the field of view. Haystack's processing system performed automatic real-time processing and threshold detection, and saved only data associated with a possible detection. Approximately 123 hours of data were collected with the radar beam parked at an elevation angle of 10 deg and an azimuth of 180 deg; 3.8 hours were collected at an elevation angle of 30 deg and an azimuth of 180 deg; and 33.9 hours were collected at an elevation angle of 90 deg. A computer model of the response of the Haystack radar while in beam park operation is described. The model calculates the probability of detection and collection area for all sizes and orbital inclinations of debris visible to the radar. At this time, the approximate sizes of the detected objects have not been determined. However, the observed detection rates and cumulative signal to noise ratio distributions agree well with the results of the radar response model using NASA's current orbital debris environment.

Stansbery, Eugene G.↗
DOE OSTI2017

TagIt: An Integrated Indexing and Search Service for File Systems

Data services such as search, discovery, and management in scalable distributed environments have traditionally been decoupled from the underlying file systems, and are often deployed using external databases and indexing services. However, modern data production rates, looming data movement costs, and the lack of metadata, entail revisiting the decoupled file system-data services design philosophy.In this paper, we present TagIt, a scalable data management service framework aimed at scientific datasets, which is tightly integrated into a shared-nothing distributed file system. A key feature of TagIt is a scalable, distributed metadata indexing framework, using which we implement a flexible tagging capability to support data discovery. The tags can also be associated with an active operator, for pre-processing, filtering, or automatic metadata extraction, which we seamlessly offload to file servers in a load-aware fashion. Our evaluation shows that TagIt can expedite data search by up to 10X over the extant decoupled approach.

Sim, Hyogi↗
NASA NTRS1967

Automatic telemetry checkout system

Telemetry checkout station is designed to automatically perform measurements on the vehicle telemetry. Its features include real-time digitizing and computer controlled station setup, data processing, and self-check. The station can handle a wide variety of automatic tests by changing its computer programs.

George, W. V.↗
NASA NTRSDate not supplied

Global 3D Data Visualization and Analysis Platform with Advanced Machine Learning Capabilities in Support of Lunar Exploration

Introduction: The science goals for NASA’s Artemis program include: a) Understanding the character and origin of lunar polar volatiles, b) Conducting experimental science in the lunar environment and c) Investigating and mitigating exploration risks. The permanently shadowed regions (PSRs) on the Lunar south pole are expected to host large quantities of water-ice and volatiles that are important for sustainable Lunar exploration. There are several missions such as onboard Korea Pathfinder Lunar Orbiter (KPLO: Korean name Danuri) with onboard ShadowCam camera, Astrobotic Peregrine Mission One [4], and other efforts underway to obtain high resolution topographic, minerals, volatiles and other information on the moon. We envision a need in immediate future for platforms to integrate these data sets, provide rendering and visualization capabilities in the context of a 3D Lunar globe for easier information access and analysis. NASA's Celestial Mapping System (CMS) is developed to address the need for 3D tools for planetary science investigations, mission planning, in-situ operations, in a 3D-first design constructed around a unified view of a planetary globe. At present CMS provides many critical functionalities that include: 1) equipment planning and optimized placement on Lunar surface 2) line-of-sight (LOS) analysis 3) powerful measurement tools based on 3D terrain with realistic 3D models to represent rovers, astronauts and equipment 4) visualization of derived mapping products (e.g. resource maps), and 5) a data engine for hosting new observations that are not available in other contemporary lunar data tools. Planetary Data Ingestion: CMS can consume and analyze data from locally hosted and external third party sources. It is compatible with Open Geospatial Consortium (OGC) data and file standards and currently integrates datasets from the Astrogeology Science Center of USGS. This includes global and local data acquired from NASA (LRO, Clementine, Lunar Orbiter) and JAXA (SELENE/Kaguya), with capability of integrating more datasets. In addition, users can specify other WMS-hosted data endpoints, which CMS can then query and stream data from automatically. To set-up an automated process for ingestion and accurate rendering, visualization and analysis of external 3rd party planetary datasets within CMS, we initiated the process of ingesting unique dataset of super-enhanced images of the permanently shadowed regions (PSRs) at the lunar poles which were produced by the Hyper-effective nOise Removal U-net Software (HORUS) tool. This tool was developed to enhance the extremely low-light images of the interior of PSRs and provide the ability to see within these regions at and discern surface features (i.e. boulders and craters) down to 3 meters in size. We focused on the Nobile region on the Lunar south pole, selected site for VIPER mission and stitched several images to create a high-resolution map within one of the PSR of Nobile crater. Figure 1 shows the dark PSR zone form the original NAC layer of LRO as the base layer (left image) and the illuminated areas within that crater (center) which was created by ingesting and merging several of HORUS generated images. At present we employ a semi-automated process to ensure spatial accuracy and merger of several overlapping zones. However, we are in the process of completely automating this process by employing AI based techniques that would rank, sort, and stack the images based on their information density. The georectification of the images would employ selected features. Analysis on Ingested Planetary Datasets: Once an external planetary data-set is successfully ingested, georectified and merged seamlessly as a data-layer; CMS’ numerous analysis tools can be used on this data. A Line of Sight (LOS) tool has been developed for CMS which analyzes terrain profiles and obstructions to determine visibility for remote observers. Figure 1 (right image) shows the viewshed analysis on the same PSR in the Nobile region. The yellow pin shows the observer location outside the PSR. The yellow area shows the visible part of PSR. The obstructed area with no visibility for the observer is shown in red. The Measurements tool allows the user to take area and distance measurements of features on the terrain using various shapes. Measurement type can be specified in a number of ways: Line, Path, Polygon, Circle, Ellipse, Square, Rectangle or Freehand. Once the shape is specified, elevation information can then be extracted along each of these shapes. Figure 2 (left) shows the measurements performed on a crater n illuminated PSR in Nobile region. The equipment placement tool allows the user to place a 3D equipment model at a desired location and analyze its coverage area. The equipment placement tool is coupled with LOS to determine the coverage. Figure 2 (right) shows an equipment placed on the Lunar terrain and it’s coverage area. The red rays are blocked sight lines and the green rays are non-obstructed sight lines with the cyan lines showing the point of intersection with the terrain. More details are provided in the video demonstrations in Reference 5. Overcoming Polar Distortions: 3D geospatial applications exhibit significant distortions in polar imagery due to several reasons: 1) distortions in the source imagery, 2) incompatible tessellation algorithms at the poles, and 3) map projections. We are leveraging new tessellation algorithms and reprojecting data using projections that are better suited for Lunar poles. The goal is to seamlessly switch to polar projections while maintaining 3D view and navigation.

Maps↗
NASA NTRSDate not supplied

Global 3D Data Visualization and Analysis Platform With Advanced Machine Learning Capabilities in Support of Lunar Exploration

Introduction: The science goals for NASA’s Artemis program include: a) Understanding the character and origin of lunar polar volatiles, b) Conducting experimental science in the lunar environment and c) Investigating and mitigating exploration risks [1]. The permanently shadowed regions (PSRs) on the Lunar south pole are expected to host large quantities of water-ice and volatiles that are important for sustainable Lunar exploration [2]. There are several missions such as onboard Korea Pathfinder Lunar Orbiter (KPLO: Korean name Danuri) with onboard ShadowCam camera [3], Astrobotic Peregrine Mission One [4], and other efforts underway to obtain high resolution topographic, minerals, volatiles and other information on the moon. We envision a need in immediate future for platforms to integrate these data sets, provide rendering and visualization capabilities in the context of a 3D Lunar globe for easier information access and analysis. NASA's Celestial Mapping System (CMS) [5] is developed to address the need for 3D tools for planetary science investigations, mission planning, in-situ operations, in a 3D-first design constructed around a unified view of a planetary globe. At present CMS provides many critical functionalities that include: 1) equipment planning and optimized placement on Lunar surface 2) line-of-sight (LOS) analysis 3) powerful measurement tools based on 3D terrain with realistic 3D models to represent rovers, astronauts and equipment 4) visualization of derived mapping products (e.g. resource maps), and 5) a data engine for hosting new observations that are not available in other contemporary lunar data tools [5, 7]. Planetary Data Ingestion: CMS can consume and analyze data from locally hosted and external third party sources. It is compatible with Open Geospatial Consortium (OGC) data and file standards and currently integrates datasets from the Astrogeology Science Center of USGS. This includes global and local data acquired from NASA (LRO, Clementine, Lunar Orbiter) and JAXA (SELENE/Kaguya), with capability of integrating more datasets. In addition, users can specify other WMS-hosted data endpoints, which CMS can then query and stream data from automatically. To set-up an automated process for ingestion and accurate rendering, visualization and analysis of external 3rd party planetary datasets within CMS, we initiated the process of ingesting unique dataset of super-enhanced images of the permanently shadowed regions (PSRs) at the lunar poles which were produced by the Hyper-effective nOise Removal U-net Software (HORUS) tool [8]. This tool was developed to enhance the extremely low-light images of the interior of PSRs and provide the ability to see within these regions at and discern surface features (i.e. boulders and craters) down to 3 meters in size. We focused on the Nobile region on the Lunar south pole, selected site for VIPER mission and stitched several images to create a high-resolution map within one of the PSR of Nobile crater. Figure 1 shows the dark PSR zone form the original NAC layer of LRO as the base layer (left image) and the illuminated areas within that crater (center) which was created by ingesting and merging several of HORUS generated images. At present we employ a semi-automated process to ensure spatial accuracy and merger of several overlapping zones. However, we are in the process of completely automating this process by employing AI based techniques that would rank, sort, and stack the images based on their information density. The georectification of the images would employ selected features. Analysis on Ingested Planetary Datasets: Once an external planetary data-set is successfully ingested, georectified and merged seamlessly as a data-layer; CMS’ numerous analysis tools can be used on this data. A Line of Sight (LOS) tool has been developed for CMS which analyzes terrain profiles and obstructions to determine visibility for remote observers [5,6]. Figure 1 (right image) shows the viewshed analysis on the same PSR in the Nobile region. The yellow pin shows the observer location outside the PSR. The yellow area shows the visible part of PSR. The obstructed area with no visibility for the observer is shown in red. The Measurements tool allows the user to take area and distance measurements of features on the terrain using various shapes. Measurement type can be specified in a number of ways: Line, Path, Polygon, Circle, Ellipse, Square, Rectangle or Freehand. Once the shape is specified, elevation information can then be extracted along each of these shapes. Figure 2 (left) shows the measurements performed on a crater n illuminated PSR in Nobile region. The equipment placement tool allows the user to place a 3D equipment model at a desired location and analyze its coverage area. The equipment placement tool is coupled with LOS to determine the coverage. Figure 2 (right) shows an equipment placed on the Lunar terrain and it’s coverage area. The red rays are blocked sight lines and the green rays are non-obstructed sight lines with the cyan lines showing the point of intersection with the terrain. More details are provided in the video demonstrations in Reference 5. Overcoming Polar Distortions: 3D geospatial applications exhibit significant distortions in polar imagery due to several reasons: 1) distortions in the source imagery, 2) incompatible tessellation algorithms at the poles, and 3) map projections. We are leveraging new tessellation algorithms and reprojecting data using projections that are better suited for Lunar poles. The goal is to seamlessly switch to polar projections while maintaining 3D view and navigation.

Maps↗
DOE OSTI2024

Automated analysis of unlabeled PV data with Solar Data Tools software: Overview and feature updates

Distributed rooftop PV systems: ubiquitous, yet commonly have unlabeled data Difficult or impossible to form a performance index We developed Solar Data Tools (SDT), an open-source Python library for analyzing PV power (and irradiance) time-series data SDT enables analysis of unlabeled PV data—no model, no meteorological data, no performance index required Takes a statistical signal processing approach Data processing steps are largely pre-defined and automatic regardless of system type—from utility tracking systems to multi-pitch rooftop systems

Meyers-Im, Bennet E↗
NASA NTRS1980

Automated frequency standard stability data reduction

A design using a medium sized computer, a fiber optic communication link, and a desk top computer to reduce data collected at a remote site for analysis of frequency standard tests is described. Data generated at the Interim Frequency Standards Test Facility is collected on magnetic tape, which is then transported to a Sigma 5 computer. An Allan variance (two sample variance) is then calculated. The Allan variance (sigma vs tau) is then written to a data file in the Sigma 5. A fiber optic communication link between the Sigma 5 and a desk top computer was designed to transmit data in the Sigma 5 file to the desk top computer. A program was written for the desk top computer that will automatically access the data file in the Sigma 5, process the data and provide a data plot.

Knapp, L. J.↗
NASA NTRS1991

Landsat Legacy

The Landsat resources survey system spawned a number of companies engaged in commercial applications of remote sensing, among them International Imaging Systems (I2S). With initial NASA assistance, I2S has provided remote sensing hardware and software to several foreign countries, developed meteorological analysis systems, medical diagnostic software and scanning equipment for government and commercial use. Latest product is an advanced image-based photogrammetric system employing digital technology - not optical or mechanical systems - to generate terrain elevation data and other processing functions. Called PRI2SM, it compensates automatically for topographic relief displacement, is cheaper, faster, and easier to use and maintain. Company product line includes four major areas: image processing equipment for Earth Resources Management; meteorological analysis systems; satellite ground processing systems; and digital photogrammetric mapping systems.

Source record↗
NASA NTRS2012

Science Operations with the James Webb Space Telescope

The James Webb Space Telescope (JWST) will be a powerful space observatory whose four science instruments will deliver rich imaging and multiplexed spectroscopic datasets to the astronomical and planetary science communities. The ground segment for JWST, now being designed and built, will carry out JWST's science operations. The ground segment includes: (1) software that the scientific community will use to propose and specify new observations; (2) software that will schedule both science and calibration observations in a way that optimizes observing efficiency while managing the accumulation of momentum; (3) the infrastructure to regularly measure and maintain the telescope's wavefront; (4) orbit determination, ranging, and tracking; (5) communication via the Deep Space Network to command the observatory and retrieve scientific data; (6) onboard scripts that execute each observing program in an event-driven fashion, with occasional interruptions for targets of opportunity or time-critical observations; and (7) a system that processes and calibrates the data into science ready products, automatically recalibrates when calibrations improve, and archives the data for timely access by the principal investigator and later worldwide access by the scientific community. This ground system builds on experience from operating the Hubble Space Telescope, while solving challenges that are unique to the James Webb Space Telescope. In this paper, we describe the elements of the JWST ground system, how it will work operationally from the perspective of the observatory itself, and how a typical user will interact with the system to turn his/her idea into scientific discovery.

Rigby, Jane↗
NASA NTRS2012

The HEASARC Swift Gamma-Ray Burst Archive: The Pipeline and the Catalog

Since its launch in late 2004, the Swift satellite triggered or observed an average of one gamma-ray burst (GRB) every 3 days, for a total of 771 GRBs by 2012 January. Here, we report the development of a pipeline that semi automatically performs the data-reduction and data-analysis processes for the three instruments on board Swift (BAT, XRT, UVOT). The pipeline is written in Perl, and it uses only HEAsoft tools and can be used to perform the analysis of a majority of the point-like objects (e.g., GRBs, active galactic nuclei, pulsars) observed by Swift. We run the pipeline on the GRBs, and we present a database containing the screened data, the output products, and the results of our ongoing analysis. Furthermore, we created a catalog summarizing some GRB information, collected either by running the pipeline or from the literature. The Perl script, the database, and the catalog are available for downloading and querying at the HEASARC Web site.

BURST ARCHIVE↗
NASA NTRS1995

Results of an Automated GPS Tracking System in Support of Topex/Poseidon and GPSMet

A fully automated near real-time GPS tracking system has been developed around JPL's GIPSY/OASIS II software. The system produces <25 cm (3D rms) GPS orbits and one-half nanosecond (15 cm) clock estimates. The process starts automatically when a favorable global distribution of ground data from the IGS network (International GPS Service for Geodynamics) becomes available. Ionospherically corrected phase and pseudorange data are optimally combined to remove satellite and ground receiver clock errors, including selective availability. After the GPS orbits are determined within the data arc, they are then propagated with empirically determined dynamic force models. Real-time <2 meter (3D rms) GPS orbits are always available. As a by-product of this process, other calibration estimates such as station clocks, troposphere estimates, and Earth orientation parameters are also produced. For daily arc fits, the process requires 7-8 hours of CPU time on an HP9000/735 workstation.

GPS↗
DOE OSTI2021

Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Ghoshal, Devarshi↗
DOE OSTI2017

Adaptive Self-Tuning of Signal Detection Parameters

The quality of automatic detections from sensor networks depends on a large number of data processing parameters that interact in complex ways. The largely manual process of identifying effective parameters is painstaking and does not guarantee that the resulting controls are the optimal configuration settings, yet achieving superior automatic detection of events is closely related to these parameters. We present an automated sensor tuning (AST) system that tunes effective parameter settings for each sensor detector to the current state of the environment by leveraging cooperation within a neighborhood of sensors. After a stabilization period, the AST algorithm can adapt in near real-time to changing conditions and automatically self-tune a signal detector to identify (detect) only signals from events of interest. The overall goal is to reduce the number of missed legitimate event detections and the number of false event detections. Our current work focuses on reducing false signal detections early in the seismic signal processing pipeline, which leads to fewer false events and has a significant impact on reducing analyst time and effort. Applicable both for existing sensor performance boosting and new sensor deployment, this system provides an important new method to automatically tune complex remote sensing systems. Systems tuned in this way will achieve better performance than is currently possible by manual tuning, and with much less time and effort devoted to the tuning process. With ground truth on detections from a seismic sensor network monitoring the Mount Erebus Volcano in Antarctica, we show that AST increases the probability of detection while decreasing false alarms.

58 GEOSCIENCES↗
NASA NTRS1989

Optimum data weighting and error calibration for estimation of gravitational parameters

A new technique was developed for the weighting of data from satellite tracking systems in order to obtain an optimum least-squares solution and an error calibration for the solution parameters. Data sets from optical, electronic, and laser systems on 17 satellites in GEM-T1 Goddard Earth Model-T1 (GEM-T1) were employed toward application of this technique for gravity field parameters. Also GEM-T2 (31 satellites) was recently computed as a direct application of the method and is summarized. The method employs subset solutions of the data associated with the complete solution to agree with their error estimates. With the adjusted weights the process provides for an automatic calibration of the error estimates for the solution parameters. The data weights derived are generally much smaller than corresponding weights obtained from nominal values of observation accuracy or residuals. Independent tests show significant improvement for solutions with optimal weighting. The technique is general and may be applied to orbit parameters, station coordinates, or other parameters than the gravity model.

Lerch, Francis J.↗