Results for “near real time algorithm” — Page 4

NASA NTRS2002

Cloud Algorithm Design and Performance for the 2002 Geoscience Laser Altimeter System Mission

A satellite borne lidar instrument, the Geoscience Laser Altimeter System (GLAS), is to be launched in late 2002 and will provide continuous profiling of atmospheric clouds and aerosol on a global basis. Data processing algorithms have been developed to provide operational data products in near real time. Basic data products for cloud observations are the height of the top and bottom of single to multiple cloud layers and the lidar calibrated observed backscatter cross section up to the level of signal attenuation. In addition the optical depth and vertical profile of visible extinction cross section of many transmissive cloud layers and most haze layers are to be derived. The optical thickness is derivable in some cases from the attenuation of the molecular scattering below cloud base. In other cases an assumption of the scattering phase function is required. In both cases a estimated correction for multiple scattering is required. The data processing algorithms have been tested in part from aircraft measurements used to simulated satellite data. The GLAS lidar observations will be made from an orbit that will allow inter comparison with all other existing satellite cloud measurements.

Spinhirne, J. D.↗

NASA NTRSDate not supplied

Do Better Satellite Precipitation Algorithms Improve Landslide Hazard Assessment?

Satellites make it possible to estimate precipitation in near real time. Given the challenges of achieving global coverage by other means, these data are used widely. However, few systems for landslide hazard assessment rely on satellite precipitation estimates. This could be due in part to perceptions of accuracy, although latency, spatial resolution, and other factors may also be important. We test whether recent changes to data streams from the Global Precipitation Measurement mission (GPM) have improved its potential for use in landslide prediction. Specifically, we examine data produced by the Integrated Multi-satellitERetrievals for the GPM (IMERG) algorithm, which was upgraded to version 7 this year. IMERG relies upon other algorithms, including the Goddard Profiling Algorithm (GPROF) and the GPM Combined Radar-Radiometer Algorithm (CORRA). Many changes have been made during the switch from IMERG version 6 to version 7. These include upgrading CORRA and GPROF to version 7, to improve the accuracy of precipitation in frozen, mountainous, and coastal areas. The measured intensity of some storms has been enhanced with a new algorithm, the Scheme for Histogram Adjustment with Ranked Precipitation Estimates in the Neighborhood. Combined with many others, these changes to IMERG should improve its utility for landslide hazard assessment in a variety of contexts. To test this idea, we retrain the global Landslide Hazard Assessment for Situational Awareness (LHASA) model twice—first with data from IMERG version 6B and second with 7B. Since current daily rainfall is the most important variable in determining outcomes predicted by LHASA, it should reflect changes made to that input. First, we grid the landslides at a daily, thirty-arcsecond resolution. This serves as the response variable. At each of these sites current and antecedent rainfall are extracted, along with antecedent snow mass and soil moisture, slope, and PGA. In addition, one million grid cells are selected at random points to represent conditions under which landslides (probably) do not occur. After merging these data, we hold back 20% of the dataset for validation purposes and train a machine-learning model with the rest. We assess both the model’s overall ability to identify landslides and its ability to predict specific large landslide disasters.

Thomas A Stanley↗

NASA NTRS1976

Investigation of a Real-time Processing System for the NASA Multifrequency Microwave Radiometer

A study was conducted to investigate the data reduction and processing requirements for the multifrequency microwave radiometer system (MFMR). The objectives were to develop and evaluate algorithms and processing techniques which might provide for dedicated real time or near real time data processing and to develop a configuration design and processor recommendation to accomplish the data reduction. An analysis of the required data reduction and calibration equations was included along with the identification of sources of error which may be present in the (MFMR) data. The definition and evaluation of the significance of effects introduced by aircraft perturbation was given.

Source record↗

NASA NTRS2018

Assessing Global Surface Water Inundation Dynamics Using Combined Satellite Information from SMAP, AMSR2 and Landsat

A method to assess global land surface water (fw) inundation dynamics was developed by exploiting the enhanced fw sensitivity of L-band (1.4 GHz) passive microwave observations from the Soil Moisture Active Passive (SMAP) mission. The L-band fw (fw(sub LBand)) retrievals were derived using SMAP H-polarization brightness temperature (Tb) observations and predefined L-band reference microwave emissivities for water and land endmembers. Potential soil moisture and vegetation contributions to the microwave signal were represented from overlapping higher frequency (Tb) observations from AMSR2. The resulting (fw(sub LBand)) global record has high temporal sampling (1-3 days) and 36-km spatial resolution. The (fw(sub LBand)) annual averages corresponded favourably (R=0.84, p<0.001) with a 250-m resolution static global water map (MOD44W) aggregated at the same spatial scale, while capturing significant inundation variations worldwide. The monthly (fw(sub LBand)) averages also showed seasonal inundation changes consistent with river discharge records within six major US river basins. An uncertainty analysis indicated generally reliable (fw(sub LBand)) performance for major land cover areas and under low to moderate vegetation cover, but with lower accuracy for detecting water bodies covered by dense vegetation. Finer resolution (30-m) (fw(sub LBand)) results were obtained for three sub-regions in North America using an empirical downscaling approach and ancillary global Water Occurrence Dataset (WOD) derived from the historical Landsat record. The resulting 30-m (fw(sub LBand)) retrievals showed favourable spatial accuracy for water (70.71%) and land (98.99%) classifications and seasonal wet and dry periods when compared to independent water maps derived from Landsat-8 imagery. The new (fw(sub LBand)) algorithms and continuing SMAP and AMSR2 operations provide for near real-time, multi-scale monitoring of global surface water inundation dynamics and potential flood risk.

WOD↗

NASA NTRS2019

Real Time HABs Mapping Using NASA Glenn Hyperspectral Imager

The hyperspectral imaging system (HSI) developed by the NASA Glenn Research Center was used from 2015- 2017 to collect high spatial resolution data over Lake Erie and the Ohio River. Paired with a vicarious correction approach implemented by the Michigan Tech Research Institute, radiance data collected by the HSI system can be converted to high quality reflectance data which can be used to generate near-real time (within 24 hours) products for the monitoring of harmful algal blooms using existing algorithms. The vicarious correction method relies on imaging a spectrally constant target to normalize HSI data for atmospheric and instrument calibration signals. A large asphalt parking lot near the Western Basin of Lake Erie was spectrally characterized and was determined to be a suitable correction target. Due to the HSI deployment aboard an aircraft, it is able to provide unique insights into water quality conditions not offered by space-based solutions. Aircraft can operate under cloud cover and flight paths can be chosen and changed on-demand, allowing for far more flexibility than space-based platforms. The HSI is also able to collect data at a high spatial resolution (~1 m), allowing for the monitoring of small water bodies, the ability to detect small patches of surface scum, and the capability to monitor the proximity of blooms to targets of interest such as water intakes. With this new rapid turnaround time, airborne data can serve as a complementary monitoring tool to existing satellite platforms, targeting critical areas and responding to bloom events on-demand.

Reid W Sawtell↗

NASA NTRS2019

Real Time HABs Mapping Using NASA Glenn Hyperspectral Imager

The hyperspectral imaging system (HSI) developed by the NASA Glenn Research Center was used from 2015- 2017 to collect high spatial resolution data over Lake Erie and the Ohio River. Paired with a vicarious correction approach implemented by the Michigan Tech Research Institute, radiance data collected by the HSI system can be converted to high quality reflectance data which can be used to generate near-real time (within 24 hours) products for the monitoring of harmful algal blooms using existing algorithms. The vicarious correction method relies on imaging a spectrally constant target to normalize HSI data for atmospheric and instrument calibration signals. A large asphalt parking lot near the Western Basin of Lake Erie was spectrally characterized and was determined to be a suitable correction target. Due to the HSI deployment aboard an aircraft, it is able to provide unique insights into water quality conditions not offered by space-based solutions. Aircraft can operate under cloud cover and flight paths can be chosen and changed on-demand, allowing for far more flexibility than space-based platforms. The HSI is also able to collect data at a high spatial resolution (~1 m), allowing for the monitoring of small water bodies, the ability to detect small patches of surface scum, and the capability to monitor the proximity of blooms to targets of interest such as water intakes. With this new rapid turnaround time, airborne data can serve as a complementary monitoring tool to existing satellite platforms, targeting critical areas and responding to bloom events on-demand.

Reid W Sawtell↗

NASA NTRS1988

Optimal guidance with obstacle avoidance for nap-of-the-earth flight

The development of automatic guidance is discussed for helicopter Nap-of-the-Earth (NOE) and near-NOE flight. It deals with algorithm refinements relating to automated real-time flight path planning and to mission planning. With regard to path planning, it relates rotorcraft trajectory characteristics to the NOE computation scheme and addresses real-time computing issues and both ride quality issues and pilot-vehicle interfaces. The automated mission planning algorithm refinements include route optimization, automatic waypoint generation, interactive applications, and provisions for integrating the results into the real-time path planning software. A microcomputer based mission planning workstation was developed and is described. Further, the application of Defense Mapping Agency (DMA) digital terrain to both the mission planning workstation and to automatic guidance is both discussed and illustrated.

Pekelsma, Nicholas J.↗

NASA NTRS2015

Improved Rainfall Estimates and Predictions for 21st Century Drought Early Warning

As temperatures increase, the onset and severity of droughts is likely to become more intense. Improved tools for understanding, monitoring and predicting droughts will be a key component of 21st century climate adaption. The best drought monitoring systems will bring together accurate precipitation estimates with skillful climate and weather forecasts. Such systems combine the predictive power inherent in the current land surface state with the predictive power inherent in low frequency ocean-atmosphere dynamics. To this end, researchers at the Climate Hazards Group (CHG), in collaboration with partners at the USGS and NASA, have developed i) a long (1981-present) quasi-global (50degS-50degN, 180degW-180degE) high resolution (0.05deg) homogenous precipitation data set designed specifically for drought monitoring, ii) tools for understanding and predicting East African boreal spring droughts, and iii) an integrated land surface modeling (LSM) system that combines rainfall observations and predictions to provide effective drought early warning. This talk briefly describes these three components. Component 1: CHIRPS The Climate Hazards group InfraRed Precipitation with Stations (CHIRPS), blends station data with geostationary satellite observations to provide global near real time daily, pentadal and monthly precipitation estimates. We describe the CHIRPS algorithm and compare CHIRPS and other estimates to validation data. The CHIRPS is shown to have high correlation, low systematic errors (bias) and low mean absolute errors. Component 2: Hybrid statistical-dynamic forecast strategies East African droughts have increased in frequency, but become more predictable as Indo- Pacific SST gradients and Walker circulation disruptions intensify. We describe hybrid statistical-dynamic forecast strategies that are far superior to the raw output of coupled forecast models. These forecasts can be translated into probabilities that can be used to generate bootstrapped ensembles describing future climate conditions. Component 3: Assimilation using LSMs CHIRPS rainfall observations (component 1) and bootstrapped forecast ensembles (component 2) can be combined using LSMs to predict soil moisture deficits. We evaluate the skill such a system in East Africa, and demonstrate results for 2013.

Funk, Chris↗

NASA NTRS2015

Improved Rainfall Estimates and Predictions for 21st Century Drought Early Warning

As temperatures increase, the onset and severity of droughts is likely to become more intense. Improved tools for understanding, monitoring and predicting droughts will be a key component of 21st century climate adaption. The best drought monitoring systems will bring together accurate precipitation estimates with skillful climate and weather forecasts. Such systems combine the predictive power inherent in the current land surface state with the predictive power inherent in low frequency ocean-atmosphere dynamics. To this end, researchers at the Climate Hazards Group (CHG), in collaboration with partners at the USGS and NASA, have developed i) a long (1981-present) quasi-global (50degS-50degN, 180degW-180degE) high resolution (0.05deg) homogenous precipitation data set designed specifically for drought monitoring, ii) tools for understanding and predicting East African boreal spring droughts, and iii) an integrated land surface modeling (LSM) system that combines rainfall observations and predictions to provide effective drought early warning. This talk briefly describes these three components. Component 1: CHIRPS The Climate Hazards group InfraRed Precipitation with Stations (CHIRPS), blends station data with geostationary satellite observations to provide global near real time daily, pentadal and monthly precipitation estimates. We describe the CHIRPS algorithm and compare CHIRPS and other estimates to validation data. The CHIRPS is shown to have high correlation, low systematic errors (bias) and low mean absolute errors. Component 2: Hybrid statistical-dynamic forecast strategies East African droughts have increased in frequency, but become more predictable as Indo- Pacific SST gradients and Walker circulation disruptions intensify. We describe hybrid statistical-dynamic forecast strategies that are far superior to the raw output of coupled forecast models. These forecasts can be translated into probabilities that can be used to generate bootstrapped ensembles describing future climate conditions. Component 3: Assimilation using LSMs CHIRPS rainfall observations (component 1) and bootstrapped forecast ensembles (component 2) can be combined using LSMs to predict soil moisture deficits. We evaluate the skill such a system in East Africa, and demonstrate results for 2013.

Funk, Chris↗

DOE OSTI2022

A Wrapper to Use a Machine-Learning-Based Algorithm for Earthquake Monitoring

Seismology is one of the main sciences used to monitor volcanic activity worldwide. Fast, efficient, and accurate seismicity detectors are crucial to assess the activity level of a volcano in near–real time and to issue timely warnings. Traditional real–time seismic processing software uses phase onset pickers followed by a phase association algorithm to declare an event and estimate its location. The pickers typically do not identify whether the detected phase is a P or S arrival, which can have a negative impact on hypocentral location quality and complicates phase association. We implemented the deep–neural–network–based method PhaseNet to identify in real time P and S seismic waves on data from one– and three–component seismometers. We tuned the Earthworm binder_ew associator module to use the phase identification from PhaseNet to detect and locate the events, which we archive in a SeisComP3 database. We assessed the performance of the algorithm by comparing the results with existing catalogs built to monitor seismic and volcanic activity in Mayotte and the Lesser Antilles region. Our algorithm, which we refer to as PhaseWorm, showed promising results in both contexts and clearly outperformed the previous automatic method implemented in Mayotte. As a result, this innovative real–time processing system is now operational for seismicity monitoring in Mayotte and Martinique.

58 GEOSCIENCES↗

NASA NTRS2019

GeoNEX: A Cloud Gateway for Near Real-time Processing of Geostationary Satellite Products

The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.

GeoNEX↗

NASA NTRS2014

Improved Methodology for Surface and Atmospheric Soundings, Error Estimates, and Quality Control Procedures: the AIRS Science Team Version-6 Retrieval Algorithm

The AIRS Science Team Version-6 AIRS/AMSU retrieval algorithm is now operational at the Goddard DISC. AIRS Version-6 level-2 products are generated near real-time at the Goddard DISC and all level-2 and level-3 products are available starting from September 2002. This paper describes some of the significant improvements in retrieval methodology contained in the Version-6 retrieval algorithm compared to that previously used in Version-5. In particular, the AIRS Science Team made major improvements with regard to the algorithms used to 1) derive surface skin temperature and surface spectral emissivity; 2) generate the initial state used to start the cloud clearing and retrieval procedures; and 3) derive error estimates and use them for Quality Control. Significant improvements have also been made in the generation of cloud parameters. In addition to the basic AIRS/AMSU mode, Version-6 also operates in an AIRS Only (AO) mode which produces results almost as good as those of the full AIRS/AMSU mode. This paper also demonstrates the improvements of some AIRS Version-6 and Version-6 AO products compared to those obtained using Version-5.

Quality Control↗

NASA NTRS2003

Mining Distance Based Outliers in Near Linear Time with Randomization and a Simple Pruning Rule

Defining outliers by their distance to neighboring examples is a popular approach to finding unusual examples in a data set. Recently, much work has been conducted with the goal of finding fast algorithms for this task. We show that a simple nested loop algorithm that in the worst case is quadratic can give near linear time performance when the data is in random order and a simple pruning rule is used. We test our algorithm on real high-dimensional data sets with millions of examples and show that the near linear scaling holds over several orders of magnitude. Our average case analysis suggests that much of the efficiency is because the time to process non-outliers, which are the majority of examples, does not depend on the size of the data set.

Bay, Stephen D.↗

NASA NTRS1994

Massively Parallel Algorithms for Real-Time Wavefront Control of a Dense Adaptive Optics System

In this paper massively parallel algorithms and architectures for real-time wavefront control of a dense adaptive optic system (SELENE) are presented. We have already shown that the computation of a near optimal control algorithm for SELENE can be reduced to the solution of a discrete Poisson equation on a regular domain. Although this represents an optimal computation, due the large size of the system and the high sampling rate requirement, the implementation of this control algorithm poses a computationally challenging problem since it demands a sustained computational throughput of the order of 10 GFlops. We develop a novel algorithm, designated as Fast Invariant Imbedding algorithm, which offers a massive degree of parallelism with simple communication and synchronization requirements. Due to these features, our algorithm is significantly more efficient than other Fast Poisson Solvers for implementation on massively parallel architectures.

massively↗

DOE OSTI2025

Harnessing Quantum Information Science for Enhancing Sensors in Harsh Fossil Energy Environments

The main goals of this project are to utilize real-time quantum dynamics simulations and quantum optimal control algorithms to (1) harness near-surface nitrogen vacancy (NV) centers to detect chemical analytes in harsh fossil energy environments, and (2) design optimally constructed electromagnetic fields for initializing these near-surface NV center spins for efficient sensor performance and detectivity.

20 FOSSIL-FUELED POWER PLANTS↗

NASA NTRS1990

ERBE and CERES broadband scanning radiometers

Broadband scanning radiometers have been used extensively on earth-orbiting satellites to measure the Earth's outgoing radiation. The resulting estimates of longwave and shortwave fluxes have played an important role in helping to understand the Earth's radiant energy balance or budget. The Clouds and the Earth Radiant Energy System (CERES) experiment is expected to include instruments with three broadband scanning radiometers. The design of the CERES instrument will draw heavily from the flight-proven Earth Radiation Budget Experiment (ERBE) scanner instrument technology and will benefit from the several years of ERBE experience in mission operations and data processing. The discussion starts with a description of the scientific objectives of ERBE and CERES. The design and operational characteristics of the ERBE and CERES instrument are compared and the two ground-based data processing systems are compared. Finally, aspects of the CERES data processing which might be performed in near real-time aboard a spacecraft platform are discussed, and the types of algorithms and input data requirements for the onboard processing system are identified.

Weaver, William L.↗

NASA NTRS2005

Accuracy of Geophysical Parameters Derived from AIRS/AMSU as a Function of Fractional Cloud Cover

AIRS was launched on EOS Aqua on May 4,2002, together with AMSU A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sounding system. The primary products of AIRS/AMSU are twice daily global fields of atmospheric temperature-humidity profiles, ozone profiles, sea/land surface skin temperature, and cloud related parameters including OLR. The sounding goals of AIRS are to produce 1 km tropospheric layer mean temperatures with an rms error of 1K, and layer precipitable water with an rms error of 20%, in cases with up to 80% effective cloud cover. The basic theory used to analyze AIRS/AMSU/HSB data in the presence of clouds, called the at-launch algorithm, was described previously. Pre-launch simulation studies using this algorithm indicated that these results should be achievable. Some modifications have been made to the at-launch retrieval algorithm as described in this paper. Sample fields of parameters retrieved from AIRS/AMSU/HSB data are presented and validated as a function of retrieved fractional cloud cover. As in simulation, the degradation of retrieval accuracy with increasing cloud cover is small. HSB failed in February 2005, and consequently HSB channel radiances are not used in the results shown in this paper. The AIRS/AMSU retrieval algorithm described in this paper, called Version 4, become operational at the Goddard DAAC in April 2005 and is being used to analyze near-real time AIRS/AMSU data. Historical AIRS/AMSU data, going backwards from March 2005 through September 2002, is also being analyzed by the DAAC using the Version 4 algorithm.

Susskind, Joel↗

DOE OSTI2021

A novel implicit hybrid machine learning model and its application for reinforcement learning

A novel methodology to develop implicit hybrid models is presented. PyTorch is used to integrate physics-based equations with machine learning models. Automatic differentiation of the hybrid model is leveraged to solve the implicit equations. Iterative solving enables gradient based updates to the machine learning model. The novel methodology is compared to an explicit hybrid approach on a continuously stirred tank reactor (CSTR). The novel method results in a lower modelling error. Both hybrid models effectively train with noisy data. To test the implicit hybrid model, it is employed as a reinforcement learning (RL) training model. The RL algorithm trained on the hybrid model outperforms real time optimization of the CSTR and performs nearly as well as RL trained directly on the CSTR and a traditional gradient based approach. Training RL directly on the CSTR requires over 60,000 system interactions compared to 6000 historical data points for hybrid model development.

42 ENGINEERING↗