Forecasts and appraisals for management evaluation, volume 2
Forecasting methodology for Apollo project - weight-performance data, mathematical models, probable error relationships, and manual to computational system
MATHEMATICAL MODEL↗
SEARCH · Engineering Papers
Results for “forecast error”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Tropical forecasting - Predictability perspective
Results are presented of classical predictability studies and forecast experiments with observed initial conditions to show the nature of initial error growth and final error equilibration for the tropics and midlatitudes, separately. It is found that the theoretical upper limit of tropical circulation predictability is far less than for midlatitudes. The error growth for a complete general circulation model is compared to a dry version of the same model in which there is no prognostic equation for moisture, and diabatic heat sources are prescribed. It is found that the growth rate of synoptic-scale errors for the dry model is significantly smaller than for the moist model, suggesting that the interactions between dynamics and moist processes are among the important causes of atmospheric flow predictability degradation. Results are then presented of numerical experiments showing that correct specification of the slowly varying boundary condition of SST produces significant improvement in the prediction of time-averaged circulation and rainfall over the tropics.
Shukla, J.↗
Integrated Precision Landing Performance and Technology Assessments of a Human-Scale Mars Lander Using a Generalized Simulation Framework
Human-scale missions to Mars will likely require multiple landers delivered precisely to designated locations. The current NASA human Mars reference architecture assumes delivery of three 25 t payloads from a 1- or 5-Sol orbit to the surface with a landing precision of 50 m to ensure logistics are located near the habitat. While initial navigation estimates improve with on-orbit ground tracking, errors increase during post-deorbit coast. Likewise, Mars atmospheric variability and forecasting uncertainty means that the entry vehicle guidance, navigation, and control systems must be robust to accommodate landing during any time of day or Mars year, including during dust storms. Precision landing technologies are currently being assessed to determine if onboard navigation sensors are sufficient to enable the landing accuracy required or if additional navigation aids such as surface or orbiting beacons will be needed. This study evaluates the system performance requirements to meet the desired landing accuracy for the reference vehicle design and entry, descent, and landing concept of operations. A detailed six degree-of-freedom integrated performance simulation framework is used to perform the assessment and demonstrate that under current assumptions, onboard navigation sensors are sufficient to support precision landing.
spacecraft navigation↗
Integrated Precision Landing Performance and Technology Assessments of a Human- Scale Mars Lander Using a Generalized Simulation Framework
Human-scale missions to Mars will likely require multiple landers delivered precisely to designated locations. The current NASA human Mars reference architecture assumes delivery of three 25 t payloads from a 1- or 5-Sol orbit to the surface with a landing precision of 50 m to ensure logistics are located near the habitat. While initial navigation estimates improve with on-orbit ground tracking, errors increase during post-deorbit coast. Likewise, Mars atmospheric variability and forecasting uncertainty means that the entry vehicle guidance, navigation, and control systems must be robust to accommodate landing during any time of day or Mars year, including during dust storms. Precision landing technologies are currently being assessed to determine if onboard navigation sensors are sufficient to enable the landing accuracy required or if additional navigation aids such as surface or orbiting beacons will be needed. This study evaluates the system performance requirements to meet the desired landing accuracy for the reference vehicle design and entry, descent, and landing concept of operations. A detailed six degree-of-freedom integrated performance simulation framework is used to perform the assessment and demonstrate that under current assumptions, onboard navigation sensors are sufficient to support precision landing.
Chen, Po-Ting↗
Improving Forecast Skill by Assimilation of Quality Controlled AIRS Version 5 Temperature Soundings
The AIRS Science Team Version 5 retrieval algorithm has been finalized and is now operational at the Goddard DAAC in the processing (and reprocessing) of all AIRS data. The AIRS Science Team Version 5 retrieval algorithm contains two significant improvements over Version 4: 1) Improved physics allows for use of AIRS observations in the entire 4.3 micron CO2 absorption band in the retrieval of temperature profile T(p) during both day and night. Tropospheric sounding 15 micron CO2 observations are now used primarily in the generation of cloud cleared radiances R(sub i). This approach allows for the generation of accurate values of R(sub i) and T(p) under most cloud conditions. 2) Another very significant improvement in Version 5 is the ability to generate accurate case-by-case, level-by-level error estimates for the atmospheric temperature profile, as well as for channel-by-channel error estimates for R(sub i). These error estimates are used for Quality Control of the retrieved products. We have conducted forecast impact experiments assimilating AIRS temperature profiles with different levels of Quality Control using the NASA GEOS-5 data assimilation system. Assimilation of Quality Controlled T(p) resulted in significantly improved forecast skill compared to that obtained from analyses obtained when all data used operationally by NCEP, except for AIRS data, is assimilated. We also conducted an experiment assimilating AIRS radiances uncontaminated by clouds, as done operationally by ECMWF and NCEP. Forecast resulting from assimilated AIRS radiances were of poorer quality than those obtained assimilating AIRS temperatures.
Susskind, Joel↗
Advanced Intelligent System Application to Load Forecasting and Control for Hybrid Electric Bus
The primary motivation for this research emanates from providing a decision support system to the electric bus operators in the municipal and urban localities which will guide the operators to maintain an optimal compromise among the noise level, pollution level, fuel usage etc. This study is backed up by our previous studies on study of battery characteristics, permanent magnet DC motor studies and electric traction motor size studies completed in the first year. The operator of the Hybrid Electric Car must determine optimal power management schedule to meet a given load demand for different weather and road conditions. The decision support system for the bus operator comprises three sub-tasks viz. forecast of the electrical load for the route to be traversed divided into specified time periods (few minutes); deriving an optimal 'plan' or 'preschedule' based on the load forecast for the entire time-horizon (i.e., for all time periods) ahead of time; and finally employing corrective control action to monitor and modify the optimal plan in real-time. A fully connected artificial neural network (ANN) model is developed for forecasting the kW requirement for hybrid electric bus based on inputs like climatic conditions, passenger load, road inclination, etc. The ANN model is trained using back-propagation algorithm employing improved optimization techniques like projected Lagrangian technique. The pre-scheduler is based on a Goal-Programming (GP) optimization model with noise, pollution and fuel usage as the three objectives. GP has the capability of analyzing the trade-off among the conflicting objectives and arriving at the optimal activity levels, e.g., throttle settings. The corrective control action or the third sub-task is formulated as an optimal control model with inputs from the real-time data base as well as the GP model to minimize the error (or deviation) from the optimal plan. These three activities linked with the ANN forecaster proving the output to the GP model which in turn produces the pre-schedule of the optimal control model. Some preliminary results based on a hypothetical test case will be presented for the load forecasting module. The computer codes for the three modules will be made available fe adoption by bus operating agencies. Sample results will be provided using these models. The software will be a useful tool for supporting the control systems for the Electric Bus project of NASA.
Momoh, James↗
Constraining Numerical Geodynamo Modeling with Surface Observations
Numerical dynamo solutions have traditionally been generated entirely by a set of self-consistent differential equations that govern the spatial-temporal variation of the magnetic field, velocity field and other fields related to dynamo processes. In particular, those solutions are obtained with parameters very different from those appropriate for the Earth s core. Geophysical application of the numerical results therefore depends on correct understanding of the differences (errors) between the model outputs and the true states (truth) in the outer core. Part of the truth can be observed at the surface in the form of poloidal magnetic field. To understand these differences, or errors, we generate new initial model state (analysis) by assimilating sequentially the model outputs with the surface geomagnetic observations using an optimal interpolation scheme. The time evolution of the core state is then controlled by our MoSST core dynamics model. The final outputs (forecasts) are then compared with the surface observations as a means to test the success of the assimilation. We use the surface geomagnetic data back to year 1900 for our studies, with 5-year forecast and 20-year analysis periods. We intend to use the result; to understand time variation of the errors with the assimilation sequences, and the impact of the assimilation on other unobservable quantities, such as the toroidal field and the fluid velocity in the core.
Kuang, Weijia↗
Sampling and Representativeness for a Spaceborne Wind Lidar
The capability of an Earth-orbiting lidar to produce a reliable windfield sampling with an error of 1-3 ms(exp -1) and that is relevant to numerical forecasting and climate studies is discussed. The spatial and temporal resolutions range from 100 km to 1000 km and 3 hr to 1 month respectively. In this respect cloud obstruction is of great concern, as it can prevent the lidar probing whole parts of the atmosphere, for both short and long periods of time. A worldwide analysis of cloudiness from either visual observations made from the ground or else from the Stratospheric Aerosol and Gas Experiment (SAGE) instrument, has shown that the average cloud cover is 60 percent. This analysis is only valid at large scales (time and space) compatible with the currently operating satellites. However, a lidar footprint is only about a hundred meters or less, and it was observed from ground based lidars that, even when the cloudiness is 100 percent, some lidar shots get through. A complete analysis of the probability of probing through clouds by a spaceborne lidar is presently beyond our capacities. Hence, no small scale cloudiness data set is yet available representing all kinds of meteorological situations at all latitudes.
Lieutaud, F.↗
Low-frequency predictability of the Dynamical Extended-Range Forecast Experiment
The objective of the study was to analyze data from the Dynamical Extended-Range Forecast Experiment conducted from January 1986 to March 1987, and to evaluate differences between analysis and forecasts with emphasis on the tropical 30-50-day oscillation. The diagnostic toll used is the projection of analysis and forecast data onto the normal modes of a primitive equation model. Examination of zonal-wind anomalies in the tropics shows that the forecast model predicts propagation of intraseasonal variations more accurately for slow propagation rates. The forecast amplitude is generally weaker than the analyzed amplitude. Analyzed kinetic energy and error fields exhibit similar horizontal scales for internal and external modes. External Rossby-mode components maximize in the extratropics while Rossby internal modes exhibit patterns that extend over the entire globe.
Nogues-Peagle, Julia↗
Designing an Optimal Ensemble Strategy for GMAO S2S Forecast System
GMAO Sub/Seasonal prediction system (S2S) is being readied for a major upgrade to GEOS-S2S Version 3. An important factor in successful extended range forecast is the definition of an ensemble For initialization of the ensemble we propose a combination of lagged and burst initial conditions. We plan to run a relatively large ensemble of 40 members for sub-seasonal forecast (up to 3 months), at which point we sub-sample the ensemble, and continue the forecast with 10 members (up to 12 months). Here we present the results of the extensive testing of various ways to generate the perturbations to the initial conditions and the validation of the stratified sampling strategy we chose.To generate perturbations for the burst ensemble members we used scaled differences of pairs of analysis states separated by 1-10 days, randomly chosen from a corresponding season. We considered perturbing separately only the atmospheric fields or only the ocean or both of the forecast initial conditions. Considering varying separation times between the analysis states, we were able to produce perturbations sampling various modes of variability. Focusing on the ENSO SST indices, we found that all types of perturbations are important for the ensemble spread.Our ensemble size for sub-seasonal forecasts was determined as to maximize the skill of predicting some of the leading modes of boreal winter atmospheric modes, NAO, PNA and AO. It is not feasible to run equally large ensemble for seasonal forecasts. Using a stratified sampling procedure we can identify the emerging directions of error growth. By comparing the stratified ensemble with randomly sampled ensemble of the same size, we were able to show that the former better estimates the mean of the original large ensemble.
Borovikov, Anna↗
Gaussian Process for Flight Delay Prediction: Learning a Stochastic Process
This paper presents a machine-learning approach to predict flight delays. Whereas neural networks are extensively studied for predictive capabilities, they involve non-intuitive design and extensive analysis, particularly in training and optimization processes. Instead, the proposed framework employs Gaussian Processes as a supervised learning technique for flight delay prediction. This data-driven approach trains the model using prior information, specifically the mean and covariance tied to existing data. The proposed Gaussian Process Regression (GPR) model employs the day of flight as a pivotal feature for delay forecasting. We analyze flights from various routes and gauge the accuracy of the presented learning technique by comparing the predicted delays with the actual ones. Given the inherent challenges in precisely forecasting delays, we predict the delays with a 95 % confidence interval. Also, an error propagation analysis in the prediction horizon is carried out to determine the optimal time frame for prediction. The proposed method for flight delay prediction is important as airlines can strategize flight operations and issue timely advisories.
stochastic↗
Quantifying Errors in 3D CME Parameters Derived from Synthetic Data Using White-Light Reconstruction Techniques
Current efforts in space weather forecasting of CMEs have been focused on predicting their arrival time and magnetic structure. To make these predictions, methods have been developed to derive the true CME speed, size, position, and mass, among others. Difficulties in determining the input parameters for CME forecasting models arise from the lack of direct measurements of the coronal magnetic fields and uncertainties in estimating the CME 3D geometric and kinematic parameters after eruption. White-light coronagraph images are usually employed by a variety of CME reconstruction techniques that assume more or less complex geometries. This is the first study from our International Space Science Institute (ISSI) team “Understanding Our Capabilities in Observing and Modeling Coronal Mass Ejections”, in which we explore how subjectivity affects the 3D CME parameters that are obtained from the Graduated Cylindrical Shell (GCS) reconstruction technique, which is widely used in CME research. To be able to quantify such uncertainties, the “true” values that are being fitted should be known, which are impossible to derive from observational data. We have designed two different synthetic scenarios where the “true” geometric parameters are known in order to quantify such uncertainties for the first time. We explore this by using two sets of synthetic data: 1) Using the ray-tracing option from the GCS model software itself, and 2) Using 3D magnetohydrodynamic (MHD) simulation data from the Magnetohydrodynamic Algorithm outside a Sphere code. Our experiment includes different viewing configurations using single and multiple viewpoints. CME reconstructions using a single viewpoint had the largest errors and error ranges overall for both synthetic GCS and simulated MHD white-light data. As the number of viewpoints increased from one to two, the errors decreased by approximately 4° in latitude, 22° in longitude, 14° in tilt, and 10° in half-angle. Our results quantitatively show the critical need for at least two viewpoints to be able to reduce the uncertainty in deriving CME parameters. We did not find a significant decrease in errors when going from two to three viewpoints for our specific hypothetical three spacecraft scenario using synthetic GCS white-light data. As we expected, considering all configurations and numbers of viewpoints, the mean absolute errors in the measured CME parameters are generally significantly higher in the case of the simulated MHD white-light data compared to those from the synthetic white-light images generated by the GCS model. We found the following CME parameter error bars as a starting point for quantifying the minimum error in CME parameters from white-light reconstructions: Δθ (latitude)=6° +2° -3° , Δϕ (longitude)=11° +18° -6° , Δγ (tilt)=25° +8° -7° , Δx (half-angle)=10° +12° -6° , Δh (height)=0.6 +1.2 -0.4 R ⨀ , and Δκ (ratio)=0.1 +0.03 -0.02 .
Coronal mass ejections↗
Quantifying Errors in 3D CME Parameters Derived From Synthetic Data Using White-Light Reconstruction Techniques
Current efforts in space weather forecasting of CMEs have been focused on predicting their arrival time and magnetic structure. To make these predictions, methods have been developed to derive the true CME speed, size, position, and mass, among others. Difficulties in determining the input parameters for CME forecasting models arise from the lack of direct measurements of the coronal magnetic fields and uncertainties in estimating the CME 3D geometric and kinematic parameters after eruption. White-light coronagraph images are usually employed by a variety of CME reconstruction techniques that assume more or less complex geometries. This is the first study from our International Space Science Institute (ISSI) team “Understanding Our Capabilities in Observing and Modeling Coronal Mass Ejections”, in which we explore how subjectivity affects the 3D CME parameters that are obtained from the Graduated Cylindrical Shell (GCS) reconstruction technique, which is widely used in CME research. To be able to quantify such uncertainties, the “true” values that are being fitted should be known, which are impossible to derive from observational data. We have designed two different synthetic scenarios where the “true” geometric parameters are known in order to quantify such uncertainties for the first time. We explore this by using two sets of synthetic data: 1) Using the ray-tracing option from the GCS model software itself, and 2) Using 3D magnetohydrodynamic (MHD) simulation data from the Magnetohydrodynamic Algorithm outside a Sphere code. Our experiment includes different viewing configurations using single and multiple viewpoints. CME reconstructions using a single viewpoint had the largest errors and error ranges overall for both synthetic GCS and simulated MHD white-light data. As the number of viewpoints increased from one to two, the errors decreased by approximately 4° in latitude, 22° in longitude, 14° in tilt, and 10° in half-angle. Our results quantitatively show the critical need for at least two viewpoints to be able to reduce the uncertainty in deriving CME parameters. We did not find a significant decrease in errors when going from two to three viewpoints for our specific hypothetical three spacecraft scenario using synthetic GCS white-light data. As we expected, considering all configurations and numbers of viewpoints, the mean absolute errors in the measured CME parameters are generally significantly higher in the case of the simulated MHD white-light data compared to those from the synthetic white-light images generated by the GCS model. We found the following CME parameter error bars as a starting point for quantifying the minimum error in CME parameters from white-light reconstructions: Δθ (latitude)=6° +2° -3° , Δϕ (longitude)=11° +18° -6° , Δγ (tilt)=25° +8° -7° , Δx (half-angle)=10° +12° -6° , Δh (height)=0.6 +1.2 -0.4 R ⨀ , and Δκ (ratio)=0.1 +0.03 -0.02 .
Coronal mass ejections↗
An Assessment of the Skill of GEOS-5 Seasonal Forecasts
The seasonal forecast skill of the NASA Global Modeling and Assimilation Office coupled global climate model (CGCM) is evaluated based on an ensemble of 9-month lead forecasts for the period 1993 to 2010. The results from the current version (V2) of the CGCM consisting of the GEOS-5 AGM coupled to the MOM4 ocean model are compared with those from an earlier version (V1) in which the AGCM (the NSIPP model) was coupled to the Poseidon Ocean Model. It was found that the correlation skill of the Sea Surface Temperature (SST) forecasts is generally better in V2, especially over the sub-tropical and tropical central and eastern Pacific, Atlantic, and Indian Ocean. Furthermore, the improvement in skill in V2 mainly comes from better forecasts of the developing phase of ENSO from boreal spring to summer. The skill of ENSO forecasts initiated during the boreal winter season, however, shows no improvement in terms of correlation skill, and is in fact slightly worse in terms of root mean square error (RMSE). The degradation of skill is found to be due to an excessive ENSO amplitude. For V1, the ENSO amplitude is too strong in forecasts starting in boreal spring and summer, which causes large RMSE in the forecast. For V2, the ENSO amplitude is slightly stronger than that in observations and V1 for forecasts starting in boreal winter season. An analysis of the terms in the SST tendency equation, shows that this is mainly due to an excessive zonal advective feedback. In addition, V2 forecasts that are initiated during boreal winter season, exhibit a slower phase transition of El Nino, which is consistent with larger amplitude of ENSO after the ENSO peak season. It is found that this is due to weak discharge of equatorial Warm Water Volume (WWV). In both observations and V1, the discharge of equatorial WWV leads the equatorial geostrophic easterly current so as to damp the El Nino starting in January. This process is delayed by about 2 months in V2 due to the slower phase transition of the equatorial zonal current from westerly to easterly.
GMAO↗
Forecasting the Arrival Time of Coronal Mass Ejections: Analysis of the CCMC CME Scoreboard
Accurate forecasting of the properties of coronal mass ejections (CMEs) as they approach Earth is now recognized as an important strategic objective for both NOAA and NASA. The time of arrival of such events is a key parameter, one that had been anticipated to be relatively straightforward to constrain. In this study, we analyze forecasts submitted to the Community Coordinated Modeling Center at NASA's Goddard Space Flight Center over the last 6 years to answer the following questions: (1) How well do these models forecast the arrival time of CME-driven shocks? (2) What are the uncertainties associated with these forecasts? (3) Which model(s) perform best? (4) Have the models become more accurate during the past 6 years? We analyze all forecasts made by 32 models from 2013 through mid-2018, and additionally focus on 28 events, all of which were forecasted by six models. We find that the models are generally able to predict CME-shock arrival times, in an average sense, to within ±10 hr, but with standard deviations often exceeding 20 hr. The best performers, on the other hand, maintained a mean error (bias) of 1 hr, a mean absolute error of 13 hr, and a precision (standard deviation) of 15 hr. Finally, there is no evidence that the forecasts have become more accurate during this interval. We discuss the intrinsic simplifications of thevarious models analyzed, the limitations of this investigation, and suggest possible paths to improve these forecasts in the future.
space weather↗
Differing Air Traffic Controller Responses to Similar Trajectory Prediction Errors
A Human-In-The-Loop simulation was conducted in January of 2013 in the Airspace Operations Laboratory at NASA's Ames Research Center. The simulation airspace included two en route sectors feeding the northwest corner of Atlanta's Terminal Radar Approach Control. The focus of this paper is on how uncertainties in the study's trajectory predictions impacted the controllers ability to perform their duties. Of particular interest is how the controllers interacted with the delay information displayed in the meter list and data block while managing the arrival flows. Due to wind forecasts with 30-knot over-predictions and 30-knot under-predictions, delay value computations included errors of similar magnitude, albeit in opposite directions. However, when performing their duties in the presence of these errors, did the controllers issue clearances of similar magnitude, albeit in opposite directions?
trajectory prediction uncertainty↗
Differing Air Traffic Controller Responses to Similar Trajectory Prediction Errors: An Interrupted Time-Series Analysis of Controller Behavior
A Human-In-The-Loop simulation was conducted in January of 2013 in the Airspace Operations Laboratory at NASAs Ames Research Center. The simulation airspace included two en route sectors feeding the northwest corner of Atlantas Terminal Radar Approach Control. The focus of this paper is on how uncertainties in the studys trajectory predictions impacted the controllers ability to perform their duties. Of particular interest is how the controllers interacted with the delay information displayed in the meter list and data block while managing the arrival flows. Due to wind forecasts with 20-knot over-predictions and 20-knot under-predictions, delay value computations included errors of similar magnitude, albeit in opposite directions. However, when performing their duties in the presence of these errors, did the controllers issue clearances of similar magnitude, albeit in opposite directions?
human-automation interaction↗
Modeling of Multiscale Solar Dynamics for Understanding Drivers of Space Weather
Understanding the solar dynamics is critical for improving our capabilities to forecast the evolution of space weather conditions. We take advantage of currently available computational capabilities to model solar dynamics from the deep interior to the corona and investigate mechanisms that may drive space weather conditions. The simulations are performed using the 3D radiative MHD code StellarBox. Comparison of synthetic spectroscopic observables obtained from numerical simulations and actual observations allows us to uncover physical processes associated with observed phenomena. To facilitate a transition from modeling short-term physical phenomena to developing a reliable forecast-oriented model, we suggest using the data assimilation approach. It allows us to cross-analyze dynamo model solutions and observations and to consider possible uncertainties and errors. In this presentation, we briefly summarize current multi-scale modeling capabilities and results and discuss ongoing developments to build a reliable physics-based forecast-oriented model of solar activity.
Irina N Kitiashvili↗