Van Allen Probes observations of unusually low frequency whistler mode waves observed in association with moderate magnetic storms: Statistical study
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
This dataset provides multi-year, ground-truth visual observations of surface water flow/no-flow conditions within the East-Taylor Watershed, Colorado, collected during June and July of 2025 and 2026. In June and July 2025, on-the-ground visual observations of flow/no-flow were collected as part of the Watershed Function Scientific Focus Area (SFA) and Rocky Mountain Biological Laboratory (RMBL) Colorado Headwaters Ecological Spectroscopy Study (CHESS) campaign (further details are provided within the CHESS Project Description). We obtained 377 water observations of flow/no-flow within the East-Taylor Watershed, Colorado. These ground-truth observations were collected to validate classification maps from remote sensing data and model results within the East-Taylor Watershed. In 2025, flow/no-flow measurements were collected using a field-based app for the CHESS Campaign (Zerion iForm). Within the field app, a water observation form was created to collect coordinates and metadata about the observation. Information collected for the water observation points included information about visually-assessed streamflow presence/absence (standard question obtained from Colorado State University’s StreamTracker project), flow estimate, stream or ponded area width, canopy cover, manganese films, iron seeps, and beaver activity. For 2025 water observations, this dataset contains: (1) a data file with the water observations and coordinates (2025_Water_Observations.csv); (2) a Keyhole Markup Language Zipped (KMZ) with the water observation locations and metadata (2025_Water_Observations_Locations.kmz); (3) photos (.jpg and .jpeg) of the water observation points, organized by location, contained within 2025_Water_Observations_FieldPhotographs.zip file; and (4) water observation protocols and figures (2025_Water_Observation_Protocols.pdf). In June and July 2026, on-the-ground visual observations of flow/no-flow were collected as part of the Watershed Function SFA project. We obtained 365 water observations of flow/no-flow within the East-Taylor Watershed, Colorado. The 2026 observations focused on collecting repeat measurements at the 2025 flow/no-flow observation locations conducted as part of the CHESS campaign. These ground-truth observations were collected to understand differences in flow/no-flow in 2026, given the unprecedented 2026 drought in Colorado. In 2026, flow/no-flow measurements were collected using ArcGIS (Geographic Information System) Survey123. Within the field app, a water observation form was created to collect coordinates and metadata about the observation. Information collected for the water observation points included repeat information from the 2025 water observation effort, including visually-assessed streamflow presence/absence (standard question obtained from Colorado State University’s StreamTracker project), flow estimate, stream or ponded area width, canopy cover, manganese films, iron seeps, beaver activity, and a new metadata component of estimated stream depth (for select locations). For 2026 water observations, this dataset contains: (1) a data file with the water observations and coordinates (2026_Water_Observations.csv); (2) a Keyhole Markup Language Zipped (KMZ) with the water observation locations and metadata (2026_Water_Observations_Locations.kmz); (3) photos (.jpg) of the water observation points, organized by location, contained within 2026_Water_Observations_FieldPhotographs.zip file; and (4) water observation protocols and figures (2026_Water_Observation_Protocols.pdf). For 2025 and 2026 water observations, this dataset contains: (1) a location metadata file (locations.csv); (6) a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata; and (7) a data dictionary (dd.csv) file that contains column/row headers used throughout the files along with a definition, units, and data type. CHESS Project Description: The Colorado Headwaters Ecological Spectroscopy Study (CHESS) comprised a multi-week airborne remote sensing and field observation campaign in the Upper Gunnison Basin, Colorado, conducted in June and July of 2025. Airborne remote sensing was conducted by the National Ecological Observatory Network Airborne Observation Platform (NEON AOP), concurrent with a field campaign run by the Rocky Mountain Biological Laboratory (RMBL), the Lawrence Berkeley National Laboratory (LBNL) and SLAC National Accelerator Laboratory Watershed Function Science Focus Area (SFA), and NASA-JPL (Jet Propulsion Laboratory) Earth Surface Mineral Dust Source Investigation (EMIT) program. Between June 10 and July 18, 2025, the NEON AOP flight team collected high-resolution aerial imaging spectroscopy and Light Detection and Ranging (LiDAR) data over three domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). In coordination with the flights, a field campaign acquired ground-truth observations, including observations of vegetation composition, foliar traits, forest demography, and subsurface properties in 18 core sampling areas within the domains. Additional surface water observations were taken at over 380 point locations. All CHESS campaign datasets can be found within the CHESS ESS-DIVE data portal: https://data.ess-dive.lbl.gov/portals/chess. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231. 2026-09-02: This dataset was updated to include 2026 water observation measurements. The 2025 observation files were also updated to ensure a consistent file naming convention across water observation years.
The accuracy of PBL height simulation is a key issue in many applications including forecasting near surface meteorology and air quality, however, it is a very challenging problem due to the lack of not only comprehensive, global Planetary Boundary Layer (PBL) observations but also a strategy and infrastructure to utilize PBL height data from a variety of sensors. Following the designation of PBL as an incubation class observable in the 2017 Decadal Survey, the PBL Incubation Study Team Report [14] made clear that “a future global PBL observing system requires modeling and data assimilation as essential components.” There is an urgent need for global modeling development in order to utilize Program of Record (POR) observations, assess their impacts, and identify gaps to be filled by future PBL missions. Our overall objective is to develop PBL data assimilation capabilities in the NASA Global Earth Observing System (GEOS), focusing on PBL height from multiple observing systems, to support the assessment and use of future PBL observations. The NASA GEOS system is composed of the GEOS global atmospheric general circulation model (AGCM) and the atmospheric data assimilation system (ADAS). The PBL parameterizations include the “Lock” K-profile scheme driven by surface and cloud-top buoyancy fluxes ([4]), and the “Louis” local scheme for stable conditions based on the Richardson number ([5]). Above the mixed layer defined by the Lock surface plume, shallow cumulus convection is represented by the mass flux scheme of [9]. Additional parameterizations are summarized in [1]. The ADAS employs the hybrid 4D Ensemble- Variational (EnVar) configuration ([15]), with the ensemble providing flow-dependent background error covariance information. The resultant analysis increments are fed back to the forecast model through the 4D incremental analysis update (IAU) approach ([11]). In this study, PBL height data are being or have been generated from radiosondes, GNSS RO, satellite (CATS, CALIPSO and ICESat-2) and ground-based (MPLNET) lidars, and wind profiler. Investigations have been conducted to specify quality marks for PBL height retrievals for the data assimilation purpose. These PBL height data have different strengths and weaknesses ([2], [3], [6], [7], [8], [10]), and the satellite PBL height data provide better global coverage and complement in-situ PBL height data. Radiosondes offer high accuracy and in situ measurement of temperature and humidity profiles, but with poor spatio-temporal sampling. The in-situ observing systems like MPLNET and wind profiler provide long history of PBL height records at each station. The GNSS RO based PBL height is retrieved based on the sharp gradients in refractivity profile that represent the fine vertical structure of temperature and moisture changes above the PBL. However, not all RO refractivity profiles reach the surface depending on location and regime, and RO refractivity retrievals can be negatively biased below 2km. The PBL height data from satellite lidars provide high resolution along track PBL height retrievals, but over land they are affected by previous day convective PBL aerosol and strongly associated with mixing layer and retrievals cannot be made below thick, attenuating clouds. A successful assimilation of PBL height data requires a thorough understanding of the observing method and the retrieval algorithm for each observing system in order to use the PBL height data from multiple observing systems properly. Due to the sensitivity of PBL height data to the observing method and choice of algorithm, it is important to use a model definition appropriate for each observation type to compute differences between PBL height data and model PBL height (OmFs). The GEOS model currently includes two PBL height definitions suitable for direct comparison with observed PBL height, and additional definitions are being added in this study. Evaluation of different model PBL height definitions is underway. Meanwhile, efforts have been made in the GEOS data assimilation system to develop PBL height data assimilation capability. PBL height data can be assimilated using two different approaches. The traditional approach is to construct an observation operator and its tangent linear and adjoint, which link control variables to PBL height data from each observing system. This observation operator can be very complicated, e.g., the lidar-based PBL height observation operator includes the backscatter lidar forward observation operator, the algorithm to derive PBL height from attenuated total backscatter, interpolation, and calculations handling the mismatch between observed and model scales. The other approach is to augment PBL height to the control variable vector, and it is adopted in this study. The latter approach was also used in previous studies, e.g., the assimilation of PBL height data from radiosonde and aircraft in the Real Time Mesoscale Analysis (RTMA) system for a dispersion modelling study ([13]); the PBL height assimilation study using lidar PBL height data at Greensburg, Kansas for a field campaign ([12]). The PBL height assimilation from multiple observing systems in this study allows us to take advantage of the diverse PBL height data that provide much better global coverage collectively under different meteorological conditions and with different temporal and spatial scales. As all the PBL heights are tightly coupled with the PBL thermodynamic variables, the strong correlations, which are provided by the 4D ensemble forecast, enable PBL height data from various sources to interact and combine coherently and provide additional information for PBL temperature and moisture fields. The results of comparisons among PBL height data from different sources and the evaluation of the model PBL height definitions with the PBL height data will be presented, and the PBL height data synergy strategies and preliminary results will also be discussed at the conference.
Assimilation systems synthesize diverse in-situ and satellite data streams into full four-dimensional state estimates by combining the strengths of each data set and also of the model. The resulting analysis provides an integrated view of the information in the various observations as well as derived estimates of unobserved quantities. Assimilation systems are particularly important for the ocean where subsurface observations, even today, are sparse and intermittent compared with the scales needed to represent ocean variability and where satellites only sense the surface. Increasingly, models and assimilation systems are being used to provide information about the current observing system and to help in the design plans for new observations. Whether it is as a user of observations or a contributor to evaluation of the observing system, ocean synthesis and assimilation systems are now an integral part of the global ocean observing and information system. Major advances have been made over the last decade under the auspices of WCRP's Climate Variability and Predictability Project (CLIVAR) and the Global Ocean Data Assimilation Experiment (GODAE). In addition to advances in the assimilation systems, there have been major developments in the observing system, with satellite altimetry, the tropical moored buoy arrays in the Pacific and Atlantic, and more recently Argo. These developments have led to significant advances in our understanding and prediction of ocean variations at both mesoscale and climate scales. Many challenges remain. Some of these challenges lie in the observations themselves, some in the assimilation systems that, even in the more recent era of unprecedented observations from satellite altimetry and Argo, provide different views of climate variations. Yet there are many examples of successful applications from ocean assimilation products. Use of these systems for assessing the observing system helps identify the strengths of each observation type, and indicates that none of the current observations is redundant. Indeed, the indication is that the ocean remains under-sampled and that further improvements in the observing system are needed for both climate monitoring and prediction. Future developments will be increasingly towards consistent analyses across components of the Earth system using, e.g., coupled atmosphere-ocean models.
Reanalyses have become important sources of data in weather and climate research. While observations are the most crucial component of the systems, few research projects consider carefully the multitudes of assimilated observations and their impact on the results. This is partly due to the diversity of observations and their individual complexity, but also due to the unfriendly nature of the data formats. Here, we discuss the NASA Modern-Era Retrospective analysis for Research and Applications (MERRA) and a companion dataset, the Gridded Innovations and Observations (GIO). GIO is simply a post-processing of the assimilated observations and their innovations (forecast error and analysis error) to a common spatio-temporal grid, following that of the MERRA analysis fields. This data includes in situ, retrieved and radiance observations that are assimilated and used in the reanalysis. While all these disparate observations and statistics are in a uniform easily accessible format, there are some limitations. Similar observations are binned to the grid, so that multiple observations are combined in the gridding process. The data is then implicitly thinned. Some details in the meta data may also be lost (e.g. aircraft or station ID). Nonetheless, the gridded observations should provide easy access to all the observations input to the reanalysis. To provide an example of the GIO data, a case study evaluating observing systems over the United States and statistics is presented, and demonstrates the evaluation of the observations and the data assimilation. The GIO data is used to collocate 200mb Radiosonde and Aircraft temperature measurements from 1979-2009. A known warm bias of the aircraft measurements is apparent compared to the radiosonde data. However, when larger quantities of aircraft data are available, they dominate the analysis and the radiosonde data become biased against the forecast. When AMSU radiances become available the radiosonde and aircraft analysis and forecast error take on an annual cycle. While this supports results of previous work that recommend bias corrections for the aircraft measurements, the interactions with AMSU radiances will also require further investigation. This also provides an example for reanalysis users in examining the available observations and their impact on the analysis. GIO data is presently available alongside the MERRA reanalysis.
An Observing System Simulation Experiment (OSSE) is a pure modeling study used when actual observations are too expensive or difficult to obtain. OSSEs are valuable tools for determining the potential impact of new observing systems on numerical weather forecasts and for evaluation of data assimilation systems (DAS). An OSSE has been developed at the NASA Global Modeling and Assimilation Office (GMAO, Errico et al 2013). The GMAO OSSE uses a 13-month integration of the European Centre for Medium- Range Weather Forecasts 2005 operational model at T511/L91 resolution for the Nature Run (NR). Synthetic observations have been updated so that they are based on real observations during the summer of 2013. The emulated observation types include AMSU-A, MHS, IASI, AIRS, and HIRS4 radiance data, GPS-RO, and conventional types including aircraft, rawinsonde, profiler, surface, and satellite winds. The synthetic satellite wind observations are colocated with the NR cloud fields, and the rawinsondes are advected during ascent using the NR wind fields. Data counts for the synthetic observations are matched as closely as possible to real data counts, as shown in Figure 2. Errors are added to the synthetic observations to emulate representativeness and instrument errors. The synthetic errors are calibrated so that the statistics of observation innovation and analysis increments in the OSSE are similar to the same statistics for assimilation of real observations, in an iterative method described by Errico et al (2013). The standard deviations of observation minus forecast (xo-H(xb)) are compared for the OSSE and real data in Figure 3. The synthetic errors include both random, uncorrelated errors, and an additional correlated error component for some observational types. Vertically correlated errors are included for conventional sounding data and GPS-RO, and channel correlated errors are introduced to AIRS and IASI (Figure 4). HIRS, AMSU-A, and MHS have a component of horizontally correlated error. The forecast model used by the GMAO OSSE is the Goddard Earth Observing System Model, Version 5 (GEOS-5) with Gridpoint Statistical Interpolation (GSI) DAS. The model version has been updated to v. 5.13.3, corresponding to the current operational model. Forecasts are run on a cube-sphere grid with 180 points along each edge of the cube (approximately 0.5 degree horizontal resolution) with 72 vertical levels. The DAS is cycled at 6-hour intervals, with 240 hour forecasts launched daily at 0000 UTC. Evaluation of the forecasting skill for July and August is currently underway. Prior versions of the GMAO OSSE have been found to have greater forecasting skill than real world forecasts. It is anticipated that similar forecast skill will be found in the updated OSSE.
Over the past several years a number of high cadence photometric observations of solar system giant planets have been acquired by various platforms. Such observations are of interest as they provide points of comparison to the already expansive set of brown dwarf variability observations and the small, but growing, set of exoplanet variability observations. By measuring how rapidly the integrated light from solar system giant planets can evolve, variability observations of substellar objects that are unlikely to ever be resolved can be placed in a fuller context. Examples of brown dwarf variability observations include extensive work from the ground (e.g., Radigen et al. 2014), Spitzer (e.g., Metchev et al. 2015), Kepler (Gizis et al. 2015), and HST (Yang et al. 2015).Variability has been measured on the planetary mass companion to the brown dwarf 2MASS 1207b (Zhou et al. 2016) and further searches are planned in thermal emission for the known directly imaged planets with ground based telescopes (Apai et al. 2016) and in reflected light with future space based telescopes. Recent solar system variability observations include Kepler monitoring of Neptune (Simon et al. 2016) and Uranus, Spitzer observations of Neptune (Stauffer et al. 2016), and Cassini observations of Jupiter (West et al. in prep). The Cassini observations are of particular interest as they measured the variability of Jupiter at a phase angle of approximately 60 deg, comparable to the viewing geometry expected for space based direct imaging of cool extrasolar Jupiters in reflected light. These solar system analog observations capture many of the characteristics seen in brown dwarf variability, including large amplitudes and rapid light curve evolution on timescales as short as a few rotation periods. Simon et al. (2016) attribute such variations at Neptune to a combination of large scale, stable cloud structures along with smaller, more rapidly varying, cloud patches. The observed brown dwarf and exoplanet variability may well arise from comparable cloud structures. In my presentation I will compare and contrast the nature of the variability observed for the various solar system and other substelar objects and present a wish list for future observations.
Observation error determines the weights of the observations and background state used in data assimilation to generate analyses. Quantifying observation error is critical for the optimal assimilation of observational data sets. Uncrewed Aircraft System (UAS) observations have shown potential benefits in filling observational gaps in the lower atmosphere; however, characterization of their error characteristics has been limited. To optimize the use of UAS observations in numerical weather prediction, UAS observation error is estimated based on the 3‐cornered hat diagnostic approach which uses three independent estimates of the atmospheric state. This approach is applied to data from the 2018 Lower Atmospheric Profiling Studies at Elevation‐a Remotely‐piloted Aircraft Team Experiment field campaign using collocated UAS and rawinsonde observations along with output from a set of convection‐permitting model simulations. The estimated observation error values for UAS temperature, wind, and relative humidity measurements were found to be only weakly dependent on height AGL with mean values equal to 0.5°C, 0.8 m s −1 , and 3%, respectively. Only the newly estimated observation error for temperature differed from that previously used to assimilate commercial aircraft observations into global models (1.0°C). However, using this reduced temperature observation error produced more accurate mesoscale analyses and forecasts of both terrain‐driven flows and convection initiation generated by colliding outflow boundaries within the San Luis Valley of Colorado.
Developments in ocean data assimilation (DA) and observing system technologies are intertwined. New observation types lead to new DA methods, and new DA methods such as Coupled Data Assimilation can change the value of existing observations or indicate where new observations can have greater utility for monitoring and prediction. Practitioners are encouraged to make better use of observations that are already available, for example in strongly coupled data assimilation where ocean observations can be used to improve atmospheric analyses and vice versa. Ocean reanalyses are useful for the analysis of climate,as well as initializing operational long-range prediction models. There are remaining challenges for ocean reanalyses due to biases and abrupt changes in the ocean observing system throughout its history, the presence of biases and drifts in models, and simplifying assumptions made in the DA methods. From a governance point of view, more support is needed to interface the observing community and the ocean DA community. For prediction applications, the ocean DA community must work with the ocean observing community to establish protocols for rapid communication of ocean observing data on NWP timescales. There is potential for new observations to enhance the observing system by supporting prediction on multiple timescales, ranging from the typical timescale of numerical weather prediction covering hours to weeks, out to multiple decades. It is highly encouraged that communication be fostered between thesecommunities to allow operational prediction centers the ability to provide guidance to the design of a sustained and adaptive observing network.
A typical pointed observation in the Infrared Astronomical Satellite (IRAS) was designed to fit into a 15 minute window. Observing constraints, however, did not always permit targets from the scientific program to be observed on exactly 15 minute centers and a few months into the mission it was realized that the automatic scheduling program left times when no observations were being made. In order to use this potentially wasted observing time, a mode of pointed observations, the 'filler' mode, with shorter observation duration was initiated. Locations were picked on an arbitrary grid, spaced relatively uniformly in ecliptic coordinates, and the telescope was pointed to one of these sites whenever a gap potentially occurred. One of us (GXN) was the coordinator of this effort (the FL sub-group). There are a total of 22 FL fields, 13 of which have more than 50 component observations. These 13 fields cover about 30 square degrees. It was proposed to concentrate on one filler field, FL29, a field with 53 coverages which thus had among the deepest coverage of those observed in this mode. This field was also observed extensively at various non-infrared wavelengths by a variety of techniques. Because the IRAS observations of FL29 were made at the limits of the survey sensitivity and at different twist angles it was necessary to make significant technical advances in the methods used in the processing and analyzing the data. In particular, if normal IRAS coaddition procedures were used, the noise resulting from coaddition of the multiple frames did not decrease in the manner expected once significantly different twist angles were included in the coaddition. The technical effort needed to reduce the observations of FL29 was completed and now the noise behavior is understood and well behaved. The reduction of the field FL29 observations is done.
Fulfilling the promise of an era of great observatories, NASA now has more than three space-based astronomical telescopes operating in different wavebands. This situation provides astronomers with a unique opportunity to simultaneously observe with multiple observatories. Yet scheduling multiple observatories simultaneously is highly inefficient when compared to single observatory observations. Thus, programs using multiple observatories are limited not due to scientific restrictions, but due to operational inefficiencies. Each year, a number of proposals are accepted by a space-based observatory for conduction of astronomical observations and gathering of science data for the study of galactic events. Since each space-based observatory uses a set of instruments designed to operate in specific energy regions, most such studies are conducted by submitting observation proposals to multiple observatories, with requests to coordinate among themselves. To assure that the proposed observations can be scheduled, each observatory's staff has to check that the observations are valid and meet all the constraints for their own observatory; in addition, they have to verify that the observations satisfy the constraints of the other observatories. Thus, coordinated observations require painstaking manual collaboration among the observatory staff at each observatory. In order to exploit new paradigms for observatory operation, the Goddard Space Flight Center's Advanced Architectures and Automation Branch has developed a prototype tool called the Visual Observation Layout Tool (VOLT). The main objective of VOLT is to provide a visual tool to automate the science planning of coordinated observations for multiple spacecraft, as well as to increase the scheduling probability of observations. However, VOLT is also useful for single observatory planning to optimize observatory control. Three space-based missions are interested in using VOLT (the Hubble Space Telescope, the Chandra X-Ray Observatory, and the Far Ultraviolet Spectroscopic Explorer). The VOLT team members have collaborated with these missions to gather requirements and obtain feedback on their mission planning processes. VOLT has been developed as a cross-platform Java client application for use by scientists and observatory science planning staff to visualize scheduling options and constraints. It also supports a lightweight graphical user interface for remote viewing via a Web front end. Additionally, it uniquely supports the ability to interact with multiple, diverse scheduling packages in order to determine windows of opportunity for observations and visually portray the constraints of each observation request. VOLT enables science data capture scenarios which are currently either impossible, or which require extensive time and manpower to coordinate amongst multiple observatories. it supports early detection of planning conflicts by generating coordinated solutions based on observatory schedulability and constraints. The project development approach has included frequent prototype demonstrations to our interested missions to obtain feedback after each release of the software. We will present an overview of our lessons learned in infusing the VOLT tool into the operations of the missions we have collaborated with and a brief demonstration of the software.
The adjoint of a data assimilation system provides a flexible and efficient tool for estimating observation impacts on short-range weather forecasts. The impacts of any or all observations can be estimated simultaneously based on a single execution of the adjoint system. The results can be easily aggregated according to data type, location, channel, etc., making this technique especially attractive for examining the impacts of new hyper-spectral satellite instruments and for conducting regular, even near-real time, monitoring of the entire observing system. In this talk, we present results from the adjoint-based observation impact monitoring tool in NASA's GEOS-5 global atmospheric data assimilation and forecast system. The tool has been running in various off-line configurations for some time, and is scheduled to run as a regular part of the real-time forecast suite beginning in autumn 20 I O. We focus on the impacts of the newest components of the satellite observing system, including AIRS, IASI and GPS. For AIRS and IASI, it is shown that the vast majority of the channels assimilated have systematic positive impacts (of varying magnitudes), although some channels degrade the forecast. Of the latter, most are moisture-sensitive or near-surface channels. The impact of GPS observations in the southern hemisphere is found to be a considerable overall benefit to the system. In addition, the spatial variability of observation impacts reveals coherent patterns of positive and negative impacts that may point to deficiencies in the use of certain observations over, for example, specific surface types. When performed in conjunction with selected observing system experiments (OSEs), the adjoint results reveal both redundancies and dependencies between observing system impacts as observations are added or removed from the assimilation system. Understanding these dependencies appears to pose a major challenge for optimizing the use of the current observational network and defining requirements for future observing systems.
Abstract Before we attempt to (approximately) learn a function between two sets of observables of a physical process, we must first decide what the inputs and outputs of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of first deciding “the right quantities” to relate through such a function, and then proceeding to learn it. This is accomplished by first processing simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: records of multiple observation processes of the system. We determine (i) what subsets of observables are common between the observation processes (and therefore observable from each other, relatable through a function); and (ii) what information is unrelated to these common observables, therefore particular to each observation process, and not contributing to the desired function. Any data-driven technique can subsequently be used to learn the input–output relation—from k-nearest neighbors and Geometric Harmonics to Gaussian Processes and Neural Networks. Two particular “twists” of the approach are discussed. The first has to do with the identifiability of particular quantities of interest from the measurements. We now construct mappings from a single set of observations from one process to entire level sets of measurements of the second process, consistent with this single set. The second attempts to relate our framework to a form of causality: if one of the observation processes measures “now,” while the second observation process measures “in the future,” the function to be learned among what is common across observation processes constitutes a dynamical model for the system evolution.
The problem of designing analytical failure-detection systems (FDS) for sensors and actuators, using observers, is addressed. The use of observers in FDS is related to the examination of the n-dimensional observer error vector which carries the necessary information on possible failures. The problem is that in practical systems, in which only some of the components of the state vector are measured, one has access only to the m-dimensional observer-output error vector, with m or = to n. In order to cope with these cases, a secondary observer is synthesized to reconstruct the entire observer-error vector from the observer output error vector. This approach leads toward the design of highly sensitive and reliable FDS, with the possibility of obtaining a unique fingerprint for every possible failure. In order to keep the observer's (or Kalman filter) false-alarm rate under a certain specified value, it is necessary to have an acceptable matching between the observer (or Kalman filter) models and the system parameters. A previously developed adaptive observer algorithm is used to maintain the desired system-observer model matching, despite initial mismatching or system parameter variations. Conditions for convergence for the adaptive process are obtained, leading to a simple adaptive law (algorithm) with the possibility of an a priori choice of fixed adaptive gains. Simulation results show good tracking performance with small observer output errors, while accurate and fast parameter identification, in both deterministic and stochastic cases, is obtained.