Engineering topics

Kuo, Kwo-Sen

Publications and source records attributed to Kuo, Kwo-Sen.

33 records · Page 2

NASA NTRS2016

Addressing the Big-Earth-Data Variety Challenge with the Hierarchical Triangular Mesh

We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. We observe that, in the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. In addition, since HTM is also an indexing scheme, when it is used to index all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. Moreover, our updated HTM encoding turns most geospatial set operations into integer interval operations, gaining further performance advantages.

SciDB↗

NASA NTRS2016

Evaluating the Impact of Data Placement to Spark and SciDB with an Earth Science Use Case

We investigate the impact of data placement for two Big Data technologies, Spark and SciDB, with a use case from Earth Science where data arrays are multidimensional. Simultaneously, this investigation provides an opportunity to evaluate the performance of the technologies involved. Two datastores, HDFS and Cassandra, are used with Spark for our comparison. It is found that Spark with Cassandra performs better than with HDFS, but SciDB performs better yet than Spark with either datastore. The investigation also underscores the value of having data aligned for the most common analysis scenarios in advance on a shared nothing architecture. Otherwise, repartitioning needs to be carried out on the fly, degrading overall performance.

Spark↗

NASA NTRS2016

The Microwave Radiative Properties of Falling Snow Derived from Nonspherical Ice Particle Models. Part II: Initial Testing Using Radar, Radiometer and In Situ Observations

In this study, two different particle models describing the structure and electromagnetic properties of snow are developed and evaluated for potential use in satellite combined radar-radiometer precipitation estimation algorithms. In the first model, snow particles are assumed to be homogeneous ice-air spheres with single-scattering properties derived from Mie theory. In the second model, snow particles are created by simulating the self-collection of pristine ice crystals into aggregate particles of different sizes, using different numbers and habits of the collected component crystals. Single-scattering properties of the resulting nonspherical snow particles are determined using the discrete dipole approximation. The size-distribution-integrated scattering properties of the spherical and nonspherical snow particles are incorporated into a dual-wavelength radar profiling algorithm that is applied to 14- and 34-GHz observations of stratiform precipitation from the ER-2 aircraft-borne High-Altitude Imaging Wind and Rain Airborne Profiler (HIWRAP) radar. The retrieved ice precipitation profiles are then input to a forward radiative transfer calculation in an attempt to simulate coincident radiance observations from the Conical Scanning Millimeter-Wave Imaging Radiometer (CoSMIR). Much greater consistency between the simulated and observed CoSMIR radiances is obtained using estimated profiles that are based upon the nonspherical crystal/aggregate snow particle model. Despite this greater consistency, there remain some discrepancies between the higher moments of the HIWRAP-retrieved precipitation size distributions and in situ distributions derived from microphysics probe observations obtained from Citation aircraft underflights of the ER-2. These discrepancies can only be eliminated if a subset of lower-density crystal/aggregate snow particles is assumed in the radar algorithm and in the interpretation of the in situ data.

Ice particles; Radiances; Radiative transfer; Airc↗

NASA NTRS2014

ldentifying Episodes of Earth Science Phenomena Using a Big-Data Technology

A significant portion of Earth Science investigations is phenomenon- (or event-) based, such as the studies of Rossby waves, volcano eruptions, tsunamis, mesoscale convective systems, and tropical cyclones. However, except for a few high-impact phenomena, e.g. tropical cyclones, comprehensive records are absent for the occurrences or events of these phenomena. Phenomenon-based studies therefore often focus on a few prominent cases while the lesser ones are overlooked. Without an automated means to gather the events, comprehensive investigation of a phenomenon is at least time-consuming if not impossible. We have constructed a prototype Automated Event Service (AES) system that is used to methodically mine custom-defined events in the reanalysis data sets of atmospheric general circulation models. Our AES will enable researchers to specify their custom, numeric event criteria using a user-friendly web interface to search the reanalysis data sets. Moreover, we have included a social component to enable dynamic formation of collaboration groups for researchers to cooperate on event definitions of common interest and for the analysis of these events. An Earth Science event (ES event) is defined here as an episode of an Earth Science phenomenon (ES phenomenon). A cumulus cloud, a thunderstorm shower, a rogue wave, a tornado, an earthquake, a tsunami, a hurricane, or an El Nino, is each an episode of a named ES phenomenon, and, from the small and insignificant to the large and potent, all are examples of ES events. An ES event has a duration (often finite) and an associated geo-location as a function of time; it's therefore an entity embedded in four-dimensional (4D) spatiotemporal space. Earth Science phenomena with the potential to cause massive economic disruption or loss of life often rivet the attention of researchers. But, broader scientific curiosity also drives the study of phenomena that pose no immediate danger, such as land/sea breezes. Due to Earth System's intricate dynamics, we are continuously discovering novel ES phenomena. We generally gain understanding of a given phenomenon by observing and studying individual events. This process usually begins by identifying the occurrences of these events. Once representative events are identified or found, we must locate associated observed or simulated data prior to commencing analysis and concerted studies of the phenomenon. Knowledge concerning the phenomenon can accumulate only after analysis has started. However, as mentioned previously, comprehensive records only exist for a very limited set of high-impact phenomena; aside from these, finding events and locating associated data currently may take a prohibitive amount of time and effort on the part of an individual investigator. The reason for the lack of comprehensive records for most of the ES phenomena is mainly due to the perception that they do not pose immediate and/or severe threat to life and property. Thus they are not consistently tracked, monitored, and catalogued. Many phenomena even lack precise and/or commonly accepted criteria for definitions. Moreover, various Earth Science observations and data have accumulated to a previously unfathomable volume; NASA Earth Observing System Data Information System (EOSDIS) alone archives several petabytes (PB) of satellite remote sensing data, which are steadily increasing. All of these factors contribute to the difficulty of methodically identifying events corresponding to a given phenomenon and significantly impede systematic investigations. We have not only envisioned AES as an environment for identifying customdefined events but also aspired for it to be an interactive environment with quick turnaround time for revisions of query criteria and results, as well as a collaborative environment where geographically distributed experts may work together on the same phenomena. A Big Data technology is thus required for the realization of such a system. In the following, we first introduce the technology selected for AES in the next section. We then demonstrate the utility of AES using a use case, Blizzard, before we conclude.

Kuo, Kwo-Sen↗

NASA NTRS2014

Anticipated Changes in Conducting Scientific Data-Analysis Research in the Big-Data Era

A Big-Data environment is one that is capable of orchestrating quick-turnaround analyses involving large volumes of data for numerous simultaneous users. Based on our experiences with a prototype Big-Data analysis environment, we anticipate some important changes in research behaviors and processes while conducting scientific data-analysis research in the near future as such Big-Data environments become the mainstream. The first anticipated change will be the reduced effort and difficulty in most parts of the data management process. A Big-Data analysis environment is likely to house most of the data required for a particular research discipline along with appropriate analysis capabilities. This will reduce the need for researchers to download local copies of data. In turn, this also reduces the need for compute and storage procurement by individual researchers or groups, as well as associated maintenance and management afterwards. It is almost certain that Big-Data environments will require a different "programming language" to fully exploit the latent potential. In addition, the process of extending the environment to provide new analysis capabilities will likely be more involved than, say, compiling a piece of new or revised code.We thus anticipate that researchers will require support from dedicated organizations associated with the environment that are composed of professional software engineers and data scientists. A major benefit will likely be that such extensions are of higherquality and broader applicability than ad hoc changes by physical scientists. Another anticipated significant change is improved collaboration among the researchers using the same environment. Since the environment is homogeneous within itself, many barriers to collaboration are minimized or eliminated. For example, data and analysis algorithms can be seamlessly shared, reused and re-purposed. In conclusion, we will be able to achieve a new level of scientific productivity in the Big-Data analysis environments.

Kuo, Kwo-Sen↗

NASA NTRS2013

Tool for Automated Retrieval of Generic Event Tracks (TARGET)

Methods have been developed to identify and track tornado-producing mesoscale convective systems (MCSs) automatically over the continental United States, in order to facilitate systematic studies of these powerful and often destructive events. Several data sources were combined to ensure event identification accuracy. Records of watches and warnings issued by National Weather Service (NWS), and tornado locations and tracks from the Tornado History Project (THP) were used to locate MCSs in high-resolution precipitation observations and GOES infrared (11-micron) Rapid Scan Operation (RSO) imagery. Thresholds are then applied to the latter two data sets to define MCS events and track their developments. MCSs produce a broad range of severe convective weather events that are significantly affecting the living conditions of the populations exposed to them. Understanding how MCSs grow and develop could help scientists improve their weather prediction models, and also provide tools to decision-makers whose goals are to protect populations and their property. Associating storm cells across frames of remotely sensed images poses a difficult problem because storms evolve, split, and merge. Any storm-tracking method should include the following processes: storm identification, storm tracking, and quantification of storm intensity and activity. The spatiotemporal coordinates of the tracks will enable researchers to obtain other coincident observations to conduct more thorough studies of these events. In addition to their tracked locations, their areal extents, precipitation intensities, and accumulations all as functions of their evolutions in time were also obtained and recorded for these events. All parameters so derived can be catalogued into a moving object database (MODB) for custom queries. The purpose of this software is to provide a generalized, cross-platform, pluggable tool for identifying events within a set of scientific data based upon specified criteria with the possibility of storing identified events into a searchable database. The core of the application uses an implementation of the connected component labeling (CCL) algorithm to identify areas of interest, then uses a set of criteria to establish spatial and temporal relationships between identified components. The CCL algorithm is used for identifying objects within images for computer vision. This application applies it to scientific data sets using arbitrary criteria. The most novel concept was applying a generalized CCL implementation to scientific data sets for establishing events both spatially and temporally. The combination of several existing concepts (pluggable components, generalized CCL algorithm, etc.) into one application is also novel. In addition, how the system is designed, i.e., its extensibility with pluggable components, and its configurability with a simple configuration file, is innovative. This allows the system to be applied to new scenarios with ease.

Clune, Thomas↗

NASA NTRS2012

Integrated Instrument Simulator Suites for Earth Science

The NASA Earth Observing System Simulators Suite (NEOS3) is a modular framework of forward simulations tools for remote sensing of Earth's Atmosphere from space. It was initiated as the Instrument Simulator Suite for Atmospheric Remote Sensing (ISSARS) under the NASA Advanced Information Systems Technology (AIST) program of the Earth Science Technology Office (ESTO) to enable science users to perform simulations based on advanced atmospheric and simple land surface models, and to rapidly integrate in a broad framework any experimental or innovative tools that they may have developed in this context. The name was changed to NEOS3 when the project was expanded to include more advanced modeling tools for the surface contributions, accounting for scattering and emission properties of layered surface (e.g., soil moisture, vegetation, snow and ice, subsurface layers). NEOS3 relies on a web-based graphic user interface, and a three-stage processing strategy to generate simulated measurements. The user has full control over a wide range of customizations both in terms of a priori assumptions and in terms of specific solvers or models used to calculate the measured signals.This presentation will demonstrate the general architecture, the configuration procedures and illustrate some sample products and the fundamental interface requirements for modules candidate for integration.

emission↗

NASA NTRS2012

Integrated Instrument Simulator Suites for Earth Science

radiative transfer↗

NASA NTRS2012

Leveraging Data Intensive Computing to Support Automated Event Services

A large portion of Earth Science investigations is phenomenon- or event-based, such as the studies of Rossby waves, mesoscale convective systems, and tropical cyclones. However, except for a few high-impact phenomena, e.g. tropical cyclones, comprehensive records are absent for the occurrences or events of these phenomena. Phenomenon-based studies therefore often focus on a few prominent cases while the lesser ones are overlooked. Without an automated means to gather the events, comprehensive investigation of a phenomenon is at least time-consuming if not impossible. An Earth Science event (ES event) is defined here as an episode of an Earth Science phenomenon. A cumulus cloud, a thunderstorm shower, a rogue wave, a tornado, an earthquake, a tsunami, a hurricane, or an EI Nino, is each an episode of a named ES phenomenon," and, from the small and insignificant to the large and potent, all are examples of ES events. An ES event has a finite duration and an associated geolocation as a function of time; its therefore an entity in four-dimensional . (4D) spatiotemporal space. The interests of Earth scientists typically rivet on Earth Science phenomena with potential to cause massive economic disruption or loss of life, but broader scientific curiosity also drives the study of phenomena that pose no immediate danger. We generally gain understanding of a given phenomenon by observing and studying individual events - usually beginning by identifying the occurrences of these events. Once representative events are identified or found, we must locate associated observed or simulated data prior to commencing analysis and concerted studies of the phenomenon. Knowledge concerning the phenomenon can accumulate only after analysis has started. However, except for a few high-impact phenomena. such as tropical cyclones and tornadoes, finding events and locating associated data currently may take a prohibitive amount of time and effort on the part of an individual investigator. And even for these high-impact phenomena, the availability of comprehensive records is still only a recent development. A major reason for the lack of comprehensive ,records for the majority of the ES phenomena is the perception that they do not pose immediate and/or severe threat to life and property and are thus not consistently tracked. monitored, and catalogued. Many phenomena even lack commonly accepted criteria for definitions. However. the lack of comprehensive records is also due to the increasingly prohibitive volume of observations and model data that must be examined. NASA Earth Observing System Data Information System (EOSDIS) alone archives several petabytes (PB) of satellite remote sensing data and steadily increases. All of these factors contribute to the difficulty of methodically identifying events corresponding to a given phenomenon and significantly impede systematic investigations. In the following we present a couple motivating scenarios, demonstrating the issues faced by Earth scientists studying ES phenomena.

Clune, Thomas L.↗

NASA NTRS2011

NASA's Integrated Instrument Simulator Suite for Atmospheric Remote Sensing from Spaceborne Platforms (ISSARS) and Its Role for the ACE and GPM Missions

Forward simulation is an indispensable tool for evaluation of precipitation retrieval algorithms as well as for studying snow/ice microphysics and their radiative properties. The main challenge of the implementation arises due to the size of the problem domain. To overcome this hurdle, assumptions need to be made to simplify compiles cloud microphysics. It is important that these assumptions are applied consistently throughout the simulation process. ISSARS addresses this issue by providing a computationally efficient and modular framework that can integrate currently existing models and is also capable of expanding for future development. ISSARS is designed to accommodate the simulation needs of the Aerosol/Clouds/Ecosystems (ACE) mission and the Global Precipitation Measurement (GPM) mission: radars, microwave radiometers, and optical instruments such as lidars and polarimeter. ISSARS's computation is performed in three stages: input reconditioning (IRM), electromagnetic properties (scattering/emission/absorption) calculation (SEAM), and instrument simulation (ISM). The computation is implemented as a web service while its configuration can be accessed through a web-based interface.

real-aperture instruments↗

NASA NTRS2008

Seeking Augmented Information Content Concerning Diurnal Precipitation Achieved by Combining TRMM-PR and CloudSat-CPR Radar Data Sets

The CloudSat satellite's Cloud Profiling Radar (CPR) is a highly sensitive 94 GHz (W-band) nadir viewing radar system flown in retrograde sun synchronous orbit useful for determining the vertical structure of cloud hydrometeors down to sensitivity of approx. -30 dBZ reflectivity factor. Given this sensitivity, it is possible to unambiguously measure precipitation rates in clouds over a spectrum extending from approx. 0.08 - 3.0 mm hr (sup -1) down to altitudes of 0.5 km with approx.0.25 km vertical binning. This enables an effective means to measure a great deal of the drizzle and light rain spectrum. However, because of its near-polar sunsynchronous orbit, CloudSat cannot sample the diurnal cycle of precipitation, nor with its nadir-only CPR view can it obtain a high duty cycle in sampling precipitation at fixed local times over fixed positions. On the other hand, the TRMM satellite, which is flown in a non-sunsynchronous 35-degree inclined orbit carrying the 13.8 GHz KU-band Precipitation Radar (PR) scanning through nadir over an approx. 225 km swath, can sample both the diurnal cycle and with a much improved duty cycle relative to CloudSat. Moreover, the PR and CPR have the same 0.25 km vertical binning capability. The PR's greatest shortcoming is its approx. +17 dBZ sensitivity, which eliminates the possibility of measuring rain rates below -0.3-0.5 mm hr(sup -1), which can involve rainfall accumulations of up to 50% of the total over some regions. This begs the question of whether by combining CPR and PR data sets, whether it is possible to obtain an augmented measurement of the diurnal precipitation cycle. By collecting complimentary datasets during CloudSat and TRMM satellite orbit crossings within a delta t = 45-min proximity window, it is possible to demonstrate that whenever TRMM detects a precipitation signal, the correlations along the vertical axis between the reflectivities acquired from the CPR and PR are in inverse proportion to the magnitude of the delta t proximity window. By taking advantage of these underlying correlations, it is possible to develop a functional which can be used to broaden the reflectivity spectrum, concomitantly the rain rate spectrum, of the PR measurements based on the inherently broader reflectivity spectrum of the CPR at the lower reflectivity end of the spectrum, concomitantly at lighter rain rates. With the functional in place, it is then possible to produce synthetic CloudSat precipitation imagery over the PR track and thus over the diurnal time period. These augmented data are then used to study the spectral-vertical diurnal properties of precipitation over oceanic regions observed by TRMM.

Smith, Eric A.↗

NASA NTRS2007

Managing Uncertainty Due to a Fundamental Error Source Arising from Scatterer Distribution Complexity in Radar Remote Sensing of Precipitation

The assumption that cloud and rain drops are spatially distributed according to a Poisson distribution within a scattering volume probed by a radar being used to estimate precipitation has represented bedrock theory in establishing 'rules of the game' for pulse averaging--the process needed to beat down noise to an acceptable level in the measurement of radar reflectivity factor. Based on relatively recent observations of 'realistic' spatial distributions of hydrometeor scatterers in a cloudy atmosphere motivates a renewed examination of the consequences of using a too simplified assumption underlying volume scattering--particularly in regards to the standard pulse averaging rule. Our investigation addresses two extremes, simple to complex, insofar as allowed for complexities in an underlying scatterer distribution. It is demonstrated that as the spatial distribution ranges from Poisson (a narrow distribution) to multi-fractal (much broader distribution), uncertainty in a measurement increases if the rule for pulse averaging goes unchanged from its Poisson distribution reference county. [A bounded cascade is used for the multi-fractal distribution, a regularly observed distribution vis-a-vis cloud liquid water content.] The resultant measurement uncertainty leads to a fundamental source of error in the estimation of rain rate from radar measurements, one that has been disregarded since the early 1950s when radar sets first began to be used for rainfall measuring. It is shown how this source of error can be 'managed'--under the assumption that number of data analysis experiments would be carried out, experiments involving pulse-by-pulse measurements obtained from a radar set modified to output individual pulses of reflectivity factor. For practical applications, a new parameter called normalized k-sample intensity invariance is developed to enable defining the required pulse average count according to a preferred degree of uncertainty.

Smith, Eric A.↗

NASA NTRS2007

Detailed Analysis of Indian Summer Monsoon Rainfall Processes with Modern/High-Quality Satellite Observations

We examine, in detail, Indian Summer Monsoon rainfall processes using modernhigh quality satellite precipitation measurements. The focus here is on measurements derived from three NASA cloud and precipitation satellite missionslinstruments (TRMM/PR&TMI, AQUNAMSRE, and CLOUDSATICPR), and a fourth TRMM Project-generated multi-satellite precipitation measurement dataset (viz., TRMM standard algorithm 3b42) -- all from a period beginning in 1998 up to the present. It is emphasized that the 3b42 algorithm blends passive microwave (PMW) radiometer-based precipitation estimates from LEO satellites with infi-ared (IR) precipitation estimates from a world network of CEO satellites (representing -15% of the complete space-time coverage) All of these observations are first cross-calibrated to precipitation estimates taken from standard TRMM combined PR-TMI algorithm 2b31, and second adjusted at the large scale based on monthly-averaged rain-gage measurements. The blended approach takes advantage of direct estimates of precipitation from the PMW radiometerequipped LEO satellites -- but which suffer fi-om sampling limitations -- in combination with less accurate IR estimates from the optical-infrared imaging cameras on GEO satellites -- but which provide continuous diurnal sampling. The advantages of the current technologies are evident in the continuity and coverage properties inherent to the resultant precipitation datasets that have been an outgrowth of these stable measuring and retrieval technologies. There is a wealth of information contained in the current satellite measurements of precipitation regarding the salient precipitation properties of the Indian Summer Monsoon. Using different datasets obtained from the measuring systems noted above, we have analyzed the observations cast in the form of: (1) spatially distributed means and variances over the hierarchy of relevant time scales (hourly I diurnally, daily, monthly, seasonally I intra-seasonally, and inter-annually), (2) time series at these different time scales taken as area-averages over the hierarchy of relevant space scales (Indian sub-Division, Indian sub-continent, and Circumambient Indian Ocean), (3) principal autocorrelation and cross-correlation structures over various monsoon space-time domains, (4) diurnally modulated amplitude-phase properties of rain rates over different monsoon space-time domains, (5) foremost rain rate probability distributions intrinsic to monsoon precipitation, and (6) behavior of extreme events including occurrences of flood and drought episodes throughout the course of inter-annual monsoon processes.

Smith, Eric A.↗

NASA NTRS1993

The three-dimensional structure of cumulus clouds over the ocean. 1: Structural analysis

Thermal channel (channel 6, 10.4-12.5 micrometers) images of five Landsat thematic mapper cumulus scenes over the ocean are examined. These images are thresholded using the standard International Satellite Cloud Climatology Project (ISCCP) thermal threshold algorithm. The individual clouds in the cloud fields are segmented to obtain their structural statistics which include size distribution, orientation angle, horizontal aspect ratio, and perimeter-to-area (PtA) relationship. The cloud size distributions exhibit a double power law with the smaller clouds having a smaller absolute exponent. The cloud orientation angles, horizontal aspect ratios, and PtA exponents are found in good agreement with earlier studies. A technique also is developed to recognize individual cells within a cloud so that statistics of cloud cellular structure can be obtained. Cell structural statistics are computed for each cloud. Unicellular clouds are generally smaller (less than or equal to 1 km) and have smaller PtA exponents, while multicellular clouds are larger (greater than or equal to 1 km) and have larger PtA exponents. Cell structural statistics are similar to those of the smaller clouds. When each cell is approximated as a quadric surface using a linear least squares fit, most cells have the shape of a hyperboloid of one sheet, but about 15% of the cells are best modeled by a hyperboloid of two sheets. Less than 1% of the clouds are ellipsoidal. The number of cells in a cloud increases slightly faster than linearly with increasing cloud size. The mean nearest neighbor distance between cells in a cloud, however, appears to increase linearly with increasing cloud size and to reach a maximum when the cloud effective diameter is about 10 km; then it decreases with increasing cloud size. Sensitivity studies of threshold and lapse rate show that neither has a significant impact upon the results. A goodness-of-fit ratio is used to provide a quantitative measure of the individual cloud results. Significantly improved results are obtained after applying a smoothing operator, suggesting the eliminating subresolution scale variations with higher spatial resolution may yield even better shape analyses.

Kuo, Kwo-Sen↗

NASA NTRS1990

Cloud and surface textural features in polar regions

The study examines the textural signatures of clouds, ice-covered mountains, solid and broken sea ice and floes, and open water. The textural features are computed from sum and difference histogram and gray-level difference vector statistics defined at various pixel displacement distances derived from Landsat multispectral scanner data. Polar cloudiness, snow-covered mountainous regions, solid sea ice, glaciers, and open water have distinguishable texture features. This suggests that textural measures can be successfully applied to the detection of clouds over snow-covered mountains, an ability of considerable importance for the modeling of snow-melt runoff. However, broken stratocumulus cloud decks and thin cirrus over broken sea ice remain difficult to distinguish texturally. It is concluded that even with high spatial resolution imagery, it may not be possible to distinguish broken stratocumulus and thin clouds from sea ice in the marginal ice zone using the visible channel textural features alone.

Welch, Ronald M.↗