Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

How Long Do Satellites Need to Overlap? Evaluation of Climate Data Stability from Overlapping Satellite Records

Sensors on satellites provide unprecedented understanding of the Earth's climate system by measuring incoming solar radiation, as well as both passive and active observations of the entire Earth with outstanding spatial and temporal coverage. A common challenge with satellite observations is to quantify their ability to provide well-calibrated, long-term, stable records of the parameters they measure. Ground-based intercomparisons offer some insight, while reference observations and internal calibrations give further assistance for understanding long-term stability. A valuable tool for evaluating and developing long-term records from satellites is the examination of data from overlapping satellite missions. This paper addresses how the length of overlap affects the ability to identify an offset or a drift in the overlap of data between two sensors. Ozone and temperature data sets are used as examples showing that overlap data can differ by latitude and can change over time. New results are presented for the general case of sensor overlap by using Solar Radiation and Climate Experiment (SORCE) Spectral Irradiance Monitor (SIM) and Solar Stellar Irradiance Comparison Experiment (SOLSTICE) solar irradiance data as an example. To achieve a 1 % uncertainty in estimating the offset for these two instruments' measurement of the Mg II core (280 nm) requires approximately 5 months of overlap. For relative drift to be identified within 0.1 %/yr uncertainty (0.00008 W/sq m/nm/yr), the overlap for these two satellites would need to be 2.5 years. Additional overlap of satellite measurements is needed if, as is the case for solar monitoring, unexpected jumps occur adding uncertainty to both offsets and drifts; the additional length of time needed to account for a single jump in the overlap data may be as large as 50 % of the original overlap period in order to achieve the same desired confidence in the stability of the merged data set. Results presented here are directly applicable to satellite Earth observations. Approaches for Earth observations offer additional challenges due to the complexity of the observations, but Earth observations may also benefit from ancillary observations taken from ground-based and in situ sources. Difficult choices need to be made when monitoring approaches are considered; we outline some attempts at optimizing networks based on economic principles. The careful evaluation of monitoring overlap is important to the appropriate application of observational resources and to the usefulness of current and future observations.

climate data records↗

Data Albums: An Event Driven Search, Aggregation and Curation Tool for Earth Science

One of the largest continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available. Approaches used in Earth science research such as case study analysis and climatology studies involve gathering discovering and gathering diverse data sets and information to support the research goals. Research based on case studies involves a detailed description of specific weather events using data from different sources, to characterize physical processes in play for a specific event. Climatology-based research tends to focus on the representativeness of a given event, by studying the characteristics and distribution of a large number of events. This allows researchers to generalize characteristics such as spatio-temporal distribution, intensity, annual cycle, duration, etc. To gather relevant data and information for case studies and climatology analysis is both tedious and time consuming. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the datasets of interest can obtain the specific files they need using these systems. However, in cases where researchers are interested in studying a significant event, they have to manually assemble a variety of datasets relevant to it by searching the different distributed data systems. In these cases, a search process needs to be organized around the event rather than observing instruments. In addition, the existing data systems assume users have sufficient knowledge regarding the domain vocabulary to be able to effectively utilize their catalogs. These systems do not support new or interdisciplinary researchers who may be unfamiliar with the domain terminology. This paper presents a specialized search, aggregation and curation tool for Earth science to address these existing challenges. The search tool automatically creates curated "Data Albums", aggregated collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; and information about the event contained in news reports, images or videos to supplement research analysis. Curation in the tool is driven via an ontology based relevancy ranking algorithm to filter out non-relevant information and data.

Ramachandran, Rahul↗

The evolution of the FIGARO data reduction system

The Figaro data reduction system originated at Caltech around 1983. It was based on concepts being developed in the U.K. by the Starlink organization, particularly the use of hierarchical self-defining data structures and the abstraction of most user-interaction into a set of 'parameter system' routines. Since 1984 it has continued to be developed at AAO, in collaboration with Starlink and Caltech. It was adopted as Starlink's main spectroscopic data reduction package, although it is by no means limited to spectra; it has operations for images and data cubes and even a few (very specialized) for four-dimensional data hypercubes. It continued to be used at Caltech and will be used at the Keck. It is also in use at a variety of other organizations around the world. Figaro was originally a system for VMS Vaxes. Recently it was ported (at Caltech) to run on SUN's, and work is underway at the University of New South Wales on a DecStation version. It is hoped to coordinate all this work into a unified release, but coordination of the development of a system by organizations covering three continents poses a number of interesting administrative problems. The hierarchical data structures used by Figaro allow it to handle a variety of types of data, and to add new items to data structures. Error and data quality information was added to the basic file format used, error information being particularly useful for infrared data. Cooperating sets of programs can add specific sub-structures to data files to carry information that they understand (polarimetry data containing multiple data arrays, for example), without this affecting the way other programs handle the files. Complex instrument-specific ancillary information can be added to data files written at a telescope and can be used by programs that understand the instrumental details in order to produce properly calibrated data files. Once this preliminary data processing was done the resulting files contain 'ordinary' spectra or images that can be processed by programs that are not instrument-specific. The structures holding the instrumental information can then be discarded from the files. Much effort has gone into trying to make it easy to write Figaro programs; data access subroutines are now available to handle access to all the conventional items found in Figaro files (main data arrays, error information, quality information etc), and programs that only need to access such items can be very simple indeed. A large number of Figaro users do indeed write their own Figaro applications using these routines. The fact that Figaro programs are written as callable subroutines getting information from the user through a small set of parameter routines means that they can be invoked in numerous ways; they are normally linked and run as individual programs (called by a small main routine that is generated automatically), but are also available linked to run under the ADAM data acquisition system and there is an interface that lets them be called as part of a user-written Fortran program. The long-term future of Figaro probably depends to a large extent on how successfully it manages the transition from being a VMS-only system to being a multi-platform system.

Shortridge, K.↗

A Complete Set of Radiative and Auger Rates for K-vacancy States in Fe XVIII-Fe-XXV

A complete set of level energies, wavelengths, A-values, and total and partial Auger rates have been computed for transitions involving the K-vacancy states within the n = 2 complex of Fe XVIII-Fe XXV. Three different standard numerical packages are used for this purpose, namel y AUTOSTRUCTURE, the Breit-Pauli R-matrix suite (BPRM) and HFR, which allow reliable estimates of the physical effects involved and of the accuracy of the resulting data sets. The Breit interaction is taken i nto account because its contributions to the small A-values and partial Auger rates cannot be neglected with increasing electron occupancy. Semiempirical adjustments can also lead to large differences in both the radiative and Auger decay data of strongly mixed levels. Several experimental level energies and wavelengths are questioned, and significant discrepancies are found with previously computed decay rates th at are attributed to numerical problems. The statistical accuracy of the present level energies and wavelengths is ranked at +/-3 eV and +/ -2 mA, respectively, and that for A-values and partial Auger rates greater than lO(exp 13)/s at better than 20%.

Palmeri, P.↗

Automatic Detection and Classification of Aurora in THEMIS All‐Sky Images

We report a novel machine-learning algorithm for automatically detecting and classifying aurora in all–sky images (ASI) that is largely trained without requiring ground–truth labels. By including a small number of labeled images, we are able to automatically label all of the approximately 700 million images in the Time History of Events and Macroscale Interactions during Substorms (THEMIS) ASI data set from 2008 to 2022. We use a two–stage approach. In the first stage, we adapt the Simple framework for Contrastive Learning of Representations (SimCLR) algorithm to learn latent representations of THEMIS all–sky images. We then finetune a classifier network on the latent representations our model learns of the manually labeled Oslo aurora THEMIS (OATH) data set. We demonstrate that this two–stage approach achieves excellent classification results on data for which there is no current ML classification benchmark. The outcome of this work will facilitate efficient information retrieval for researchers interested in specific categories of aurora and will enable large scale statistical studies and machine learning analyses of THEMIS all–sky images that have not previously been possible. To demonstrate possible ways to utilize this database, we performed a statistical analysis of the occurrence rates of auroral labels with respect to solar wind parameters, interplanetary magnetic field vector, and geomagnetic indices. We further investigate the occurrence rates of auroral phenomena in the annotated data set and their geoeffectiveness by utilizing the co–located THEMIS ground magnetometer data set.

Jeremiah W Johnson↗

Multivariate statistical analysis software technologies for astrophysical research involving large data bases

We developed a package to process and analyze the data from the digital version of the Second Palomar Sky Survey. This system, called SKICAT, incorporates the latest in machine learning and expert systems software technology, in order to classify the detected objects objectively and uniformly, and facilitate handling of the enormous data sets from digital sky surveys and other sources. The system provides a powerful, integrated environment for the manipulation and scientific investigation of catalogs from virtually any source. It serves three principal functions: image catalog construction, catalog management, and catalog analysis. Through use of the GID3* Decision Tree artificial induction software, SKICAT automates the process of classifying objects within CCD and digitized plate images. To exploit these catalogs, the system also provides tools to merge them into a large, complete database which may be easily queried and modified when new data or better methods of calibrating or classifying become available. The most innovative feature of SKICAT is the facility it provides to experiment with and apply the latest in machine learning technology to the tasks of catalog construction and analysis. SKICAT provides a unique environment for implementing these tools for any number of future scientific purposes. Initial scientific verification and performance tests have been made using galaxy counts and measurements of galaxy clustering from small subsets of the survey data, and a search for very high redshift quasars. All of the tests were successful, and produced new and interesting scientific results. Attachments to this report give detailed accounts of the technical aspects for multivariate statistical analysis of small and moderate-size data sets, called STATPROG. The package was tested extensively on a number of real scientific applications, and has produced real, published results.

Djorgovski, S. George↗

Multivariate Statistical Analysis Software Technologies for Astrophysical Research Involving Large Data Bases

We developed a package to process and analyze the data from the digital version of the Second Palomar Sky Survey. This system, called SKICAT, incorporates the latest in machine learning and expert systems software technology, in order to classify the detected objects objectively and uniformly, and facilitate handling of the enormous data sets from digital sky surveys and other sources. The system provides a powerful, integrated environment for the manipulation and scientific investigation of catalogs from virtually any source. It serves three principal functions: image catalog construction, catalog management, and catalog analysis. Through use of the GID3* Decision Tree artificial induction software, SKICAT automates the process of classifying objects within CCD and digitized plate images. To exploit these catalogs, the system also provides tools to merge them into a large, complex database which may be easily queried and modified when new data or better methods of calibrating or classifying become available. The most innovative feature of SKICAT is the facility it provides to experiment with and apply the latest in machine learning technology to the tasks of catalog construction and analysis. SKICAT provides a unique environment for implementing these tools for any number of future scientific purposes. Initial scientific verification and performance tests have been made using galaxy counts and measurements of galaxy clustering from small subsets of the survey data, and a search for very high redshift quasars. All of the tests were successful and produced new and interesting scientific results. Attachments to this report give detailed accounts of the technical aspects of the SKICAT system, and of some of the scientific results achieved to date. We also developed a user-friendly package for multivariate statistical analysis of small and moderate-size data sets, called STATPROG. The package was tested extensively on a number of real scientific applications and has produced real, published results.

Djorgovski, S. G.↗

Direct Validation of the Wall Interference Correction System of the Ames 11-Foot Transonic Wind Tunnel

Data from the test of a large semispan model was used to perform a direct validation of a wall interference correction system for a transonic slotted wall wind tunnel. At first, different sets of uncorrected aerodynamic coefficients were generated by physically changing the boundary condition of the test section walls. Then, wall interference corrections were computed and applied to all data points. Finally, an interpolation of the corrected aerodynamic coefficients was performed. This interpolation made sure that the corrected Mach number of a given run would be constant. Overall, the agreement between corresponding interpolated lift, drag, and pitching moment coefficient sets was very good. Buoyancy corrections were also investigated. These studies showed that the accuracy goal of one drag count may only be achieved if reliable estimates of the wall interference induced buoyancy correction are available during a test.

Ulbrich, Norbert↗

Global Distribution of Aerosols Over the Open Ocean as Derived from the Coastal Zone Color Scanner

Climatological maps of monthly mean aerosol radiance levels derived from the coastal zone color scanner (CZCS) were constructed for the world's ocean basins. This is the first study to use the 7.5.-year CZCS data set to examine the distribution and seasonality of aerosols over the open ocean on a global scale. Examination of our satellite images found the most prominent large-scale patch of elevated aerosol radiances in each month off the coast of northwest Africa. The well-known, large-scale plumes of elevated aerosol levels in the Arabian Sea, the northwest Pacific, and off the east coast of North America were also successfully captured. Radiance data were extracted from 13 major open-ocean zones, ranging from the subpolar to equatorial regions. Results from these extractions revealed the aerosol load in both subpolar and subtropical zones to be higher in the Northern Hemisphere than in the Southern Hemisphere. Aerosol radiances in the subtropics of both hemispheres were about 2 times higher in summer than in winter. In subpolar regions, aerosol radiances in late spring/early summer were almost 3 times that observed in winter. In general, the aerosol signal was higher during the warmer months and lower during the cooler months, irrespective of location. A comparison between our mean monthly aerosol radiance maps with mean monthly chlorophyll maps (also from CZCS) showed similar seasonality between aerosol and chlorophyll levels in the subpolar zones of both hemispheres, i.e., high levels in summer, low levels in winter. In the subtropics of both hemispheres, however, chlorophyll levels were higher in winter months which coincided with a depressed aerosol signal. Our results indicate that the near-IR channel on ocean color sensors can be used to successfully capture well-known, large-scale aerosol plumes on a global scale and that future ocean color sensors may provide a platform for long-term synoptic studies of combined aerosol-phytoplankton productivity interactions.

Stegmann, P. M.↗

Combined MODIS/AMSR-E SST Composites for Regional Weather Applications

Recent applications of a high resolution MODlS composite SST product have clearly shown the importance of developing high-resolution SST data sets for coastal applications and modeling. In general, coupling between the oceans and atmospheres has been closely linked to SST gradients and fronts, indicating a need for high resolution SSTs, specifically in the areas of large gradients associated with coastal regions. Thus an accurate determination of SST gradients has become critical for determining the appropriate air-sea coupling and the influence on ocean modeling. Recent research is focused on improving the accuracy and spatial coverage of the current operational MODIS SST composite product provided by the Short-term Prediction Research and Transition (SPORT) project and distributed to the community. GHRSST-PP MODlS data and microwave AMSR-E data are being combined to produce composite data sets for both the West Coast and East Coast of the United States, including the Gulf of Mexico. The use of 1 km MODIS data has explicit advantages over other SST products including its global coverage and high resolution. The AMSR-E data will reduce the latency of the composites. A strategy for utilizing the error characteristics contained in the GHRSST data has been developed. This strategy will include using the error characteristics directly to calculate weights in the SST composites, uncertainty maps based on the composite biases and RMS errors, and latency products calculated in the compositing process. Recent accomplishments include the development of an enhanced compositing approach based on the error-weighted combination of recent clear MODIS SST values, where the error contributions come from measurement error, potential cloud contamination, and data latency sources. Future plans call for the inclusion of AMSR-E SST values with appropriate weights based upon measurement accuracy, MODIS-AMSR-E SST bias, and latency.

Jedlovec, Gary↗

An ALMA Study of the Massive Molecular Clump N159W-North in the Large Magellanic Cloud: A Possible Gas Flow Penetrating One of the Most Massive Protocluster Systems in the Local Group

Massive dense clumps in the Large Magellanic Cloud can be an important laboratory to explore the formation ofpopulous clusters. We report multiscale ALMA observations of the N159W-North clump, which is the most CO-intense region in the galaxy. High-resolution CO isotope and 1.3 mm continuum observations with an angularresolution of∼025(∼0.07 pc)revealed more thanfive protostellar sources with CO outflows within the mainridge clump. One of the thermal continuum sources, MMS-2, shows an especially massive/dense nature whosetotal H2mass and peak column density are∼104Meand∼1024cm−2, respectively, and harbors massive(∼100Me)starless core candidates identified as its internal substructures. The main ridge containing this sourcecan be categorized as one of the most massive protocluster systems in the Local Group. The CO high-resolutionobservations found several distinctfilamentary clouds extending southward from the star-forming spots. The CO(1–0)data set with a largerfield of view reveals a conical,∼30 pc long complex extending toward the northerndirection. These features indicate that a large-scale gas compression event may have produced the massive star-forming complex. Based on the striking similarity between the N159W-North complex and the other twopreviously reported high-mass star-forming clouds in the nearby regions, we propose a“teardrops inflow model”that explains the synchronized, extreme star formation across>50 pc, including one of the most massiveprotocluster clumps in the Local Group.

Kazuki Tokuda↗

Vertical Transport Processes for Inert and Scavenged Species: TRACE-A Measurements

The TRACE-A mission of the NASA DC-8 aircraft made a large-scale survey of the tropical and subtropical atmosphere in September and October of 1992. Both In-situ measurements of CO (G. Sachsen NASA Langley) and aerosol size (J. Browell group, NASA Langley) provide excellent data sets with which to constrain vertical transport by planetary boundary layer mixing and deep-cloud cumulus convection. Lidar profiles of aerosol-induced scattering and ozone (also by Bremen) are somewhat require more subtle interpretation as tracers, but the vertical information on layering largely compensates for these complexities. The reason this DC-8 dataset is so useful is that very large areas of biomass burning over Africa and South America provide surface sources of appropriate sizes with which to characterize vertical and horizontal motions; the major limitation of our source description is that biomass burning patterns move considerably every few days, and daily burning inventories are a matter of concurrent, intensive research. We use the Penn State / NCAR MM5 model in an assimilation mode on the synoptic and intercontinental scale, and assess the success it shows in vertical transport descriptions. We find that the general level of emissions suggested by the climatological approach (Will. Has, U. of Montana) appears to be approximately correct, possibly a bit low, for this October, 1992, time period. Vertical transport in planetary boundary layer mixing to 5.5 kin was observed and reproduced in our simulations. Furthermore we find evidence that Blackader "transilient" or matrix-transport scheme is needed, but may require some adaptation in our tracer model: CO seems to exhibit very high values at the top of the planetary boundary layer, a process that stretches the eddy-diffusion parameterization. We will report on progress in improving the deep convective transport of carbon monoxide: the Grail scheme as we used it at 100 kin resolution did not transport enough material to the upper troposphere. We expect to be able to attribute this to either parameterization reasons (inadequacy of this parameterization at the large 100km scale) or other reasons. Nevertheless, the qualitative nature of deep transport by clouds shows up well in the simulations. As for scavengable species, the simulations predict tens of micrograms per standard cubic meter of smoke aerosol in the boundary layer. In a straightforward illustration of our simple bulk-mass scavenging parameterization, to one or two micrograms per standard cubic meter of smoke aerosol in the free troposphere just above the source regions: very high concentrations for the free troposphere. We expect to report on comparisons of these predictions to a variety of observations.

Chatfield, Robert B.↗

Mapping of lithologic and structural units using multispectral imagery

The author has identified the following significant results. ERTS-1 MSS imagery covering the Afar-Triangle/Ethiopia and adjacent regions (Ethiopian Plateau, Somali Plateau, and parts of Yemen and Saudi Arabi) was applied to the mapping of lithologic and structural units of the test area at a scale 1:1,000,000. Results of the geological evaluation of the ERTS-1 imagery of the Afar have proven the usefullness of this type of satellite data for regional geological mapping. Evaluation of the ERTS images also resulted in new aspects of the structural setting and tectonic development of the Afar-Triangle, where three large rift systems, the oceanic rifts of the Red Sea and Gulf of Aden and the continental East African rift system, seem to meet each other. Surface structures mapped by ERTS do not indicate that the oceanic rift of the Gulf of Aden (Sheba Ridge) continues into the area of continental crust west of the Gulf of Tadjura. ERTS data show that the Wonji fault belt of the African rift system does not enter or cut through the central Afar. The Aysha-Horst is not a Horst but an autochthonous spur of the Somali Plateau.

Kronberg, P.↗

Tectonic stress in the plates

In the present paper, the basic set of global intraplate stress orientation data is plotted and tabulated. Although the global intraplate stress field is complicated, several large-scale patterns can be seen. Much of stable North America is characterized by an E-W to NE-SW trend for the maximum compressive stress. South American lithosphere beneath the Andes, and perhaps farther east in the stable interior, has horizontal compressive stresses trending E-W to NW-SE. Western Europe north of the Alps is characterized by a NW-SE trending maximum horizontal compression, while Asia has the maximum horizontal compressive stress trending more nearly N-S, especially near the Himalayan front.

Richardson, R. M.↗

Regional analysis of earthquake occurrence and seismic energy release

The historic temporal variation in earthquake occurrence and seismic energy release on a regional basis throughtout the world were studied. The regionalization scheme employed divided the world into large areas based either on seismic and tectonic considerations (Flinn-Engdahl Scheme) or geographic (longitude and latitude) criteria. The data set is the wide earthquake catalog of the National Geophysical Solar-Terrestrial Data Center. An apparent relationship exists between the maximum energy released in a limited time within a seismic region and the average or background energy per year averaged over a long time period. In terms of average or peak energy release, the most seismic regions of the world during the 50 to 81 year period ending in 1977 were Japanese, Andean South American, and the Alaska-Aleutian Arc regions. The year to year fluctuations in regional seismic energy release are greater, by orders of magnitude, than the corresponding variations in the world-wide seismic energy release. The b values of seismic regions range from 0.7 to 1.4 where earthquake magnitude is in the range 6.0 to 7.5.

Cohen, S. C.↗

A computer graphics display technique for the examination of aircraft design data

An interactive computer graphics technique has been developed for quickly sorting and interpreting large amounts of aerodynamic data. It utilizes a graphic representation rather than numbers. The geometry package represents the vehicle as a set of panels. These panels are ordered in groups of ascending values (e.g., equilibrium temperatures). The groups are then displayed successively on a CRT building up to the complete vehicle. A zoom feature allows for displaying only the panels with values between certain limits. The addition of color allows a one-time display thus eliminating the need for a display build up.

Talcott, N. A., Jr.↗

Dynamics and energetics of the South Pacific Convergence Zone during FGGE SOP-1

The major research objectives are to diagnose the physical processes responsible for the maintenance of the South Pacific Convergence Zone (SPCZ) and to examine the role of the SPCZ in the large-scale circulation patterns of the Southern Hemisphere. To accomplish these objectives researchers used several data sources which include: a modified set of Level III-b upper air analyses, originally produced by ECMWF (Vincent, 1982); subjectively analyzed surface analyses for the South Pacific based on island station reports (Vincent, 1985); outgoing longwave radiation values supplied to us by NOAA/NESDIS; and equivalent black body temperatures and precipitation rates derived by Robertson. In the past year researchers found that wave number four plays an inportant role in the Southern Hemisphere tropics during the 15-day period when the sPCZ was a dominant feature, particularly with regard to the baroclinic conversion of potential to kinetic energy (Huang and Vincent, 1985). The convectively-active SPCZ area was found to make a significant contribution to this conversion process; thus, it appears that baroclinic effects and latent heating are important in maintaining the SPCZ. Recently efforts concentrated on two research tasks, an examination of cyclone activity within the SPCZ (Kann, 1985; Vincent, 1985; Vincent and Kann, 1985) and a study of the heat and moisture budgets in the South Pacific (Miller, et al., 1985). It was found that cyclonic disturbances occurred with regularity in the Zone from 10 to 17 January.

Vincent, D. G.↗

Pattern recognition of satellite cloud imagery for improved weather prediction

The major accomplishment was the successful development of a method for extracting time derivative information from geostationary meteorological satellite imagery. This research is a proof-of-concept study which demonstrates the feasibility of using pattern recognition techniques and a statistical cloud classification method to estimate time rate of change of large-scale meteorological fields from remote sensing data. The cloud classification methodology is based on typical shape function analysis of parameter sets characterizing the cloud fields. The three specific technical objectives, all of which were successfully achieved, are as follows: develop and test a cloud classification technique based on pattern recognition methods, suitable for the analysis of visible and infrared geostationary satellite VISSR imagery; develop and test a methodology for intercomparing successive images using the cloud classification technique, so as to obtain estimates of the time rate of change of meteorological fields; and implement this technique in a testbed system incorporating an interactive graphics terminal to determine the feasibility of extracting time derivative information suitable for comparison with numerical weather prediction products.

Gautier, Catherine↗