Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Benchmarking Memory Performance with the Data Cube Operator

Data movement across a computer memory hierarchy and across computational grids is known to be a limiting factor for applications processing large data sets. We use the Data Cube Operator on an Arithmetic Data Set, called ADC, to benchmark capabilities of computers and of computational grids to handle large distributed data sets. We present a prototype implementation of a parallel algorithm for computation of the operatol: The algorithm follows a known approach for computing views from the smallest parent. The ADC stresses all levels of grid memory and storage by producing some of 2d views of an Arithmetic Data Set of d-tuples described by a small number of integers. We control data intensity of the ADC by selecting the tuple parameters, the sizes of the views, and the number of realized views. Benchmarking results of memory performance of a number of computer architectures and of a small computational grid are presented.

Frumkin, Michael A.↗

Radiative properties of quantum emitters in boron nitride from excited state calculations and Bayesian analysis

Abstract Point defects in hexagonal boron nitride (hBN) have attracted growing attention as bright single-photon emitters. However, understanding of their atomic structure and radiative properties remains incomplete. Here we study the excited states and radiative lifetimes of over 20 native defects and carbon or oxygen impurities in hBN using ab initio density functional theory and GW plus Bethe-Salpeter equation calculations, generating a large data set of their emission energy, polarization and lifetime. We find a wide variability across quantum emitters, with exciton energies ranging from 0.3 to 4 eV and radiative lifetimes from ns to ms for different defect structures. Through a Bayesian statistical analysis, we identify various high-likelihood charge-neutral defect emitters, among which the native V N N B defect is predicted to possess emission energy and radiative lifetime in agreement with experiments. Our work advances the microscopic understanding of hBN single-photon emitters and introduces a computational framework to characterize and identify quantum emitters in 2D materials.

Chemistry↗

Services + Components = Data Intensive Scientific Workflow Applications with MeDICi

Scientific applications are often structured as workflows that execute a series of distributed software modules to analyze large data sets. Such workflows are typically constructed using general-purpose scripting languages to coordinate the execution of the various modules and to exchange data sets between them. While such scripts provide a cost-effective approach for simple workflows, as the workflow structure becomes complex and evolves, the scripts quickly become complex and difficult to modify. This makes them a major barrier to easily and quickly deploying new algorithms and exploiting new, scalable hardware platforms. In this paper, we describe the MeDICi Workflow technology that is specifically designed to reduce the complexity of workflow application development, and to efficiently handle data intensive workflow applications. MeDICi integrates standard component-based and service-based technologies, and employs an efficient integration mechanism to ensure large data sets can be efficiently processed. We illustrate the use of MeDICi with a climate data processing example that we have built, and describe some of the new features

99 GENERAL AND MISCELLANEOUS↗

GPU-based relative fuzzy connectedness image segmentation

Purpose:Recently, clinical radiological research and practice are becoming increasingly quantitative. Further, images continue to increase in size and volume. For quantitative radiology to become practical, it is crucial that image segmentation algorithms and their implementations are rapid and yield practical run time on very large data sets. The purpose of this paper is to present a parallel version of an algorithm that belongs to the family of fuzzy connectedness (FC) algorithms, to achieve an interactive speed for segmenting large medical image data sets. Methods: The most common FC segmentations, optimizing an Script-Small-L {sub {infinity}}-based energy, are known as relative fuzzy connectedness (RFC) and iterative relative fuzzy connectedness (IRFC). Both RFC and IRFC objects (of which IRFC contains RFC) can be found via linear time algorithms, linear with respect to the image size. The new algorithm, P-ORFC (for parallel optimal RFC), which is implemented by using NVIDIA's Compute Unified Device Architecture (CUDA) platform, considerably improves the computational speed of the above mentioned CPU based IRFC algorithm. Results: Experiments based on four data sets of small, medium, large, and super data size, achieved speedup factors of 32.8 Multiplication-Sign , 22.9 Multiplication-Sign , 20.9 Multiplication-Sign , and 17.5 Multiplication-Sign , correspondingly, on the NVIDIA Tesla C1060 platform. Although the output of P-ORFC need not precisely match that of IRFC output, it is very close to it and, as the authors prove, always lies between the RFC and IRFC objects. Conclusions: A parallel version of a top-of-the-line algorithm in the family of FC has been developed on the NVIDIA GPUs. An interactive speed of segmentation has been achieved, even for the largest medical image data set. Such GPU implementations may play a crucial role in automatic anatomy recognition in clinical radiology.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Exploring Milkyway Halo Substructures with Large-Area Sky Surveys

Over the last two decades, our understanding of the Milky Way has been improved thanks to large data sets arising from large-area digital sky surveys. The stellar halo is now known to be inhabited by a variety of spatial and kinematic stellar substructures, including stellar streams and stellar clouds, all of which are predicted by hierarchical Lambda Cold Dark Matter models of galaxy formation. In this dissertation, we first present the analysis of spectroscopic observations of individual stars from the two candidate structures discovered using an M-giant catalog from the Two Micron All-Sky Survey. The follow-up observations show that one of the candidates is a genuine structure which might be associated with the Galactic Anticenter Stellar Structure, while the other one is a false detection due to the systematic photometric errors in the survey or dust extinction in low Galactic latitudes. We then presented the discovery of an excess of main sequence turn-off stars in the direction of the constellations of Eridanus and Phoenix from the first-year data of the Dark Energy Survey (DES) – a five-year, 5,000 deg2 optical imaging survey in the Southern Hemisphere. The Eridanus-Phoenix (EriPhe) overdensity is centered around l ~ 285° and b ~ -60° and the Poisson significance of the detection is at least 9σ. The EriPhe overdensity has a cloud-like morphology and the extent is at least ~ 4 kpc by ~ 3 kpc in projection, with a heliocentric distance of about d ~ 16 kpc. The EriPhe overdensity is morphologically similar to the previously-discovered Virgo overdensity and Hercules-Aquila cloud. These three overdensities lie along a polar plane separated by ~ 120° and may share a common origin. In addition to the scientific discoveries, we also present the work to improve the photometric calibration in DES using auxiliary calibration systems, since the photometric errors can cause false detection in first the halo substructure. We present a detailed description of the two auxiliary calibration systems built at Texas A&M University. We then discuss how the auxiliary systems in DES can be used to improve the photometric calibration of the systematic chromatic errors – source color-dependent systematic errors that are caused by variations in the wavelength dependence of the atmospheric transmission and the instrumental throughput.

79 ASTRONOMY AND ASTROPHYSICS↗

The MCDB System for Management and Analysis of Petabyte-Scale Uncertain Data

Analysts working with very large data sets often use statistical models to “guess” at unknown, inaccurate, or missing information associated with the data. For example, a distant object viewed through an optical lens will have its position slightly shifted by imperfections in the lens. Thus, rather than considering the object’s observed position to be absolutely correct, it makes sense to take into account the lens’s imperfections to obtain a probabilistic guess as to the object’s true position. For another example, it might be important to associate some sort of error distribution with each of the individual sensors in an array of magnetometers. This error distribution may be complex and include spatially-driven covariances, because errors in nearby sensors are likely to be correlated (caused, for example, by the presence of some nearby, fixed metal object). This project is concerned with the design and implementation of a prototype data management system called the Monte Carlo Database System, or MCDB for short. MCDB allows an expert-level analyst or statistician to attach arbitrary stochastic models to very large data sets in order to “guess” the values for unknown or inaccurate data, such as the actual position of the observed object in the lens example above. When the resulting data set is analyzed, the underlying stochastic models are used to generate hundreds or thousands of possible data set instances, and each of those possible instances is analyzed separately by MCDB. Thus, MCDB does not just give a single answer to the analysis, but it actually gives an empirical distribution of query results that embody the underlying uncertainty, and can in turn be analyzed using standard statistical techniques. The stochastic models in MCDB are implemented as user-defined, external C++ libraries called Variable Generation functions (VG functions for short). Because the VG function interface is exceedingly general, it allows MCDB to be used in a very wide variety of application domains, in conjunction with virtually any stochastic model. Ultimately, we ended up calling the system that we developed over the course of the project SimSQL, to denote the fact that it allowed for stochastic simulation of data. The software developed during the lifetime of the project is available for download from the SimSQL project website: http://cmj4.web.rice.edu/SimSQL/SimSQL.html.

42 ENGINEERING↗

The System for Classification of Low-Pressure Systems (SyCLoPS): An All-In-One Objective Framework for Large-Scale Data Sets

We propose the first unified objective framework (SyCLoPS) for detecting and classifying all types of low-pressure systems (LPSs) in a given data set. We use the state-of-the-art automated feature tracking software TempestExtremes (TE) to detect and track LPS features globally in ERA5 and compute 16 parameters from commonly found atmospheric variables for classification. A Python classifier is implemented to classify all LPSs at once. The framework assigns 16 different labels (classes) to each LPS data point and designates four different types of high-impact LPS tracks, including tracks of tropical cyclone (TC), monsoonal system, subtropical storm and polar low. The classification process involves disentangling high-altitude and drier LPSs, differentiating tropical and non-tropical LPSs using novel criteria, and optimizing for the detection of the four types of high-impact LPS. A comparison of our labels with those in the International Best Track Archive for Climate Stewardship (IBTrACS) revealed an overall accuracy of 95% in distinguishing between tropical systems, extratropical cyclones, and disturbances. SyCLoPS produces a better TC detection skill compared to the previous algorithms, highlighted by an approximately 6% reduction in the false alarm rate compared to the previous TE algorithm. The vertical cross section composite of the four types of high-impact LPS we detect each shows distinct structural characteristics. Finally, we demonstrate that SyCLoPS is valuable for investigating various aspects of LPSs in climate data, such as the evolution of a single LPS track, patterns of LPS frequencies, and precipitation or wind influence associated with a particular LPS class.

54 ENVIRONMENTAL SCIENCES↗

IMPROVED VARIABLE STAR SEARCH IN LARGE PHOTOMETRIC DATA SETS: NEW VARIABLES IN CoRoT FIELD LRa02 DETECTED BY BEST II

The CoRoT field LRa02 has been observed with the Berlin Exoplanet Search Telescope II (BEST II) during the southern summer 2007/2008. A first analysis of stellar variability led to the publication of 345 newly discovered variable stars. Now, a deeper analysis of this data set was used to optimize the variability search procedure. Several methods and parameters have been tested in order to improve the selection process compared to the widely used J index for variability ranking. This paper describes an empirical approach to treat systematic trends in photometric data based upon the analysis of variance statistics that can significantly decrease the rate of false detections. Finally, the process of reanalysis and method improvement has virtually doubled the number of variable stars compared to the first analysis by Kabath et al. A supplementary catalog of 272 previously unknown periodic variables plus 52 stars with suspected variability is presented. Improved ephemerides are given for 19 known variables in the field. In addition, the BEST II results are compared with CoRoT data and its automatic variability classification.

79 ASTRONOMY AND ASTROPHYSICS↗

CoreCruncher : Fast and Robust Construction of Core Genomes in Large Prokaryotic Data Sets

The core genome represents the set of genes shared by all, or nearly all, strains of a given population or species of prokaryotes. Inferring the core genome is integral to many genomic analyses, however, most methods rely on the comparison of all the pairs of genomes; a step that is becoming increasingly difficult given the massive accumulation of genomic data. Here, we present CoreCruncher; a program that robustly and rapidly constructs core genomes across hundreds or thousands of genomes. CoreCruncher does not compute all pairwise genome comparisons and uses a heuristic based on the distributions of identity scores to classify sequences as orthologs or paralogs/xenologs. Although it is much faster than current methods, our results indicate that our approach is more conservative than other tools and less sensitive to the presence of paralogs and xenologs. CoreCruncher is freely available from: https://github.com/lbobay/CoreCruncher. CoreCruncher is written in Python 3.7 and can also run on Python 2.7 without modification. It requires the python library Numpy and either Usearch or Blast. Certain options require the programs muscle or mafft.

59 BASIC BIOLOGICAL SCIENCES↗

Distributed Queries of Large Numerical Data Sets

We have extended a previously developed high-level data model, which combines numerical quantities and meta-data into a unified hybrid model, to distributed data. An elegant query language based on SQL is extended further to allow queries against such a distributed hybrid data base. The extension is realized by allowing statements in a non-SQL programming language to be embedded in SQL view definitions.

Nemes, Richard M.↗

Integrating Large Scale Data Sets to Develop Predictive Hypotheses of Low-Dose Radiation-Induced Health Effects

Over one hundred years of radiation biology research has revealed much about the DNA damages induced by the deposition of energy from exposure to ionizing radiation and the subsequent cellular responses. However, there are still significant gaps in our understanding of how these might lead to detrimental health effects, particularly at low doses (100 mGy (milligray)). Recent advances in high throughput omics technologies enable interrogation of induced radiation effects at the genomic, proteomic and metabolomic levels. These include changes in gene expression, protein modifications, e.g., phosphorylation, acetylation, and methylation, and metabolic changes. We will discuss the integration of data obtained from multiple omics platforms to understand radiation dose, and dose rate effects in a complex human tissue model as a function of time. We will use as an example our results on the low dose responses in a 3D human skin model.

ionizing radiation↗