Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “very large databases”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

The Effect of Spectral Resolution of Light Sources on Photopic and a-opic Quantities

There is a growing interest in including light source spectrum in advanced lighting software tools and simulations. Given that traditional lighting software tools have been used primarily for calculating photometric quantities, their simplifying assumptions may not be suitable for calculating other a-opic quantities. Commonly used simulation tools use three values to represent the three primary colors (red, green, and blue), but others have expanded the number of bands (i.e., the spectral resolution) to 9, 27, or 81 in an attempt to more accurately represent variations in light source spectrum and downstream spectrally-derived quantities. It remains unclear, however, to what extent spectral resolution affects calculated quantities. To address this gap, a numerical analysis was completed using a large spectral power distribution database (n = 1,302 light sources). Calculated illuminance, a-opic irradiance, luminous efficacy of radiation (LER), melanopic efficacy of radiation (MER), and melanopic to photopic (M/P) ratio were compared for spectral resolutions of 3 and 9 bands compared to a baseline of 81 bands. Across all examined lighting quantities, considerable errors—a mean absolute percent error of 18.4%—were found when using 3 band calculations. These were reduced to 4% for 9 band calculations. The errors varied by metric and light source type. The results suggest that the use of 9 bands can more accurately characterize performance of different light source types across a range of metrics compared to using 3 bands.

Abboushi, Belal K.↗

Conquering Data Chaos: Research Data Management with Kubernetes

Managing massive volumes of data and effectively making it accessible to researchers poses significant challenges and is a barrier to scientific discovery. In many cases, critical data is locked up in unwieldy file formats or one-off databases and is too large to effectively process on a single machine. This talk explores the role of Kubernetes, an open-source container orchestration platform, in addressing research data management challenges. I will discuss how we are using a set of publicly available open-source and home-grown tools in the National Renewable Energy Lab (NREL) Data, Analysis, and Visualization (DAV) group to help researchers overcome data-related bottlenecks. The talk will begin by providing an overview of the data challenges faced in research data management, including data storage, processing, and analysis. I will highlight Kubernetes' ability to handle large-scale data by leveraging containerization and distributed computing, including distributed storage. Kubernetes allows researchers to encapsulate data processing infrastructure and workflows into portable containers, enabling reproducibility and ease of deployment. Kubernetes can then schedule and manage the resource allocation of these containers to enable efficient utilization of limited computing resources, leading to more efficient data processing and analysis. I will discuss some limitations of traditional, siloed approaches to dealing with data and emphasize the need for solutions which foster collaboration. I will highlight how we are using Kubernetes at NREL to facilitate data sharing and cooperation among research teams. Kubernetes' flexible architecture enables the deployment of shared computing environments, such as Apache Superset, where researchers can seamlessly access and analyze shared datasets. Providing the ability to have one research team easily consume data generated by another, utilizing Kubernetes' as a central data platform, is one of the major wins we've encountered by adopting the platform. Finally, I will showcase real-world use cases from NREL where we have used Kubernetes to solve some persistent data challenges involving large volumes of sensor and monitoring data. I will discuss the challenges we encountered when creating our cluster and making it available as a production-ready resource. I will also discuss the specific suite of tools, including Postgres and Apache Druid for columnar and timeseries data, and Redpanda Kafka for streaming data we have deployed in our infrastructure, and the process that went into the selection of these tools.

collaborative environment↗

PROGRESS TOWARDS NEXT GENERATION, WAVEFORM BASED THREE-DIMENSIONAL MODELS AND METRICS TO IMPROVE NUCLEAR EXPLOSION MONITORING IN THE MIDDLE EAST

Efforts to update current wave speed models of the Middle East require a thoroughly tested database of sources and recordings. Recordings of seismic waves traversing the region from Tibet to the Red Sea will be the principal metric in guiding improvements to the current wave speed model. Precise characterizations of the earthquakes, specifically depths and faulting mechanisms, are essential to avoid mapping source errors into the refined wave speed model. Errors associated with the source are manifested in amplitude and phase changes. Source depths and paths near nodal planes are particularly error prone as small changes may severely affect the resulting wavefield. Once sources are quantified, regions requiring refinement will be highlighted using adjoint tomography methods based on spectral element simulations [Komatitsch and Tromp (1999)]. An initial database of 250 regional Middle Eastern events from 1990-2007, was inverted for depth and focal mechanism using teleseismic arrivals [Kikuchi and Kanamori (1982)] and regional surface and body waves [Zhao and Helmberger (1994)]. From this initial database, we reinterpreted a large, well recorded subset of 201 events through a direct comparison between data and synthetics based upon a centroid moment tensor inversion [Liu et al. (2004)]. Evaluation was done using both a 1D reference model [Dziewonski and Anderson (1981)] at periods greater than 80 seconds and a 3D model [Kustowski et al. (2008)] at periods of 25 seconds and longer. The final source reinterpretations will be within the 3D model, as this is the initial starting point for the adjoint tomography. Transitioning from a 1D to 3D wave speed model shows dramatic improvements when comparisons are done at shorter periods, (25 s). Synthetics from the 1D model were created through mode summations while those from the 3D simulations were created using the spectral element method. To further assess errors in source depth and focal mechanism, comparisons between the three methods were made. These comparisons help to identify problematic stations and sources which may bias the final solution. Estimates of standard errors were generated for each event's source depth and focal mechanism to identify poorly constrained events. A final, well characterized set of sources and stations will be then used to iteratively improve the wave speed model of the Middle East. After a few iterations during the adjoint inversion process, the sources will be reexamined and relocated to further reduce mapping of source errors into structural features. Finally, efforts continue in developing the infrastructure required to 'quickly' generate event kernels at the n-th iteration and invert for a new, (n+1)-th, wave speed model of the Middle East. While development of the infrastructure proceeds, initial tests using a limited number of events shows the 3D model, while showing vast improvement compared to the 1D model, still requires substantial modifications. Employing our new, full source set and iterating the adjoint inversions at successively shorter periods will lead to significant changes and refined wave speed structures of the Middle East.

58 GEOSCIENCES↗

MASSCLEANage-STELLAR CLUSTER AGES FROM INTEGRATED COLORS

We present the recently updated and expanded MASSCLEANcolors, a database of 70 million Monte Carlo models selected to match the properties (metallicity, ages, and masses) of stellar clusters found in the Large Magellanic Cloud (LMC). This database shows the rather extreme and non-Gaussian distribution of integrated colors and magnitudes expected with different cluster age and mass and the enormous age degeneracy of integrated colors when mass is unknown. This degeneracy could lead to catastrophic failures in estimating age with standard simple stellar population models, particularly if most of the clusters are of intermediate or low mass, like in the LMC. Utilizing the MASSCLEANcolors database, we have developed MASSCLEANage, a statistical inference package which assigns the most likely age and mass (solved simultaneously) to a cluster based only on its integrated broadband photometric properties. Finally, we use MASSCLEANage to derive the age and mass of LMC clusters based on integrated photometry alone. First, we compare our cluster ages against those obtained for the same seven clusters using more accurate integrated spectroscopy. We find improved agreement with the integrated spectroscopy ages over the original photometric ages. A close examination of our results demonstrates the necessity of solving simultaneously for mass and age to reduce degeneracies in the cluster ages derived via integrated colors. We then selected an additional subset of 30 photometric clusters with previously well-constrained ages and independently derive their age using the MASSCLEANage with the same photometry with very good agreement. The MASSCLEANage program is freely available under GNU General Public License.

79 ASTRONOMY AND ASTROPHYSICS↗

An ab initio electronic transport database for inorganic materials

Electronic transport in materials is governed by a series of tensorial properties such as conductivity, Seebeck coefficient, and effective mass. These quantities are paramount to the understanding of materials in many fields from thermoelectrics to electronics and photovoltaics. Transport properties can be calculated from a material’s band structure using the Boltzmann transport theory framework. We present here the largest computational database of electronic transport properties based on a large set of 48,000 materials originating from the Materials Project database. Our results were obtained through the interpolation approach developed in the BoltzTraP software, assuming a constant relaxation time. We present the workflow to generate the data, the data validation procedure, and the database structure. In conclusion, our aim is to target the large community of scientists developing materials selection strategies and performing studies involving transport properties.

36 MATERIALS SCIENCE↗

Mining data in a dynamic PRA framework

Computational (also known as Dynamic) Probabilistic Risk Assessment (PRA) methods employs system simulation codes coupled with stochastic analysis tools in order to determine probabilities of certain outcomes such as system failure. In contrast to classical PRA methods (i.e., Event-Tree and Fault-Tree) in which timing and sequencing of events is set by the analyst, accident progression is dictated by the system control logic and its interaction with system temporal evolution. Due to the nature of the problem, Simulation-Based PRA methods require large amount of computationally expensive simulation runs. In addition, these methods usually generate a large number of simulation runs (database storage may be on the order of gigabytes or higher). In this paper we investigate and apply several methods and algorithms to analyze these large amounts of time-dependent data. Furthermore, the scope of this article is to present a broad overview of methods and algorithms that can be used to improve data quality and to analyze and extract information from large data sets containing time dependent data. In this context, “extracting information” means constructing input-output correlations, finding commonalities, and identifying outliers.

97 MATHEMATICS AND COMPUTING↗

A WASHINGTON PHOTOMETRIC SURVEY OF THE LARGE MAGELLANIC CLOUD FIELD STAR POPULATION

We present photometry for an unprecedented database of some 5.5 million stars distributed throughout the Large Magellanic Cloud main body, from 21 fields covering a total area of 7.6 deg{sup 2}, obtained from Washington CT{sub 1} T{sub 2} CTIO 4 m MOSAIC data. Extensive artificial star tests over the whole mosaic image data set and the observed behavior of the photometric errors with magnitude demonstrate the accuracy of the morphology and clearly delineate the position of the main features in the color-magnitude diagrams (CMDs). The representative T{sub 1}(MS TO) mags are on average {approx}0.5 mag brighter than the T{sub 1} mags for the 100% completeness level of the respective field, allowing us to derive an accurate age estimate. We have analyzed the CMD Hess diagrams and used the peaks in star counts at the main sequence turnoff and red clump (RC) locations to age date the most dominant sub-population (or 'representative' population) in the stellar population mix. The metallicity of this representative population is estimated from the locus of the most populous red giant branch track. We use these results to derive age and metallicity estimates for all of our fields. The analyzed fields span age and metallicity ranges covering most of the galaxy's lifetime and chemical enrichment, i.e., ages and metallicities between {approx}1 and 13 Gyr and {approx}-0.2 and -1.2 dex, respectively. We show that the dispersions associated with the mean ages and metallicities represent in general a satisfactory estimate of the age/metallicity spread ({approx}1-3 Gyr/0.2-0.3 dex), although a few subfields have a slightly larger age/metallicity spread. Finally, we revisit the study of the vertical structure (VS) phenomenon, a striking feature composed of stars that extend from the bottom, bluest end of the RC to {approx}0.45 mag fainter. We confirm that the VS phenomenon is not clearly seen in most of the studied fields and suggest that its occurrence is linked to some other condition(s) in addition to the appropriate age, metallicity, and the necessary red giant star density.

79 ASTRONOMY AND ASTROPHYSICS↗

Data Analysis Approaches for the Risk-Informed Safety Margins Characterization Toolkit

In the past decades, several numerical simulation codes have been employed to simulate accident dynamics (e.g., RELAP5-3D, RELAP-7, MELCOR, MAAP). In order to evaluate the impact of uncertainties into accident dynamics, several stochastic methodologies have been coupled with these codes. These stochastic methods range from classical Monte-Carlo and Latin Hypercube sampling to stochastic polynomial methods. Similar approaches have been introduced into the risk and safety community where stochastic methods (such as RAVEN, ADAPT, MCDET, ADS) have been coupled with safety analysis codes in order to evaluate the safety impact of timing and sequencing of events. These approaches are usually called Dynamic PRA or simulation-based PRA methods. These uncertainties and safety methods usually generate a large number of simulation runs (database storage may be on the order of gigabytes or higher). The scope of this paper is to present a broad overview of methods and algorithms that can be used to analyze and extract information from large data sets containing time dependent data. In this context, “extracting information” means constructing input-output correlations, finding commonalities, and identifying outliers. Some of the algorithms presented here have been developed or are under development within the RAVEN statistical framework.

97 MATHEMATICS AND COMPUTING↗

Implementation of a data management software system for SSME test history data

The implementation of a software system for managing Space Shuttle Main Engine (SSME) test/flight historical data is presented. The software system uses the database management system RIM7 for primary data storage and routine data management, but includes several FORTRAN programs, described here, which provide customized access to the RIM7 database. The consolidation, modification, and transfer of data from the database THIST, to the RIM7 database THISRM is discussed. The RIM7 utility modules for generating some standard reports from THISRM and performing some routine updating and maintenance are briefly described. The FORTRAN accessing programs described include programs for initial loading of large data sets into the database, capturing data from files for database inclusion, and producing specialized statistical reports which cannot be provided by the RIM7 report generator utility. An expert system tutorial, constructed using the expert system shell product INSIGHT2, is described. Finally, a potential expert system, which would analyze data in the database, is outlined. This system could use INSIGHT2 as well and would take advantage of RIM7's compatibility with the microcomputer database system RBase 5000.

Abernethy, Kenneth↗

Analysis of Time Dependent Data and PRA

In the past decades, several numerical simulation codes have been employed to simulate accident dynamics (e.g., RELAP5-3D, MELCOR, MAAP). In order to evaluate the impact of uncertainties into accident dynamics, several stochastic methodologies have been coupled with these codes. These stochastic methods range from classical Monte-Carlo and Latin Hypercube sampling to stochastic polynomial methods. Similar approaches have been introduced into the risk and safety community where stochastic methods (such as RAVEN, ADAPT, MCDET, ADS) have been coupled with safety analysis codes in order to evaluate the safety impact of timing and sequencing of events. These approaches are usually called Dynamic PRA or simulation-based PRA methods. These uncertainties and safety methods usually generate a large number of simulation runs (database storage may be on the order of gigabytes or higher). The scope of this paper is to present a broad overview of methods and algorithms that can be used to analyze and extract information from large data sets containing time dependent data. In this context, 'extracting information' means constructing input-output correlations, finding commonalities, and identifying outliers. Some of the algorithms presented here have been developed or are under development within the RAVEN statistical framework. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A Comprehensive Opacities/Atomic Database for the Analysis of Astrophysical Spectra and Modeling

The main goals of this ADP award have been accomplished. The electronic database TOPBASE, consisting of the large volume of atomic data from the Opacity Project, has been installed and is operative at a NASA site at the Laboratory for High Energy Astrophysics Science Research Center (HEASRC) at the Goddard Space Flight Center. The database will be continually maintained and updated by the PI and collaborators. TOPBASE is publicly accessible from IP: topbase.gsfc.nasa.gov. During the last six months (since the previous progress report), considerable work has been carried out to: (1) put in the new data for low ionization stages of iron: Fe I - V, beginning with Fe II, (2) high-energy photoionization cross sections computed by Dr. Hong Lin Zhang (consultant on the Project) were 'merged' with the current Opacity Project data and input into TOPbase; (3) plans laid out for a further extension of TOPbase to include TIPbase, the database for collisional data to complement the radiative data in TOPbase.

Pradhan, Anil K.↗

Algal functional annotation tool

Abstract BACKGROUND: Progress in genome sequencing is proceeding at an exponential pace, and several new algal genomes are becoming available every year. One of the challenges facing the community is the association of protein sequences encoded in the genomes with biological function. While most genome assembly projects generate annotations for predicted protein sequences, they are usually limited and integrate functional terms from a limited number of databases. Another challenge is the use of annotations to interpret large lists of 'interesting' genes generated by genome-scale datasets. Previously, these gene lists had to be analyzed across several independent biological databases, often on a gene-by-gene basis. In contrast, several annotation databases, such as DAVID, integrate data from multiple functional databases and reveal underlying biological themes of large gene lists. While several such databases have been constructed for animals, none is currently available for the study of algae. Due to renewed interest in algae as potential sources of biofuels and the emergence of multiple algal genome sequences, a significant need has arisen for such a database to process the growing compendiums of algal genomic data. DESCRIPTION: The Algal Functional Annotation Tool is a web-based comprehensive analysis suite integrating annotation data from several pathway, ontology, and protein family databases. The current version provides annotation for the model alga Chlamydomonas reinhardtii, and in the future will include additional genomes. The site allows users to interpret large gene lists by identifying associated functional terms, and their enrichment. Additionally, expression data for several experimental conditions were compiled and analyzed to provide an expression-based enrichment search. A tool to search for functionally-related genes based on gene expression across these conditions is also provided. Other features include dynamic visualization of genes on KEGG pathway maps and batch gene identifier conversion. CONCLUSIONS: The Algal Functional Annotation Tool aims to provide an integrated data-mining environment for algal genomics by combining data from multiple annotation databases into a centralized tool. This site is designed to expedite the process of functional annotation and the interpretation of gene lists, such as those derived from high-throughput RNA-seq experiments. The tool is publicly available at http://pathways.mcdb.ucla.edu.

Source record↗

Performance analysis of data intensive cloud systems based on data management and replication: a survey

As we delve deeper into the ‘Digital Age’, we witness an explosive growth in the volume, velocity, and variety of the data available on the Internet. For example, in 2012 about 2.5 quintillion bytes of data was created on a daily basis that originated from myriad of sources and applications including mobiledevices, sensors, individual archives, social networks, Internet of Things, enterprises, cameras, software logs, etc. Such ‘Data Explosions’ has led to one of the most challenging research issues of the current Information and Communication Technology era: how to optimally manage (e.g., store, replicated, filter, and the like) such large amount of data and identify new ways to analyze large amounts of data for unlocking information. It is clear that such large data streams cannot be managed by setting up on-premises enterprise database systems as it leads to a large up-front cost in buying and administering the hardware and software systems. Therefore, next generation data management systems must be deployed on cloud. The cloud computing paradigm provides scalable and elastic resources, such as data and services accessible over the Internet Every Cloud Service Provider must assure that data is efficiently processed and distributed in a way that does not compromise end-users’ Quality of Service (QoS) in terms of data availability, data search delay, data analysis delay, and the like. In the aforementioned perspective, data replication is used in the cloud for improving the performance (e.g., read and write delay) of applications that access data. Through replication a data intensive application or system can achieve high availability, better fault tolerance, and data recovery. In this paper, we survey data management and replication approaches (from 2007 to 2011) that are developed by both industrial and research communities. The focus of the survey is to discuss and characterize the existing approaches of data replication and management that tackle the resource usage and QoS provisioning with different levels of efficiencies. Moreover, the breakdown of both influential expressions (data replication and management) to provide different QoS attributes is deliberated. Furthermore, the performance advantages and disadvantages of data replication and management approaches in the cloud computing environments are analyzed. Open issues and future challenges related to data consistency, scalability, load balancing, processing and placement are also reported.

Malik, Saif Ur Rehman↗

Evaluation of Use Cases and Types of Databases for Hosting Wind Power Data

The goal of the National Wind Power Production Data Dashboard project is to develop a publicly available platform to model, process, and share wind power with uncertainty quantification for the current and future onshore and offshore wind plants across the continental United States. As of 2021, more than 70,000 utility-scale wind turbines have been installed across the United States and this number is expected to grow in the next few years. A large-scale wind power production database is needed for stakeholders (policy makers, operators, and researchers) to access easily for their decision-making. Currently, meeting that need is a challenge. Most wind power operators and system operators focus on regions of their own interest, and the data are usually inaccessible to the public.

17 WIND ENERGY↗

New approaches to an old problem: addressing spatial gaps in the World Stress Map

Abstract A well-recognized characteristic of the World Stress Map (WSM) database is the continued presence of large spatial gaps in the distribution of the data records despite the more than 40-year development history of the database. The current release has more than 30 000 high-quality (A–C) data records (often referred to as ‘stress indicators’), but while some continental areas (such as Australia) have seen a significant increase in spatial converge with the latest release, other continental regions (Africa, central Asia, most of South America) remain markedly sparse. In this contribution we (1) review the current state of the spatial distribution of stress indicators in the continental regions (above sea-level); (2) quantify the clustering of the stress indicators in the latest WSM release using the Hopkins statistic as a way to explore the current spatial distribution of the indicators and assess future WSM releases; and (3) present three approaches (joint inversion, seismic anisotropy and InSAR) that provide a way to fill in the gaps (both in the S Hmax orientation and principal stress magnitudes) in regions that lack active seismicity and where borehole drilling is cost prohibitive. These three approaches have the potential to guide procedures for improving a priori estimates of the ambient stress field in the Earth's crust and reduce the uncertainty in predicting both the magnitude and orientation of the principal tectonic stresses.

58 GEOSCIENCES↗

A high performance, ad-hoc, fuzzy query processing system for relational databases

Database queries involving imprecise or fuzzy predicates are currently an evolving area of academic and industrial research. Such queries place severe stress on the indexing and I/O subsystems of conventional database environments since they involve the search of large numbers of records. The Datacycle architecture and research prototype is a database environment that uses filtering technology to perform an efficient, exhaustive search of an entire database. It has recently been modified to include fuzzy predicates in its query processing. The approach obviates the need for complex index structures, provides unlimited query throughput, permits the use of ad-hoc fuzzy membership functions, and provides a deterministic response time largely independent of query complexity and load. This paper describes the Datacycle prototype implementation of fuzzy queries and some recent performance results.

Mansfield, William H., Jr.↗

Global auroral imaging: Dynamics of Explorer Miss ion

An automated contour analysis of auroral images from the DESAI database is being developed. The goal is to be able to feed DESAI images to the software and let it automatically determine the inner and outer boundaries of the auroral oval of a given set of DESAI images. With these contours established, the software will determine the enclosed polar cap area and (model based) integrated magnetic flux through that area. Through the use of this automated system, it will be possible to analyze a large fraction of the DESAI database and correlate it with various other solar-terrestrial space plasma physics parameters and indices.

Frank, L. A.↗

Future Projection of Tropical Upper-Tropospheric Troughs and Implications for Tropical Cyclone Activity

Summertime tropical upper-tropospheric troughs (TUTTs) provide a unified framework to better understand how extratropical and tropical forcings jointly modulate basin-scale tropical cyclone (TC) activity. In this study, we examine future changes in TUTTs and their implications for TC activity. Multimodel ensemble-mean projections from 45 Coupled Model Intercomparison Project phase 6 (CMIP6) models suggest a contraction of the Pacific TUTT and an expansion of the Atlantic TUTT as the climate warms. Consistently, future changes in environment-based TC indices indicate that the large-scale conditions will become more favorable for TC genesis and intensification over the central North Pacific but less favorable over the tropical North Atlantic and Gulf of Mexico. Utilizing a TC-permitting large-ensemble dataset [i.e., the Database for Policy Decision-Making for Future Climate Change (d4PDF)] that adequately captures the observed interannual TUTT–TC relationships, we further confirm the impacts of projected TUTT changes on the TC activity in a warmer climate. In contrast, the TUTT–TC relationship is poorly represented in most CMIP6 High-Resolution Model Intercomparison Project (HighResMIP) models; such deficiencies call for caution when assessing future TC risk based on explicitly tracked TCs in these models. Additionally, CMIP6 projections show large intermodel spread in TUTT changes, implying uncertainty in projected TC activity, especially over the central-to-eastern Pacific and the North Atlantic. This intermodel spread is associated with interhemispheric sea surface temperature warming asymmetry, which leads to a meridional shift of the intertropical convergence zone (ITCZ) and the simultaneous weakening or strengthening of TUTTs in the North Pacific and North Atlantic. The potential contributions of anthropogenic aerosol forcing and oceanic circulation to this interhemispheric warming asymmetry are briefly examined.

Climate Change↗