Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “cloud database”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

MLClouds [SWR-24-24]

The National Solar Radiation Database (NSRDB) is NREL’s flagship solar data resource. With over 20 years of high-resolution surface irradiance data covering most of the western hemisphere, the NSRDB is a crucial public data asset. A fundamental input to accurate surface irradiance in the NSRDB is high quality cloud property data. Cloud properties are used in radiative transfer calculations and are sourced from satellite imagery. Improving the accuracy of cloud property inputs is a tractable method for improving the accuracy of the irradiance data in the NSRDB. For example, in July of 2018, an average location in the Continental United States is missing cloud property data for nearly one quarter of all daylight cloudy timesteps. This project aims to improve the cloud data inputs to the NSRDB by using machine learning techniques to exploit the NSRDB’s massive data resources. More accurate cloud property input data will yield more accurate surface irradiance data in the NSRDB, providing direct benefit to researchers at NREL and to public data users everywhere.

Buster, Grant↗
DOE OSTI2015

High Performance Data Analytics on a Commodity Private Cloud

Sandia is researching novel methods of analysis for large datasets. The term “Big Data” has been used frequently in recent years to describe a discipline of computer science that attempts to distil an increasing quantity of data into a result or meaningful subset that can be easily visualized and acted upon by human consumers. These data are frequently characterized by one or more of the following: high volume, velocity, variety, and veracity. Sandia scientists and researchers are no strangers to large datasets, which often typify the result of high-performance computing workloads. The challenge for Sandia data scientists, however, comes at the arrival of exascale computing, which has become mandated by executive order for national computing centers. Because of power and networking limitations, exascale computing precludes the movement of datasets at exascale from data production to data analysis. Such analysis must be done in-situ, to mitigate the time and power costs of network transport. Currently, many of Sandia’s data analytics environments exist as independent entities, often specialized for individual use cases, interconnected to other data analytics, high-performance computing, and storage systems. Sandia is procuring, installing, and researching future platforms for such data analytics. A memory footprint of 256 GB per node, fourteen data rate (FDR) InfiniBand networking, PCIe connected non-volatile memory (NVMe) data storage, and an object data store will characterize the coming institutional data analytics platforms. The combined systems will contain 300 compute nodes. A cloud-inspired software stack resulting from collaboration between the Sandia centers will allow the systems to support a flexible mix of user and system-facing workloads and use cases including code development and testing in R and Python, Apache Hadoop and Spark, distributed in-memory databases, graph analytics, emulytics, system log and predictive analytics, and data visualization for institutional and mission customers. These systems will arrive at the end of FY 2015 and production is planned by the spring of FY 2016.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Cloud Atlas: Hubble Space Telescope Near-infrared Spectral Library of Brown Dwarfs, Planetary-mass Companions, and Hot Jupiters

Bayesian atmospheric retrieval tools can place constraints on the properties of brown dwarfs' and hot Jupiters' atmospheres. To fully exploit these methods, high signal-to-noise spectral libraries with well-understood uncertainties are essential. We present a high signal-to-noise spectral library (1.10–1.69 μm) of the thermal emission of 76 brown dwarfs and hot Jupiters. All our spectra have been acquired with the Hubble Space Telescope’s Wide Field Camera 3 instrument and its G141 grism. The near-infrared spectral types of these objects range from L4 to Y1. Eight of our targets have estimated masses below the deuterium-burning limit. We analyze the database to identify peculiar objects and/or multiple systems, concluding that this sample includes two very-low-surface-gravity objects and five intermediate-surface-gravity objects. In addition, spectral indices designed to search for composite-atmosphere brown dwarfs indicate that eight objects in our sample are strong candidates to have such atmospheres. None of these objects are overluminous, so their composite atmospheres are unlikely to be companion-induced artifacts. Five of the eight confirmed candidates have been reported as photometrically variable, suggesting that composite atmospheric indices are useful in identifying brown dwarfs with strongly heterogeneous cloud covers. We compare hot Jupiters and brown dwarfs in a near-infrared color–magnitude diagram. We confirm that the coldest hot Jupiters in our sample have spectra similar to mid-L dwarfs, and the hottest hot Jupiters have spectra similar to those of M-dwarfs. Our sample provides a uniform data set of a broad range of ultracool atmospheres, allowing large-scale comparative studies and providing an HST legacy spectral library.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

Variable stars in metal-rich globular clusters. IV. Long-period variables in NGC 6496

We present VI-band photometry for stars in the metal-rich globular cluster NGC 6496. Our time-series data were cadenced to search for long-period variables (LPVs) over a span of nearly two years, and our variability search yielded the discovery of 13 new variable stars, of which 6 are LPVs, 2 are suspected LPVs, and 5 are short-period eclipsing binaries. An additional star was found in the ASAS database, and we clarify its type and period. We argue that all of the eclipsing binaries are field stars, while five to six of the LPVs are members of NGC 6496. We compare the period–luminosity distribution of these LPVs with those of LPVs in the Large Magellanic Cloud and 47 Tucanae, and with theoretical pulsation models. We also present a VI color–magnitude diagram, display the evolutionary states of the variables, and match isochrones to determine a reddening of E(B−V)= 0.21±0.02 mag and apparent distance modulus of 15.60±0.15 mag.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Reinspection of a Clinical Proteomics Tumor Analysis Consortium (CPTAC) Dataset with Cloud Computing Reveals Abundant Post-Translational Modifications and Protein Sequence Variants

The Clinical Proteomic Tumor Analysis Consortium (CPTAC) has provided some of the most in-depth analyses of the phenotypes of human tumors ever constructed. Today, the majority of proteomic data analysis is still performed using software housed on desktop computers which limits the number of sequence variants and post-translational modifications that can be considered. The original CPTAC studies limited the search for PTMs to only samples that were chemically enriched for those modified peptides. Similarly, the only sequence variants considered were those with strong evidence at the exon or transcript level. In this multi-institutional collaborative reanalysis, we utilized unbiased protein databases containing millions of human sequence variants in conjunction with hundreds of common post-translational modifications. Using these tools, we identified tens of thousands of high-confidence PTMs and sequence variants. We identified 4132 phosphorylated peptides in nonenriched samples, 93% of which were confirmed in the samples which were chemically enriched for phosphopeptides. In addition, our results also cover 90% of the high-confidence variants reported by the original proteogenomics study, without the need for sample specific next-generation sequencing. Finally, we report fivefold more somatic and germline variants that have an independent evidence at the peptide level, including mutations in ERRB2 and BCAS1. In this reanalysis of CPTAC proteomic data with cloud computing, we present an openly available and searchable web resource of the highest-coverage proteomic profiling of human tumors described to date.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2017

Evaluation of the National Solar Radiation Database (NSRDB): 1998-2015

This paper validates the performance of the physics-based Physical Solar Model (PSM) data set in the National Solar Radiation Data Base (NSRDB) to quantify the accuracy of the magnitude and the spatial and temporal variability of the solar radiation data. Achieving higher penetrations of solar energy on the electric grid and reducing integration costs requires accurate knowledge of the available solar resource. Understanding the impacts of clouds and other meteorological constituents on the solar resource and quantifying intra-/inter-hour, seasonal, and interannual variability are essential for accurately designing utility-scale solar energy projects. Solar resource information can be obtained from ground-based measurement stations and/or from modeled data sets. The availability of measurements is scarce, both temporally and spatially, because it is expensive to maintain a high-density solar radiation measurement network that collects good quality data for long periods of time. On the other hand, high temporal and spatial resolution gridded satellite data can be used to estimate surface radiation for long periods of time and is extremely useful for solar energy development. Because of the advantages of satellite-based solar resource assessment, the National Renewable Energy Laboratory developed the PSM. The PSM produced gridded solar irradiance -- global horizontal irradiance (GHI), direct normal irradiance (DNI), and diffuse horizontal irradiance -- for the NSRDB at a 4-km by 4-km spatial resolution and half-hourly temporal resolution covering the 18 years from 1998-2015. The NSRDB also contains additional ancillary meteorological data sets, such as temperature, relative humidity, surface pressure, dew point, and wind speed. Details of the model and data are available at https://nsrdb.nrel.gov. The results described in this paper show that the hourly-averaged satellite-derived data have a systematic (bias) error of approximately +5% for GHI and less than +10% for DNI; however, the scatter (root mean square error [RMSE]) difference is higher for the hourly averages.

14 SOLAR ENERGY↗
DOE OSTI2017

Evaluation of the National Solar Radiation Database (NSRDB): 1998-2015

This paper validates the performance of the physics-based Physical Solar Model (PSM) data set in the National Solar Radiation Data Base (NSRDB) to quantify the accuracy of the magnitude and the spatial and temporal variability of the solar radiation data. Achieving higher penetrations of solar energy on the electric grid and reducing integration costs requires accurate knowledge of the available solar resource. Understanding the impacts of clouds and other meteorological constituents on the solar resource and quantifying intra-/inter-hour, seasonal, and interannual variability are essential for accurately designing utility-scale solar energy projects. Solar resource information can be obtained from ground-based measurement stations and/or from modeled data sets. The availability of measurements is scarce, both temporally and spatially, because it is expensive to maintain a high-density solar radiation measurement network that collects good quality data for long periods of time. On the other hand, high temporal and spatial resolution gridded satellite data can be used to estimate surface radiation for long periods of time and is extremely useful for solar energy development. Because of the advantages of satellite-based solar resource assessment, the National Renewable Energy Laboratory developed the PSM. The PSM produced gridded solar irradiance -- global horizontal irradiance (GHI), direct normal irradiance (DNI), and diffuse horizontal irradiance -- for the NSRDB at a 4-km by 4-km spatial resolution and half-hourly temporal resolution covering the 18 years from 1998-2015. The NSRDB also contains additional ancillary meteorological data sets, such as temperature, relative humidity, surface pressure, dew point, and wind speed. Details of the model and data are available at https://nsrdb.nrel.gov. The results described in this paper show that the hourly-averaged satellite-derived data have a systematic (bias) error of approximately +5% for GHI and less than +10% for DNI; however, the scatter (root mean square error [RMSE]) difference is higher for the hourly averages.

14 SOLAR ENERGY↗
DOE OSTI2017

Use of MERRA-2 in the National Solar Radiation Database and Beyond

The National Solar Radiation Database (NSRDB) is a flagship product of NREL that provides solar radiation and ancillary meteorological information through a GIS based portal. This data is provided at a 4kmx4km spatial and 30 minute temporal resolution covering the period between 1998-2015. The gridded data that is distributed by the NSRDB is derived from satellite measurements using the Physical Solar Model (PSM) that contains a 2-stage approach. This 2-stage approach consists of first retrieving cloud properties using measurement from the GOES series of satellites and using that information in a radiative transfer model to estimate solar radiation at the surface. In addition to the satellite data the model requires ancillary meteorological information that is provided mainly by NASA's Modern Era Retrospecitve Analysis for Research and Applications (MERRA-2) 2 model output. This presentation provides an insight into how the NSRDB is developed using the PSM and how the various sources of data including the MERRA-2 data is used during the process.

14 SOLAR ENERGY↗
DOE OSTI2021

Hamilton: Flexible, Open Source $10 Wireless Sensor System for Energy Efficient Building Operation

Sensors for improving building performance are rapidly populating the market, driven in part by the drive to reduce greenhouse gas emissions resulting from energy production as well as improve the interior environment for healthy and more productive spaces. UC Berkeley has led wireless sensor development over the past 25 years (e.g., Telos mote), with the Hamilton (named after Alexander Hamilton on the US $10 bill) as the most recent. The Hamilton sensor was designed as a low-cost high-performance sensor that is modular and interoperable. The objective of the Hamilton project was to create, evaluate and establish the technological foundations for secure and easy to deploy building energy efficiency applications utilizing pervasive, low-cost wireless sensors integrated with traditional Building Management Systems (BMS), consumer-sector building components, and powerful data analytics. The project included iterative hardware design, incorporating a high-performance database (BTrDb, http://btrdb.io/), creating and iterating the development of secure data middleware (BOSSwave, WAVE/WAVEMQ), working with and pushing the development of an open-source tiny operating system RiotOS, and implementing and improving protocols such as Thread/OpenThread and TCP/IP. The hardware benefited from careful design to drive down the cost; the design included a System-on-a-Chip (SoC), chip antenna, single crystal and five passive components. Careful design of the operating system created a low-power design to enable a long life with small batteries. The hardware included several sensors: temperature, radiant temperature, relative humidity, magnetometer, accelerometer, and light, with an optional occupancy (Passive InfraRed) sensor. The project was the basis of several applications, both internal to the research team and other researchers and professionals at other institutions. Several applications used the sensor hardware as the basis for other complex devices. Other applications used the sensors to improve building performance through interoperating with the building Heating Ventilation and Air-Conditioning (HVAC) system, such as using occupancy and/or distributed temperature sensing to reduce HVAC zone energy while still providing thermal comfort and to reduce peak loads in small commercial buildings. We demonstrated cloud-based energy analytics, implemented a schedule and a Model Predictive Controller in a small commercial building to optimize HVAC energy, occupancy and electricity price. Initial integration of these technological innovations was performed through the creation of execution containers containing the WAVE agent and various driver, proxy, or building system function logic. The research added to the understanding of efficient sensor hardware, secure middleware, time-series data management (high performance database), efficient communication protocols, and interoperating with applications and building systems. The project showed the technical effectiveness and economic feasibility of creating a low-cost, modular, and easy-to-deploy sensor. Through conversations with multiple end users, the research team discovered that many customers wanted data management and services in addition to the sensors. HamiltonIOT developed packages of sensors, border router, and data services to provide a seamless “plug-and-play” sensor deployment. Some customers were willing to pay for higher quality sensors (such as light); some customers wanted a robust enclosure (waterproof).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

AERO-MAP: a data compilation and modeling approach to understand spatial variability in fine- and coarse-mode aerosol composition

Abstract. Aerosol particles are an important part of the Earth climate system, and their concentrations are spatially and temporally heterogeneous, as well as being variable in size and composition. Particles can interact with incoming solar radiation and outgoing longwave radiation, change cloud properties, affect photochemistry, impact surface air quality, change the albedo of snow and ice, and modulate carbon dioxide uptake by the land and ocean. High particulate matter concentrations at the surface represent an important public health hazard. There are substantial data sets describing aerosol particles in the literature or in public health databases, but they have not been compiled for easy use by the climate and air quality modeling community. Here, we present a new compilation of PM2.5 and PM10 surface observations, including measurements of aerosol composition, focusing on the spatial variability across different observational stations. Climate modelers are constantly looking for multiple independent lines of evidence to verify their models, and in situ surface concentration measurements, taken at the level of human settlement, present a valuable source of information about aerosols and their human impacts complementarily to the column averages or integrals often retrieved from satellites. We demonstrate a method for comparing the data sets to outputs from global climate models that are the basis for projections of future climate and large-scale aerosol transport patterns that influence local air quality. Annual trends and seasonal cycles are discussed briefly and are included in the compilation. Overall, most of the planet or even the land fraction does not have sufficient observations of surface concentrations – and, especially, particle composition – to characterize and understand the current distribution of particles. Climate models without ammonium nitrate aerosols omit ∼ 10 % of the globally averaged surface concentration of aerosol particles in both PM2.5 and PM10 size fractions, with up to 50 % of the surface concentrations not being included in some regions. In these regions, climate model aerosol forcing projections are likely to be incorrect as they do not include important trends in short-lived climate forcers.

Mahowald, Natalie M. (ORCID:000000022873997X)↗
DOE OSTI2020

Probabilistic Forecast of All-Sky Solar Radiation Using Enhanced WRF-Solar: Preprint

This study presents enhancements of the Weather Research and Forecasting model with solar extensions (WRF-Solar) to provide probabilistic forecasts of solar radiation. Our approach builds ensemble WRF-Solar runs by introducing stochastic perturbations of variables that produce the largest uncertainties in predicting surface irradiance and clouds. The key variables are identified using tangent linear sensitivity analysis of six physics packages responsible for all-sky irradiance variability. An optimal strategy to stochastically perturb the selected variables is developed and applied to WRF-Solar to generate ensemble members for day-ahead solar prediction. The National Solar Radiation Database (NSRDB) is used to validate the ensemble forecast at arbitrary locations on the model grid. Preliminary results indicate that the proposed technique can potentially produce WRF-Solar ensembles providing reliable information of solar prediction uncertainty. This study describes the implemented methodology and initial results as well as future research to improve ensemble-based probabilistic forecasts with WRF-Solar.

all-sky↗
DOE OSTI2021

SoDa: An Irradiance-Based Synthetic Solar Data Generation Tool (SoDa) v0.1

SoDa is an irradiance-based synthetic Solar Data generation tool to generate realistic sub-minute solar photovoltaic (PV) power time series, that emulate the weather pattern for a certain geographical location. Our tool relies on the National Solar Radiation Database (NSRDB) to obtain irradiance and weather data patterns for the site. Irradiance is mapped onto a PV model estimate of a solar plant's 30-min power output, based on the configuration of the panel. We use a stochastic model with a switching behavior due to different weather regimes as provided by the cloud type label in the NSRDB, with parameters for the cloudy states trained on the high-resolution solar power measurements from a Phasor Measurement Unit (PMU).

Carreno, IgnacioLosada↗
DOE OSTI2017

Could Blobs Fuel Storage-Based Convergence between HPC and Big Data?

The increasingly growing data sets processed on HPC platforms raise major challenges for the underlying storage layer. A promising alternative to POSIX-IO- compliant file systems are simpler blobs (binary large objects), or object storage systems. Such systems offer lower overhead and better performance at the cost of largely unused features such as file hierarchies or permissions. Similarly, blobs are increasingly considered for replacing distributed file systems for big data analytics or as a base for storage abstractions such as key-value stores or time-series databases. This growing interest in such object storage on HPC and big data platforms raises the question: Are blobs the right level of abstraction to enable storage-based convergence between HPC and Big Data? In this paper we study the impact of blob-based storage for real-world applications on HPC and cloud environments. The results show that blobbased storage convergence is possible, leading to a significant performance improvement on both platforms

Matri, Pierre↗
DOE OSTI2023

Deep Learning for Modeling Enhanced Geothermal Systems

Enhanced Geothermal Systems (EGS) offer a vast potential to expand the use of geothermal energy. Heat is extracted from this engineered system by injecting cold water into a subsurface fractures, which are in contact with the hot dry rock, and pulled through the production wells. Creating EGS requires improving the natural permeability of hot crystalline rocks. To develop economically viable EGS reservoirs, significant technical barriers (e.g., better stimulation technologies without adequate water and/or permeability) and non-technical barriers (e.g., land access and permitting) must be overcome. In this short conference paper, we present a workflow to address a part of this challenge – “How to develop economically viable EGS using existing technologies?”. Our workflow called the GeoThermalCloud (GTC) for EGS, leverages recent advances in machine learning, deep learning, and cloud computing. This GTC framework is open-source and available at https://github.com/SmartTensors/GeoThermalCloud.jl. The GTC framework provides trained deep learning (DL) models to estimate the net present value of a given EGS design scenario. The Geothermal Design Tool (https://github.com/GeoDesignTool/GeoDT.git), a fast and simplified multi-physics solver, is used to develop a database for training DL models. The database consists of EGS design parameters (inputs to DL model) and their net present value (output of DL model) in uncertain geologic systems. The EGS design parameters for constructing this training database are based on Utah FORGE but include the options of more wells and deeper depths. The DL models are trained by ingesting the EGS design parameters and estimating the corresponding net present value. Such an emulation allows us to screen various EGS designs quickly and identify good development strategies by coupling them with optimization techniques. Our preliminary results show promise in DL emulation of net present value. However, a lot more work is needed to improve the predictive capability of DL models (i.e., extensive hyperparameter tuning is necessary). This will be the primary focus of our future work.

artificial neural networks, geothermal↗
DOE OSTI2023

What's New in the NSRDB

The National Solar Radiation Database (NSRDB) provides solar resource data across the globe at a high temporal and spatial resolution. This data is primarily used in solar energy modeling. The NSRDB is updated annually for the United States and North, Central and South America and the data is currently available from 1998-2021. In 2022 the NSRDB was updated using the latest version of the underlying Physical Solar Model (PSM). This update includes improved surface albedo and gap-filling of cloud properties. The inclusion of these updates reduced the uncertainty in the data compared to previous versions of the NSRDB. The Himawari and Meteosat Indian Ocean Data Coverage (IODC) satellites were added to the Geostationary Operational Environmental Satellite (GOES) and made our coverage global. While standard data from the GOES continues to be served at an hourly 4km x 4km resolution, full resolution data has also been made available to the user. The NSRDB now contains over 200Tb of data with nearly 40Tb being added annually. We provide significant flexibility for data download depending on the amount of data required by the users. In this paper we provide an update on the current status on the NSRDB.

photovoltaic systems↗
DOE OSTI2022

The Impact of Stochastic Perturbations in Physics Variables for Predicting Surface Solar Irradiance

We present a probabilistic framework tailored for solar energy applications referred to as the Weather Research and Forecasting-Solar ensemble prediction system (WRF-Solar EPS). WRF-Solar EPS has been developed by introducing stochastic perturbations into the most relevant physical variables for solar irradiance predictions. In this study, we comprehensively discuss the impact of the stochastic perturbations of WRF-Solar EPS on solar irradiance forecasting compared to a deterministic WRF-Solar prediction (WRF-Solar DET), a stochastic ensemble using the stochastic kinetic energy backscatter scheme (SKEBS), and a WRF-Solar multi-physics ensemble (WRF-Solar PHYS). The performances of the four forecasts are evaluated using irradiance retrievals from the National Solar Radiation Database (NSRDB) over the contiguous United States. We focus on the predictability of the day-ahead solar irradiance forecasts during the year of 2018. The results show that the ensemble forecasts improve the quality of the forecasts, compared to the deterministic prediction system, by accounting for the uncertainty derived by the ensemble members. However, the three ensemble systems are under-dispersive, producing unreliable and overconfident forecasts due to a lack of calibration. In particular, WRF-Solar EPS produces less optically thick clouds than the other forecasts, which explains the larger positive bias in WRF-Solar EPS (31.7 W/m 2 ) than in the other models (22.7–23.6 W/m 2 ). This study confirms that the WRF-Solar EPS reduced the forecast error by 7.5% in terms of the mean absolute error (MAE) compared to WRF-Solar DET, and provides in-depth comparisons of forecast abilities with the conventional scientific probabilistic approaches (i.e., SKEBS and a multi-physics ensemble). Guidelines for improving the performance of WRF-Solar EPS in the future are provided.

14 SOLAR ENERGY↗
DOE OSTI2024

Modeling diurnal and annual ethylene generation from solar-driven electrochemical CO 2 reduction devices

Integrated solar fuels devices for CO 2 reduction (CO 2 R) are a promising technology class towards reducing carbon emissions. Designing integrated CO 2 R solar fuels devices requires careful co-design of electrochemical and photovoltaic components as well as consideration of the diurnal and seasonal effects of solar irradiance, temperature, and other meteorological factors expected for ‘on-sun’ deployment. Using a photovoltaic-electrochemical (PV-EC) platform, we developed a temperature and potential-dependent diurnal and annual model using experimentally-determined CO 2 R performance of Cu-based electrocatalysts, local meteorological data from the National Solar Radiation Database (NSRD), and modeled performance of commercial c-Si PVs. Here we simulated gaseous diurnal product outputs with and without the effects of ambient temperature. From these outputs, we observed seasonal variation in gaseous product generation, with up to two-fold increases in ethylene productivity between the Winter and Summer, analyzed the consequences of dynamic cloud coverage, and identified periods where device cooling/heating mechanisms could be implemented to maximize ethylene generation. Finally, we modeled the annual ethylene generation for a scaled 1 MW solar farm at three different locations (Beijing, CN; Sydney, AUS; Barstow, CA) to determine the consequences of local meteorological climates on PV-EC CO 2 R product output, recording a maximum ethylene output of 18.5 tonne per year at Barstow. Overall, this model presents a critical tool for streamlining the translation of experimental solar-driven electrochemical research to real-world implementation.

14 SOLAR ENERGY↗