Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Impact Analysis of Future Electric Vehicles Using Model of Real Distribution Feeders

Globally, the number of electric vehicles (EVs) continues to increase. This on-going trend poses challenges for power distribution systems. The inclusion of electric vehicle supply equipment (EVSE) should be compatible with a changing energy system, whose structure is becoming increasingly distributed. The impact of future EVs should be analyzed and considered in the system planning and upgrades. In this study, impact analysis of future EVs is conducted for a utility company of U.S. West Coast. Real data of distribution feeders is converted into the GridLAB-D model for running power flow analysis. An estimation of the additional loading of EVs in 2050 is provided by the utility company. Simple mitigation methods are tested to improve voltage profiles and complete this case study.

Xie, Jing↗
DOE OSTI2024

Tensor decompositions for count data that leverage stochastic and deterministic optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the maximum likelihood estimator (MLE) of the Poisson CPD model. Here, this work presents two new algorithms that extend state-of-the-art local methods for Poisson CPD. Hybrid GCP-CPAPR combines Generalized Canonical Decomposition (GCP) with stochastic optimization and CP Alternating Poisson Regression (CPAPR), a deterministic algorithm, to increase the probability of converging to the MLE over either method used alone. Restarted CPAPR with SVDrop uses a heuristic based on the singular values of the CPD model unfoldings to identify convergence toward optimizers that are not the MLE and restarts within the feasible domain of the optimization problem, thus reducing overall computational cost when using a multi-start strategy. We provide empirical evidence that indicates our approaches outperform existing methods with respect to converging to the Poisson CPD MLE.

CPAPR↗
DOE OSTI2023

Tensor Decompositions for Count Data that Leverage Stochastic and Deterministic Optimization

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the global maximum likelihood estimator from local minima. Simultaneously, a recent trend in theoretical computer science and numerical linear algebra leverages randomization to solve very large, hard problems. The typical approach is to use randomization for a fast approximation and determinism for refinement to yield effective algorithms with theoretical guarantees. Two popular algorithms for Poisson CPD reflect that emergent dichotomy: CP Alternating Poisson Regression is a deterministic algorithm and Generalized Canonical Polyadic decomposition makes use of stochastic algorithms in several variants. This work extends recent work to develop two new methods that leverage randomized and deterministic algorithms for improved accuracy and performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

FL-DISCO: Federated Generative Adversarial Network for Graph-based Molecule Drug Discovery: Special Session Paper

The outbreak of the global COVID-19 pandemic emphasizes the importance of collaborative drug discovery for high effectiveness; however, due to the stringent data regulation, data privacy becomes an imminent issue needing to be addressed to enable collaborative drug discovery. In addition to the data privacy issue, the efficiency of drug discovery is another key objective since infectious diseases spread exponentially and effectively conducting drug discovery could save lives. Advanced Artificial Intelligence (AI) techniques are promising to solve these problems: (1) Federated Learning (FL) is born to keep data privacy while learning data from distributed clients; (2) graph neural network (GNN) can extract structural properties of molecules whose underlying architecture is the connected atoms; and (3) generative adversarial network (GAN) can generate novel molecules while retaining the properties learned from the training data. In this work, we make the first attempt to build a holistic collaborative and privacy-preserving FL framework, namely FL- DISCO, which integrates GAN and GNN to generate molecular graphs. Experimental results demonstrate the effectiveness of FL- DISCO on: (1) IID data for ESOL and QM9, where FL-DISCO can generate highly novel compounds with high drug-likeliness, uniqueness and LogP scores compared to the baseline; (2) non- IID data for ESOL and QM9, where FL-DISCO generates 100% novel compounds with high validity and LogP scores compared to the baseline. We also demonstrate how different fractions of clients, generator and discriminator architectures affect our evaluation scores.

Manu, Daniel↗
DOE OSTI2024

Harnessing Machine Learning and Data Fusion for Accurate Undocumented Well Identification in Satellite Images

This study utilizes satellite data to detect undocumented oil and gas wells, which pose significant environmental concerns, including greenhouse gas emissions. Three key findings emerge from the study. Firstly, the problem of imbalanced data is addressed by recommending oversampling techniques like Rotation–GaussianBlur–Solarization data augmentation (RGS), the Synthetic Minority Over-Sampling Technique (SMOTE), or ADASYN (an extension of SMOTE) over undersampling techniques. The performance of borderline SMOTE is less effective than that of the rest of the oversampling techniques, as its performance relies heavily on the quality and distribution of data near the decision boundary. Secondly, incorporating pre-trained models trained on large-scale datasets enhances the models’ generalization ability, with models trained on one county’s dataset demonstrating high overall accuracy, recall, and F1 scores that can be extended to other areas. This transferability of models allows for wider application. Lastly, including persistent homology (PH) as an additional input improves performance for in-distribution testing but may affect the model’s generalization for out-of-distribution testing. A careful consideration of PH’s impact on overall performance and generalizability is recommended. Overall, this study provides a robust approach to identifying undocumented oil and gas wells, contributing to the acceleration of a net-zero economy and supporting environmental sustainability efforts.

SMOTE↗
DOE OSTI2021

Enabling modern data discovery for atmospheric measurements

The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Application of a Machine Learning Algorithm in Generating an Evapotranspiration Data Product From Coupled Thermal Infrared and Microwave Satellite Observations

Land surface evapotranspiration (ET) is one of the main energy sources for atmospheric dynamics and a critical component of the local, regional, and global water cycles. Consequently, accurate measurement or estimation of ET is one of the most active topics in hydro-climatology research. With massive and spatially distributed observational data sets of land surface properties and environmental conditions being collected from the ground, airborne or space-borne platforms daily over the past few decades, many research teams have started to use big data science to advance the ET estimation methods. The Geostationary satellite Evapotranspiration and Drought (GET-D) product system was developed at the National Oceanic and Atmospheric Administration (NOAA) in 2016 to generate daily ET and drought maps operationally. The primary inputs of the current GET-D system are the thermal infrared (TIR) observations from NOAA GOES satellite series. Because of the cloud contamination to the TIR observations, the spatial coverage of the daily GET-D ET product has been severely impacted. Based on the most recent advances, we have tested a machine learning algorithm to estimate all-weather land surface temperature (LST) from TIR and microwave (MW) combined satellite observations. With the regression tree machine learning approach, we can combine the high accuracy and high spatial resolution of GOES TIR data with the better spatial coverage of passive microwave observations and LST simulations from a land surface model (LSM). The regression tree model combines the three LST data sources for both clear and cloudy days, which enables the GET-D system to derive an all-weather ET product. This paper reports how the all-weather LST and ET are generated in the upgraded GET-D system and provides an evaluation of these LST and ET estimates with ground measurements. The results demonstrate that the regression tree machine learning method is feasible and effective for generating daily ET under all weather conditions with satisfactory accuracy from the big volume of satellite observations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Constraining the Stellar-to-Halo Mass Relation with Galaxy Clustering and Weak Lensing from DES Year 3 Data

We develop a framework to study the relation between the stellar mass of a galaxy and the total mass of its host dark matter halo using galaxy clustering and galaxy-galaxy lensing measurements. We model a wide range of scales, roughly from $\sim 100 \; {\rm kpc}$ to $\sim 100 \; {\rm Mpc}$, using a theoretical framework based on the Halo Occupation Distribution and data from Year 3 of the Dark Energy Survey (DES) dataset. The new advances of this work include: 1) the generation and validation of a new stellar mass-selected galaxy sample in the range of $\log M_\star/M_\odot \sim 9.6$ to $\sim 11.5$; 2) the joint-modeling framework of galaxy clustering and galaxy-galaxy lensing that is able to describe our stellar mass-selected sample deep into the 1-halo regime; and 3) stellar-to-halo mass relation (SHMR) constraints from this dataset. In general, our SHMR constraints agree well with existing literature with various weak lensing measurements. We constrain the free parameters in the SHMR functional form $\log M_\star (M_h) = \log(εM_1) + f\left[ \log\left( M_h / M_1 \right) \right] - f(0)$, with $f(x) \equiv -\log(10^{αx}+1) + δ[\log(1+\exp(x))]^γ/ [1+\exp(10^{-x})]$, to be $\log M_1 = 11.506^{+0.325}_{-0.404}$, $\log ε= -1.632^{+0.306}_{-0.181}$, $α= -1.638^{+0.108}_{-0.099}$, $γ= 0.596^{+0.251}_{-0.210}$ and $δ= 3.810^{+2.045}_{-1.811}$. The inferred average satellite fraction is within $\sim 5-35\%$ for our fiducial results and we do not see any clear trends with redshift or stellar mass. Furthermore, we find that the inferred average galaxy bias values follow the generally expected trends with stellar mass and redshift. Our study is the first SHMR in DES in this mass range, and we expect the stellar mass sample to be of general interest for other science cases.

Zacharegkas, G. [Argonne; Chicago U., KICP] (ORCID↗
DOE OSTI2023

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

This report details recent progress for the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows”. We refer to the project as IPPD/2, reflecting the 2017 renewal under expanded scope and partners In IPPD/2, we increased our research scope to include data motion. We are focusing on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. This new work on data motion will augment and complement IPPD/2’s research that focused on the computational aspects of tasks. We leverage and extend our existing tools and demonstrate our work on the Belle II workflow suite as well as on workflows from NSLS-II. The highlights of our work are as follows: Provenance for Workflows: Provenance is used to provide information enabling quality control, re-run computational workflows, and reproduce results. IPPD/2 has been building a scalable provenance management system that enables the capture of provenance from the high-level workflow through all relevant system levels in one integrated environment. Leveraging this work, our recent efforts have included using provenance as an enabling technique. Workload characterization: Leveraging provenance and analysis, we characterize data movement within network, storage, and memory over a variety of workloads. This characterization enables an understanding by performance analysts and application developers of the range of behaviors that could be expected. Performance Prediction for Workflows: The goal of modeling distributed workflows is to understand performance bottlenecks and enable more intelligent task scheduling to optimize selected metrics of interest (e.g., task throughput or output data rate). IPPD/2 has utilized both analytical and AI/ML modeling methodologies for performance modeling. Advanced Scheduling and Fault Modeling for Workflows: Scheduling of large-scale scientific workflows on geographically distributed resources is a challenging problem. To improve workflow throughput, we combined novel scheduling algorithms with task predictions from performance modeling and fault modeling. Dynamically Alleviating Bottlenecks in Workflows: Exploiting our provenance, analysis, and modeling efforts, we have explored and developed several techniques for dynamically detecting and alleviating bottlenecks in data movement. In particular, we have spent considerable effort demonstrating our techniques on production-like workflow configurations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Multi Time-scale Imputation aided State Estimation in Distribution System

With the transition to a smart grid, we are witnessing a significant growth in sensor deployments and smart metering infrastructure in the distribution system. However, information from these sensors and meters are typically unevenly sampled at different time-scales and are incomplete. It is critical to effectively aggregate these information sources for situational awareness. In order to reconcile the heterogeneous multi-scale time-series data, we present a multi-task Gaussian process framework. This framework exploits the spatio-temporal correlation across the time-series data to impute data at any desired timescale while providing confidence bounds on the imputations. The value of the imputed data for distribution system operation is illustrated via a matrix completion based state estimation strategy. Results on the IEEE 37 bus distribution system reveals the superior performance of the proposed approach relative to linear interpolation approaches.

Dahale, Shweta↗
DOE OSTI2026

Modeling the Effect of the Heliospheric Magnetic Field on Cosmic Ray Muon Shadows

Shadows cast in the cosmic ray (CR) muon sky by the Sun were located using muon data from the MINOS far detector in Northern Minnesota. The shadows were observed independently across three time periods; near solar minimum, near solar maximum, and over the entire 13 year span of the data. A distribution of muon positions for each shadow was then sampled to simulate CR motions near the Sun using the Parker spiral model of the Heliospheric Magnetic Field (HMF) and a dipole model of the Geomagnetic Field (GMF). The resulting particle distributions were then compared to their position with respect to the Sun. Results show that the Parker spiral model is most consistent with the solar minimum shadow and least consistent with the solar maximum shadow, as expected. The simple Parker spiral is more consistent with the data for a harder CR spectrum than is actually present, indicating the need for a more detailed HMF model. Plausible modifications to the Parker spiral model which would affect the overall shift of the Sun's CR shadow are discussed.

Adamson, P. [Fermilab]↗
DOE OSTI2024

Development of Real-Time High-Density Pulsar Data Transmission and Processing for Grid Synchronization

Taking advantage of the extreme stability of the pulsar period, it can serve as the timing source for grid synchronization to compensate for the timing drift instigated by the loss of GPS signal. Nevertheless, the real-time transmission and processing of the pulsar data suffer from its high-frequency data rate, varying from megahertz to gigahertz, resulting in reduced computing speed and increased time delay. To mitigate this issue, the hardware and software frameworks are implemented for the high-density pulsar data transmission and processing for grid synchronization in this research. Initially, the high-density pulsar data is transferred using open-source software. The complementary duty cycle timing module is designed to coordinate the operation of the dual-channel high-speed interface and software. Subsequently, the multiple-threading is applied to the receiving, parsing, and splicing pulsar data. Next, the pulsar signal extraction method is implemented based on the polyphase filterbank and time of arrival estimation. Ultimately, real-time performance verification experiments are carried out for different components under two hardware platforms. Finally, the results demonstrate that only 0.482 s is required for processing 4 Gigabyte data through multiple-threading, which is 3.8 times faster than the single thread. The pulsar signal extraction can also be executed within 707 ms for 4.8 seconds of data, thereby indicating that real-time requirements can be met.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Prototype energy models for data centers

Data centers in the United States consume about two percent of the nation’s electricity. Because heat gains from IT equipment drive cooling demand, data centers offer unique opportunities for energy savings. However, no prototype energy model for data centers is available in the suite of existing U.S. Department of Energy’s Commercial Prototype Building Models. Here we present the development of two new data center prototype models and their implementation in OpenStudio and EnergyPlus. The small-size data center model represents a computer room in a building served by computer room air conditioners (CRACs); while the large-sized model represents stand-alone data centers served by computer room air handlers (CRAHs) with a central chiller plant. For each data center model, two levels of IT equipment (ITE) load density were considered, to cover the wide range of IT power density of data centers: 40 and 100 W/ft 2 (430 and 1076 W/m 2 ) for the computer room, and 100 and 500 W/ft 2 (1076 and 5382 W/m 2 ) for the stand-alone data center. All other assumptions, such as building envelope, lighting, HVAC efficiencies and schedules, were based on the minimal requirements of ASHRAE Standard 90.1 at various vintages. We introduced a novel concept of supply and return air approach temperatures to capture the essential effects of non-uniform airflow and temperature distribution in data centers. The approach temperatures were pre-computed by computational fluid dynamics (CFD) simulations for various configurations of ITE loads and airflow containment management in data centers. A new feature was developed in EnergyPlus to implement the approach temperature method. A case study was conducted to demonstrate the use of the data center models. The two data center models cover all U.S. climate zones and can be used to evaluate energy saving measures for data centers, as well as to support development of data center energy efficiency codes and standards.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Drought index using micrometeorological data from Embrapa weather station at Adolpho Ducke Reserve in Manaus, Brazil

This dataset includes daily resolution time series data including precipitation, minimum and maximum daily air temperature, and air relative humidity downloaded from the Embrapa InfoClima portal (https://www.cnpaf.embrapa.br/infoclima/), with data for the Adolpho Ducke Reserve climatological station in Manaus, Brazil, for the period of January 1, 1980 to December 31, 2016. Using this precipitation record, a Standardized Precipitation Index (SPI) was calculated and added to the dataset, using daily resolution for 180 day intervals and with 20 years for calibration (1980-1999) and adopting a gamma distribution. These data were applied as a proxy for analyzes of precipitation and SPI for the Manaus ZF2 Research station, located approximately 50 Km North of the Adolpho Ducke Reserve. Data are included in a .csv file, and the text file Drought-Indices-Embrapa-Ducke_met_spi_headers.txt provides descriptions of the data file headers.

54 ENVIRONMENTAL SCIENCES↗