Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A practical approach to using the Genomic Standards Consortium MIxS reporting standard for comparative genomics and metagenomics

Comparative analysis of (meta)genomes necessitates aggregation, integration, and synthesis of well-annotated data using standards. The Genomic Standards Consortium (GSC) collaborates with the research community to develop and maintain the Minimal Information about any (x) Sequence (MIxS) reporting standard for genomic data. To facilitate use of the GSC’s MIxS reporting standard, we provide a description of the structure and terminology, how to navigate ontologies for required terms in MIxS, and demonstrate practical usage through a soil metagenome example.

standards, metadata, genome, metagenome, schema, v↗
DOE OSTI2020

Scalable line and plane relaxation in a parallel structured multigrid solver

The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevel cycle. Results show an effective reduction in communication costs for coarse-grid problems, and result in a speedup of 8.7x in smoothing routines shown in Fig. 12 using this approach. This produces a significant improvement to strong scalability while maintaining favorable weak scaling behavior. Finally, a parallel scaling study using a series of refined meshes is included that demonstrates the effectiveness of this approach in an application of interest.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Light absorption and plasmon – exciton interaction in three-layer nanorods with a gold core and outer shell composed of molecular J- and H-aggregates of dyes

Optical properties of hybrid rod-like nanoparticles, consisting of a gold core, an intermediate passive organic layer (spacer) and outer layer of ordered molecular cyanine dye aggregates, are experimentally and theoretically investigated. It is shown that these dyes can form not only ordered J-aggregates but also H-aggregates (differing by the packing angle of dye molecules in an aggregate and having other spectral characteristics) in the outer shell of the hybrid nanostructure. Absorption spectra of synthesised three-layer nanorods are recorded, and their sizes are determined. The optical properties of the composite nanostructures under study are found to differ significantly, depending on the type of the molecular aggregate formed in the outer shell. The experimental data are quantitatively explained based on computer simulation using the finite-difference time-domain (FDTD) method, and characteristic features of the plasmon – exciton interaction in the systems under study are revealed. (nanophotonics)

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis

The four-index integral transform is a fundamental and computationally demanding calculation used in many computational chemistry suites such as NWChem. It transforms a four-dimensional tensor from an atomic basis to a molecular basis. This transformation is most efficiently implemented as a sequence of four tensor contractions that each contract a four-dimensional tensor with a two-dimensional transformation matrix. Differing degrees of permutation symmetry in the intermediate and final tensors in the sequence of contractions cause intermediate tensors to be much larger than the final tensor and limit the number of electronic states in the modeled systems. Loop fusion, in conjunction with tiling, can be very effective in reducing the total space requirement, as well as data movement. However, the large number of possible choices for loop fusion and tiling, and data/computation distribution across a parallel system, make it challenging to develop an optimized parallel implementation for the four-index integral transform. We develop a novel approach to address this problem, using lower bounds modeling of data movement complexity. We establish relationships between available aggregate physical memory in a parallel computer system and ineffective fusion configurations, enabling their pruning and consequent identification of effective choices and a characterization of optimality criteria. This work has resulted in the development of a significantly improved implementation of the four-index transform that enables higher performance and the ability to model larger electronic systems than the current implementation in the NWChem quantum chemistry software suite.

Rajbhandari, Samyam↗
DOE OSTI2012

Hard Disk/Solid State Drive Synergy in Support of Data-Intensive Computing [Slides]

Data-intensive applications are becoming increasingly common in high-performance computing. Examples include combustion simulation, human genome analysis, and satellite image processing. Efficient access of data sets is critical to the performance of these applications. Because of the size of the data today's economically feasible approach is to store the data files on an array of hard disks or data servers equipped with hard disks and managed by a parallel file system such as PVFS or Lustre wherein the data is striped over a (large) number of disks for high aggregate I/O throughout. With file striping, a request for a segment of logically contiguous file space is decomposed into multiple sub-requests, each to a different server. While the data unit for this striping is usually reasonably large to benefit disk efficiency, the first and/or last sub-requests can be much smaller than the striping unit if the request does not align with the striping pattern, severely compromising hard disk efficiency and thus application performance. We propose to exploit solid state drives (SSD), whose efficiency is much less sensitive to small random accesses, to enable the alignment of requests to disk with the data striping pattern. In this scheme hard disks mainly serve large, aligned, sequential requests, with SSDs serving small or unaligned requests, thus respecting the relative cost, performance, and durability characteristics of the two media, and thereby achieving synergy in performance/cost. We will describe the design of the proposed scheme, its implementation on CCS-7's Darwin cluster, and performance results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE <1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (<1 °C) accuracy for most locations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

FFT-based model for irradiated aggregate microstructures in concrete

The concrete biological shield of light water reactors is exposed to neutron and gamma irradiation throughout its lifetime, which results in the long-term degradation of the concrete’s mechanical properties. Under neutron irradiation, the concrete’s aggregates are subjected to radiation-induced volumetric expansion (RIVE), which strongly depends on the mineral content of the aggregate and exhibits the largest expansion in silicate-bearing minerals. In this work, the authors used the fast Fourier transform-based code Microstructure-Oriented Scientific Analysis of Irradiated Concrete (MOSAIC) in 2D to model the expansion of five different aggregates provided by the Japan Concrete Aging Management Program (JCAMP). Comparable rock specimens were irradiated at the JEEP-II test reactor. The model uses realistic aggregate microstructure reconstruction based on high-resolution characterization images. The model accounts for anisotropic RIVE, thermal expansion, and the associated initiation and propagation of damage. The RIVE models are calibrated based on expansion data in the literature. The authors assume that damage occurs exclusively at interfaces between the particles that compose an aggregate and that these interfaces also exhibit swelling. Using a micromechanical model, the evolution of Young’s modulus with RIVE is calculated for each aggregate and compared with Russian irradiation data. The modeled linear expansion agrees well with the experimentally measured expansion. Furthermore, the model also predicts that anisotropic RIVE and thermal expansion result in an earlier onset of damage with neutron fluence than in the isotropic case.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Thermodynamics, morphology, and kinetics of early-stage self-assembly of π-conjugated oligopeptides

Synthetic oligopeptides containing π-conjugated cores self-assemble novel materials with attractive electronic and photophysical properties. All-atom, explicit solvent molecular dynamics simulations of Asp-Phe-Ala-Gly-OPV3-Gly-Ala-Phe-Asp peptides were used to parameterize an implicit solvent model to simulate early-stage self-assembly. Under low-pH conditions, peptides assemble into β-sheet-like stacks with strongly favorable monomer association free energies of ΔF ≈ -25kBT. Aggregation at high-pH produces disordered aggregates destabilized by Coulombic repulsion between negatively charged Asp termini (ΔF ≈ -5kBT). In simulations of hundreds of monomers over 70 ns we observe the spontaneous formation of up to undecameric aggregates under low-pH conditions. Modeling assembly as a continuous-time Markov process, we infer transition rates between different aggregate sizes and microsecond relaxation times for early-stage assembly. Our data suggests a hierarchical model of assembly in which peptides coalesce into small clusters over tens of nanoseconds followed by structural ripening and diffusion limited aggregation on longer time scales. This work provides new molecular-level understanding of early-stage assembly, and a means to study the impact of peptide sequence and aromatic core chemistry upon the thermodynamics, assembly kinetics, and morphology of the supramolecular aggregates.

36 MATERIALS SCIENCE↗
DOE OSTI2012

A comparison of thermal zone aggregation methods

The impact of increasing energy prices on building operation budgets has fueled demand for more energy-efficient structures. Existing building energy simulation tools generate an immense amount of data yet comparatively little knowledge. This paper introduces a framework that allows aggregation-based model reduction to operate on geometric building information models. The resulting aggregation sequence provides designers with faster simulations and affords insight into complex multi-scale thermal interactions. A comparison of the trade-off between simulation speed and accuracy for three hierarchical cluster partitioning methods concludes the discussion.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A Unified Ice Microphysical Parameterization for Numerical Models: Development and Testing with In-situ Data

The primary goal of the project is to develop and test new microphysical parameterizations of ice crystal growth. Over the past 7 years, we have developed a bulk ice microphysical model that evolves a measure of particle shape and density during vapor diffusional growth. The method allows smaller, pristine ice crystals to evolve into larger snow crystals in a more natural fashion than traditional microphysical schemes. In doing so, the new method avoids problems that occur when ice is transferred among pre-defined classes. The new model was developed and tested for vapor grown ice only, and so the goal of this project is to develop bulk parameterizations of the missing physics for the model including the prediction of surface deposition coefficients, riming, and aggregation. The parameterizations are being developed using prior laboratory data as a constraint, and then will be tested in simulations of case studies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2012

Long-term Differences in Tillage and Land Use Affect Intra-aggregate Pore Heterogeneity

Recent advances in computed tomography provide measurement tools to study internal structures of soil aggregates at micrometer resolutions and to improve our understanding of specific mechanisms of various soil processes. Fractal analysis is one of the data analysis tools that can be helpful in evaluating heterogeneity of the intra-aggregate internal structures. The goal of this study was to examine how long-term tillage and land use differences affect intra-aggregate pore heterogeneity. The specific objectives were: (i) to develop an approach to enhance utility of box-counting fractal dimension in characterizing intra-aggregate pore heterogeneity; (ii) to examine intra-aggregate pores in macro-aggregates (4-6 mm in size) using the computed tomography scanning and fractal analysis, and (iii) to compare heterogeneity of intra-aggregate pore space in aggregates from loamy Alfisol soil subjected to 20 yr of contrasting management practices, namely, conventional tillage (chisel plow) (CT), no-till (NT), and native succession vegetation (NS). Three-dimensional images of the intact aggregates were obtained with a resolution of 14.6 {micro}m at the Advanced Photon Source, Argonne National Laboratory, Argonne, IL. Proposed box-counting fractal dimension normalization was successfully implemented to estimate heterogeneity of pore voxel distributions without bias associated with different porosities in soil aggregates. The aggregates from all three studied treatments had higher porosity associated with large (>100 {micro}m) pores present in their centers than in their exteriors. Pores 15 to 60 {micro}m were equally abundant throughout entire aggregates but their distributions were more heterogeneous in aggregate interiors. The CT aggregates had greater numbers of pores 15 to 60 {micro}m than NT and NS. Distribution of pore voxels belonging to large pores was most heterogeneous in the aggregates from NS, followed by NT and by CT. This result was consistent with presence of very large pores associated with former root channels in NT and NS aggregates. Our findings indicate that mechanisms of macro-aggregate formation might differ in their importance in different land use and management practices.

43 PARTICLE ACCELERATORS↗
DOE OSTI2014

Radiation effects in concrete for nuclear power plants, Part II: Perspective from micromechanical modeling

The need to understand and characterize the effects of neutron irradiation on concrete has become urgent because of the possible extension of service life of many nuclear power generating stations. Current knowledge is primarily based on a collection of data obtained in test reactors. These results are inherently difficult to interpret because materials and testing conditions are inconsistent. A micromechanical approach based on the Hashin composite sphere model is presented to derive a first-order separation of the effects of radiation on cement paste and aggregate, and, also, on their interaction. Although the scarcity of available data limits the validation of the model, it appears that, without negating a possible gamma-ray induced effect, the neutron-induced damage and swelling of aggregate plays a predominant role on the overall concrete expansion and the damage of the cement paste. Finally, the radiation-induced volumetric expansion (RIVE) effects can also be aided by temperature elevation and shrinkage in the cement paste.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

NGEE Arctic Meteorological Data from Instrumented PCDC Stations, Council Site, Seward Peninsula, Alaska, 2018 - 2023

This dataset provides meteorological data collected at two instrumented Power, Communication, and Data Collection (PCDC) Stations, located at Council Road Mile Marker 71 (CN_MM71) and Council Road Mile Marker 68 (Rocky Top). Measurements began in June of 2018 at CN_MM71 and in September 2018 at Rocky Top and ended in 2023. Raw data included. Meteorological data include air temperature, relative humidity, wind speed and direction, and snow depth. Measurements have been aggregated to both 10-minute and 30-minute reporting intervals provided in four comma-separated (*.csv) data files.• The 30-minute data files include only the primary meteorological variables, leaf wetness, and snow depth.. The 30-minute reporting interval matches other coincident site data collections.• The 10-minute files provide the primary meteorological variables and numerous system operational parameters such as battery charge and solar and wind turbine status. This 10-minute data product is valuable for analyzing solar and wind charging system performance in arctic landscapes.Daily photographic images (09:00, 12:00, and 15:00 AKST/AKDT) of the PCDC Stations and the surrounding landscape that are captured by an adjacent web camera, are also provided.The daily 12:00 images from CN_MM71 camera have been compiled into time-lapse movies:• October 2018 – June 12, 2019 = Council_winter_2018_2019.mov• January 2019 – December 2019 = Council_2019.mp4The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Day-Ahead Forecasting with Federated LSTM to Plan Energy Sharing in a Community Microgrid

Energy balancing in microgrids is a key enabler of resilience. Community microgrids located close to each other have the added benefit of networking and sharing surplus energy, if available. Such complex decision-making runs on optimization that requires reliable short-term (up to very-short-term) forecasts of energy generation and consumption for scheduling or trading. Each microgrid may also opt to not expose their sensitive data such as consumption patterns of individual businesses or residences. This paper investigates a federated approach to dayahead forecasting that trains naive long short-term memory (LSTM) at each business in a microgrid and aggregates weights at the microgrid controller using proximal regularization. This approach ensures that the controller has access only to energy surplus/deficit and not the actual generation or consumption values, avoiding unwanted exposure of sensitive data. A community microgrid in Adjuntas, Puerto Rico with 3 businesses is selected as a case study with a laboratory-scale computing setup. A central LSTM forecaster, where sensitive data from businesses are aggregated at the controller, is implemented as a baseline for qualifying the results. This work serves as a proof-of-concept for scaling the approach to networked and nested microgrids with more complex control options.

Sundararajan, Aditya [ORNL] (ORCID:000000033577854↗
DOE OSTI2021

Tri-Level Scheduling Model Considering Residential Demand Flexibility of Aggregated HVACs and EVs Under Distribution LMP

Residential loads, especially heating, ventilation and air conditioners (HVACs) and electric vehicles (EVs), have great potentials to provide demand flexibility which is an attribute of grid-interactive efficient buildings (GEB). Under this new paradigm, first, EV and HVAC aggregator models are developed in this paper to represent the fleet of GEBs, in which the aggregated parameters are obtained based on a new approach of data generation and least squares parameter estimation (DG-LSPE), which can deal with heterogeneous HVACs. Then, a tri-level bidding and dispatching framework is established based on competitive distribution operation with distribution locational marginal price (DLMP). Furthermore, the first two levels form a bilevel model to optimize the aggregators’ payment and to represent the interdependency between load aggregators and the distribution system operator (DSO) using DLMP, and the third level is to dispatch the optimal load aggregation to all residents by the proposed priority list-based demand dispatching algorithm. Finally, case studies on a modified IEEE 33-Bus system illustrate three main technical reasons of payment reduction due to demand flexibility: load shift, DLMP step changes, and power losses. They can be used as general guidelines for better decision-making for future planning and operation of demand response programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Using observational mean-flow data to drive large-eddy simulations of a diurnal cycle at the SWiFT site

Reproducing realistic date- and site-specific unsteady wind conditions in large-eddy simulations is becoming increasingly useful in wind energy. How to run a large-eddy simulation to match observed conditions, however, remains an open research question. One approach that has received considerable attention is mesoscale-to-microscale coupling, in which information about the mesoscale weather, most commonly acquired from a mesoscale numerical weather model, is passed on to a microscale model. In this paper, we demonstrate how the recently developed profile-assimilation technique, a form of mesoscale-to-microscale coupling, can be used to drive large-eddy simulations solely based on observed mean-flow profiles at a single location, bypassing the need for auxiliary mesoscale simulations. The new approach is evaluated for a diurnal cycle at the Scaled Wind Farm Technology site. Observed mean-flow profiles from the ground up to a height of 2 km are reconstructed by aggregating measurements from multiple instruments, and gaps in the data are infilled with natural neighbor interpolation. We perform nine simulations using various forcing approaches to deal with data limitations. The results show that it is indeed possible to drive microscale large-eddy simulation with observations using the profile-assimilation technique, notwithstanding large gaps in virtual potential temperature measurements. However, profile assimilation with vertical smoothing of the error between the desired and actual profiles is required. Without that smoothing, the microscale simulations develop unrealistically high turbulence levels under many situations. Finally, we show that simulated mesoscale data can account for missing observations, although care is needed as both data sources are not necessarily compatible.

17 WIND ENERGY↗
DOE OSTI2016

Multilayer spectral graph clustering via convex layer aggregation

Multilayer graphs are commonly used for representing different relations between entities and handling heterogeneous data processing tasks. New challenges arise in multilayer graph clustering for assigning clusters to a common multilayer node set and for combining information from each layer. This paper presents a theoretical framework for multilayer spectral graph clustering of the nodes via convex layer aggregation. Under a novel multilayer signal plus noise model, we provide a phase transition analysis that establishes the existence of a critical value on the noise level that permits reliable cluster separation. The analysis also specifies analytical upper and lower bounds on the critical value, where the bounds become exact when the clusters have identical sizes. Numerical experiments on synthetic multilayer graphs are conducted to validate the phase transition analysis and study the effect of layer weights and noise levels on clustering reliability.

97 MATHEMATICS AND COMPUTING↗