Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Big Biomedical data as the key resource for discovery science

Modern biomedical data collection is generating exponentially more data in a multitude of formats. This flood of complex data poses significant opportunities to discover and understand the critical interplay among such diverse domains as genomics, proteomics, metabolomics, and phenomics, including imaging, biometrics, and clinical data. The Big Data for Discovery Science Center is taking an “-ome to home” approach to discover linkages between these disparate data sources by mining existing databases of proteomic and genomic data, brain images, and clinical assessments. In support of this work, the authors developed new technological capabilities that make it easy for researchers to manage, aggregate, manipulate, integrate, and model large amounts of distributed data. Guided by biological domain expertise, the Center’s computational resources and software will reveal relationships and patterns, aiding researchers in identifying biomarkers for the most confounding conditions and diseases, such as Parkinson’s and Alzheimer’s.

Alzheimer's disease (ID)↗
DOE OSTI2014

Small Angle Neutron Scattering (SANS) Studies on the Structural Evolution of Pyromellitamide Self-assembled Gels

The kinetics of aggregation of two pyromellitamide gelators; tetrabutyl- (C4) and tetrahexylpyromellitamide (C6), in deuterated cyclohexane has been investigated by small angle neutron scattering (SANS) for up to six days. The purpose of this study was to improve our understanding of how self-assembled gels are formed. Short-term (< 3 hour) time scales revealed multiple phases with the data for the tetrabutylpyromellitamide C4 indicating one dimensional stacking and aggregation corresponding to a multi-fiber braided cluster arrangement that is about 35 Å in diameter. The corresponding tetrahexylpyromellitamide C6 data suggests that the C6 also forms one-dimensional stacks but that these aggregate to a thicker multi-fiber braided cluster that have a diameter of 61.8 Å. Over a longer period of time, the radius, persistence length and contour length all continue to increase in 6 days after cooling. This data suggests that structural changes in self-assembled gels occur over a period exceeding several days and that fairly subtle changes in the structure (e.g. tail-length) can influence the packing of molecules in self-assembled gels on the single-to-few fiber bundle stage.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Two-Year Operational Evaluation Of A Consumer Electronics-Based Data Acquisition System For Equipment Monitoring

Oak Ridge National Laboratory (ORNL) has performed a two-year evaluation of the performance of an in-house developed consumer electronics-based data acquisition system (DAS). The main advantage of this approach compared to conventional instrumentation grade systems is cost; instrument grade data acquisition systems average costs range from $800 to $2,000 per channel compared to a range of $200 - $400 per channel for a consumer electronics-based system. The DAS is operated as a full-time in-situ vibration monitor. The resulting data stream is streamed over the ORNL network, at an aggregate rate of approximately 2 megabytes/s, to a Linux server. The server includes the capability to implement event-triggered data stores, as well as real-time files for the implementation of continuous display monitoring of the spectra. Detailed spectral analysis is performed post event. The DAS is installed on a large industrial chiller and cooling water pump associated with ORNL’s Titan supercomputer. These mechanical systems include rotating components that operate at fundamental frequencies within the range of 30 Hz to over 3 KHz. Evaluation of the DAS data over a two-year operating period leads to the conclusion that for many industrial processes this system could form the basis for a cost effective means of obtaining operating health data in real time from rotating machinery. The deployment has also shown that the DAS technology is reliable. Furthermore, because the cost of the DAS is low, the other significant advantage of this approach is that the DAS can be deployed in a dedicated manner and operated on a full-time basis.

Wetherington Jr, G↗
DOE OSTI2025

Pricing Strategy of Electric Vehicle Aggregators Based on Locational Marginal Price to Minimize Photovoltaic (PV) Curtailment

The global climate crisis demands urgent action to mitigate global warming. Using renewable energy sources, such as solar and wind power, for electricity generation is crucial. This shift from centralized to distributed power systems, however, brings challenges, including voltage fluctuations and renewable energy curtailment. The rapid growth of the electric vehicle (EV) industry adds complexity, increasing overall electricity demand and straining the power supply during peak charging times. This paper proposes a scheduling strategy for EV aggregators to reduce renewable energy curtailment and stabilize grid operation by strategically scheduling EV charging. Using Multi -Agent Transport Simulation (MATSim), a traffic simulation tool, EV driving data in Denver, Colorado, USA, were modeled. The EV aggregator adjusts charging fees based on locational marginal prices, encouraging EVs to charge at different stations according to pricing. Simulations on an IEEE 33-bus system with distributed energy resources and EV charging stations validate the proposed algorithm, demonstrating its effectiveness in reducing curtailment by 12.55% and stabilizing grid operation.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Field-scale dynamics of planting dates in the US Corn Belt from 2000 to 2020

Crop planting dates are a dynamic feature of agricultural systems that respond to short- and long-term climate signals, crop and cultivar selection, and technology changes. Planting date records are essential for yield gap analyses, accurate crop modeling, and tracking farmer adaptations to weather and climate change. Although planting dates have high variation at local scales due to heterogeneity in farm resources and decision-making, available long-term data on planting dates is largely restricted to aggregated regional statistics or, at best, satellite-derived datasets with limited spatiotemporal extent and at resolutions unable to distinguish individual fields (> 250 m). Here, we generated retrospective annual field-scale (30 m) planting date maps for both maize and soybeans spanning 2000-2020 across a 12 state region in the United States Corn Belt based on Landsat satellite data and a large ground sample of over 28,000 maize and soybean fields. Using training data from 2015-2020 for model selection, we found that planting date predictions improved with harmonic regression of Landsat data and additional annual weather covariates. The preferred random forests model approximately doubled performance compared to a null model based on state median planting dates, capturing 47% of field-level variation for maize (mean absolute error, MAE = 7.4 days) and 44% for soybeans (MAE = 7.5 days) against held-out ground truth test data for 2008-2014. We also evaluated the full 2000-2020 dataset with state agricultural statistics, finding strong agreement with median planting dates for maize (R 2 = 0.76, MAE = 4.4 days) and slightly lower agreement for soybeans (R 2 = 0.65, MAE = 5.4 days) when aggregated to the state level. We then used this new dataset to analyze environmental determinants of planting dates at a finer-scale than previously possible, controlling for unobserved variation at the sub-state district level. We found that during 2000-2020, each standard deviation increase in rainfall delayed planting by ~ 2.5 days, and fields with higher soil productivity ratings tended to be planted earlier. We did not find meaningful trends over the last two decades in planting dates for maize or soybeans, in contrast to trends towards earlier planting dates late last century and predicted for this period in climate adaptation studies. We hypothesize increases in early season rainfall may have inhibited these shifts towards earlier planting. Remotely sensed planting dates will be a useful tool for yield gap analyses, crop simulation modeling, and ongoing assessment of climate adaptation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Uncovering the intrinsic variability of gamma-ray bursts

We develop a robust technique to determine the minimum variability timescale for gamma-ray burst (GRB) light curves, utilizing Haar wavelets. Our approach averages over the data for a given GRB, providing an aggregate measure of signal variation while also retaining sensitivity to narrow pulses within complicated time series. In contrast to previous studies using wavelets, which simply define the minimum timescale in reference to the measurement noise floor, our approach identifies the signature of temporally smooth features in the wavelet scaleogram and then additionally identifies a break in the scaleogram on longer timescales as a signature of a true, temporally unsmooth light curve feature or features. We apply our technique to the large sample of Swift GRB gamma-ray light curves and for the first time—due to the presence of a large number of GRBs with measured redshift—determine the distribution of minimum variability timescales in the source frame. We find a median minimum timescale for long-duration GRBs in the source frame of Δt {sub min} = 0.5 s, with the shortest timescale found being on the order of 10 ms. This short timescale suggests a compact central engine (3 × 10{sup 3} km). We discuss further implications for the GRB fireball model and present a tantalizing correlation between the minimum timescale and redshift, which may in part be due to cosmological time dilation.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

When to use rsync

We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency &lE; 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

International collaboration framework for the calculation of performance loss rates: Data quality, benchmarks, and trends (towards a uniform methodology)

Abstract The IEA PVPS Task 13 group, experts who focus on photovoltaic performance, operation, and reliability from several leading R&D centers, universities, and industrial companies, is developing a framework for the calculation of performance loss rates of a large number of commercial and research photovoltaic (PV) power plants and their related weather data coming across various climatic zones. The general steps to calculate the performance loss rate are (i) input data cleaning and grading; (ii) data filtering; (iii) performance metric selection, corrections, and aggregation; and finally, (iv) application of a statistical modeling method to determine the performance loss rate value. In this study, several high‐quality power and irradiance datasets have been shared, and the participants of the study were asked to calculate the performance loss rate of each individual system using their preferred methodologies. The data are used for benchmarking activities and to define capabilities and uncertainties of all the various methods. The combination of data filtering, metrics (performance ratio or power based), and statistical modeling methods are benchmarked in terms of (i) their deviation from the average value and (ii) their uncertainty, standard error, and confidence intervals. It was observed that careful data filtering is an essential foundation for reliable performance loss rate calculations. Furthermore, the selection of the calculation steps filter/metric/statistical method is highly dependent on one another, and the steps should not be assessed individually.

14 SOLAR ENERGY↗
DOE OSTI2022

Assessing the large-scale drivers of precipitation in the northeastern United States via linear orthogonal decomposition

This study examines the linear orthogonal modes associated with monthly precipitation in the northeastern United States, from CESM1 LENS (35 ensemble members, 1979–2005) and two reanalysis datasets (ERA5, 1950–2018 and NOAA-CIRES-DOE 20CRv3, 1950–2015). Calendar months are aggregated together, and any linear trends in data are removed. Using region-averaged precipitation anomaly time series and monthly anomalies for several global 2D atmospheric fields, a linear orthogonal decomposition method is implemented to iteratively extract time series (based on field and geographic location) of absolute maximum correlation. Linear modes associated with this method are then projected onto the full set of 2D fields to provide physical insight into the mechanisms involved in generating precipitation. In this region, the first mode is associated with vapor transport from the Atlantic seaboard, the second mode is characterized by westward vapor transport associated with extratropical cyclones, and the third mode captures vapor transport from the Gulf of Mexico during the fall and winter. However, the third mode is less robust in the spring and summer. Results are generally consistent across the datasets, and applying multiple linear regression with the linear modes to predict the precipitation anomalies produces R-squared values of around 0.54–0.65 for CESM1 LENS, and around 0.58–0.88 for reanalysis, with the lowest values generally in the spring and late summer. The influence of low-frequency climate variability on the modes is considered for CESM1 LENS, and the modes in late winter can be predicted with some success via a combination of several, prominent large-scale teleconnection patterns.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Parametric dynamic mode decomposition for reduced order modeling

Dynamic Mode Decomposition (DMD) is a model-order reduction approach, whereby spatial modes of fixed temporal frequencies are extracted from numerical or experimental data sets. The DMD low-rank or reduced operator is typically obtained by singular value decomposition of the temporal data sets. For parameter-dependent models, as found in many multi-query applications such as uncertainty quantification or design optimization, the only parametric DMD technique developed was a stacked approach, with data sets at multiple parameter values were aggregated together, increasing the computational work needed to devise low-rank dynamical reduced-order models. Here in this paper, we present two novel approach to carry out parametric DMD: one based on the interpolation of the reduced-order DMD eigen-pair and the other based on the interpolation of the reduced DMD (Koopman) operator. Numerical results are presented for diffusion-dominated nonlinear dynamical problems, including a multiphysics radiative transfer example. All three parametric DMD approaches are compared.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Toward Molecule-Specific Geochemistry of Heavy Ends: Application to the Upstream Oil Industry

Detailed molecular analysis of the composition of petroleum is typically performed on only the light (volatile) fraction, using analyses involving gas chromatography (GC). Heavier (nonvolatile) components such as kerogen, bitumen, and asphaltenes, which are relevant to conventional reservoirs and particularly to unconventional gas shale and tight oil formations, are not amenable to analysis by GC and instead are typically studied by more phenomenological techniques. This report describes a suite of analytical techniques that can provide sufficient average molecular information on the heavy ends (defined as the nonvolatile fraction) to be useful for a variety of industrial applications, expanding the discipline of molecule-specific geochemistry into the non-GC amenable fractions of petroleum materials. Heavy ends are composed of the elements C, H, N, S, and O, and the bonding environments of each can be measured by various solid state spectroscopies: nuclear magnetic resonance (NMR) and infrared (IR) spectroscopies for C, H, and O; X-ray absorption near-edge structure (XANES) spectroscopy for S and N. Additionally, advances in mass spectrometry, including techniques such as laser desorption laser ionization mass spectrometry (L2MS), can be used to measure the molecular weight and dominant molecular architecture of asphaltenes. Moreover, higher-order structures can be measured: samples of kerogen with representative pore geometry can be prepared, and those pore systems are analyzed with techniques such as X-ray diffraction and neutron scattering. Colloidal structures of asphaltenes can be measured with a variety of techniques, which reveal a hierarchical aggregation structure. These analyses provide sufficient structural data to enable structure–property and structure–function relationships to be determined, which can be applied to the characterization of conventional and unconventional resources. In gas shale and tight oil formations, which contain abundant inorganic minerals as well as kerogen, diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS) is used for simultaneous analysis of mineralogy, kerogen content, and maturity, typically performed at the wellsite. In conventional reservoirs, spatial variations in the asphaltene concentration measured by downhole fluid analysis logging tools are being interpreted with a recently developed equation of state to address upstream issues such as reservoir connectivity, tar mat formation, and fault block migration. Furthermore, as more heavy end analyses are developed, further scientific knowledge and industrial applications of heavy end geochemistry are anticipated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Linking structural and rheological memory in disordered soft materials

Linking the macroscopic flow properties and nanoscopic structure is a fundamental challenge to understanding, predicting, and designing disordered soft materials. Under small stresses, these materials are soft solids, while larger loads can lead to yielding and the acquisition of plastic strain, which adds complexity to the task. In this work, we connect the transient structure and rheological memory of a colloidal gel under cyclic shearing across a range of amplitudes via a generalized memory function using rheo-X-ray photon correlation spectroscopy (rheo-XPCS). Our rheo-XPCS data show that the nanometer scale aggregate-level structure recorrelates whenever the change in recoverable strain over some interval is zero. The macroscopic recoverable strain is therefore a measure of the nano-scale structural memory. We further show that yielding in disordered colloidal materials is strongly heterogeneous and that memories of prior deformation can exist even after the material has been subjected to flow.

Kamani, Krutarth M. [Univ. of Illinois at Urbana-C↗
DOE OSTI2013

ULTRAVIOLET EXTINCTION AT HIGH GALACTIC LATITUDES

In order to study the properties and effects of high Galactic latitude dust, we present an analysis of 373,303 galaxies selected from the Galaxy Evolution Explorer All-Sky Survey and Wide-field Infrared Explorer All-Sky Data Release. By examining the variation in aggregate ultraviolet colors and number density of these galaxies, we measure the extinction curve at high latitude. We additionally consider a population of spectroscopically selected galaxies from the Sloan Digital Sky Survey to measure extinction in the optical. We find that dust at high latitude is neither quantitatively nor qualitatively consistent with standard reddening laws. Extinction in the FUV and NUV is {approx}10% and {approx}35% higher than expected, with significant variation across the sky. We find that no single R{sub V} parameter fits both the optical and ultraviolet extinction at high latitude, and that while both show detectable variation across the sky, these variations are not related. We propose that the overall trends we detect likely stem from an increase in very small silicate grains in the interstellar medium.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Pattern Discovery in Internet Background Radiation

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

Iglesias, Felix↗
DOE OSTI2014

MO-G-18C-05: Real-Time Prediction in Free-Breathing Perfusion MRI

Purpose: The aim is to minimize frame-wise difference errors caused by respiratory motion and eliminate the need for breath-holds in magnetic resonance imaging (MRI) sequences with long acquisitions and repeat times (TRs). The technique is being applied to perfusion MRI using arterial spin labeling (ASL). Methods: Respiratory motion prediction (RMP) using navigator echoes was implemented in ASL. A least-square method was used to extract the respiratory motion information from the 1D navigator. A generalized artificial neutral network (ANN) with three layers was developed to simultaneously predict 10 time points forward in time and correct for respiratory motion during MRI acquisition. During the training phase, the parameters of the ANN were optimized to minimize the aggregated prediction error based on acquired navigator data. During realtime prediction, the trained ANN was applied to the most recent estimated displacement trajectory to determine in real-time the amount of spatial Results: The respiratory motion information extracted from the least-square method can accurately represent the navigator profiles, with a normalized chi-square value of 0.037±0.015 across the training phase. During the 60-second training phase, the ANN successfully learned the respiratory motion pattern from the navigator training data. During real-time prediction, the ANN received displacement estimates and predicted the motion in the continuum of a 1.0 s prediction window. The ANN prediction was able to provide corrections for different respiratory states (i.e., inhalation/exhalation) during real-time scanning with a mean absolute error of < 1.8 mm. Conclusion: A new technique enabling free-breathing acquisition during MRI is being developed. A generalized ANN development has demonstrated its efficacy in predicting a continuum of motion profile for volumetric imaging based on navigator inputs. Future work will enhance the robustness of ANN and verify its effectiveness with human subjects. Research supported by National Institutes of Health National Cancer Institute Grant R01 CA159471-01.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

LBNL-Load

This software package is a set of Python-based tools for forecasting future time series of electrical power demand for a granular set of customer types in California, based on historical data on individual customers' demand, combined with aggregate forecasts of demand growth.

Gerke, Brian↗
DOE OSTI2013

Technological Cost-Reduction Pathways for Axial-Flow Turbines in the Marine Hydrokinetic Environment.

This report considers and prioritizes potential technical cost reduction pathways for axial flow turbines designed for tidal, river, and ocean current resources. This report focuses on technical research and development cost reduction pathways related to the device technology rather than environmental monitoring or permitting opportunities. Three sources of information were utilized to understand current cost drivers and develop a list of potential cost reduction pathways: a literature review of technical work related to axial flow turbines, the U.S. Department of Energy Reference Model effort, and informal webinars and other targeted interactions with industry developers. Data from these various information sources were aggregated and prioritized with respect to potential impact on the lifetime levelized cost of energy. The four most promising cost reduction pathways include structural design optimization; improved deployment, maintenance, and recovery; system simplicity and reliability; and array optimization.

Laird, Daniel L.↗