Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions

The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.

Durden, David J.↗
DOE OSTI2022

Skopi: a simulation package for diffractive imaging of noncrystalline biomolecules

X-ray free-electron lasers (XFELs) have the ability to produce ultra-bright femtosecond X-ray pulses for coherent diffraction imaging of biomolecules. While the development of methods and algorithms for macromolecular crystallography is now mature, XFEL experiments involving aerosolized or solvated biomolecular samples offer new challenges in terms of both experimental design and data processing. Skopi is a simulation package that can generate single-hit diffraction images for reconstruction algorithms, multi-hit diffraction images of aggregated particles for training machine learning classifiers using labeled data, diffraction images of randomly distributed particles for fluctuation X-ray scattering algorithms, and diffraction images of reference and target particles for holographic reconstruction algorithms. Skopi is a resource to aid feasibility studies and advance the development of algorithms for noncrystalline experiments at XFEL facilities.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2018

Narrowband Propagation Statistics of Aeronautical Mobile-Ground Links in the L- and C-Bands

To provide for the safe integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS), command and control (C2) links must be highly reliable. Hence, protected aviation spectrum is required to support such links for UAS that are integrated into controlled non-segregated airspace. For air-ground (i.e., non-satellite) links, protected aviation spectrum to support C2 links is available in the 960-1164 MHz (L) and 5030-5091 MHz (C) bands. The performance of any C2 system is critically dependent upon the characteristics of the air-ground (AG) channel. Therefore, as part of its UAS Integration in the NAS (UAS in the NAS) project, the U.S. National Aeronautics and Space Administration (NASA) performed a series of air-ground propagation flight tests to collect AG channel data for model development and analysis of potential C2 communications links capable of providing the required reliability. NASA's Glenn Research Center (GRC) conducted an extensive air-ground channel propagation measurement campaign (at altitude) for frequencies in the 960-977 MHz and 5030-5091 MHz ranges, for seven different terrain environments. The measurements were conducted in 2013, and produced the largest set of AG channel data ever gathered to date. This data was subsequently processed to develop models for the AG channel. The statistics collected enabled the derivation of channel model parameters for both narrowband and wideband channels. In order to make the propagation data widely available, the resulting narrowband statistics were processed and submitted to the International Telecommunications Union - Radiocommunication Sector (ITU-R) Study Group 3 Data Banks. Formats for data tables were developed, and tables of the aggregate narrowband propagation statistics for the seven ground site terrain environments were prepared, submitted to, and approved by, the ITU-R Study Group 3. This paper provides brief background on the measurement campaign, collection and processing of data, and development of the narrowband data tables. It further provides examples of the data and its use.

radiofrequency spectrum↗
NASA NTRS2018

Narrowband Propagation Statistics of Aeronautical Mobile-Ground Links in the L- and C-Bands

To provide for the safe integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS), command and control (C2) links must be highly reliable. Hence, protected aviation spectrum is required to support such links for UAS that are integrated into controlled non-segregated airspace. For air-ground (i.e., non-satellite) links, protected aviation spectrum to support C2 links is available in the 960-1164 MHz (L) and 5030-5091 MHz (C) bands. The performance of any C2 system is critically dependent upon the characteristics of the air-ground (AG) channel. Therefore, as part of its UAS Integration in the NAS (UAS in the NAS) project, the U.S. National Aeronautics and Space Administration (NASA) performed a series of air-ground propagation flight tests to collect AG channel data for model development and analysis of potential C2 communications links capable of providing the required reliability. NASA's Glenn Research Center (GRC) conducted an extensive air-ground channel propagation measurement campaign (at altitude) for frequencies in the 960-977 MHz and 5030-5091 MHz ranges, for seven different terrain environments. The measurements were conducted in 2013, and produced the largest set of AG channel data ever gathered to date. This data was subsequently processed to develop models for the AG channel. The statistics collected enabled the derivation of channel model parameters for both narrowband and wideband channels. In order to make the propagation data widely available, the resulting narrowband statistics were processed and submitted to the International Telecommunications Union Radiocommunication Sector (ITU-R) Study Group 3 Data Banks. Formats for data tables were developed, and tables of the aggregate narrowband propagation statistics for the seven ground site terrain environments were prepared, submitted to, and approved by, the ITU-R Study Group 3. This paper provides brief background on the measurement campaign, collection and processing of data, and development of the narrowband data tables. It further provides examples of the data and its use.

aircraft communications↗
DOE OSTI2015

Big Biomedical data as the key resource for discovery science

Modern biomedical data collection is generating exponentially more data in a multitude of formats. This flood of complex data poses significant opportunities to discover and understand the critical interplay among such diverse domains as genomics, proteomics, metabolomics, and phenomics, including imaging, biometrics, and clinical data. The Big Data for Discovery Science Center is taking an “-ome to home” approach to discover linkages between these disparate data sources by mining existing databases of proteomic and genomic data, brain images, and clinical assessments. In support of this work, the authors developed new technological capabilities that make it easy for researchers to manage, aggregate, manipulate, integrate, and model large amounts of distributed data. Guided by biological domain expertise, the Center’s computational resources and software will reveal relationships and patterns, aiding researchers in identifying biomarkers for the most confounding conditions and diseases, such as Parkinson’s and Alzheimer’s.

Alzheimer's disease (ID)↗
DOE OSTI2014

Small Angle Neutron Scattering (SANS) Studies on the Structural Evolution of Pyromellitamide Self-assembled Gels

The kinetics of aggregation of two pyromellitamide gelators; tetrabutyl- (C4) and tetrahexylpyromellitamide (C6), in deuterated cyclohexane has been investigated by small angle neutron scattering (SANS) for up to six days. The purpose of this study was to improve our understanding of how self-assembled gels are formed. Short-term (< 3 hour) time scales revealed multiple phases with the data for the tetrabutylpyromellitamide C4 indicating one dimensional stacking and aggregation corresponding to a multi-fiber braided cluster arrangement that is about 35 Å in diameter. The corresponding tetrahexylpyromellitamide C6 data suggests that the C6 also forms one-dimensional stacks but that these aggregate to a thicker multi-fiber braided cluster that have a diameter of 61.8 Å. Over a longer period of time, the radius, persistence length and contour length all continue to increase in 6 days after cooling. This data suggests that structural changes in self-assembled gels occur over a period exceeding several days and that fairly subtle changes in the structure (e.g. tail-length) can influence the packing of molecules in self-assembled gels on the single-to-few fiber bundle stage.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this article, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such a decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

Measured and Modeled Vineyard Canopy Development and Water Use

Two publicly available applications relevant to vineyard irrigation management are described. OpenET is a satellite-based system that applies an ensemble of remote sensing methods to enable wide-area monitoring of evapotranspiration (ET) and related measures such as vegetation canopy development (via the NDVI spectral index). Data are freely available at one-quarter acre spatial resolution, and may be automatically aggregated to the individual block level. The satellite-based daily ET data were compared with in-situ eddy covariance measurements collected by micro-meteorological instrumentation in a Central Coast vineyard over a three-year period (2020-2022). Estimation uncertainties were reasonably consistent with prior reports for GRAPEX sites in the Central Valley and North Coast. The CropManage (CM) web application is a free software tool developed and operated U.C. Cooperative Extension for ET-based irrigation scheduling of major specialty crops. CM provides specific guidance for irrigation events in terms of irrigation system runtime. Applied water recommendations are based largely on estimated ET, derived from assumed canopy cover and associated crop coefficients, since the last irrigation or rainfall event. The application was recently adapted to vineyards by accounting for presence of winter/spring cover crop, and vine water stress imposed by regulated deficit irrigation. A 2022 field campaign involved 12 Central Coast and San Joaquin Valley commercial vineyards, and OpenET data were used to help evaluate CM output. Maximum percent vine cover was compared to estimates derived from average July satellite NDVI. The difference for 10 sites lacking midseason groundcover ranged from 0-7% between datasets with average agreement near 4%. Cumulative ET estimates agreed with OpenET to within 12% at the majority of sites, while larger discrepancies at the remaining sites may require additional data collection and analysis during the 2023 season. Satellite based systems such as OpenET have the potential to help parameterize CropManage and similar agricultural decision-support systems.

Measured↗
DOE OSTI2017

Two-Year Operational Evaluation Of A Consumer Electronics-Based Data Acquisition System For Equipment Monitoring

Oak Ridge National Laboratory (ORNL) has performed a two-year evaluation of the performance of an in-house developed consumer electronics-based data acquisition system (DAS). The main advantage of this approach compared to conventional instrumentation grade systems is cost; instrument grade data acquisition systems average costs range from $800 to $2,000 per channel compared to a range of $200 - $400 per channel for a consumer electronics-based system. The DAS is operated as a full-time in-situ vibration monitor. The resulting data stream is streamed over the ORNL network, at an aggregate rate of approximately 2 megabytes/s, to a Linux server. The server includes the capability to implement event-triggered data stores, as well as real-time files for the implementation of continuous display monitoring of the spectra. Detailed spectral analysis is performed post event. The DAS is installed on a large industrial chiller and cooling water pump associated with ORNL’s Titan supercomputer. These mechanical systems include rotating components that operate at fundamental frequencies within the range of 30 Hz to over 3 KHz. Evaluation of the DAS data over a two-year operating period leads to the conclusion that for many industrial processes this system could form the basis for a cost effective means of obtaining operating health data in real time from rotating machinery. The deployment has also shown that the DAS technology is reliable. Furthermore, because the cost of the DAS is low, the other significant advantage of this approach is that the DAS can be deployed in a dedicated manner and operated on a full-time basis.

Wetherington Jr, G↗
NASA NTRSDate not supplied

Hydrological Data at the NASA GES DISC: Current Capabilities and New Opportunities

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of twelve NASA Earth science data centers that document, process, archive and distribute data from Earth observation missions and projects. GES DISC maintains an archive of several hydrology datasets, including the Land Data Assimilation Systems (LDAS) and the Gravity Recovery and Climate Experiment (GRACE) Data Assimilation for Drought Monitoring (GRACE-DA-DM) data products. These datasets include model output of heat fluxes, rain, snow, soil temperature, soil moisture, and runoff; and observational forcing data, including surface pressure, temperature, precipitation, downward shortwave and longwave radiation, humidity, and wind. The temporal resolution of the hydrology data at GES DISC ranges from hourly to monthly, and spatial resolutions range from 0.1° to 1.0°. The GES DISC provides services which enable users to aggregate, temporally and spatially subset, regrid, and visualize archived data including the GES DISC Subsetter, Hydrology Data Rods, and the Geospatial Interactive Online Visualization and Analysis Infrastructure (GIOVANNI). The Hydrology Data Rods service optimally reorganizes large hydrological data sets as extended time series, providing more efficient access for the hydrological community. The time series data (aka “data rods”) were integrated into hydrology community tools, such as the Data Rods Explorer on HydroShare. Furthermore, the GES DISC is in the process of migrating its data and services to the cloud. Hydrological data available at the GES DISC are now available in the Amazon Web Services (AWS) cloud (us-west-2 region) providing users Direct S3 data access and the capability for cloud computing operations. In this presentation, the hydrology data products and services currently available at the GES DISC will be summarized. Also discussed are the migration to the cloud, user support through this transition, and the status of migrating the data rods service to the cloud.

Ashley Heath↗
DOE OSTI2025

Pricing Strategy of Electric Vehicle Aggregators Based on Locational Marginal Price to Minimize Photovoltaic (PV) Curtailment

The global climate crisis demands urgent action to mitigate global warming. Using renewable energy sources, such as solar and wind power, for electricity generation is crucial. This shift from centralized to distributed power systems, however, brings challenges, including voltage fluctuations and renewable energy curtailment. The rapid growth of the electric vehicle (EV) industry adds complexity, increasing overall electricity demand and straining the power supply during peak charging times. This paper proposes a scheduling strategy for EV aggregators to reduce renewable energy curtailment and stabilize grid operation by strategically scheduling EV charging. Using Multi -Agent Transport Simulation (MATSim), a traffic simulation tool, EV driving data in Denver, Colorado, USA, were modeled. The EV aggregator adjusts charging fees based on locational marginal prices, encouraging EVs to charge at different stations according to pricing. Simulations on an IEEE 33-bus system with distributed energy resources and EV charging stations validate the proposed algorithm, demonstrating its effectiveness in reducing curtailment by 12.55% and stabilizing grid operation.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Field-scale dynamics of planting dates in the US Corn Belt from 2000 to 2020

Crop planting dates are a dynamic feature of agricultural systems that respond to short- and long-term climate signals, crop and cultivar selection, and technology changes. Planting date records are essential for yield gap analyses, accurate crop modeling, and tracking farmer adaptations to weather and climate change. Although planting dates have high variation at local scales due to heterogeneity in farm resources and decision-making, available long-term data on planting dates is largely restricted to aggregated regional statistics or, at best, satellite-derived datasets with limited spatiotemporal extent and at resolutions unable to distinguish individual fields (> 250 m). Here, we generated retrospective annual field-scale (30 m) planting date maps for both maize and soybeans spanning 2000-2020 across a 12 state region in the United States Corn Belt based on Landsat satellite data and a large ground sample of over 28,000 maize and soybean fields. Using training data from 2015-2020 for model selection, we found that planting date predictions improved with harmonic regression of Landsat data and additional annual weather covariates. The preferred random forests model approximately doubled performance compared to a null model based on state median planting dates, capturing 47% of field-level variation for maize (mean absolute error, MAE = 7.4 days) and 44% for soybeans (MAE = 7.5 days) against held-out ground truth test data for 2008-2014. We also evaluated the full 2000-2020 dataset with state agricultural statistics, finding strong agreement with median planting dates for maize (R 2 = 0.76, MAE = 4.4 days) and slightly lower agreement for soybeans (R 2 = 0.65, MAE = 5.4 days) when aggregated to the state level. We then used this new dataset to analyze environmental determinants of planting dates at a finer-scale than previously possible, controlling for unobserved variation at the sub-state district level. We found that during 2000-2020, each standard deviation increase in rainfall delayed planting by ~ 2.5 days, and fields with higher soil productivity ratings tended to be planted earlier. We did not find meaningful trends over the last two decades in planting dates for maize or soybeans, in contrast to trends towards earlier planting dates late last century and predicted for this period in climate adaptation studies. We hypothesize increases in early season rainfall may have inhibited these shifts towards earlier planting. Remotely sensed planting dates will be a useful tool for yield gap analyses, crop simulation modeling, and ongoing assessment of climate adaptation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Uncovering the intrinsic variability of gamma-ray bursts

We develop a robust technique to determine the minimum variability timescale for gamma-ray burst (GRB) light curves, utilizing Haar wavelets. Our approach averages over the data for a given GRB, providing an aggregate measure of signal variation while also retaining sensitivity to narrow pulses within complicated time series. In contrast to previous studies using wavelets, which simply define the minimum timescale in reference to the measurement noise floor, our approach identifies the signature of temporally smooth features in the wavelet scaleogram and then additionally identifies a break in the scaleogram on longer timescales as a signature of a true, temporally unsmooth light curve feature or features. We apply our technique to the large sample of Swift GRB gamma-ray light curves and for the first time—due to the presence of a large number of GRBs with measured redshift—determine the distribution of minimum variability timescales in the source frame. We find a median minimum timescale for long-duration GRBs in the source frame of Δt {sub min} = 0.5 s, with the shortest timescale found being on the order of 10 ms. This short timescale suggests a compact central engine (3 × 10{sup 3} km). We discuss further implications for the GRB fireball model and present a tantalizing correlation between the minimum timescale and redshift, which may in part be due to cosmological time dilation.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRSDate not supplied

Creating Near Real-time Hourly VIIRS Radiance Mosaic over Polar Regions from SatCORPS Global Cloud Composite (GCC)

While polar orbiting satellites are essential in providing the needed observations over polar regions for data assimilation, weather diagnoses/forecasting, and flight guidance and situation awareness in airborne field campaigns, there is limited temporal coverage. Often, polar mosaics are created by overlaying multiple satellite imagers on top of each other which can lead to transition lines and discontinuities in the data products. To help ease some of these shortcomings, we utilize the NASA Langley Satellite ClOud and Radiation Property retrieval System (SatCORPS) Mosaic algorithm for optimally fusing and merging selected imager radiances from the Visible Infrared Imaging Radiometer Suite (VIIRS) instrument onboard the Suomi National polar-orbiting Partnership (SNPP), Joint Polar Satellite System (JPSS-1), and JPSS-2 satellites, and the MODerate resolution Imaging Spectroradiometer (MODIS) instruments onboard AQUA and TERRA satellites. Since multiple satellites overpasses and instruments are used in creating hourly 3-km composites over the polar regions, selection of the best satellite data for each 3-km composite pixel is based on aggregated ranking of satellite resolution, time, viewing zenith angle, and distance from terminator. This ranking system provides a smoother transition and improves data discontinuity in the merged output resulting in a more seamless polar region dataset. We will discuss the use of this near real-time dataset to support polar airborne field campaigns and SatCORPS satellite-derived Global Cloud Composite (GCC) generation.

Satellite composites↗
NASA NTRS1993

Strategic planning for aircraft noise route impact analysis: A three dimensional approach

The strategic routing of aircraft through navigable and controlled airspace to minimize adverse noise impact over sensitive areas is critical in the proper management and planning of the U.S. based airport system. A major objective of this phase of research is to identify, inventory, characterize, and analyze the various environmental, land planning, and regulatory data bases, along with potential three dimensional software and hardware systems that can be potentially applied for an impact assessment of any existing or planned air route. There are eight data bases that have to be assembled and developed in order to develop three dimensional aircraft route impact methodology. These data bases which cover geographical information systems, sound metrics, land use, airspace operational control measures, federal regulations and advisories, census data, and environmental attributes have been examined and aggregated. A three dimensional format is necessary for planning, analyzing space and possible noise impact, and formulating potential resolutions. The need to develop this three dimensional approach is essential due to the finite capacity of airspace for managing and planning a route system, including airport facilities. It appears that these data bases can be integrated effectively into a strategic aircraft noise routing system which should be developed as soon as possible, as part of a proactive plan applied to our FAA controlled navigable airspace for the United States.

Bragdon, C. R.↗
DOE OSTI2021

When to use rsync

We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency &lE; 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

International collaboration framework for the calculation of performance loss rates: Data quality, benchmarks, and trends (towards a uniform methodology)

Abstract The IEA PVPS Task 13 group, experts who focus on photovoltaic performance, operation, and reliability from several leading R&D centers, universities, and industrial companies, is developing a framework for the calculation of performance loss rates of a large number of commercial and research photovoltaic (PV) power plants and their related weather data coming across various climatic zones. The general steps to calculate the performance loss rate are (i) input data cleaning and grading; (ii) data filtering; (iii) performance metric selection, corrections, and aggregation; and finally, (iv) application of a statistical modeling method to determine the performance loss rate value. In this study, several high‐quality power and irradiance datasets have been shared, and the participants of the study were asked to calculate the performance loss rate of each individual system using their preferred methodologies. The data are used for benchmarking activities and to define capabilities and uncertainties of all the various methods. The combination of data filtering, metrics (performance ratio or power based), and statistical modeling methods are benchmarked in terms of (i) their deviation from the average value and (ii) their uncertainty, standard error, and confidence intervals. It was observed that careful data filtering is an essential foundation for reliable performance loss rate calculations. Furthermore, the selection of the calculation steps filter/metric/statistical method is highly dependent on one another, and the steps should not be assessed individually.

14 SOLAR ENERGY↗
NASA NTRS2023

Fifteen Years of Millimeter Accuracy Lunar Laser Ranging with APOLLO: Data Reduction and Calibration

The Apache Point Lunar Laser-ranging Operation (APOLLO) has been collecting lunar range measurements for 15 yr at millimeter accuracy. The median nightly range uncertainty since 2006 is 1.7 mm. A recently added Absolute Calibration System (ACS), providing an independent assessment of APOLLO system accuracy and the capability to correct lunar range data, revealed a ∼0.4% (10 ps) systematic error in the calibration of one piece of hardware that has been present for the entire history of APOLLO. The application of ACS-based timing corrections suggests systematic errors are reduced to <1 mm, such that overall data accuracy and precision are both ∼1 mm. This paper describes the processing of APOLLO/ACS data that converts photon-by-photon range measurements into the aggregated normal points that are used for science analyses. Additionally, we present methodologies to estimate timing corrections for range data lacking contemporaneous ACS photons, including range data collected prior to installation of the ACS. We also provide access to the full 15 yr archive of APOLLO normal points (2006 April 6–2020 December 27).

LLR↗