Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Multi-Decadal Pathfinder Data Sets of Global Land Biophysical Variables from AVHRR and MODIS and their Use in GCM Studies of Biogeophysics and Biogeochemistry

The problem of how the scale, or spatial resolution, of reflectance data impacts retrievals of vegetation leaf area index (LAI) and fraction absorbed photosynthetically active radiation (PAR) has been investigated. We define the goal of scaling as the process by which it is established that LAI and FPAR values derived from coarse resolution sensor data equal the arithmetic average of values derived independently from fine resolution sensor data. The increasing probability of land cover mixtures with decreasing resolution is defined as heterogeneity, which is a key concept in scaling studies. The effect of pixel heterogeneity on spectral reflectances and LAI/FPAR retrievals is investigated with 1 km Advanced Very High Resolution Radiometer (AVHRR) data aggregated to different coarse spatial resolutions. It is shown that LAI retrieval errors at coarse resolution are inversely related to the proportion of the dominant land cover in such pixel. Further, large errors in LAI retrievals are incurred when forests are minority biomes in non-forest pixels compared to when forest biomes are mixed with one another, and vice-versa. A physically based technique for scaling with explicit spatial resolution dependent radiative transfer formulation is developed. The successful application of this theory to scaling LAI retrievals from AVHRR data of different resolutions is demonstrated

Myneni, Ranga↗
DOE OSTI2020

Multiscale and Multivariate Transportation System Visualization for Shopping District Traffic and Regional Traffic

In this paper, we present a suite of visualization techniques for sensor-based transportation system data at different scales to facilitate the exploration of interconnected traffic dynamics at intersections and highways. Additionally, these techniques are designed for analyzing multivariate traffic data from radar-based highway sensors and camera-based intersection sensors recording turn movements and vehicle speed, in the Chattanooga Metropolitan Area, with the capability of (a) revealing multiscale mobility patterns using different levels of data aggregation (e.g., individual sensor for microscale, multiple sensors along a corridor for mesoscale, and a larger number of sensors across the region for macroscale visualization) at different intervals (e.g., 5-min intervals, time of day, full day, and day-of-the-week), and (b) exploring the spatial variation of multiple traffic-related variables (e.g., volumes, speeds, turn movements, and traffic light colors) provided by the sensors. We close with a case study to demonstrate the effectiveness of our multiscale and multivariate visualization techniques. At microscale, we focused on intersection data from a shopping district around Shallowford Road in East Chattanooga. For mesoscale visualization, we studied the Shallowford Road corridor and an adjacent stretch of I-75. At macroscale, we included highway data from the Chattanooga Metropolitan Area. All visualizations were integrated into a web-based situational awareness tool to promote user access and interaction. At a minimum, each visualization provides the option for selecting dates for real-time (depending on sensor availability) and historical data, and additional information on hovering, though most provide more detailed information, including different views of the selected data, or interactive highlights.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2018

Robust Period Estimation Using Mutual Information for Multiband Light Curves in the Synoptic Survey Era

The Large Synoptic Survey Telescope (LSST) will produce an unprecedented amount of light curves using six optical bands. Robust and efficient methods that can aggregate data from multidimensional sparsely sampled time-series are needed. Here, we present a new method for light curve period estimation based on quadratic mutual information (QMI). The presented method does not assume a particular model for the light curve nor its underlying probability density and it is robust to non-Gaussian noise and outliers. By combining the QMI from several bands the true period can be estimated even when no single-band QMI yields the period. Period recovery performance as a function of average magnitude and sample size is measured using 30,000 synthetic multiband light curves of RR Lyrae and Cepheid variables generated by the LSST Operations and Catalog simulators. The results show that aggregating information from several bands is highly beneficial in LSST sparsely sampled time-series, obtaining an absolute increase in period recovery rate up to 50%. We also show that the QMI is more robust to noise and light curve length (sample size) than the multiband generalizations of the Lomb–Scargle and AoV periodograms, recovering the true period in 10%–30% more cases than its competitors. A python package with efficient Cython implementations of the QMI and other methods is provided.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2007

An Intriguing Convex Break in the EGRET SED of Mrk 421

Based upon analysis of the entire EGRET data from Mrk 421, it is found that the time-averaged spectra are inconsistent with the predictions of current theoretical models that have had success in describing simultaneous X-ray/TeV observations, and suggest additional components in the GeV band, as well as complex time variability. Current theoretical pictures explain the GeV emission as comptonization of the synchrotron photons in the jet, and predict hard spectra that should join smoothly with the TeV emission. Our analysis shows that the situation is more complex. The spectrum ranges from hard to soft during individual epochs, and shows a convext break in the aggregated data. We also present the mission-averaged EGRET spectrum for PKS 2155-304, which shows a similar (but not as pronounced) convex curvature. We discuss a series of possible explanations for the 10(exp 22) - 10(exp 23) HZ declining part of the EGRET nu F(sub nu), spectrum for Mrk 421, and suggest that it is synchrotron emission from the high energy tail of the electron population that produces the X-rays during the highest X-ray states. Such multi-MeV photons are produced by electrons accelerated close to the limit of diffusive shock acceleration. Simultaneous GLAST and X-ray observations of high X-ray states will address the issue of the convex curvature in the future.

Nandikotkur, Giridhar↗
DOE OSTI2025

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗
DOE OSTI2022

Prototype hardware design and testing of the global common module for the global trigger subsystem of the ATLAS phase-II upgrade

We report the High-Luminosity Large Hadron Collider (HL-LHC) will deliver more than ten times the integrated luminosity of the previous runs combined. Meeting its stricter throughput requirements poses new challenges to the Trigger and Data Acquisition (TDAQ) systems of the LHC experiments. Introduced in the framework of the ATLAS experiment’s HL upgrade, the Global Trigger (GT) is a new subsystem which will perform offline-like algorithms on full-granularity calorimeter data. The implementation of the GT’s functionality is firmware-focused and is composed of three layers: multiplexing (or data aggregating), global event processing, and demultiplexing interface to the central trigger processor. Each layer will be composed of several, similar nodes, hosted on replicas of identical hardware, the Global Common Module (GCM), an ATCA front board which is designed to be adopted throughout the entire GT subsystem. This article proceeds from the TWEPP 2021 conference and presents the GCM hardware design, performed in 2020, and focuses on some key results of its extensive testing performed in 2021.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2017

Smashing the Stovepipe: Leveraging the GMSEC Open Architecture and Advanced IT Automation to Rapidly Prototype, Develop and Deploy Next-Generation Multi-Mission Ground Systems

Satellite/Payload Ground Systems - Typically highly-customized to a specific mission's use cases - Utilize hundreds (or thousands!) of specialized point-to-point interfaces for data flows / file transfers Documentation and tracking of these complex interfaces requires extensive time to develop and extremely high staffing costs Implementation and testing of these interfaces are even more cost-prohibitive, and documentation often lags behind implementation resulting in inconsistencies down the road With expanding threat vectors, IT Security, Information Assurance and Operational Security have become key Ground System architecture drivers New Federal security-related directives are generated on a daily basis, imposing new requirements on current / existing ground systems - These mandated activities and data calls typically carry little or no additional funding for implementation As a result, Ground System Sustaining Engineering groups and Information Technology staff continually struggle to keep up with the rolling tide of security Advancing security concerns and shrinking budgets are pushing these large stove-piped ground systems to begin sharing resources - I.e. Operational / SysAdmin staff, IT security baselines, architecture decisions or even networks / hosting infrastructure Refactoring these existing ground systems into multi-mission assets proves extremely challenging due to what is typically very tight coupling between legacy components As a result, many "Multi-Mission" ops. environments end up simply sharing compute resources and networks due to the difficulty of refactoring into true multi-mission systems Utilizing continuous integration / rapid system deployment technologies in conjunction with an open architecture messaging approach allows System Engineers and Architects to worry less about the low-level details of interfaces between components and configuration of systems GMSEC messaging is inherently designed to support multi-mission requirements, and allows components to aggregate data across multiple homogeneous or heterogeneous satellites or payloads - The highly-successful Goddard Science and Planetary Operations Control Center (SPOCC) utilizes GMSEC as the hub for it's automation and situational awareness capability Shifts focus towards getting GS to a final configuration-managed baseline, as well as multi-mission / big-picture capabilities that help increase situational awareness, promote cross-mission sharing and establish enhanced fleet management capabilities across all levels of the enterprise.

GMSEC↗
DOE OSTI2021

Evaluating the Benefits of Bayesian Hierarchical Methods for Analyzing Heterogeneous Environmental Datasets: A Case Study of Marine Organic Carbon Fluxes

Large compilations of heterogeneous environmental observations are increasingly available as public databases, allowing researchers to test hypotheses across datasets. Statistical complexities arise when analyzing compiled data due to unbalanced spatial sampling, variable environmental context, mixed measurement techniques, and other reasons. Hierarchical Bayesian modeling is increasingly used in environmental science to describe these complexities, however few studies explicitly compare the utility of hierarchical Bayesian models to simpler and more commonly applied methods. Here we demonstrate the utility of the hierarchical Bayesian approach with application to a large compiled environmental dataset consisting of 5,741 marine vertical organic carbon flux observations from 407 sampling locations spanning eight biomes across the global ocean. We fit a global scale Bayesian hierarchical model that describes the vertical profile of organic carbon flux with depth. Profile parameters within a particular biome are assumed to share a common deviation from the global mean profile. Individual station-level parameters are then modeled as deviations from the common biome-level profile. The hierarchical approach is shown to have several benefits over simpler and more common data aggregation methods. First, the hierarchical approach avoids statistical complexities introduced due to unbalanced sampling and allows for flexible incorporation of spatial heterogeneitites in model parameters. Second, the hierarchical approach uses the whole dataset simultaneously to fit the model parameters which shares information across datasets and reduces the uncertainty up to 95% in individual profiles. Third, the Bayesian approach incorporates prior scientific information about model parameters; for example, the non-negativity of chemical concentrations or mass-balance, which we apply here. We explicitly quantify each of these properties in turn. We emphasize the generality of the hierarchical Bayesian approach for diverse environmental applications and its increasing feasibility for large datasets due to recent developments in Markov Chain Monte Carlo algorithms and easy-to-use high-level software implementations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Quantifying microbial control of soil organic matter dynamics at macrosystem scales

Soil organic matter (SOM) stocks, decomposition and persistence are largely the product of controls that act locally. Yet the controls are shaped and interact at multiple spatiotemporal scales, from which macrosystem patterns in SOM emerge. Theory on SOM turnover recognizes the resulting spatial and temporal conditionality in the effect sizes of controls that play out across macrosystems, and couples them through evolutionary and community assembly processes. For example, climate history shapes plant functional traits, which in turn interact with contemporary climate to influence SOM dynamics. Selection and assembly also shape the functional traits of soil decomposer communities, but it is less clear how in turn these traits influence temporal macrosystem patterns in SOM turnover. Here, we review evidence that establishes the expectation that selection and assembly should generate decomposer communities across macrosystems that have distinct functional effects on SOM dynamics. Representation of this knowledge in soil biogeochemical models affects the magnitude and direction of projected SOM responses under global change. Yet there is high uncertainty and low confidence in these projections. To address these issues, we make the case that a coordinated set of empirical practices are required which necessitate (1) greater use of statistical approaches in biogeochemistry that are suited to causative inference; (2) long-term, macrosystem-scale, observational and experimental networks to reveal conditionality in effect sizes, and embedded correlation, in controls on SOM turnover; and (3) use of multiple measurement grains to capture local- and macroscale variation in controls and outcomes, to avoid obscuring causative understanding through data aggregation. Here, when employed together, along with process-based models to synthesize knowledge and guide further empirical work, we believe these practices will rapidly advance understanding of microbial controls on SOM and improve carbon cycle projections that guide policies on climate adaptation and mitigation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

A 4×8-Gbps VCSEL array driver ASIC and integration with a custom array transmitter module for the LHC front-end transmission

This paper describes the design, fabrication and experiment results of a 4×8-Gbps Vertical-Cavity Surface-Emitting Laser (VCSEL) array driver ASIC with the adjustable active-shunt peaking technique and the novel balanced output structure under the Silicon-on-Sapphire (SOS) process, and a custom array optical transmitter module, featuring a compact size of 10 mm×15 mm×5.3 mm. Both the array driver ASIC and the module have been fully tested after integration as a complete parallel transmitter. Optical eye diagram of each channel passes the eye mask at 8 Gbps/ch with adjacent channel working simultaneously with a power consumption of 150 mW/ch. As a result, the optical transmission of Bit-Error Rate (BER) less than 10E-12 is achieved at an aggregated data rate of 4×8-Gbps.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2018

Evaluation of thermodynamic equations of state across chemistry and structure in the materials project

Thermodynamic equations of state (EOS) for crystalline solids describe material behaviors under changes in pressure, volume, entropy and temperature, making them fundamental to scientific research in a wide range of fields including geophysics, energy storage and development of novel materials. Despite over a century of theoretical development and experimental testing of energy–volume (E–V) EOS for solids, there is still a lack of consensus with regard to which equation is indeed optimal, as well as to what metric is most appropriate for making this judgment. In this study, several metrics were used to evaluate quality of fit for 8 different EOS across 87 elements and over 100 compounds which appear in the literature. Our findings do not indicate a clear “best” EOS, but we identify three which consistently perform well relative to the rest of the set. Furthermore, we find that for the aggregate data set, the RMSrD is not strongly correlated with the nature of the compound, e.g., whether it is a metal, insulator, or semiconductor, nor the bulk modulus for any of the EOS, indicating that a single equation can be used across a broad range of classes of materials.

36 MATERIALS SCIENCE↗
DOE OSTI2019

A global, empirical, harmonised dataset of soil organic carbon changes under perennial crops

A global, unified dataset on Soil Organic Carbon (SOC) changes under perennial crops has not existed till now. We present a global, harmonised database on SOC change resulting from perennial crop cultivation. It contains information about 1605 paired-comparison empirical values (some of which are aggregated data) from 180 different peer-reviewed studies, 709 sites, on 58 different perennial crop types, from 32 countries in temperate, tropical and boreal areas; including species used for food, bioenergy and bio-products. The database also contains information on climate, soil characteristics, management and topography. This is the first such global compilation and will act as a baseline for SOC changes in perennial crops. It will be key to supporting global modelling of land use and carbon cycle feedbacks, and supporting agricultural policy development.

Science & Technology - Other Topics↗
DOE OSTI2023

CreelCat, a Catalog of United States Inland Creel and Angler Survey Data

The United States Inland Creel and Angler Survey Catalog (CreelCat) contains a national compilation of angler and creel survey data collected by natural resource management agencies across the United States (including Washington, D.C. and Puerto Rico). These surveys are used to help inform the management of recreational fisheries, by collecting information about anglers including what they are catching and harvesting, the amount of effort they expend, their angling preferences, and demographic information. As of May 1, 2023, CreelCat houses over 14,729 surveys from 33 states, Puerto Rico, and Washington, D.C., comprising 235 data fields across 8 tables. These tables contain 235,015 records of fish catch and harvest metrics, 27,250 angler preference metrics, 14,729 records of survey characteristics, 13,576 records of effort metrics, and 409 records of angler demographics. Though individual creel surveys are often deployed to meet local science and management objectives, creel data aggregated across jurisdictions has the potential to address larger scale research and management needs.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Distributional Equity in the Employment and Wage Impacts of Energy Transitions

We use restricted-access, geocoded data on the near-universe of workers in 23 US states to quantify the impact of wind energy development on local earnings and employment, by race, ethnicity, sex, and educational attainment. We find significant impacts that persist for several years beyond the project construction phase. Our estimates are larger than those from previous studies but still small relative to typical economic multipliers for fiscal spending or investment in other industries. We find the largest percentage increases for black workers and workers who either do not have a high school diploma or who have a college degree. We also find the economic gains for men to be much larger than those for women. Finally, we find estimates from data aggregated to the county level to be significantly lower than our worker-level estimates. We suggest a number of areas for further study building off the justice implications of our findings.

employment↗
DOE OSTI2016

RNAcentral: A comprehensive database of non-coding RNA sequences

RNAcentral is a database of non-coding RNA (ncRNA) sequences that aggregates data from specialised ncRNA resources and provides a single entry point for accessing ncRNA sequences of all ncRNA types from all organisms. Since its launch in 2014, RNAcentral has integrated twelve new resources, taking the total number of collaborating database to 22, and began importing new types of data, such as modified nucleotides from MODOMICS and PDB. We created new species-specific identifiers that refer to unique RNA sequences within a context of single species. Furthermore, the website has been subject to continuous improvements focusing on text and sequence similarity searches as well as genome browsing functionality.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

A Learning-based Power Management Method for Networked Microgrids Under Incomplete Information

In this paper, we present a model-free reinforcement learning (RL)-based technique to solve the power management problem of networked microgrids (MGs), when the utility has only limited or no detailed knowledge of the MG asset models.The proposed method consists of two levels: at the first level, using an approximate RL strategy and aggregated data, a utility agent determines the locational price signals to maximize its profit under incomplete information on the MGs’ private data behind their points of common coupling (PCC). The proposed RL strategy enables the utility to adapt to changing system conditions and learn from previous experiences, while respecting the data ownership of the MGs. At the second level, the MGs receive the price signals to dispatch their generation/storage assets individually over look-ahead decision time window, while considering power flow constraints. The performance of the proposed RL technique has been verified under realistic operational scenarios.

Zhang, Qianzhi↗
DOE OSTI2014

Poster — Thur Eve — 74: Distributed, asynchronous, reactive dosimetric and outcomes analysis using DICOMautomaton

Many have speculated about the future of computational technology in clinical radiation oncology. It has been advocated that the next generation of computational infrastructure will improve on the current generation by incorporating richer aspects of automation, more heavily and seamlessly featuring distributed and parallel computation, and providing more flexibility toward aggregate data analysis. In this report we describe how a recently created — but currently existing — analysis framework (DICOMautomaton) incorporates these aspects. DICOMautomaton supports a variety of use cases but is especially suited for dosimetric outcomes correlation analysis, investigation and comparison of radiotherapy treatment efficacy, and dose-volume computation. We describe: how it overcomes computational bottlenecks by distributing workload across a network of machines; how modern, asynchronous computational techniques are used to reduce blocking and avoid unnecessary computation; and how issues of out-of-date data are addressed using reactive programming techniques and data dependency chains. We describe internal architecture of the software and give a detailed demonstration of how DICOMautomaton could be used to search for correlations between dosimetric and outcomes data.

60 APPLIED LIFE SCIENCES↗