Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Spatial and Temporal Characterization of Activity in Public Space, 2019–2020

The data reported here characterize spatial and temporal variation in the ratio of short-to-long-duration visits in public places (i.e., points of interest) in the United States for each week between January 2019 and December 2020. The underlying data on anonymized and aggregated foot traffic to public places is curated by SafeGraph, a geospatial data provider. In this work, we report the estimated number and duration of “short” (i.e., <4 hours) and “long” (i.e., >4 hours) visits to public places at the US census block group level. Long visits are shown to be a good proxy for workers based on formal economic data. We propose that short visits are more likely to represent nonobligate activities: people visiting a public place for leisure, shopping, entertainment, or civic or cultural engagement. Our work constructs a ratio of short to long visits, which can be used to inform population estimates for nonworker use of public space. These data may be useful for understanding how people’s use of public space has changed during the COVID-19 pandemic and, more generally, for understanding activity patterns in public.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2015

A STACKED ANALYSIS OF 115 PULSARS OBSERVED BY THE FERMI LAT

Due to the low gamma-ray fluxes from pulsars above 50 GeV and the small collecting area of space-based telescopes, the gamma-ray emission discovered by the Fermi Large Area Telescope (LAT) in ∼150 pulsars is largely unexplored at these energies. In this regime, the uncertainties on the spectral data points and/or the constraints from upper limits are not sufficient to provide robust tests of competing emission models in individual pulsars. The discovery of power-law-type emission from the Crab pulsar at energies exceeding 100 GeV provides a compelling justification for exploration of other pulsars at these energies. We applied the method of aperture photometry to measure pulsar emission spectra from Fermi-LAT data and present a stacked analysis of 115 pulsars selected from the Second Fermi-LAT catalog of gamma-ray pulsars. This analysis, which uses an average of ∼4.2 yr of data per pulsar, aggregates low-level emission which cannot be resolved in individual objects but can be detected in an ensemble. We find no significant stacked excess at energies above 50 GeV. An upper limit of 30% of the Crab pulsar level is found for the average flux from 115 pulsars in the 100–177 GeV energy range at the 95% confidence level. Stacked searches exclusive to the young pulsar sample, the millisecond pulsar sample, and several other promising sub-samples also return no significant excesses above 50 GeV.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

MARIAH PCAP data for Validation Demonstration

This dataset holds simulated PCAP (packet capture) data from the SCEPTRE validation demonstration model as a set of pairwise communications between devices via specific protocols. All connections should be assumed to be symmetric, as this data is an aggregation of the true PCAP. A mapping is also provided associating each IP address with its true device type.

cyber-physical system↗
DOE OSTI2023

Biogeophysical Effects of Land-Use and Land-Cover Change Not Detectable in Warmest Month

Land-use and land-cover changes (hereafter simply “land use”) alter climates biogeophysically by affecting surface fluxes of energy and water. Yet, near-surface temperature responses to land use across observational versus model-based studies and spatial-temporal scales can be inconsistent. Here we assess the prevalence of the historical land use signal of daily maximum temperatures averaged over the warmest month of the year (t LU ) using regularized optimal fingerprinting for detection and attribution. We use observations from the Climatic Research Unit and Berkeley Earth alongside historical simulations with and without land use from phase 6 of the Coupled Model Intercomparison Project to reconstruct an experiment representing the effects of land use on climate. To assess the signal of land use at spatially resolved continental and global scales, we aggregate all input data across reference regions and continents, respectively. At both scales, land use does not comprise a significantly detectable set of forcings for two of four Earth system models and their multimodel mean. Furthermore, using a principal component analysis, we find that t LU is mostly composed of the nonlocal effects of land use rather than its local effects. These findings show that, at scales relevant for climate attribution, uncertainties in Earth system model representations of land use are too high relative to the effects of internal variability to confidently assess land use.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Adoption of AI in the Utility T&D Sector: Use Cases, Consequence, Assessment and Benefits

Digital transformation and utilization of artificial intelligence (AI) in the electric grid are fundamentally changing the industry’s approach to common problems and enabling a broader paradigm shift in grid planning and operations. The change in approach is circularly both enabling and driving modernization, with load growth and reliable management of data center and AI infrastructure shifting away from planning approaches with relatively predictable behaviors and toward a mix of consumer and industrial choices that surpass human cognitive abilities to process. This movement has potential to condition humans to not understand the system on which the AI depends, while requiring it for development of the necessary infrastructure. Approaches which would address most likely grid conditions and events, such as faults, aging of equipment, and weather, now must also account for large loads which shift not based upon weather or time of day, but the computational load. Quantifying computational load is independent of the traditional grid forecasting variables, where a data center’s aggregate load is determined by user and AI system behavior and decoupled from normal grid planning and operations. AI is both the cause and solution for these challenges, with new grid planning tools integrating massive amounts of decisions into frameworks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Comparison of Radiosonde Datasets: SondeHub and Integrated Global Radiosonde Archive

SondeHub aggregates radiosonde telemetry data uploaded from community-run radiosonde receiver stations. This radiosonde telemetry dataset is open-source, available to anyone through Amazon S3. There are also other public radiosonde datasets such as National Centers for Environmental Information (NCEI)’s Integrated Global Radiosonde Archive (IGRA). While there are many similarities between the two datasets, there are many differences as well due to the nature of the two datasets: one is community-run, while the other is managed by a government agency. This report presents the result of analyzing and comparing the two datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

3D reconstruction identifies loci linked to variation in angle of individual sorghum leaves

Selection for yield at high planting density has reshaped the leaf canopy of maize, improving photosynthetic productivity in high density settings. Further optimization of canopy architecture may be possible. However, measuring leaf angles, the widely studied component trait of leaf canopy architecture, by hand is a labor and time intensive process. Here, we use multiple, calibrated, 2D images to reconstruct the 3D geometry of individual sorghum plants using a voxel carving based algorithm. Automatic skeletonization and segmentation of these 3D geometries enable quantification of the angle of each leaf for each plant. The resulting measurements are both heritable and correlated with manually collected leaf angles. This automated and scaleable reconstruction approach was employed to measure leaf-by-leaf angles for a population of 366 sorghum plants at multiple time points, resulting in 971 successful reconstructions and 3,376 leaf angle measurements from individual leaves. A genome wide association study conducted using aggregated leaf angle data identified a known large effect leaf angle gene, several previously identified leaf angle QTL from a sorghum NAM population, and novel signals. Genome wide association studies conducted separately for three individual sorghum leaves identified a number of the same signals, a previously unreported signal shared across multiple leaves, and signals near the sorghum orthologs of two maize genes known to influence leaf angle. Automated measurement of individual leaves and mapping variants associated with leaf angle reduce the barriers to engineering ideal canopy architectures in sorghum and other grain crops.

3D reconstruction↗
DOE OSTI2021

Evaluation of Stripping Resistance of Organoclay-Modified Asphalt Binder and Aggregate Systems Using an Optical Contact Angle Analyzer

Organoclays, often referred to as nanoclays (NCs), are organically modified phyllosilicates. They are derived from naturally occurring clay minerals and possess unique characteristics in improving the physical and mechanical properties of polymers and rubbers. Recently, NCs have drawn interest in the modification of asphalt binders as micro-scale fillers. A few recent studies have revealed that the addition of NCs in asphalt binders improved binders’ stiffness, aging characteristics, and fatigue resistance. This study aims to evaluate the changes in moisture resistance of different NC-modified asphalt binders through the surface free energy (SFE) technique. This technique estimates SFE properties of binders and aggregates from static contact angle (SCA) data measured from an optical contact angle analyzer (OCA). Further, this study examines the changes in chemical compositions (functional groups) of binders due to the addition of selected NCs by using the Fourier Transformation Infrared (FTIR) technique. To this end, a commonly used performance grade (PG) binder (PG 64-22) modified with four different types (shape and size) of NCs, namely, Cloisite® 15 (C-15), Cloisite® 20 (C-20), Cloisite® Na+ (C-Na), and Cloisite® Ca++ (C-Ca) were evaluated in this study. The SFE data of five different types of aggregates from Oklahoma, namely, Davis Limestone (DL), Snyder Granite (SG), Dolese-Cooperton Limestone (DCL), Hanson-Davis Rhyolite (HDR), and Martin-Marietta-Mill-Creek Granite (MMMG), were used to perform their compatibility with the aforementioned NC-modified binders. The FTIR spectra revealed the presence of various alcohols, ethers, and esters in NC-modified asphalt binders. The moisture susceptibility analysis shows that the addition of NC results in an increase of SFE and cohesive energy of an asphalt binder, which are desired for improved moisture resistance. The addition of NC also shows improved compatibility in cases of all aggregates. Among the five aggregates, MMMCG showed the highest compatibility with all binders, followed by DCL, HDR, DL, and SG. In regard to NCs, the C-15 sample showed the highest compatibility followed by C-20, C-Na, and C-Ca.

Hossain, Zahid↗
DOE OSTI2016

Global‐view coefficients: a data management solution for parallel quantum Monte Carlo applications

Summary Quantum Monte Carlo (QMC) applications perform simulation with respect to an initial state of the quantum mechanical system, which is often captured by using a cubic B‐spline basis. This representation is stored as a read‐only table of coefficients and accesses to the table are generated at random as part of the Monte Carlo simulation. Current QMC applications, such as QWalk and QMCPACK, replicate this table at every process or node, which limits scalability because increasing the number of processors does not enable larger systems to be run. We present a partitioned global address space approach to transparently managing this data using Global Arrays in a manner that allows the memory of multiple nodes to be aggregated. We develop an automated data management system that significantly reduces communication overheads, enabling new capabilities for QMC codes. Experimental results with QWalk and QMCPACK demonstrate the effectiveness of the data management system. Copyright © 2016 John Wiley & Sons, Ltd.

Niu, Qingpeng↗
DOE OSTI2014

WE-G-9A-01: Radiation Oncology Outcomes Informatics

The construction of databases and support software to enable routine and systematic aggregation, analysis and reporting of patient outcomes data is emerging as an important area. “How have results for our patients been affected by the improvements we have made in our practice and in the technologies we use?” To answer this type of fundamental question about the overall pattern of efficacy observed, it is necessary to systematically gather and analyze data on all patients treated within a clinic. Clinical trials answer, in great depth and detail, questions about outcomes for the subsets of patients enrolled in a given trial. However, routine aggregation and analysis of key treatment parameter data and outcomes information for all patients is necessary to recognize emergent patterns that would be of interest from a public health or practice perspective and could better inform design of clinical trials or the evolution of best practice principals. To address these questions, Radiation Oncology outcomes databases need to be constructed to enable combination essential data from a broad group of data types including: diagnosis and staging, dose volume histogram metrics, patient reported outcomes, toxicity metrics, performance status, treatment plan parameters, demographics, DICOM data and demographics. Developing viable solutions to automate aggregation and analysis of this data requires multidisciplinary efforts to define nomenclatures, modify clinical processes and develop software and database tools requires detailed understanding of both clinical and technical issues. This session will cover the developing area of Radiation Oncology Outcomes Informatics. Learning Objectives: Audience will be able to speak to the technical requirements (software, database, web services) which must be considered in designing an outcomes database. Audience will be able to understand the content and the role of patient reported outcomes as compared to traditional toxicity measures. Audience will be understand approaches, clinical process changes, consensus building efforts and standardizations which must be addressed to succeed in a multi-disciplinary effort to aggregate data for all patients. Audience will be able to discuss technical and process issues related to pooling data among institutions in the context of collaborative studies among the presenting institutions.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Combined Transmission and Distribution Test System to Study High Penetration of Distributed Solar Generation

Transmission and distribution (T&D) standard test systems are currently defined separately, and are critical for researchers and engineers to test new solutions. Growing installation of energy resources in distribution and sub-transmission systems is creating higher interactions between T&D systems. This paper proposes a standard test system to study T&D interactions for high penetration of solar photovoltaic (PV) generation. The proposed test system can be used to study and propose solutions to problems such as reverse power flows and voltage control affected by PV variability. The proposed test system combines the transmission IEEE 118 bus system, with the distribution IEEE 123-node feeder. The original test systems are significantly enhanced by adding high penetration of solar PV generation with corresponding full-year minute-resolution chronological data. Conventional generation chronology is achieved by adding dispatch and unit commitment data for AC optimal power flow solutions. And chronological load data is added in both aggregated and disaggregated fashions. Sample applications of the data are provided highlighting voltage control challenges, and the proposed system is made available openly online.

Samaan, Nader A.↗
DOE OSTI2021

NeuralCubes: Deep Representations for Visual Data Exploration

Visual exploration of large multi-dimensional datasets has seen tremendous progress in recent years, allowing users to express rich data queries that produce informative visual summaries, all in real time. Techniques based on data cubes are some of the most promising approaches. However, these techniques usually require a large memory footprint for large datasets. To tackle this problem, we present NeuralCubes: neural networks that predict results for aggregate queries, similar to data cubes. NeuralCubes learns a function that takes as input a given query, for instance, a geographic region and temporal interval, and outputs the result of the query. The learned function serves as a real-time, low-memory approximator for aggregation queries. Our models are small enough to be sent to the client side (e.g. the web browser for a web-based application) for evaluation, enabling data exploration of large datasets without database/network connection. Here, we demonstrate the effectiveness of NeuralCubes through extensive experiments on a variety of datasets and discuss how NeuralCubes opens up opportunities for new types of visualization and interaction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Massively scalable Kerr comb-driven silicon photonic link

Abstract The growth of computing needs for artificial intelligence and machine learning is critically challenging data communications in today’s data-centre systems. Data movement, dominated by energy costs and limited ‘chip-escape’ bandwidth densities, is perhaps the singular factor determining the scalability of future systems. Using light to send information between compute nodes in such systems can dramatically increase the available bandwidth while simultaneously decreasing energy consumption. Through wavelength-division multiplexing with chip-based microresonator Kerr frequency combs, independent information channels can be encoded onto many distinct colours of light in the same optical fibre for massively parallel data transmission with low energy. Although previous high-bandwidth demonstrations have relied on benchtop equipment for filtering and modulating Kerr comb wavelength channels, data-centre interconnects require a compact on-chip form factor for these operations. Here we demonstrate a massively scalable chip-based silicon photonic data link using a Kerr comb source enabled by a new link architecture and experimentally show aggregate single-fibre data transmission of 512 Gb s −1 across 32 independent wavelength channels. The demonstrated architecture is fundamentally scalable to hundreds of wavelength channels, enabling massively parallel terabit-scale optical interconnects for future green hyperscale data centres.

Rizzo, Anthony (ORCID:000000034752797X)↗
DOE OSTI2023

Data-driven insights into the fluorescence of asphaltene aggregates using extended Frenkel exciton model

The origin of the fluorescence redshift during asphaltene aggregation remains debated due to the great diversity of asphaltene molecules, while the extended Frenkel exciton model provided a theoretical framework for studying multi-chromophore systems, such as asphaltene aggregates. We investigated the fluorescence energy of hundreds of asphaltene dimers based on 133 experimental revealed asphaltene monomer structures. Our result shows that the dimer's fluorescence energy is always lower than both of its monomers regardless of its intermolecular conformation, with its redshift dominated by superexchange. The dimer oscillator strength predominantly depends on the monomer with the lower fluorescence energy, and the involvement of charge-transfer states and the cancellation between monomer transition dipole moments suppressed the fluorescence especially when two monomers have similar fluorescence energy. The above-mentioned findings hold for all asphaltene dimers despite their diversity, which offers a theoretical interpretation for comprehending the relationship between asphaltene aggregation and its fluorescence variation. Furthermore, this work provides theoretical guidance to other research areas related to organic planar conjugated systems with multiple chromophores.

Chemistry↗
DOE OSTI2018

Method and apparatus for selective and power-aware memory error protection and memory management

A method for providing selective memory error protection responsive to a predictable failure notification associated with at least one portion of a memory in a computing system includes: obtaining an active error correcting code (ECC) configuration corresponding to the portion of the memory; determining whether the active ECC configuration is sufficient to correct at least one error in the portion of the memory affected by the predictable failure notification; when the active ECC configuration is insufficient to correct the error, determining whether data corruption can be tolerated by an application running on the computing system; when data corruption cannot be tolerated by the application, determining whether a stronger ECC level is available and, if a stronger ECC level is available, increasing a strength of the active ECC configuration; and when data corruption can be tolerated, performing page reassignment and aggregation of non-critical data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Presentation layer of CMS Online Monitoring System

The Compact Muon Solenoid (CMS) is one of the experiments at the CERN Large Hadron Collider (LHC). The CMS Online Monitoring system (OMS) is an upgrade and successor to the CMS Web-Based Monitoring (WBM)system, which is an essential tool for shift crew members, detector subsystem experts, operations coordinators, and those performing physics analyses. The CMS OMS is divided into aggregation and presentation layers. Communication between layers uses RESTful JSON:API compliant requests. The aggregation layer is responsible for collecting data from heterogeneous sources, storage of transformed and pre-calculated (aggregated) values and exposure of data via the RESTful API. The presentation layer displays detector information via a modern, user-friendly and customizable web interface. The CMS OMS user interface is composed of a set of cutting-edge software frameworks and tools to display non-event data to any authenticated CMS user worldwide. The web interface tree-like component structure comprises (top-down): workspaces, folders, pages, controllers and portlets. A clear hierarchy gives the required flexibility and control for content organization. Each bottom element instantiates a portlet and is a reusable component that displays a single aspect of data, like a table, a plot, an article, etc. Pages consist of multiple different portlets and can be customized at runtime by using a drag-and-drop technique. This is how a single page can easily include information from multiple online sources. Different pages give access to a summary of the current status of the experiment, as well as convenient access to historical data. This paper describes the CMS OMS architecture, core concepts and technologies of the presentation layer.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

CORE: A Global Aggregation Service for Open Access Papers

This paper introduces CORE, a widely used scholarly service, which provides access to the world’s largest collection of open access research publications, acquired from a global network of repositories and journals. CORE was created with the goal of enabling text and data mining of scientific literature and thus supporting scientific discovery, but it is now used in a wide range of use cases within higher education, industry, not-for-profit organisations, as well as by the general public. Through the provided services, CORE powers innovative use cases, such as plagiarism detection, in market-leading third-party organisations. CORE has played a pivotal role in the global move towards universal open access by making scientific knowledge more easily and freely discoverable. In this paper, we describe CORE’s continuously growing dataset and the motivation behind its creation, present the challenges associated with systematically gathering research papers from thousands of data providers worldwide at scale, and introduce the novel solutions that were developed to overcome these challenges. The paper then provides an in-depth discussion of the services and tools built on top of the aggregated data and finally examines several use cases that have leveraged the CORE dataset and services.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗