Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Evaluation of Stripping Resistance of Organoclay-Modified Asphalt Binder and Aggregate Systems Using an Optical Contact Angle Analyzer

Organoclays, often referred to as nanoclays (NCs), are organically modified phyllosilicates. They are derived from naturally occurring clay minerals and possess unique characteristics in improving the physical and mechanical properties of polymers and rubbers. Recently, NCs have drawn interest in the modification of asphalt binders as micro-scale fillers. A few recent studies have revealed that the addition of NCs in asphalt binders improved binders’ stiffness, aging characteristics, and fatigue resistance. This study aims to evaluate the changes in moisture resistance of different NC-modified asphalt binders through the surface free energy (SFE) technique. This technique estimates SFE properties of binders and aggregates from static contact angle (SCA) data measured from an optical contact angle analyzer (OCA). Further, this study examines the changes in chemical compositions (functional groups) of binders due to the addition of selected NCs by using the Fourier Transformation Infrared (FTIR) technique. To this end, a commonly used performance grade (PG) binder (PG 64-22) modified with four different types (shape and size) of NCs, namely, Cloisite® 15 (C-15), Cloisite® 20 (C-20), Cloisite® Na+ (C-Na), and Cloisite® Ca++ (C-Ca) were evaluated in this study. The SFE data of five different types of aggregates from Oklahoma, namely, Davis Limestone (DL), Snyder Granite (SG), Dolese-Cooperton Limestone (DCL), Hanson-Davis Rhyolite (HDR), and Martin-Marietta-Mill-Creek Granite (MMMG), were used to perform their compatibility with the aforementioned NC-modified binders. The FTIR spectra revealed the presence of various alcohols, ethers, and esters in NC-modified asphalt binders. The moisture susceptibility analysis shows that the addition of NC results in an increase of SFE and cohesive energy of an asphalt binder, which are desired for improved moisture resistance. The addition of NC also shows improved compatibility in cases of all aggregates. Among the five aggregates, MMMCG showed the highest compatibility with all binders, followed by DCL, HDR, DL, and SG. In regard to NCs, the C-15 sample showed the highest compatibility followed by C-20, C-Na, and C-Ca.

Hossain, Zahid↗
DOE OSTI2016

Global‐view coefficients: a data management solution for parallel quantum Monte Carlo applications

Summary Quantum Monte Carlo (QMC) applications perform simulation with respect to an initial state of the quantum mechanical system, which is often captured by using a cubic B‐spline basis. This representation is stored as a read‐only table of coefficients and accesses to the table are generated at random as part of the Monte Carlo simulation. Current QMC applications, such as QWalk and QMCPACK, replicate this table at every process or node, which limits scalability because increasing the number of processors does not enable larger systems to be run. We present a partitioned global address space approach to transparently managing this data using Global Arrays in a manner that allows the memory of multiple nodes to be aggregated. We develop an automated data management system that significantly reduces communication overheads, enabling new capabilities for QMC codes. Experimental results with QWalk and QMCPACK demonstrate the effectiveness of the data management system. Copyright © 2016 John Wiley & Sons, Ltd.

Niu, Qingpeng↗
DOE OSTI2014

WE-G-9A-01: Radiation Oncology Outcomes Informatics

The construction of databases and support software to enable routine and systematic aggregation, analysis and reporting of patient outcomes data is emerging as an important area. “How have results for our patients been affected by the improvements we have made in our practice and in the technologies we use?” To answer this type of fundamental question about the overall pattern of efficacy observed, it is necessary to systematically gather and analyze data on all patients treated within a clinic. Clinical trials answer, in great depth and detail, questions about outcomes for the subsets of patients enrolled in a given trial. However, routine aggregation and analysis of key treatment parameter data and outcomes information for all patients is necessary to recognize emergent patterns that would be of interest from a public health or practice perspective and could better inform design of clinical trials or the evolution of best practice principals. To address these questions, Radiation Oncology outcomes databases need to be constructed to enable combination essential data from a broad group of data types including: diagnosis and staging, dose volume histogram metrics, patient reported outcomes, toxicity metrics, performance status, treatment plan parameters, demographics, DICOM data and demographics. Developing viable solutions to automate aggregation and analysis of this data requires multidisciplinary efforts to define nomenclatures, modify clinical processes and develop software and database tools requires detailed understanding of both clinical and technical issues. This session will cover the developing area of Radiation Oncology Outcomes Informatics. Learning Objectives: Audience will be able to speak to the technical requirements (software, database, web services) which must be considered in designing an outcomes database. Audience will be able to understand the content and the role of patient reported outcomes as compared to traditional toxicity measures. Audience will be understand approaches, clinical process changes, consensus building efforts and standardizations which must be addressed to succeed in a multi-disciplinary effort to aggregate data for all patients. Audience will be able to discuss technical and process issues related to pooling data among institutions in the context of collaborative studies among the presenting institutions.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Combined Transmission and Distribution Test System to Study High Penetration of Distributed Solar Generation

Transmission and distribution (T&D) standard test systems are currently defined separately, and are critical for researchers and engineers to test new solutions. Growing installation of energy resources in distribution and sub-transmission systems is creating higher interactions between T&D systems. This paper proposes a standard test system to study T&D interactions for high penetration of solar photovoltaic (PV) generation. The proposed test system can be used to study and propose solutions to problems such as reverse power flows and voltage control affected by PV variability. The proposed test system combines the transmission IEEE 118 bus system, with the distribution IEEE 123-node feeder. The original test systems are significantly enhanced by adding high penetration of solar PV generation with corresponding full-year minute-resolution chronological data. Conventional generation chronology is achieved by adding dispatch and unit commitment data for AC optimal power flow solutions. And chronological load data is added in both aggregated and disaggregated fashions. Sample applications of the data are provided highlighting voltage control challenges, and the proposed system is made available openly online.

Samaan, Nader A.↗
DOE OSTI2021

NeuralCubes: Deep Representations for Visual Data Exploration

Visual exploration of large multi-dimensional datasets has seen tremendous progress in recent years, allowing users to express rich data queries that produce informative visual summaries, all in real time. Techniques based on data cubes are some of the most promising approaches. However, these techniques usually require a large memory footprint for large datasets. To tackle this problem, we present NeuralCubes: neural networks that predict results for aggregate queries, similar to data cubes. NeuralCubes learns a function that takes as input a given query, for instance, a geographic region and temporal interval, and outputs the result of the query. The learned function serves as a real-time, low-memory approximator for aggregation queries. Our models are small enough to be sent to the client side (e.g. the web browser for a web-based application) for evaluation, enabling data exploration of large datasets without database/network connection. Here, we demonstrate the effectiveness of NeuralCubes through extensive experiments on a variety of datasets and discuss how NeuralCubes opens up opportunities for new types of visualization and interaction.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2012

Revamping Spacecraft Operational Intelligence with Splunk

So what is Splunk? Instead of giving the technical details, which you can find online, I'll tell you what it did for me. Splunk slapped everything into one place, with one uniform format, and gave me the ability to forget about all these annoying details of where it is, how to parse it, and all that. Instead, I only need to interact with Splunk to find the data I need. This sounds simple and obvious, but it's surprising what you can do once you all of your data is indexed in one place. By having your data organized, querying becomes much easier. Let's say that I want to search telemetry for a sensor_name gtemp_1 h and to return all data that is at most five minutes old. And because Splunk can hook into a real ]time stream, this data will always be up-to-date. Extending the previous example, I can now aggregate all types of data into one view based in time. In this picture, I've got transaction logs, telemetry, and downlinked files all in one page, organized by time. Even though the raw data looks completely than this, I've defined interfaces that transform it into this uniform format. This gives me a more complete picture for the question what was the spacecraft doing at this particular time? And because querying data is simple, I can start with a big block of data and whiddle it down to what I need, rather than hunting around for the individual pieces of data that I need. When we have all the data we need, we can begin widdling down the data with Splunk's Unix-like search syntax. These three examples highlights my trial-and-error attempts to find large temperature changes. I begin by showing the first 5 temperatures, only to find that they're sorted chronologically, rather than from highest temperatures to lowest temperatures. The next line shows sorting temperatures by their values, but I find that that fs not really what I want either. I want to know the delta temperatures between readings. Looking through Splunk's user manual, I find the delta function, which lets me dynamically generate new information to use in my query. With that extra piece of information, I can now return only the telemetry readings where the temperature changed by at least 10. One other useful feature I'll mention is that all of these queries can be run through Splunk's API. So any scripting language you can think of can plug right in and make these queries. This gives us the ability to build a lot of new tools.

operational intelligence↗
NASA NTRSDate not supplied

Global Variability in Sonic Boom Exposure due to Macroscopic Effects

Supersonic flight over land has been prohibited since 1973 due to the loudness of sonic booms. NASA is building the X-59 aircraft as part of its Quesst mission to demonstrate low-loudness shaped sonic booms, or “sonic thumps.” The Quesst mission will gather human perception data via a series of community noise surveys across the USA. The noise dose and perceptual response data will be provided to the International Civil Aviation Organization (ICAO) and the Federal Aviation Administration for use in determining potential future supersonic aircraft noise certification standards, effectively changing the prohibition from a speed limit to a noise limit. These noise regulations must be globally effective, as long travel distances see the largest benefit to supersonic flight. The state of the atmosphere through which a sonic boom travels affects the size of the region exposed to sound, the “carpet width” (CW), as well as the loudness. The focus of this dissertation is to understand and quantify the expected loudness and CW of sonic booms due to the macroscopic atmospheric effects around the world. A pair of large-scale propagation simulation studies were conducted using the NASA PCBoom code to compare predicted sonic boom loudness and CW statistics first across the USA and then across the world. For the USA study, near-field data of the X-59 in steady cruise was propagated at 4 cardinal headings at 138 locations through 5 years of Climate Forecast System Version 2 (CFSv2) atmospheric profiles. Results of a bootstrap forest predictor screening model indicated the importance of climate zone, latitude, ground elevation, season, and heading. It also noted the unimportance of time of day for predicting loudness and CW. The data is visualized in aggregate, and then broken out geographically, by season and heading, and by climate zone. Multiple linear regression models were fit to the data from the 138 locations so that estimates of the loudness and CW can be produced anywhere in the US. The results can aid in planning when and where to fly the X-59. For the global study, near-field data from three aircraft, the X-59 in a quiet and loud configuration, B-58, and Concorde, were propagated at four cardinal headings through data from three atmospheric models, the CFSv2, the Global Forecast System (GFS), and the ECMWF Reanalysis Version 5 (ERA5), at 100 global locations over 1 year. Results of a bootstrap forest predictor screening model indicated the importance of climate zone, ground elevation, season, and heading. Similar to the US study, the model indicated time of day was not an important predictor. The model also indicated that choice of weather model was not important, so the atmospheric model data are effectively interchangeable. The ERA5 model was chosen for use in an extension of the study to include 18 additional locations to ensure sampling of every climate zone. Loudness and CW results are shown in aggregate, and split geographically and by heading, season, and climate. Multiple linear regression models were fit to the data from the 118 locations so that estimates of loudness and CW can be produced around the world. N-waves and shaped booms did not have the same global variability. Koppen-Geiger climate zones were used as the climate zone definition for the global study. These are available as present-day and future climate projections. Making use of the multiple linear regression models, the future climate zones were input to estimate the effect of the changing climate on sonic boom loudness and CW. Results indicate that a changing climate would have little impact on the effectiveness of noise regulations.

X-59↗
NASA NTRS2008

Defect Categorization: Making Use of a Decade of Widely Varying Historical Data

This paper describes our experience in aggregating a number of historical datasets containing inspection defect data using different categorizing schemes. Our goal was to make use of the historical data by creating models to guide future development projects. We describe our approach to reconciling the different choices used in the historical datasets to categorize defects, and the challenges we faced. We also present a set of recommendations for others involved in classifying defects.

Shull, Forrest↗
DOE OSTI2023

Massively scalable Kerr comb-driven silicon photonic link

Abstract The growth of computing needs for artificial intelligence and machine learning is critically challenging data communications in today’s data-centre systems. Data movement, dominated by energy costs and limited ‘chip-escape’ bandwidth densities, is perhaps the singular factor determining the scalability of future systems. Using light to send information between compute nodes in such systems can dramatically increase the available bandwidth while simultaneously decreasing energy consumption. Through wavelength-division multiplexing with chip-based microresonator Kerr frequency combs, independent information channels can be encoded onto many distinct colours of light in the same optical fibre for massively parallel data transmission with low energy. Although previous high-bandwidth demonstrations have relied on benchtop equipment for filtering and modulating Kerr comb wavelength channels, data-centre interconnects require a compact on-chip form factor for these operations. Here we demonstrate a massively scalable chip-based silicon photonic data link using a Kerr comb source enabled by a new link architecture and experimentally show aggregate single-fibre data transmission of 512 Gb s −1 across 32 independent wavelength channels. The demonstrated architecture is fundamentally scalable to hundreds of wavelength channels, enabling massively parallel terabit-scale optical interconnects for future green hyperscale data centres.

Rizzo, Anthony (ORCID:000000034752797X)↗
DOE OSTI2023

Data-driven insights into the fluorescence of asphaltene aggregates using extended Frenkel exciton model

The origin of the fluorescence redshift during asphaltene aggregation remains debated due to the great diversity of asphaltene molecules, while the extended Frenkel exciton model provided a theoretical framework for studying multi-chromophore systems, such as asphaltene aggregates. We investigated the fluorescence energy of hundreds of asphaltene dimers based on 133 experimental revealed asphaltene monomer structures. Our result shows that the dimer's fluorescence energy is always lower than both of its monomers regardless of its intermolecular conformation, with its redshift dominated by superexchange. The dimer oscillator strength predominantly depends on the monomer with the lower fluorescence energy, and the involvement of charge-transfer states and the cancellation between monomer transition dipole moments suppressed the fluorescence especially when two monomers have similar fluorescence energy. The above-mentioned findings hold for all asphaltene dimers despite their diversity, which offers a theoretical interpretation for comprehending the relationship between asphaltene aggregation and its fluorescence variation. Furthermore, this work provides theoretical guidance to other research areas related to organic planar conjugated systems with multiple chromophores.

Chemistry↗
DOE OSTI2018

Method and apparatus for selective and power-aware memory error protection and memory management

A method for providing selective memory error protection responsive to a predictable failure notification associated with at least one portion of a memory in a computing system includes: obtaining an active error correcting code (ECC) configuration corresponding to the portion of the memory; determining whether the active ECC configuration is sufficient to correct at least one error in the portion of the memory affected by the predictable failure notification; when the active ECC configuration is insufficient to correct the error, determining whether data corruption can be tolerated by an application running on the computing system; when data corruption cannot be tolerated by the application, determining whether a stronger ECC level is available and, if a stronger ECC level is available, increasing a strength of the active ECC configuration; and when data corruption can be tolerated, performing page reassignment and aggregation of non-critical data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1977

Ground truth management system to support multispectral scanner /MSS/ digital analysis

A computerized geographic information system for management of ground truth has been designed and implemented to relate MSS classification results to in situ observations. The ground truth system transforms, generalizes and rectifies ground observations to conform to the pixel size and shape of high resolution MSS aircraft data. These observations can then be aggregated for comparison to lower resolution sensor data. Construction of a digital ground truth array allows direct pixel by pixel comparison between classification results of MSS data and ground truth. By making comparisons, analysts can identify spatial distribution of error within the MSS data as well as usual figures of merit for the classifications. Use of the ground truth system permits investigators to compare a variety of environmental or anthropogenic data, such as soil color or tillage patterns, with classification results and allows direct inclusion of such data into classification operations. To illustrate the system, examples from classification of simulated Thematic Mapper data for agricultural test sites in North Dakota and Kansas are provided.

Coiner, J. C.↗
NASA NTRS1993

Optical spectroscopy of a highly fluorescent aggregate of bacteriochlorophyll c

Bacteriochlorophyll (BChl) c and a similar model compound, Mg-methyl bacteriopheophorbide d, form several types of aggregates in nonpolar solvents. One of these aggregates is highly fluorescent, with a quantum yield higher than that of the monomer. This aggregate is also unusual in that it shows a rise time in its fluorescence emission decay at certain wavelengths, which is ascribed to a change in conformation of the aggregate. An analysis of fluorescence depolarization data is consistent with either a linear aggregate of four or five monomers or preferably a cyclic arrangement of three dimers.

NASA Discipline Exobiology↗
NASA NTRS2012

Security Data Warehouse Application

The Security Data Warehouse (SDW) is used to aggregate and correlate all JSC IT security data. This includes IT asset inventory such as operating systems and patch levels, users, user logins, remote access dial-in and VPN, and vulnerability tracking and reporting. The correlation of this data allows for an integrated understanding of current security issues and systems by providing this data in a format that associates it to an individual host. The cornerstone of the SDW is its unique host-mapping algorithm that has undergone extensive field tests, and provides a high degree of accuracy. The algorithm comprises two parts. The first part employs fuzzy logic to derive a best-guess host assignment using incomplete sensor data. The second part is logic to identify and correct errors in the database, based on subsequent, more complete data. Host records are automatically split or merged, as appropriate. The process had to be refined and thoroughly tested before the SDW deployment was feasible. Complexity was increased by adding the dimension of time. The SDW correlates all data with its relationship to time. This lends support to forensic investigations, audits, and overall situational awareness. Another important feature of the SDW architecture is that all of the underlying complexities of the data model and host-mapping algorithm are encapsulated in an easy-to-use and understandable Perl language Application Programming Interface (API). This allows the SDW to be quickly augmented with additional sensors using minimal coding and testing. It also supports rapid generation of ad hoc reports and integration with other information systems.

Vernon, Lynn R.↗
DOE OSTI2019

Presentation layer of CMS Online Monitoring System

The Compact Muon Solenoid (CMS) is one of the experiments at the CERN Large Hadron Collider (LHC). The CMS Online Monitoring system (OMS) is an upgrade and successor to the CMS Web-Based Monitoring (WBM)system, which is an essential tool for shift crew members, detector subsystem experts, operations coordinators, and those performing physics analyses. The CMS OMS is divided into aggregation and presentation layers. Communication between layers uses RESTful JSON:API compliant requests. The aggregation layer is responsible for collecting data from heterogeneous sources, storage of transformed and pre-calculated (aggregated) values and exposure of data via the RESTful API. The presentation layer displays detector information via a modern, user-friendly and customizable web interface. The CMS OMS user interface is composed of a set of cutting-edge software frameworks and tools to display non-event data to any authenticated CMS user worldwide. The web interface tree-like component structure comprises (top-down): workspaces, folders, pages, controllers and portlets. A clear hierarchy gives the required flexibility and control for content organization. Each bottom element instantiates a portlet and is a reusable component that displays a single aspect of data, like a table, a plot, an article, etc. Pages consist of multiple different portlets and can be customized at runtime by using a drag-and-drop technique. This is how a single page can easily include information from multiple online sources. Different pages give access to a summary of the current status of the experiment, as well as convenient access to historical data. This paper describes the CMS OMS architecture, core concepts and technologies of the presentation layer.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

CORE: A Global Aggregation Service for Open Access Papers

This paper introduces CORE, a widely used scholarly service, which provides access to the world’s largest collection of open access research publications, acquired from a global network of repositories and journals. CORE was created with the goal of enabling text and data mining of scientific literature and thus supporting scientific discovery, but it is now used in a wide range of use cases within higher education, industry, not-for-profit organisations, as well as by the general public. Through the provided services, CORE powers innovative use cases, such as plagiarism detection, in market-leading third-party organisations. CORE has played a pivotal role in the global move towards universal open access by making scientific knowledge more easily and freely discoverable. In this paper, we describe CORE’s continuously growing dataset and the motivation behind its creation, present the challenges associated with systematically gathering research papers from thousands of data providers worldwide at scale, and introduce the novel solutions that were developed to overcome these challenges. The paper then provides an in-depth discussion of the services and tools built on top of the aggregated data and finally examines several use cases that have leveraged the CORE dataset and services.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Mapping Inquiry Tool (MapIT) Database

The Mapping Inquiry Tool (MapIT) database consists of a geodatabase and data catalog of geologic, geophysical, structural, hydrologic, and contextual data, based on the data types to support geologic carbon storage activities and other subsurface energy systems resource assessments. The database was aggregated from publicly available data across the USA from state and federal entities. The database is structured by categories including rock unit geology, boundaries, national CS datasets, geophysical data, faults and structural data, infrastructure, surface hydrology, groundwater, and more. The data described in the data catalog is also available in the Mapping Inquiry Tool (https://edx.netl.doe.gov/dataset/mapping-inquiry-tool). Version 3 of the geodatabase and data catalog have been updated as of 5/17/2024. The database was published with a limited number of layers. The Catalog V3 contains many more resources than the geodatabase, documenting all layers that will be included in MapIT, and includes links to the original sources of the data. Within the catalog, in the final column, there is information about if the file is included in the geodatabase or not. Use the links provided in the catalog to download data directly from the original source if not included in the geodatabase. Four resources are included in this submission: 1. Geodatabase 2. ReadMe file 3. Catalog of data layers and additional data resources 4. Web link to a resource describing the motivation and reviewing the content of the geodatabase - DOE NETL Carbon Storage Site Mapping Inquiry Tool Database

carbon storage↗