Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data diversity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Modeling the Spectral Diversity of Quasars in the Sixteenth Data Release from the Sloan Digital Sky Survey

Abstract We present a new approach to capturing the broad diversity of emission-line and continuum properties in quasar spectra. We identify populations of spectrally similar quasars through pixel-level clustering on 12,968 high signal-to-noise ratio (S/N) spectra from the Sloan Digital Sky Survey (SDSS) in the redshift range of 1.57 < z < 2.4. Our clustering analysis finds 396 quasar spectra that are not assigned to any population, 15 misclassified spectra, and 6 quasars with incorrect redshifts. We compress the quasar populations into a library of 684 high-S/N composite spectra, anchored in redshift space by the Mg ii emission line. Principal component analysis on the library results in an eigenspectrum basis spanning 1067–4007 Å. We model independent samples of SDSS quasar spectra with the eigenbasis, allowing for a free redshift parameter. Our models achieve a median reduced χ 2 on non–broad absorption line quasar spectra that is reduced by 8.5% relative to models using the eigenspectra from the SDSS spectroscopic pipeline. A significant contribution to the relative improvement is from the ability to reconstruct the range of emission-line variation. The redshift estimates from our model are consistent with the Mg ii emission-line redshift with an average offset that displays 51.4% less redshift-dependent variation relative to the SDSS eigenspectra. Our method for developing quasar spectra models can improve automated classification and predict the intrinsic spectrum in regions affected by intervening absorbers such as Ly α , C iv , and Mg ii , thus benefiting studies of large-scale structure.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2006

Advanced Visualization of Experimental Data in Real Time Using LiveView3D

LiveView3D is a software application that imports and displays a variety of wind tunnel derived data in an interactive virtual environment in real time. LiveView3D combines the use of streaming video fed into a three-dimensional virtual representation of the test configuration with networked communications to the test facility Data Acquisition System (DAS). This unified approach to real time data visualization provides a unique opportunity to comprehend very large sets of diverse forms of data in a real time situation, as well as in post-test analysis. This paper describes how LiveView3D has been implemented to visualize diverse forms of aerodynamic data gathered during wind tunnel experiments, most notably at the NASA Langley Research Center Unitary Plan Wind Tunnel (UPWT). Planned future developments of the LiveView3D system are also addressed.

Schwartz, Richard J.↗
DOE OSTI2019

Broker for Assimilation, Synthesis and Integration of eNvironmental Diverse, Distributed Datasets v1.1.0

BASIN-3D (Broker for Assimilation, Synthesis and Integration of eNvironmental Diverse, Distributed Datasets) uses a data brokering approach to synthesize diverse earth science data from a variety of remote sources in real-time without the need for additional storage. BASIN-3D is an extendable Django web framework application using a generalized data synthesis model that makes the synthesized data available via REpresentational State Transfer (REST) Application Programming Interface (API). We have currently implemented our data synthesis model to represent sensor-based time series earth science observations across a hierarchy of spatial locations. Supporting our data synthesis model is a plugin framework that allows developers to map data sources of interest to the BASIN-3D synthesis model

Varadharajan, Charuleka↗
DOE OSTI2016

A Nonrelational Data Warehouse for the Analysis of Field and Laboratory Data From Multiple Heterogeneous Photovoltaic Test Sites

A nonrelational, distributed computing, data warehouse, and analytics environment (Energy-CRADLE) was developed for the analysis of field and laboratory data from multiple heterogeneous photovoltaic (PV) test sites. This data informatics and analytics infrastructure was designed to process diverse formats of PV performance data and climatic telemetry time-series data collected from a PV outdoor test network, i.e., the Solar Durability and Lifetime Extension global SunFarm network, as well as point-in-time laboratory spectral and image measurements of PV material samples. Using Hadoop/HBase for the distributed data warehouse, Energy-CRADLE does not have a predefined data table schema, which enables ingestion of data in diverse and changing formats. For easy data ingestion and data retrieval, Energy-CRADLE utilizes Hadoop streaming to enable Python MapReduce and provides a graphical user interface, i.e., py-CRADLE. By developing the Hadoop distributed computing platform and the HBase NoSQL database schema for solar energy, Energy-CRADLE exemplifies an integrated, scalable, secure, and user-friendly data informatics and analytics system for PV researchers. Furthermore, an example of Energy-CRADLE enabled scalable, data-driven, analytics is presented, where machine learning is used for anomaly detection across 2.2 million real-world current-voltage (I–V) curves of PV modules in three distinct Koppen–Geiger climatic zones.

14 SOLAR ENERGY↗
DOE OSTI2012

Abrolhos Bank Reef Health Evaluated by Means of Water Quality, Microbial Diversity, Benthic Cover, and Fish Biomass Data

The health of the coral reefs of the Abrolhos Bank (southwestern Atlantic) was characterized with a holistic approach using measurements of four ecosystem components: (i) inorganic and organic nutrient concentrations, [1] fish biomass, [1] macroalgal and coral cover and (iv) microbial community composition and abundance. The possible benefits of protection from fishing were particularly evaluated by comparing sites with varying levels of protection. Two reefs within the wellenforced no-take area of the National Marine Park of Abrolhos (Parcel dos Abrolhos and California) were compared with two unprotected coastal reefs (Sebastia˜o Gomes and Pedra de Leste) and one legally protected but poorly enforced coastal reef (the ‘‘paper park’’ of Timbebas Reef). The fish biomass was lower and the fleshy macroalgal cover was higher in the unprotected reefs compared with the protected areas. The unprotected and protected reefs had similar seawater chemistry. Lower vibrio CFU counts were observed in the fully protected area of California Reef. Metagenome analysis showed that the unprotected reefs had a higher abundance of archaeal and viral sequences and more bacterial pathogens, while the protected reefs had a higher abundance of genes related to photosynthesis. Similar to other reef systems in the world, there was evidence that reductions in the biomass of herbivorous fishes and the consequent increase in macroalgal cover in the Abrolhos Bank may be affecting microbial diversity and abundance. Through the integration of different types of ecological data, the present study showed that protection from fishing may lead to greater reef health. The data presented herein suggest that protected coral reefs have higher microbial diversity, with the most degraded reef (Sebastia&otilde Gomes) showing a marked reduction in microbial species richness. It is concluded that ecological conditions in unprotected reefs may promote the growth and rapid evolution of opportunistic microbial pathogens.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

A guide to the BRAIN Initiative Cell Census Network data ecosystem

Characterizing cellular diversity at different levels of biological organization and across data modalities is a prerequisite to understanding the function of cell types in the brain. Classification of neurons is also essential to manipulate cell types in controlled ways and to understand their variation and vulnerability in brain disorders. The BRAIN Initiative Cell Census Network (BICCN) is an integrated network of data-generating centers, data archives, and data standards developers, with the goal of systematic multimodal brain cell type profiling and characterization. Emphasis of the BICCN is on the whole mouse brain with demonstration of prototype feasibility for human and nonhuman primate (NHP) brains. Here, we provide a guide to the cellular and spatial approaches employed by the BICCN, and to accessing and using these data and extensive resources, including the BRAIN Cell Data Center (BCDC), which serves to manage and integrate data across the ecosystem. We illustrate the power of the BICCN data ecosystem through vignettes highlighting several BICCN analysis and visualization tools. Finally, we present emerging standards that have been developed or adopted toward Findable, Accessible, Interoperable, and Reusable (FAIR) neuroscience. The combined BICCN ecosystem provides a comprehensive resource for the exploration and analysis of cell types in the brain.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2001

Spectral Data Assimilation for Merging Satellite Ocean Color Imagery

The Sensor Intercomparison and Merger for Biological and Interdisciplinary Oceanic Studies (SIMBIOS) project was initiated 'to develop a methodology and operational capability to combine data products from various ocean color missions in a manner that ensures the best possible global coverage and best exploits the complementary missions of the sensors'. While the merging of data from multiple sources has already been implemented for sea surface temperature (SST), altimetry or clouds, this is a new topic in ocean color science. Merging ocean color data has obvious advantage in terms of spatial and temporal coverage of the global ocean but data merging should also result in new, more diverse and improved data products with lower uncertainties. The merging of ocean color satellite data is generally considered at the level of the global gridded products, i.e., NASA Level-3 and, more specifically, for the merging of chlorophyll a concentration data, [Ch1]. This approach limits computational issues and deals with a unique, simple quantity, [Ch1]. However, ocean color data merging can also be conducted at the level of water-leaving radiances, LwN(lamda), using either empirical or semi-analytical algorithms. Although more challenging, this latter approach is potentially extremely powerful as it can achieve good consistency in the final data products and can take advantage of both the specificities and differences of each source of data. The demonstration of the feasibility of such approach is the main objective of our SIMBIOS work. Here, we use a semi-analytical algorithm to merge Rrs (or, equivalently LwN) data from different sources to estimate [Ch1]. The model also allows for the retrieval of other quantities such as the combined absorption of colored detrital particulate and dissolved organic matter at 443 nm, a(sub cdm)(443) and the particulate backscattering, b(sub bp)(443).

Siegel, David A.↗
NASA NTRS2003

Optimization of Ocean Color Algorithms: Application to Satellite Data Merging

The objective of our program is to develop and validate a procedure for ocean color data merging which is one of the major goals of the SIMBIOS project. The need for a merging capability is dictated by the fact that since the launch of MODIS on the Terra platform and over the next decade, several global ocean color missions from various space agencies are or will be operational simultaneously. The apparent redundancy in simultaneous ocean color missions can actually be exploited to various benefits. The most obvious benefit is improved coverage. The patchy and uneven daily coverage from any single sensor can be improved by using a combination of sensors. Beside improved coverage of the global Ocean the merging of Ocean color data should also result in new, improved, more diverse and better data products with lower uncertainties. Ultimately, ocean color data merging should result in the development of a unified, scientific quality, ocean color time series, from SeaWiFS to NPOESS and beyond. Various approaches can be used for ocean color data merging and several have been tested within the frame of the SIMBIOS program. As part of the SIMBIOS Program, we have developed a merging method for ocean color data. Conversely to other methods our approach does not combine end-products like the subsurface chlorophyll concentration (chl) from different sensors to generate a unified product. Instead, our procedure uses the normalized water-leaving radiances (L(sub WN)(lambda)) from single or multiple sensors and uses them in the inversion of a semi-analytical ocean color model that allows the retrieval of several ocean color variables simultaneously. Beside ensuring simultaneity and consistency of the retrievals (all products are derived from a single algorithm), this model-based approach has various benefits over techniques that blend end-products (e.g. chlorophyll): 1) it works with single or multiple data sources regardless of their specific bands, 2) it exploits band redundancies and band differences, 3) it accounts for uncertainties in the (L(sub WN)(lambda)) data and, 4) it provides uncertainty estimates for the retrieved variables.

Maritorena, Stephane↗
NASA NTRS2003

Optimization Of Ocean Color Algorithms: Application To Satellite And In Situ Data Merging

The objective of our program is to develop and validate a procedure for ocean color data merging which is one of the major goals of the SIMBIOS project (McClain et al., 1995). The need for a merging capability is dictated by the fact that since the launch of MODIS on the Terra platform and over the next decade, several global ocean color missions from various space agencies are or will be operational simultaneously. The apparent redundancy in simultaneous ocean color missions can actually be exploited to various benefits. The most obvious benefit is improved coverage (Gregg et al., 1998; Gregg & Woodward, 1998). The patchy and uneven daily coverage from any single sensor can be improved by using a combination of sensors. Beside improved coverage of the global ocean the merging of ocean color data should also result in new, improved, more diverse and better data products with lower uncertainties. Ultimately, ocean color data merging should result in the development of a unified, scientific quality, ocean color time series, from SeaWiFS to NPOESS and beyond. Various approaches can be used for ocean color data merging and several have been tested within the frame of the SIMBIOS program (see e.g. Kwiatkowska & Fargion, 2003, Franz et al., 2003). As part of the SIMBIOS Program, we have developed a merging method for ocean color data. Conversely to other methods our approach does not combine end-products like the subsurface chlorophyll concentration (chl) from different sensors to generate a unified product. Instead, our procedure uses the normalized waterleaving radiances (LwN( )) from single or multiple sensors and uses them in the inversion of a semianalytical ocean color model that allows the retrieval of several ocean color variables simultaneously. Beside ensuring simultaneity and consistency of the retrievals (all products are derived from a single algorithm), this model-based approach has various benefits over techniques that blend end-products (e.g. chlorophyll): 1) it works with single or multiple data sources regardless of their specific bands, 2) it exploits band redundancies and band differences, 3) it accounts for uncertainties in the LwN( ) data and, 4) it provides uncertainty estimates for the retrieved variables.

Maritorena, Stephane↗
NASA NTRS1984

A program to form a multidisciplinary data base and analysis for dynamic systems

Diverse sets of experimental data and analysis programs have been assembled for the purpose of facilitating research in systems identification, parameter estimation and state estimation techniques. The data base analysis programs are organized to make it easy to compare alternative approaches. Additional data and alternative forms of analysis will be included as they become available.

Taylor, L. W.↗
NASA NTRSDate not supplied

Improving GES Disc Data Search and Discovery Through AI Metadata Augmentation

NASA’s Goddard Earth Science (GES) Data and Information Services Center (DISC) is one of twelve data centers in NASA's Science Mission Directorate (SMD), providing vital earth science data to a diverse user base. To enhance the discoverability of this data, GES DISC employs a keyword search system, which leverages scientific keywords embedded in dataset metadata. However, the evolving nature of scientific applications of our data necessitates regular review and augmentation of these keywords. To address this, we developed a service to automatically predict missing science keywords in the metadata. This service constructs a knowledge graph from the latest GES DISC metadata within NASA’s Common Metadata Repository (CMR). Using an open-source library, we trained a machine learning model to predict absent science keywords in the metadata. Our preliminary results indicate that the model has high levels of accuracy at predicting science keywords in the dataset metadata when exposed to data not included in its training. These predicted keywords were then evaluated by GES DISC data curation scientists and compared against other AI tools for metadata augmentation. We aim to enhance the overall usability and accessibility of NASA’s earth science data by implementing this tool in our data curation processes.

Kendall Gilbert↗
DOE OSTI2023

Geodesic-based relaxation for deep canonical correlation analysis

Deep canonical correlation analysis (DCCA) is often applied to paired data samples from diverse sources to extract meaningful common information. However, when the data sources are heterogeneous, some of the useful information may be complementary but not exactly common. In spite of this fact, existing techniques learn maximally correlated representations from multiple views and are formulated so that they aim to yield identical latent subspaces for each view. This approach is sub-optimal in estimating the true signal subspaces for heterogeneous data sources. We propose a residual relaxation for deep canonical correlation analysis (RDCCA) based on a subspace distance metric, which generalizes the existing problem formulation and extracts representations that are better estimates of the actual, non-identical subspaces. We demonstrate that when using such a relaxation, the learned representations are closer to the true ones and that RDCCA outperforms CCA and DCCA in scenarios with heterogeneous data.

Kuschel, Maurice↗
DOE OSTI2023

A Machine Learning–Based Tire Life Prediction Framework for Increasing Life of Commercial Vehicle Tires

In the commercial freight industry, tire retreading decisions are often conservative due to limited knowledge of a tire’s remaining service life. This practice leads to increased costs and material waste. This paper proposes a machine learning–based approach for estimating tire casing life and retreadability, focusing on usage data rather than wear information. This approach could extend the tire’s lifespan and reduce landfill waste. Data integration from diverse tire casing measurement sources presents challenges, including imbalanced removal data. Our methodology addresses these challenges by using historical inspection, telematics, and finite element modeling (FEM) datasets. We introduce “Tire Casing Energy” as a comprehensive usage input and apply a Variance-Reduction Synthetic Minority Oversampling Technique (VR-SMOTE) for data imbalance rectification. A random forest model is used to estimate the state of the tire casing and the casing removal probability, with Bayesian optimization applied for hyperparameter tuning, enhancing model accuracy. Here, the proposed prediction framework is able to differentiate different truck fleets and tire locations based on their usage parameters. With the aid of this machine learning model, the importance and sensitivity of different tire usage parameters can be obtained, which is beneficial to maximize tire life.

Data balancing↗
DOE OSTI2011

Modeling Human Behavior to Anticipate Insider Attacks

The insider threat ranks among the most pressing cybersecurity challenges that threaten government and industry information infrastructures. To date, no systematic methods have been developed that provide a complete and effective approach to prevent data leakage, espionage and sabotage. Current practice is forensic in nature, relegating to the analyst the bulk of the responsibility to monitor, analyze, and correlate an overwhelming amount of data. We describe a predictive modeling framework that integrates a diverse set of data sources from the cyber domain as well as inferred psychological/motivational factors that may underlie malicious insider exploits. This comprehensive threat assessment approach provides automated support for the detection of high-risk behavioral “triggers” to help focus the analyst’s attention and inform the analysis. Designed to be domain independent, the system may be applied to many different threat and warning analysis/sensemaking problems.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2016

Advancing User Supports with a Structured How-To Knowledge Base for Earth Science Data

It is a challenge to access and process fast growing Earth science data from satellites and numerical models, which may be archived in very different data format and structures. NASA data centers, managed by the Earth Observing System Data and Information System (EOSDIS), have developed a rich and diverse set of data services and tools with features intended to simplify finding, downloading, and working with these data. Although most data services and tools have user guides, many users still experience difficulties with accessing or reading data due to varying levels of familiarity with data services, tools, and/or formats. A type of structured online document, data recipe, were created in beginning 2013 by Goddard Earth Science Data and Information Services Center (GES DISC). A data recipe is the How-To document created by using the fixed template, containing step-by-step instructions with screenshots and examples of accessing and working with real data. The recipes has been found to be very helpful, especially to first-time-users of particular data services, tools, or data products. Online traffic to the data recipe pages is significant to some recipes. In 2014, the NASA Earth Science Data System Working Group (ESDSWG) for data recipes was established, aimed to initiate an EOSDIS-wide campaign for leveraging the distributed knowledge within EOSDIS and its user communities regarding their respective services and tools. The ESDSWG data recipe group started with inventory and analysis of existing EOSDIS-wide online help documents, and provided recommendations and guidelines and for writing and grouping data recipes. This presentation will overview activities of creating How-To documents at GES DISC and ESDSWG. We encourage feedback and contribution from users for improving the data How-To knowledge base.

how-to↗
NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗