Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data diversity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Raw diffraction data preservation and reuse: Overview, update on practicalities and metadata requirements

A topical review is presented of the rapidly developing interest in and storage options for the preservation and reuse of raw data within the scientific domain of the IUCr and its Commissions, each of which operates within a great diversity of instrumentation. A résumé is included of the case for raw diffraction data deposition. An overall context is set by highlighting the initiatives of science policy makers towards an `Open Science' model within which crystallographers will increasingly work in the future; this will bring new funding opportunities but also new codes of procedure within open science frameworks. Skills education and training for crystallographers will need to be expanded. Overall, there are now the means and the organization for the preservation of raw crystallographic diffraction data via different types of archive, such as at universities, discipline-specific repositories (Integrated Resource for Reproducibility in Macromolecular Crystallography, Structural Biology Data Grid), general public data repositories (Zenodo, ResearchGate) and centralized neutron and X-ray facilities. Formulation of improved metadata descriptors for the raw data types of each of the IUCr Commissions is in progress; some detailed examples are provided. Lastly, a number of specific case studies are presented, including an example research thread that provides complete open access to raw data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS2019

NASA GES DISC Giovanni: Current and Future

Giovanni (Geospatial Interactive Online Visualization and Analysis Infrastructure), developed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), has established a reputation among NASA users for easy access, analysis, and visualization of NASA Earth science data. Currently, Giovanni supports over 1900 variables in eight disciplinary areas. Like any other enterprise application, Giovanni faces big data challenges, such as servicing increasingly large data volumes and more complex data types, while at the same time addressing the demands of a more diverse user community, e.g., placing requests for long-term time series from multiple spatially and temporally dense data records. I will present how Giovanni has been evolving from an on-premises, monolithic software application towards a cloud-enabled implementation to address these challenges.

Analytics↗
NASA NTRS1991

Data acquisition, preprocessing and analysis for the Virginia Tech OLYMPUS experiment

Virginia Tech is conducting a slant path propagation experiment using the 12, 20, and 30 GHz OLYMPUS beacons. Beacon signal measurements are made using separate terminals for each frequency. In addition, short baseline diversity measurements are collected through a mobile 20 GHz terminal. Data collection is performed with a custom data acquisition and control system. Raw data are preprocessed to remove equipment biases and discontinuities prior to analysis. Preprocessed data are then statistically analyzed to investigate parameters such as frequency scaling, fade slope and duration, and scintillation intensity.

Remaklus, P. Will↗
DOE OSTI2023

Evaluation of distributed process-based hydrologic model performance using only a priori information to define model inputs

Fully distributed, integrated surface–subsurface hydrological models (ISSHMs) have seen renewed interest due to availability of better software, high performance computing facilities, and high-resolution, spatially extensive data products. ISSHMs are valuable as tools for advancing system understanding as they can resolve multiple processes defined on the plot scale including three-dimensional interaction of surface water and groundwater. Here, we evaluated the performance of an ISSHM, the Advanced Terrestrial Simulator (ATS), on seven diverse catchments across the continental US using widely available data products to define model inputs without calibration. We compare the ATS-simulated streamflow and evapotranspiration with gauge observations and MODIS-derived evapotranspiration, respectively. Using the Kling-Gupta Efficiency (KGE) as metric, ATS with default data products performed reasonably well at 6 of 7 catchments for streamflow. However, in one of those 6 catchments ATS had poor performance on baseflow and ATS’s overall performance was thus judged to be inadequate despite the acceptable KGE. ATS performance for evapotranspiration was good in all 7 catchments using default data products. In the two catchments where ATS streamflow performance using default data products was not acceptable, the performance was significantly improved by using local information on subsurface properties below the soil. We also compare the model-simulated streamflow and evapotranspiration with the Sacramento soil moisture accounting (SAC-SMA) model, a semi-distributed model that was calibrated on a catchment-by-catchment basis. Uncalibrated ATS performance is comparable to the calibrated SAC-SMA model in terms of streamflow while ATS performance is similar to or better (much better in certain catchments) in reproducing MODIS-derived evapotranspiration. Reasonably good performance of ATS without catchment-specific calibration provides new confidence in the ISSHM class of models and community data products as tools for advancing understanding of watershed function in a changing environment.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Uncovering I/O demands on HPC platforms: Peeking under the hood of Santos Dumont

High-Performance Computing (HPC) platforms are required to solve the most diverse large-scale scientific problems in various research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific softwares, which have different requirements. These include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle mixed workload when storing data from the applications. Understanding the set of applications and their performance running in a supercomputer is paramount to understanding the storage system's usage, pinpointing possible bottlenecks, and guiding optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, identifying inefficient usage, problematic performance factors, and providing guidelines on how to tackle those issues.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1999

Improving Assimilated Global Data Sets using TMI Rainfall and Columnar Moisture Observations

A global analysis that optimally combine observations from diverse sources with physical models of atmospheric and land processes can provide a comprehensive description of the climate systems. Currently, such data products contain significant errors in primary hydrological fields such as precipitation and evaporation, especially in the tropics. In this study, we show that assimilating precipitation and total precipitable water (TPW) retrievals derived from the TRMM Microwave Imager (TMI) improves not only the hydrological cycle but also key climate parameters such as clouds, radiation, and the large-scale circulation produced by the Goddard Earth Observing System (GEOS) data assimilation system (DAS). In particular, assimilating TMI rain improves clouds and radiation in areas of active convection, as well as the latent heating distribution and the large-scale motion field in the tropics, while assimilating TMI TPW heating distribution and the large-scale motion field in the tropics, while assimilating TMI TPW retrievals leads to reduced moisture biases and improved radiative fluxes in clear-sky regions. The improved analysis also improves short-range forecasts in the tropics. Ensemble forecasts initialized with the GEOS analysis incorporating TMI rain rates and TPW yield smaller biases in tropical precipitation forecasts beyond 1 day and better 500 hPa geopotential height forecasts up to 5 days. Results of this study demonstrate the potential of using high-quality space-borne rainfall and moisture observations to improve the quality of assimilated global data for climate analysis and weather forecasting applications

Hou, Arthur Y.↗
DOE OSTI2018

Linking Associations of Rare Low-Abundance Species to Their Environments by Association Networks

Studies of microbial communities by targeted sequencing of rRNA genes lead to recovering numerous rare low-abundance taxa with unknown biological roles. We propose to study associations of such rare organisms with their environments by a computational framework based on transformation of the data into qualitative variables. Namely, we analyze the sparse table of putative species or OTUs (operational taxonomic units) and samples generated in such studies, also known as an OTU table, by collecting statistics on co-occurrences of the species and on shared species richness across samples. Based on the statistics we built two association networks, of the rare putative species and of the samples respectively, using a known computational technique, Association networks (Anets) developed for analysis of qualitative data. Clusters of samples and clusters of OTUs are then integrated and combined with metadata of the study to produce a map of associated putative species in their environments. We tested and validated the framework on two types of microbiomes, of human body sites and that of the Populus tree root systems. We show that in both studies the associations of OTUs can separate samples according to environmental or physiological characteristics of the studied systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

RCSB protein data Bank: Next‐generation advanced search for exploration of experimental structures and computed structure models

Abstract The Protein Data Bank (PDB), established in 1971, is the primary global, open‐access archive for experimentally determined 3D macromolecular structures (proteins, RNA, DNA). The research‐focused RCSB.org web‐portal provides access to these data alongside more than one million machine‐learning‐predicted structure models, greatly expanding the available structural landscape. Rapid growth of both experimental and computational structures has increased the need for powerful yet accessible search tools that serve a broad and diverse scientific community. Herein, we describe a redesigned RCSB Protein Data Bank RCSB.org Advanced Search capability that supports intuitive discovery of 3D structures through a unified interface. This interface integrates annotation‐, sequence‐, and 3D structure‐based searches, embeds an interactive 3D viewer, and incorporates curated biological knowledge, such as catalytic site definitions from Mechanism and Catalytic Site Atlas and ligand‐guided structural motifs, for constructing geometry‐driven queries. A new Chemical Search tool allows definition of chemical queries via an integrated drawing tool or standard identifiers, seamlessly combining them with annotation filters. By allowing query definition directly within spatial and chemical contexts, these search interfaces reduce the need for detailed knowledge of residue numbering, chain identifiers, or external cheminformatics software. This capability enables efficient exploration of structures, chemical diversity, and structure–function relationships across all life domains. The redesigned interfaces can be accessed directly at rcsb.org/search/advanced for Advanced Search and rcsb.org/search/chemical for Chemical Search.

Rose, Yana [Research Collaboratory for Structural ↗
DOE OSTI2024

4th Big Data for Nuclear Power Plants Workshop 2023

The Ohio State University and Idaho National Laboratory organized the 4 th Big Data for Nuclear Power Plants Workshop in November, 2023 in Columbus, Ohio. Workshop topics were chosen to understand the challenges and gaps that need to be addressed to maximize the impact of data on the nuclear industry, as well as the associated applications and risks. Discussions were focused around six specific application areas: Operation and Maintenance; Machine Learning in Nuclear Materials and Advanced Manufacturing; Cybersecurity; High-Performance Computing and Massive Computation; Big Data and Digital Twins; and Nuclear Non-Proliferation. The opportunities, challenges, and risks identified in the six focus areas explored in this workshop are diverse, but some common themes emerge, such as the importance of data integrity, quality, coverage, privacy, and traceability. Big data and AI/ML tools can be leveraged to reduce costs, optimize human tasking, and reduce human error across various application areas. In order for the nuclear industry to benefit from big data and advanced analytic capabilities, it is essential to address challenges and risks, such as data privacy, model reliability, and computational resource availability. Learning from other industries that have successfully implemented big data and AI/ML technologies, like the aerospace industry, can help the nuclear industry successfully integrate these technologies.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1983

Application of Heat Capacity Mapping Mission data to regional geologic analysis for mineral and energy resource evaluation

Heat Capacity Mapping Mission thermal-inertia images of a diversity of terrains and geologic settings were examined in conjunction with topographic, geologic, geophysical, and LANDSAT data. The images were found to have attributes similar to bedrock maps. In the Cascades region, two new features were identified and a method was developed to characterize regional terranes using linear feature data. Two northeast-trending Lineaments were discovered in the Overthrust Belt of Montana and Idaho. The longer of the two extends from the Idaho-Oregon border, through the Idaho batholith and across the Lewis thrust. It coincides, along segments, with mapped faults and an aeromagnetic pattern change. A major lineament crossing the Colorado Plateau and the Southern Rocky Mountians was detected on several thermal-inertial images and evidence was found for the existence of a geologic discontinuity. Vegetation-covered areas in Richfield and the Silver City quadrangle (Arizona and New Mexico) displayed thermal-inertia differences within heavily vegetation areas although no apreciable correlation was found between vegetation cover and thermal inertia. Resistant ridges and knolls have high thermal inertias and thermal-inertia contrasts occurred at lithologic and fault contacts. In the heavy vegetated Pinaleno Mountains, Arizona, a Lithologic unit obscured on LANDSAT MSS data due to the vegetation cover, exhibited a thermal-inertia contrast with its surroundings.

Watson, K.↗
DOE OSTI2024

Individual Data Sparsity in Smart Thermostat Big Data: Impacts on Modeling Thermostat Use Behavior Dynamics

This study explores the impacts of the sparsity of individual thermostat interaction data on modeling thermostat use behavior dynamics using a dataset of over 100,000 smart thermostats. In developing a data-driven model of Thermal Frustration Theory (TFT), we investigate the challenges and trade-offs in clustering occupant data to enhance predictive accuracy. Our findings reveal that a single, aggregated model fails to capture the diversity of occupant behaviors, resulting in extremely poor prediction performance. Conversely, excessive clustering exacerbates data sparsity, undermining model reliability. By identifying an optimal clustering strategy, we achieve a balance that significantly improves the prediction of manual setpoint changes during demand response (DR) events, enhancing energy management and occupant comfort

Fannon, David↗
DOE OSTI2023

BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. Furthermore, we observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Rapid phylogenetic and functional classification of short genomic fragments with signature peptides

Background: Classification is difficult for shotgun metagenomics data from environments such as soils, where the diversity of sequences is high and where reference sequences from close relatives may not exist. Approaches based on sequence-similarity scores must deal with the confounding effects that inheritance and functional pressures exert on the relation between scores and phylogenetic distance, while approaches based on sequence alignment and tree-building are typically limited to a small fraction of gene families. We describe an approach based on finding one or more exact matches between a read and a precomputed set of peptide 10-mers. Results: At even the largest phylogenetic distances, thousands of 10-mer peptide exact matches can be found between pairs of bacterial genomes. Genes that share one or more peptide 10-mers typically have high reciprocal BLAST scores. Among a set of 403 representative bacterial genomes, some 20 million 10-mer peptides were found to be shared. We assign each of these peptides as a signature of a particular node in a phylogenetic reference tree based on the RNA polymerase genes. We classify the phylogeny of a genomic fragment (e.g., read) at the most specific node on the reference tree that is consistent with the phylogeny of observed signature peptides it contains. Using both synthetic data from four newly-sequenced soil-bacterium genomes and ten real soil metagenomics data sets, we demonstrate a sensitivity and specificity comparable to that of the MEGAN metagenomics analysis package using BLASTX against the NR database. Phylogenetic and functional similarity metrics applied to real metagenomics data indicates a signal-to-noise ratio of approximately 400 for distinguishing among environments. Our method assigns ~6.6 Gbp/hr on a single CPU, compared with 25 kbp/hr for methods based on BLASTX against the NR database. Conclusions: Classification by exact matching against a precomputed list of signature peptides provides comparable results to existing techniques for reads longer than about 300 bp and does not degrade severely with shorter reads. Orders of magnitude faster than existing methods, the approach is suitable now for inclusion in analysis pipelines and appears to be extensible in several different directions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Outage probability and average BER estimation of FSO system employing wavelength diversity

Free space optical (FSO) communication has gained significant attention among researchers for high bandwidth and secured data transfer applications. However, its performance is limited by the local atmospheric conditions. This paper focusses on an FSO system employing a diversity method to compensate signal loss occurred due to atmospheric turbulence conditions varying from moderate to strong. We derive a closed-form expression for two important parameters, viz.: average bit error rate (BER) and outage probability over gamma–gamma distribution channel. The numerical results for different combining schemes are presented in this paper. Enhancement in reliability and performance of FSO communication system with wavelength diversity is investigated. Effect of different wavelengths 850 nm, 1310 nm and 1550 nm on average BER is analysed and minimum BER is achieved for optimal combining at 1550 nm at diversity order R=3.

42 ENGINEERING↗
DOE OSTI2023

A Processing and Analytics System for Microscopy Data Workflows: The Pycroscopy Ecosystem of Packages

Major advancements in fields as diverse as biology and quantum computing have relied on a multitude of microscopy techniques. Despite the considerable proliferation of these instruments, significant bottlenecks remain in terms of processing, analysis, storage, and retrieval of the acquired datasets. Aside from lack of file standards, individual domain-specific analysis packages are often disjoint from the underlying datasets, and thus keeping track of analysis and processing steps remains tedious for the end-user, hampering reproducibility. Here, in this study, the pycroscopy ecosystem of packages is introduced, an open-source python-based ecosystem underpinned by a common data model. The data model, termed the N-dimensional spectral imaging data format, is realized in pycroscopy's sidpy package. This package is built on top of dask arrays, thus leveraging dask array attributes, but expanding them to accelerate microscopy relevant analysis and visualization. Several examples of the use of the pycroscopy ecosystem to create workflows for data ingestion and analysis of scanning transmission electron microscopy (STEM) and scanning probe microscopy data are shown. Adoption of such standardized routines will be critical to usher in the next generation of autonomous instruments where processing, computation, and meta-data storage will be critical to overall experimental operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Review of Intrusion Detection Methods and Tools for Distributed Energy Resources

Recent trends in the growth of distributed energy resources (DER) in the electric grid and newfound malware frameworks that target internet of things (IoT) devices is driving an urgent need for more reliable and effective methods for intrusion detection and prevention. Cybersecurity intrusion detection systems (IDSs) are responsible for detecting threats by monitoring and analyzing network data, which can originate either from networking equipment or end-devices. Creating intrusion detection systems for PV/DER networks is a challenging undertaking because of the diversity of the attack types and intermittency and variability in the data. Distinguishing malicious events from other sources of anomalies or system faults is particularly difficult. New approaches are needed that not only sense anomalies in the power system but also determine causational factors for the detected events. In this report, a range of IDS approaches were summarized along with their pros and cons. Using the review of IDS approaches and subsequent gap analysis for application to DER systems, a preliminary hybrid IDS approach to protect PV/DER communications is formed in the conclusion of this report to inform ongoing and future research regarding the cybersecurity and resilience enhancement of DER systems.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2020

Modeling occupancy-driven building loads for large and diversified building stocks through the use of parametric schedules

Building energy modeling provides a fundamental tool to assess the potential for energy efficiency to contribute to reducing world energy consumption and global emissions. Occupancy-related operations are a key source of uncertainty for building energy analysis, particularly for aggregated building stocks. At a district or city level, it is critical to estimate aggregated power load profiles for sizing power grid infrastructure, power plant capacity allocation, and energy efficiency measures. The stochastic nature of behavior-related operations complicates the creation of models that accurately capture building load profiles for entire building stocks. This research introduces a new methodology called parametric schedules to model occupancy-driven schedules for large and diverse building stocks. In contrast to computationally expensive methodologies proposed in the literature, our work does not use a recursive time-consuming step. Occupancy is estimated by the extrapolation of operation times directly from metered electric consumption data; occupancy-related schedules are stochastically assigned to each building model, guaranteeing diversity of operation times in the stock. Our procedure has been tested on a large, diversified data-set of 25,000 commercial buildings in Los Angeles, California. It proved to be able to adequately represent the stochastic schedules diversity of the stock and to refine the stock calibration process by 1%. This innovative approach represents a useful asset for utility companies, grid operators, urban planners, and balancing authorities, seeking to improve building stock modeling and better estimate the impact of energy conservation measures. – This work is part of a larger stock modeling tool called ComStock, which is under development by NREL.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2005

Data-Driven Software Framework for Web-Based ISS Telescience

Software that enables authorized users to monitor and control scientific payloads aboard the International Space Station (ISS) from diverse terrestrial locations equipped with Internet connections is undergoing development. This software reflects a data-driven approach to distributed operations. A Web-based software framework leverages prior developments in Java and Extensible Markup Language (XML) to create portable code and portable data, to which one can gain access via Web-browser software on almost any common computer. Open-source software is used extensively to minimize cost; the framework also accommodates enterprise-class server software to satisfy needs for high performance and security. To accommodate the diversity of ISS experiments and users, the framework emphasizes openness and extensibility. Users can take advantage of available viewer software to create their own client programs according to their particular preferences, and can upload these programs for custom processing of data, generation of views, and planning of experiments. The same software system, possibly augmented with a subset of data and additional software tools, could be used for public outreach by enabling public users to replay telescience experiments, conduct their experiments with simulated payloads, and create their own client programs and other custom software.

Tso, Kam S.↗