SEARCH · Engineering Papers

Results for “data diversity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2019

AI Meets Exascale Computing: Advancing Cancer Research With Large-Scale High Performance Computing

The application of data science in cancer research has been boosted by major advances in three primary areas: (1) Data: diversity, amount, and availability of biomedical data; (2) Advances in Artificial Intelligence (AI) and Machine Learning (ML) algorithms that enable learning from complex, large-scale data; and (3) Advances in computer architectures allowing unprecedented acceleration of simulation and machine learning algorithms. These advances help build in silico ML models that can provide transformative insights from data including: molecular dynamics simulations, next-generation sequencing, omics, imaging, and unstructured clinical text documents. Unique challenges persist, however, in building ML models related to cancer, including: (1) access, sharing, labeling, and integration of multimodal and multi-institutional data across different cancer types; (2) developing AI models for cancer research capable of scaling on next generation high performance computers; and (3) assessing robustness and reliability in the AI models. In this paper, we review the National Cancer Institute (NCI) -Department of Energy (DOE) collaboration, Joint Design of Advanced Computing Solutions for Cancer (JDACS4C), a multi-institution collaborative effort focused on advancing computing and data technologies to accelerate cancer research on three levels: molecular, cellular, and population. This collaboration integrates various types of generated data, pre-exascale compute resources, and advances in ML models to increase understanding of basic cancer biology, identify promising new treatment options, predict outcomes, and eventually prescribe specialized treatments for patients with cancer.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2014

Current Trends in the Detection of Sociocultural Signatures: Data-Driven Models

The harvesting of behavioral data and their analysis through evidence-based reasoning enable the detection of sociocultural signatures in their context to support situation awareness and decision making. Harvested data are used as training materials from which to infer computational models of sociocultural behaviors or calibrate parameters for such models. Harvested data also serve as evidence input that the models use to provide insights about observed and future behaviors for targets of interest. The harvested data is often the result of assembling diverse data types and aggregating them into a form suitable for analysis. Data need to be analyzed to bring out the categories of content that are relevant to the domain being addressed in order to train or run a model. If, for example, we are modeling the intent of a group to engage in violent behavior using messages that the group has broadcasted, then these messages need to be processed to extract and measure indicators of violent intent. The extracted indicators and the associated measurements (e.g. rates or counts of occurrence) can then be used to train/calibrate and run computational models that assess the propensity for violence expressed in the source message. Ubiquitous access to the Internet, mobile telephony and technologies such as digital photography and digital video have enabled social media application platforms such as Facebook, YouTube, and Twitter that are altering the nature of human social interaction. The fast increasing pace of online social interaction introduces new challenges and opportunities for gathering sociocultural data. Challenges include the development of harvesting and processing techniques tailored to new data environments and formats (e.g. Twitter, Facebook), the integration of social media content with traditional media content, and the protection of personal privacy. As these and other challenges are addressed, a new wealth of behavioral data and data analysis methods becomes available that are shaping social computing as a strongly data-driven experimental discipline with an increasingly stronger impact on the decision-making process of groups and individuals alike. In this chapter, we review current advances and trends in the detection of sociocultural signatures. Specific embodiments of the issues discussed are provided with respect to the assessment of violent intent and sociopolitical contention. We begin by reviewing current approaches to the detection of sociocultural signatures in these domains. Next, we turn to the review of novel data harvesting methods for social media content. Finally, we discuss the application of sociocultural models to social media content, and conclude by commenting on current challenges and future developments.

Sanfilippo, Antonio P.↗

NASA NTRS2022

Using airborne and DESIS imaging spectroscopy to map plant diversity across the largest contiguous tract of tallgrass prairie on earth

Grassland ecosystems are under threat globally, primarily due to land-use and land-cover changes that have adversely affected their biodiversity. Given the negative ecological impacts of biodiversity loss in grasslands, there is an urgent need for developing an operational biodiversity monitoring system that functions in these ecosystems. In this paper, we assessed the capability of airborne and spaceborne imaging spectroscopy (also known as hyperspectral imaging) to capture plant α-diversity in a large naturally-assembled grassland while considering the impact of common management practices, specifically prescribed fire. We collected a robust in-situ plant diversity data set, including species composition and percent cover from 2500 sampling points with different burn ages, from recently-burned to transitional and pre-prescribed fire at the Joseph H. Williams Tallgrass Prairie Preserve in Oklahoma, USA. We expressed in-situ plant α-diversity using the first three Hill numbers, including species richness (number of observed species in a plant community), exponential Shannon entropy index (hereafter Shannon diversity; effective number of common species, where species are weighed proportional to their percent cover), and inverse Simpson concentration index (hereafter Simpson diversity; effective number of dominant species, where more weight is given to dominant species) at four different plot sizes, including 60 m × 60 m, 120 m × 120 m, 180 m × 180 m, and 240 m × 240 m. We collected full-range airborne hyperspectral data with fine spatial resolution (1 m) and visible and near-infrared spaceborne hyperspectral data from DESIS sensor with coarse spatial resolution (30 m), and used the spectral diversity hypothesis—i.e., that the variability in spectral data is largely driven by plant diversity—to estimate α-diversity remotely. In recently-burned plots and those at the transitional stage, both airborne and spaceborne data were capable of capturing Simpson diversity—a metric that calculates the effective number of dominant species by emphasizing abundant species and discounting rare species—but not species richness or Shannon diversity. Further, neither airborne nor spaceborne hyperspectral data sets were capable of capturing plant α-diversity of 60 m × 60 m or 120 m × 120 m plots. Based on these results, three main findings emerged: (1) management practices influence grassland biodiversity patterns that can be remotely detected, (2) both fine- and coarse-resolution remotely-sensed data can detect the effective number of dominant species (e.g., Simpson diversity), and (3) attention should be given to site-specific plant diversity field data collection to appropriately interpret remote sensing results. Findings of this study indicate the feasibility of estimating Simpson diversity in naturally-assembled grasslands using forthcoming spaceborne imagers such as National Aeronautics and Space Administration's Surface Biology and Geology mission.

Hamed Gholizadeh↗

DOE OSTI2024

Enhancing Discoverability and Management of Atmospheric Data at Scale: Solutions from the ARM Data Center

The Atmospheric Radiation Measurement (ARM) is a multi-laboratory and multi-institutional U.S. Department of Energy (DOE) Office of Science National User Facility. The ARM Data Center (ADC), located at Oak Ridge National Laboratory, collects, archives, and shares vast atmospheric data crucial for climate research. The ADC manages over 7 PB of data from 460 instruments worldwide, processing it into more than 11,000 diverse data products using the Network Common Data Form (NetCDF) for machine-independent accessibility. The primary challenge addressed in this paper is the efficient management and distribution of vast and diverse datasets essential for the climate research community, enhancing accessibility through advanced tools like Data Discovery. The ADC has developed advanced infrastructure and software architecture to handle the continuous influx of heterogeneous data to enhance data discoverability, resulting in increased scientific collaboration. In 2023, users from over 34 countries downloaded and utilized ARM data, resulting in 1,455 publications. The ADC’s efforts have significantly improved the discoverability and usability of atmospheric data, fostering extensive scientific research and collaboration. This paper details the solutions implemented by the ADC team for efficient data discovery and distribution, and it demonstrates ARM’s capability of staging processed data for scientific analysis.

Shah, Chirag [ORNL] (ORCID:0000000203145737)↗

DOE OSTI2025

Building Inclusive Educational and Career Pathways in Fusion Energy

This paper was written in preparation for the Workforce Accelerator for Fusion Energy Development conference, which was funded by the National Science Foundation. Prior to the conference, participants from academia, government, industry, national laboratories, and nonprofit organizations worked together to write White papers. Held in Hampton, Virginia, in May 2024, the conference convened participants to discuss various topics in support of workforce development in fusion energy. As the fusion industry expands, efforts to recruit and retain a diverse workforce, ensure equitable access to fusion education, and gain public support will be critical to meeting workforce needs in this field. We outline strategic recommendations across three vital areas: improving diversity data collection; supporting career advancement and retention with a diversity, equity, inclusion, accessibility, and belonging focus; and strengthening community engagement and education in fusion energy.

Cote, Laleh [Lawrence Berkeley National Laboratory↗

NASA NTRSDate not supplied

Data Science and Urban Air Mobility: Challenges and Opportunities

Aviation is broadly a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Urban Air Mobility↗

NASA NTRS2020

Data Science and Urban Air Mobility: Challenges and Opportunities

Urban Air Mobility↗

NASA NTRSDate not supplied

Data Science Challenges for Urban Air Mobility

Aviation is a combination of aircraft, airspace and airports. The data science life cycle comprises of five steps - capture, maintain, process, analyze and communicate. The presentation introduces the legacy of conventional aviation research in the context of the data science life cycle to motivate the challenges with Urban Air Mobility, a field that is quite nascent. A summary of recent research will be presented to highlight the innovative ways to address the challenges. Examples provided will include the generation of synthetic data, encounter models from simulations, and leveraging novel and diverse data sets from traditional transportation and non-aviation sources, to analyze problems of operation in urban airspace. Finally, opportunities will be identified for further exploration, niche development and filling the gaps in the field of data science for UAM.

Data Science↗

NASA NTRS2020

Data Science Challenges for Urban Air Mobility

Data Science↗

NASA NTRS1990

Upper Atmosphere Data Program (UADP)

Rapid, effective access to the increasing volume of information on stratospheric trace gases, both from measurements and model calculations, is important in advancing our knowledge of the Earth's atmosphere. The data have their origins in a variety of sources, ranging from satellite experiments and 2-D model predictions to individual balloon measurements, and are found in a variety of locations and formats. This task focuses on development and operation of an electronic data base for such trace gas data. The objectives are to provide effective access to these diverse data sets; to foster electronic data access, manipulation, and display; and to support periodic assessment and intercomparison activities.

Seals, R. K., Jr.↗

NASA NTRS1984

Meteorological factors in Earth-Satellite propagation

A 5-meter paraboloidal antenna operated at 28 GHz showed gain changes of 2 dB due to rain. While precise estimation of the corresponding angle of arrival changes is difficult, they appear to have been on the order of 0.02 degrees. The attenuation at 28.6 GHz inferred from radiometry agreed well with that measured simultaneously over a satellite/Earth link at the same frequency. The radiometers so calibrated have been used to add to the available site diversity data base using a 9 km baseline. An improved empirical model of site diversity gain was obtained by applying regression techniques to available published data. A brief review of the literature has led to suggestions for two experimental programs, one dealing with multifrequency radiometry and the other with the effects of the stochastic properties of precipitation on wideband data transmission.

Levis, C. A.↗

DOE OSTI2018

Development of city buildings dataset for urban building energy modeling

Urban building energy modeling (UBEM) is becoming a proven tool to support energy efficiency programs for buildings in cities. Development of a city-scale dataset of the existing building stock is a critical step of UBEM to automatically generate energy models of urban buildings and simulate their performance. This study introduces data needs, data standards, and data sources to develop city building datasets for UBEM. First, a literature review of data needs for UBEM was conducted. Then, the capabilities of the current data standards for city building datasets were reviewed. Moreover, the existing public data sources from several pioneer cites were studied to evaluate whether they are adequate to support UBEM. The results show that most cities have adequate public data to support UBEM; however, the data are represented in different formats without standardization, and there is a lack of common keys to make the data mapping easier. Finally, a case study is presented to integrate the diverse data sources from multiple city departments of San Francisco. The data mapping process is introduced and discussed. It is recommended to use the unique building identifiers as the common keys in the data sources to simplify the data mapping process. Furthermore, the integration methods and workflow are applied to other U.S. cities for developing the city-scale datasets of their existing building stock, including San Jose, Los Angeles, and Boston.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2012

A Multi-layer, Data-driven Advanced Reasoning Tool for Intelligent Data Mining and Analysis for Smart Grids

This paper presents the multi-layer, data-driven advanced reasoning tool (M-DART), a proof-of-principle decision support tool for improved power system operation. M-DART will cross-correlate and examine different data sources to assess anomalies, infer root causes, and anneal data into actionable information. By performing higher-level reasoning “triage” of diverse data sources, M-DART focuses on early detection of emerging power system events and identifies highest priority actions for the human decision maker. M-DART represents a significant advancement over today’s grid monitoring technologies that apply offline analyses to derive model-based guidelines for online real-time operations and use isolated data processing mechanisms focusing on individual data domains. The development of the M-DART will bridge these gaps by reasoning about results obtained from multiple data sources that are enabled by the smart grid infrastructure. This hybrid approach integrates a knowledge base that is trained offline but tuned online to capture model-based relationships while revealing complex causal relationships among data from different domains.

data mining, reasoning, smart grid, AMI, power gen↗

DOE OSTI2019

Integration of transcriptomic and proteomic data identifies biological functions in cell populations from human infant lung

Systems biology uses computational approaches to integrate diverse data types to understand cell and organ behavior. Data derived from complementary technologies, for example transcriptomic and proteomic analyses, are providing new insights into development and disease. We compared mRNA and protein profiles from purified endothelial, epithelial, immune, and mesenchymal cells from normal human lung tissue. Signatures for each cell type were identified and compared at both mRNA and protein levels. Cell specific biological processes and pathways were predicted by analysis of concordant and discordant RNA-protein pairs. Cell clustering and gene set enrichment comparisons identified shared versus unique processes associated with transcriptomic and/or proteomic data. Clear cell-cell correlations between mRNA and protein data were obtained from each cell type. Approximately 40% of RNA-protein pairs were coherently expressed. While the correlation between RNA and their protein products was relatively low (approximately 0.4), cell specific signature genes involved in functional processes characteristic of each cell type, were more highly correlated with their protein products. Consistency of cell specific RNA-protein signatures indicated an essential framework for the function of each cell type. Visualization and reutilization of the protein and RNA profiles are supported by a new web application, "LungProteomics," which is incorporated into the LGEA web portal and freely accessible at (https://research.cchmc.org/pbge/lunggens/lungProteomics/human_sorted_profiles.html).

human lung, sorted cells, integrated omics, Transc↗

DOE OSTI2020

The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules

Abstract Maximum diversification of data is a central theme in building generalized and accurate machine learning (ML) models. In chemistry, ML has been used to develop models for predicting molecular properties, for example quantum mechanics (QM) calculated potential energy surfaces and atomic charge models. The ANI-1x and ANI-1ccx ML-based general-purpose potentials for organic molecules were developed through active learning; an automated data diversification process. Here, we describe the ANI-1x and ANI-1ccx data sets. To demonstrate data diversity, we visualize it with a dimensionality reduction scheme, and contrast against existing data sets. The ANI-1x data set contains multiple QM properties from 5 M density functional theory calculations, while the ANI-1ccx data set contains 500 k data points obtained with an accurate CCSD(T)/CBS extrapolation. Approximately 14 million CPU core-hours were expended to generate this data. Multiple QM calculated properties for the chemical elements C, H, N, and O are provided: energies, atomic forces, multipole moments, atomic charges, etc. We provide this data to the community to aid research and development of ML models for chemistry.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Object detection with deep learning for rare event search in the GADGET II TPC

In the pursuit of identifying rare two-particle events within the GADGET II Time Projection Chamber (TPC), this paper presents a comprehensive approach for leveraging Convolutional Neural Networks (CNNs) and various data processing methods. To address the inherent complexities of 3D TPC track reconstructions, the data is expressed in 2D projections and 1D quantities. This approach capitalizes on the diverse data modalities of the TPC, allowing for the efficient representation of the distinct features of the 3D events, with no loss in topology uniqueness. Additionally, it leverages the computational efficiency of 2D CNNs and benefits from the extensive availability of pre-trained models. Given the scarcity of real training data for the rare events of interest, simulated events are used to train the models to detect real events. To account for potential distribution shifts when predominantly depending on simulations, significant perturbations are embedded within the simulations. This produces a broad parameter space that works to account for potential physics parameter and detector response variations and uncertainties. These parameter-varied simulations are used to train sensitive 2D CNN object detectors. When combined with 1D histogram peak detection algorithms, this multi-modal detection framework is highly adept at identifying rare, two-particle events in data taken during experiment 21072 at the Facility for Rare Isotope Beams (FRIB), demonstrating a 100% recall for events of interest. Here, we present the methods and outcomes of our investigation and discuss the potential future applications of these techniques.

Convolutional neural network↗

DOE OSTI2020

CoREC: Scalable and Resilient In-memory Data Staging for In-situ Workflows

The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Well Stimulation Treatment in California: Evaluation of Disclosure Data, May 2015 – October 2019

In this document, disclosure data from 1,228 well stimulations occurring May 2015 to October 2019 in California are evaluated. This evaluation updates a previous study that was based on 618 well stimulations occurring May 2015 to June 2016 (Stringfellow, W.T., Camarillo, M.K., and Jordan, P. 2017, Status of Well Stimulation in California Since Implementation of SB-4 Regulations, Berkeley National Laboratory, Berkeley, CA). While the goal of the previous study was to provide a summary of information obtained since passage of California Senate Bill No. 4—Oil and Gas, the goal of the current evaluation is to provide a basis for studying chemical indicators that could potentially be used to detect aquifer contamination. We evaluate chemical indicators is a separate document using the data contained herein as well as data from monitoring wells located within oil fields. Compared with the data used by Stringfellow et al. 2017, the current evaluation is based on a more diverse data set. While the previous study contained data for only two oil fields with more than 30 well stimulations, the current data set contains four oil fields with more than 30 well stimulations, making these data more ideal for comparisons using statistical tests. More producers and a new oil field, Buena Vista Nose, are represented in the current data set. Some well stimulation practices remain relatively unchanged since the previous study. Well stimulation is mostly occurring in Kern County with the exception of a single well stimulation in Orange County. Almost all well stimulations are hydraulic fracturing. The current data set contains two acid fracturing treatments and one matrix acidizing treatment. The median number of chemicals added per well stimulation—excluding water and proppant—was previously 21; the number in the expanded data set is 20. Median water added per well stimulation was previously 89,000 gallons and is now 98,000 gallons. While this water use represents a 10% increase, water use for hydraulic fracturing in California is still lower than water use in other oil and gas fields. Chemical formulations appear to have shifted since the previous study. This shift may be related to a change in the predominant service company. We identified 26 chemicals for which the frequency of use has changed by more than 30%. The total number of chemicals used has also expanded. Previously, 178 unique chemicals were identified as being added to well stimulation fluids. The total number of unique chemicals is now 205. Analytical data available for base and recovered fluids has expanded. The number of base fluid samples has increased from 12 to 35 and the number of recovered fluid samples has increased from 1,078 to 2,166. The expanded data set is more ideal for an evaluation of indicator chemicals and comparisons across different formations. The analytical data for recovered fluid samples indicates that many ions, radioactive constituents, and organics are consistently observed, making these chemicals ideal for consideration as potential indicators of aquifer contamination.

54 ENVIRONMENTAL SCIENCES↗