Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data representations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Exploiting Colorimetry for Fidelity in Data Visualization

Advances in multimodal characterization methods fuel the generation of increasing immense hyper-dimensional data sets. Color mapping is employed for conveying higher dimensional data in two-dimensional (2D) representations for human consumption without relying on multiple projections. How one constructs these color maps, however, critically affects how accurately one perceives data. For simple scalar fields, perceptually uniform color maps and color selection have been shown to improve data readability and interpretation across research fields. In this work, we review core concepts underlying the design of perceptually uniform color maps and extend the concepts from scalar fields to two-dimensional vector fields and three-component composition fields frequently found in materials-chemistry research to enable high-fidelity visualization. Additionally, we develop the software tools PAPUC and CMPUC to enable researchers to utilize these colorimetry principles and employ perceptually uniform color spaces for rigorously meaningful color mapping of higher dimensional data representations. Last, we demonstrate how these approaches deliver immediate improvements in data readability and interpretation in microscopies and spectroscopies routinely used in discerning materials structure, chemistry, and properties.

36 MATERIALS SCIENCE↗

Evaluating Awkward Arrays, uproot, and coffea as a query platform for High Energy Physics Data

Query languages for High Energy Physics (HEP) are an ever present topic within the field. A query language that can efficiently represent the nested data structures that encode the statistical and physical meaning of HEP data will help analysts by ensuring their code is more clear and pertinent. As the result of a multi-year effort to develop an in-memory columnar representation of high energy physics data, the NumPy, Awkward Array, and uproot Python packages present a mature and efficient interface to HEP data. Atop that base, the coffea package adds functionality to launch queries at scale, manage and apply experiment-specific transformations to data, and present a rich object-oriented columnar data representation to the analyst. Recently, a set of Analysis Description Language (ADL) benchmarks has been established to compare HEP queries in multiple languages and frameworks. In this paper we present these benchmark queries implemented within the coffea framework and discuss their readability and performance characteristics. We find that the columnar queries perform as well or better than the implementations given in previous studies.

Gray, L.↗

Storage and Read-Optimized Data Placement Structures for High-Performance Analysis [Book Chapter]

Here, we present state-of-art structures and methods for efficient data preparation and representation for analysis. Our intent is to introduce the data science and analytics communities to open source data placements, structures, and methods. These practices can make the foundational processes of data preparation and access dramatically more efficient than typical raw file or database representations and use more conservative storage. To illustrate this, we introduce two highly efficient data placement structures. We then present a tutorial, supported by step-by-step examples, of how to create, use, and access data, structured by Parquet or ORC (Optimized Row Columnar), using Apache Spark. Finally, we illustrate the benefits of using these structures with computational and storage volume benchmarks.

97 MATHEMATICS AND COMPUTING↗

Grouper: A Compact, Streamable Triangle Mesh Data Structure

Here, we present Grouper: an all-in-one compact file format, random-access data structure, and streamable representation for large triangle meshes. Similarly to the recently published SQuad representation, Grouper represents the geometry and connectivity of a mesh by grouping vertices and triangles into fixed-size records, most of which store two adjacent triangles and a shared vertex. Unlike SQuad, however, Grouper interleaves geometry with connectivity and uses a new connectivity representation to ensure that vertices and triangles can be stored in a coherent order that enables memory-efficient sequential stream processing. We also present a linear-time construction algorithm that allows streaming out Grouper meshes using a small memory footprint while preserving the initial ordering of vertices. In this construction, we show how the problem of assigning vertices and triangles to groups reduces to a well-known NP-hard optimization problem, and present a simple yet effective heuristic solution that performs well in practice. Our array-based Grouper representation also doubles as a triangle mesh data structure that allows direct access to vertices and triangles. Storing only about two integer references per triangle-i.e., less than the three vertex references stored with each triangle in a conventional indexed mesh format-Grouper answers both incidence and adjacency queries in amortized constant time. Our compact representation enables data-parallel processing on multicore computers, instant partitioning and fast transmission for distributed processing, as well as efficient out-of-core access. We demonstrate the versatility and performance benefits of Grouper using a suite of example meshes and processing kernels.

97 MATHEMATICS AND COMPUTING↗

Development of High Resolution Land Surface Parameters for the Community Land Model

There is a growing need for high-resolution land surface parameters as land surface models are being applied at increasingly higher spatial resolution offline as well as in regional and global models. The default land surface parameters for the most recent version of the Community Land Model (i.e. CLM 4.0) are at 0.5° or coarser resolutions, released with the Community Earth System Model (CESM). Plant Functional Types (PFTs), vegetation properties such as Leaf Area Index (LAI), Stem Area Index (SAI), and non-vegetated land covers were developed using remotely sensed datasets retrieved in late 1990’s and the beginning of this century. In this study, we developed new land surface parameters for CLM 4.0, specifically PFTs, LAI, SAI and non-vegetated land cover composition, at 0.05° resolution globally based on the most recent MODIS land cover and improved MODIS LAI products. Compared to the current CLM 4.0 parameters, the new parameters produced a decreased coverage by bare soil and trees, but an increased coverage by shrub, grass, and cropland. The new parameters result in a decrease in global seasonal LAI, with the biggest decrease in boreal forests; however, the new parameters also show a large increase in LAI in tropical forest. Differences between the new and the current parameters are mainly caused by changes in the sources of remotely sensed data and the representation of land cover in the source data. Advantages and disadvantages of each dataset were discussed in order to provide guidance on the use of the data. The new high-resolution land surface parameters have been used in a coupled land-atmosphere model (WRF-CLM) applied to the western U.S. to demonstrate their use in high-resolution modeling. A remapping method from the latitude/longitude grid of the CLM data to the WRF grids with map projection was also demonstrated. Future work will include global offline CLM simulations to examine the impacts of source data resolution and subsequent land parameter changes on simulated land surface processes.

Environmental Molecular Sciences Laboratory↗

Automated Computational Thermochemistry for Butane Oxidation: A Prelude to Predictive Automated Combustion Kinetics

arge-scale implementation of high level computational theoretical chemical kinetics offers the prospect for dramatically improving the fidelity of combustion chemical modeling. As a first step toward this goal, we developed a procedure for automatically generating the thermochemical data for combustion of an arbitrary fuel. The procedure begins by producing a list of combustion relevant species from a specification of the fuel and combustion conditions of interest. Then, for each element in the list of species, the procedure determines an internal coordinate z-matrix description of its structure, the optimal torsional configuration via Monte Carlo sampling, key rovibrational properties for that optimal geometry (including anharmonic corrections from torsional mappings and/or vibrational perturbation theory), and high level estimates of the electronic and zero-point energies via arbitrarily defined composite methods. This dataset is then converted first to partition functions, then to thermodynamic properties, and finally to NASA polynomial representations of the data. The end product is an automatically generated database of electronic structure results and thermochemical data including representations in a format appropriate for combustion simulations. The utility and functioning of this predictive automated computational thermochemistry (PACT) software package is illustrated through application to the automated generation of thermochemical data for the combustion of n-butane. Butane is chosen for this demonstration as its species list is of reasonably manageable size for debugging level computations, while still presenting most of the key challenges that need to be surmounted in the consideration of larger fuels. Furthermore, its low temperature chemistry is representative of that oc-curring with larger alkanes.

Keceli, Murat↗

Particle trajectory representation learning with masked point modeling

Liquid argon time projection chambers (LArTPCs) offer millimeter-scale 3D images of particle trajectories, enabling precision studies of neutrino oscillation, detection of supernova and solar neutrinos, searches for exotic dark matter, and proton decay. Current approaches utilize supervised machine learning models, requiring extensive simulations of particle physics and detector response that can introduce bias. Self-supervised learning (SSL), a machine learning approach that learns useful representations of unlabeled data from the data itself, has significantly advanced how large datasets are utilized for representation learning; however, its potential for applications to sensory data in high precision particle physics experiments remains largely unexplored. We introduce the Point-based liquid argon masked autoencoder (PoLAr-MAE), a self-supervised framework that learns physically meaningful representations directly from unlabeled LArTPC images. PoLAr-MAE achieves remarkable data efficiency for a point-level segmentation task, outperforming fully supervised methods in low data regimes. Linear classifiers on model outputs demonstrate robust performance across multiple downstream tasks. Our results position sensor-level SSL as a practical foundation model strategy for LArTPCs.

Young, Samuel [Stanford Univ., CA (United States)]↗

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture resulting in high-dimensional, difficult to interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper we apply cutting edge techniques from TDA with the goal of gaining insight towards interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers and discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight as to how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.

topological data analysis, deep learning↗

RADEMACHER COMPLEXITY REGULARIZATION FOR CORRELATION-BASED MULTIVIEW REPRESENTATION LEARNING

Deep correlation-based multiview representation learning techniques have become increasingly popular methods for extracting highly correlated representations from multiview data. However, their ability to find highly complex mappings between the views can also lead to overfitting and overly correlated representations. In this work, we propose a regularizer for this specific problem, based on the Rademacher complexity of the DNNs, tailored for multiview correlation maximization. We demonstrate that the proposed regularization leads to less noisy representations in synthetic data and improved performance of downstream tasks in real-world multiview datasets.

Kuschel, Maurice↗

Simulation INsight and Analysis

Sina is a tool set for modern scientific data management that provides flexible, light-weight support of non-bulk data capture for retention in and queries against SQL and noSQL data stores. HPC simulations traditionally maintain their data in files. Extracting data of interest for subsequent analysis then requires the time-consuming process of traversing directories and scraping data from files in a variety of formats. Sina facilitates capturing relevant data during execution or post-processing of simulation runs for retention in and queries from a modern data store. The tools are sufficiently general to allow for the inclusion of new fields as scientists learn more about their data. Libraries, currently in C++ and Python, and a command line interface (CLI) are provided. Sina's flexibility starts with a general schema, in JSON, for the collection of non-bulk simulation data. JSON provides a flexible, human-readable representation of the data that of interest. Sina currently has a C++ library for simulations to write data to and read from a schema-compliant file for subsequent ingestion into one of the supported data stores. However, applications are free to write their data directly into a schema-compliant file. Python packages provide data ingestion, management, query, and export capabilities. A command line interface (CLI) provides simplified access to these features. A common application programming interface (API) is used to maintain and query data in any of the supported data stores, which are currently limited to SQL and Apache Cassandra (a column store). Tutorials, demonstrations, and examples illustrate aspects of the process using scripts and Jupyter notebooks.

Pauli, EstebanT↗

An Interactive Multi-instrument Database of Solar Flares

Solar flares are complicated physical phenomena that are observable in a broad range of the electromagnetic spectrum, from radio waves to γ -rays. For a more comprehensive understanding of flares, it is necessary to perform a combined multi-wavelength analysis using observations from many satellites and ground-based observatories. For an efficient data search, integration of different flare lists, and representation of observational data, we have developed the Interactive Multi-Instrument Database of Solar Flares (IMIDSF, https://solarflare.njit.edu/). The web-accessible database is fully functional and allows the user to search for uniquely identified flare events based on their physical descriptors and the availability of observations by a particular set of instruments. Currently, the data from three primary flare lists ( Geostationary Operational Environmental Satellites , RHESSI , and HEK) and a variety of other event catalogs ( Hinode , Fermi GBM, Konus- W IND, the OVSA flare catalogs, the CACTus CME catalog, the Filament eruption catalog) and observing logs ( IRIS and Nobeyama coverage) are integrated, and an additional set of physical descriptors (temperature and emission measure) is provided along with an observing summary, data links, and multi-wavelength light curves for each flare event since 2002 January. We envision that this new tool will allow researchers to significantly speed up the search of events of interest for statistical and case studies.

79 ASTRONOMY AND ASTROPHYSICS↗

Malicious Cyber Activity Detection using Zigzag Persistence

In this study we synthesize zigzag persistence from topological data analysis with autoencoder-based approaches to detect malicious cyber activity, and derive analytic insights. Cybersecurity aims to safeguard computers, networks, and servers from various forms of malicious attacks, including network damage, data theft, and activity monitoring. We focus on the cybersecurity domain and investigate the detection of malicious activity using log data. We consider the dynamics of the log data and explore the changing topology of a hypergraph representation of this data to gain insights into the underlying activity. These hypergraphs capture complex interactions between processes, together with their temporal information. To study the changing topology we use zigzag persistence, which captures how topological features persist at multiple dimensions over time. We observe that this detects malicious activity in a cyber data set. To automate this detection we implement an autoencoder trained on a vectorization of the resulting zigzag persistence barcodes. Our experimental results demonstrate the effectiveness of the autoencoder in detecting malicious activity. Overall, this study highlights the potential of zigzag persistence and its combination with temporal hypergraphs for analyzing cybersecurity log data and detecting malicious behavior.

hypergraphs, temporal hypergraph, topological data↗

A New Default Colormap for ParaView

ParaView is one of the most prominent software tools for scientific visualization used by scientists around the world. Color is a primary conduit to visually map data to its representation and, thus, enable investigation and interpretation of the data. Colormap selection has a significant impact on the data revealed; its design and selection is a critical aspect of scientific data visualization. A common choice for a user is the program’s default colormap, so careful consideration of this default is consequential. Although the current default colormap in ParaView, a succession of hues from cool blue to warm red, has served the community well, research shows that more nuanced colormap configurations increase discriminability while maintaining other critical metrics. These findings inspire us to revisit and update the default colors in ParaView. Here, in this study, we present a new ParaView default colormap, the criteria and methods of development, and example visualizations and analytic metrics.

97 MATHEMATICS AND COMPUTING↗

The circular bioeconomy: a driver for system integration

Background: Human and earth system modeling, traditionally centered on the interplay between the energy system and the atmosphere, are facing a paradigm shift. The Intergovernmental Panel on Climate Change’s mandate for comprehensive, cross-sectoral climate action emphasizes avoiding the vulnerabilities of narrow sectoral approaches. Our study explores the circular bioeconomy, highlighting the intricate interconnections among agriculture, forestry, aquaculture, technological advancements, and ecological recycling. Collectively, these sectors play a pivotal role in supplying essential resources to meet the food, material, and energy needs of a growing global population. We pose the pertinent question of what it takes to integrate these multifaceted sectors into a new era of holistic systems thinking and planning. Results: The foundation for discussion is provided by a novel graphical representation encompassing statistical data on food, materials, energy flows, and circularity. This representation aids in constructing an inventory of technological advancements and climate actions that have the potential to significantly reshape the structure and scale of the economic metabolism in the coming decades. In this context, the three dominant mega-trends—population dynamics, economic developments, and the climate crisis—compel us to address the potential consequences of the identified actions, all of which fall under the four categories of substitution, efficiency, sufficiency, and reliability measures. Substitution and efficiency measures currently dominate systems modeling. Including novel bio-based processes and circularity aspects might require only expanded system boundaries. Conversely, paradigm shifts in systems engineering are expected to center on sufficiency and reliability actions. Effectively assessing the impact of sufficiency measures will necessitate substantial progress in inter- and transdisciplinary collaboration, primarily due to their non-technological nature. In addition, placing emphasis on modeling the reliability and resilience of transformation pathways represents a distinct and emerging frontier that highlights the significance of an integrated network of networks. Conclusions: Existing and emerging circular bioeconomy practices can serve as prime examples of system integration. These practices facilitate the interconnection of complex biomass supply chain networks with other networks encompassing feedstock-independent renewable power, hydrogen, CO 2 , water, and other biotic, abiotic, and intangible resources. Elevating the prominence of these connectors will empower policymakers to steer the amplification of synergies and mitigation of tradeoffs among systems, sectors, and goals.

09 BIOMASS FUELS↗

Artificial Neural Network Representation of Dynamic Criticality Excursion Experiment Data

This paper outlines time dependent considerations in applying the artificial neural network (ANN) paradigm to dynamic nuclear criticality excursion experiment data. The ANN amongst a variety of uses, can apply to function fitting of multiple input-multiple output data sets, pattern recognition, cluster analysis, and time series prediction. The ANN is a component of Machine Learning (ML) data analysis. The practical application of ML to criticality safety has been funded through the Y-12 Plant Directed Research and Demonstration (PDRD) program to support criticality safety and emergency planning efforts. The initial effort considered function fitting of criticality excursion data. Recent efforts successfully extended the ANN to criticality safety code validation. The initial ANN work applied to criticality excursions centered around multivariate function fitting of static (tabular) data taken from the French Consequences Radiologiques d'un Accident de Criticite (CRAC) and SILENE reactor campaigns. The former report (LeCorche) summarized important tabular data for the first 44 CRAC experiments. The initial ANN was trained to specific time-independent inputs, then applied to predict peak fission rate and total fissions for a wide variety of fissile solutions process conditions. This work extends that effort to apply the ANN paradigm to dynamic prediction of time-dependent criticality excursion experiment data. A dynamic criticality excursion in fissile solution represents a complex multiphysics phenomenon. Factors include among other things, coupled neutronic, multiphase fluid flow, temperature, pressure, acoustic shock, boiling heat transfer, reactivity feedback, and radiolytic gas considerations. There have been many efforts to model (over time) as coupled ordinary and (space) as partial differential equations (ODE, PDE). The ANN representation noted here does not rely on solving explicit ODE or PDEs. Rather the intent is to demonstrate that as a time series, the criticality excursion can be predicted by training specific ANN architectures to experimental input and output time series exclusively. The rationale for extending this work to the time-dependent regime is important in developing a 'criticality excursion emulator' solely on the basis of learning characteristics from experiment dynamic data. An emulator is different than a computer code simulator driven by the solution of ODEs and PDEs given select 'tuning parameters' that match outputs to experiments. A criticality excursion emulator, derived exclusively from an ANN or ML, could predict most likely excursion characteristics, trained from historic dynamic experiment data, presented with input values within the range of those experiments. The work noted shows that generic time-series ANN architectures can be satisfactorily applied to the prediction of dynamic, extremely chaotic criticality excursion power given an initial time dependent input series. In reality, an excursion emulator could be driven by user-supplied initial (t=0), time independent or constant values to generate time dependent outputs. Thus ongoing work includes developing a custom ANN such that initial dynamic conditions would be predicted. Those results then serve as inputs to a customized ANN architecture currently in development. (author)

42 ENGINEERING↗

Getting Started with Evaluations with Means and Uncertainties (EMU 3.0)

One of the most fundamental quantities in nuclear physics is the reaction cross section. A cross section represents the probability that a nuclear reaction will resolve through a given channel given a target nucleus and a projectile with a certain energy. A nuclear evaluation is a set of discrete data and interpolation rules to convert those discrete nuclear reaction data—such as the cross section—into a continuous function at arbitrary energies. Evaluated nuclear data files that can appear in Evaluated Nuclear Data File (ENDF) and Generalized Nuclear Data Structure (GNDS) formats, storing a “most-complete” discretized representation of nuclear data, based on both experimental measurements and theory models.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Plotting XPS Narrow Scans (and other Data) with MATLAB’s Powerful Plotting Capabilities, 2. An Introduction to Handle Graphics

The graphical representation of numerical data plays a vital role in chemistry, materials science, and every other scientific field. No matter how good your results are, poorly represented data can ultimately make your work confusing. Indeed, with an increase in the amount of data that is being gathered, e.g., by the Materials Genome Initiative, or in X-ray photoelectron spectroscopy (XPS) or time-of-flight secondary ion mass spectrometry (ToF-SIMS) depth profiles, the importance of effectively representing data has increased. Good figures also matter because many researchers will not look at more than the title, abstract, and perhaps a few key figures from a paper.

42 ENGINEERING↗

All-sky interferometry with spherical harmonic transit telescopes

In this paper, we describe the spherical harmonic transit telescope through the use of a novel formalism for the analysis of transit radio telescopes. This all-sky approach bypasses the curved-sky complications of traditional interferometry and so is particularly well-suited to the analysis of wide-field radio interferometers. It enables compact and computationally efficient representations of the data and its statistics, which allow new ways of approaching important problems like map-making and foreground removal. In particular, we show how it enables the use of the Karhunen-Loève transform as a highly effective foreground filter, suppressing realistic foreground residuals for our fiducial example by at least a factor 20 below the 21 cm signal, even in highly contaminated regions of the sky. This is despite the presence of the mode-mixing inherent in real-world instruments with frequency-dependent beams. We show, using Fisher forecasting, that foreground cleaning has little effect on power spectrum constraints compared to hypothetical foreground-free measurements. Beyond providing a natural real-world data analysis framework for 21 cm telescopes now under construction and future experiments, this formalism allows accurate power spectrum forecasts to be made that include the interplay of design constraints and realistic experimental systematics with 21st century 21 cm science.

79 ASTRONOMY AND ASTROPHYSICS↗