SEARCH · Engineering Papers

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Automating Traffic Microsimulation from SYNCHRO UTDF to SUMO

Modern transportation research relies on seamlessly integrating traffic signal data with robust network representation and simulation tools. This study presents utdf2gmns, an open-source Python tool that automates conversion of the Universal Traffic Data Format, including network representation, signalized intersections, and turning volumes into the General Modeling Network Specification (GMNS) Standard. The resulting GMNS-compliant network can be converted for microsimulation in SUMO. By automatically extracting intersection control parameters and aligning them with GMNS conventions, utdf2gmns minimizes manual preprocessing and data loss. utdf2gmns also integrates with the Sigma-X engine to extract and visualize key traffic control metrics, such as phasing diagrams, turning volumes, volume-tocapacity ratios, and control delays. This streamlined workflow enables efficient scenario testing, accurate model building, and consistent data management. Validated through case studies, utdf2gmns reliably models complex urban corridors, promoting reproducibility and standardization. Documentation is available on GitHub and PyPI, supporting easy integration and community engagement.

Luo, Roy [ORNL] (ORCID:0009000312909983)↗

DOE OSTI2012

Image Reconstruction of Imatron Electron-Beam Tomography Data

In this paper we describe the development of a filtered backprojection image reconstruction algorithm for an Imatron Electron-Beam Tomography (EBT) scanner. Raw data, software to perform data correction routines, and a technical document on the data format were all provided by Imatron. This document describes the basic geometry of the EBT scanner and the steps of the image reconstruction algorithm. Finally we demonstrate that the image quality our reconstruction algorithm is comparable with the reconstructed images provided by Imatron on an airport luggage data set.

42 ENGINEERING↗

NASA NTRS1993

Visualization techniques to aid in the analysis of multi-spectral astrophysical data sets

The goal of this project was to support the scientific analysis of multi-spectral astrophysical data by means of scientific visualization. Scientific visualization offers its greatest value if it is not used as a method separate or alternative to other data analysis methods but rather in addition to these methods. Together with quantitative analysis of data, such as offered by statistical analysis, image or signal processing, visualization attempts to explore all information inherent in astrophysical data in the most effective way. Data visualization is one aspect of data analysis. Our taxonomy as developed in Section 2 includes identification and access to existing information, preprocessing and quantitative analysis of data, visual representation and the user interface as major components to the software environment of astrophysical data analysis. In pursuing our goal to provide methods and tools for scientific visualization of multi-spectral astrophysical data, we therefore looked at scientific data analysis as one whole process, adding visualization tools to an already existing environment and integrating the various components that define a scientific data analysis environment. As long as the software development process of each component is separate from all other components, users of data analysis software are constantly interrupted in their scientific work in order to convert from one data format to another, or to move from one storage medium to another, or to switch from one user interface to another. We also took an in-depth look at scientific visualization and its underlying concepts, current visualization systems, their contributions, and their shortcomings. The role of data visualization is to stimulate mental processes different from quantitative data analysis, such as the perception of spatial relationships or the discovery of patterns or anomalies while browsing through large data sets. Visualization often leads to an intuitive understanding of the meaning of data values and their relationships by sacrificing accuracy in interpreting the data values. In order to be accurate in the interpretation, data values need to be measured, computed on, and compared to theoretical or empirical models (quantitative analysis). If visualization software hampers quantitative analysis (which happens with some commercial visualization products), its use is greatly diminished for astrophysical data analysis. The software system STAR (Scientific Toolkit for Astrophysical Research) was developed as a prototype during the course of the project to better understand the pragmatic concerns raised in the project. STAR led to a better understanding on the importance of collaboration between astrophysicists and computer scientists.

Brugel, Edward W.↗

NASA NTRS1993

Visualization techniques to aid in the analysis of multispectral astrophysical data sets

The goal of this project was to support the scientific analysis of multi-spectral astrophysical data by means of scientific visualization. Scientific visualization offers its greatest value if it is not used as a method separate or alternative to other data analysis methods but rather in addition to these methods. Together with quantitative analysis of data, such as offered by statistical analysis, image or signal processing, visualization attempts to explore all information inherent in astrophysical data in the most effective way. Data visualization is one aspect of data analysis. Our taxonomy as developed in Section 2 includes identification and access to existing information, preprocessing and quantitative analysis of data, visual representation and the user interface as major components to the software environment of astrophysical data analysis. In pursuing our goal to provide methods and tools for scientific visualization of multi-spectral astrophysical data, we therefore looked at scientific data analysis as one whole process, adding visualization tools to an already existing environment and integrating the various components that define a scientific data analysis environment. As long as the software development process of each component is separate from all other components, users of data analysis software are constantly interrupted in their scientific work in order to convert from one data format to another, or to move from one storage medium to another, or to switch from one user interface to another. We also took an in-depth look at scientific visualization and its underlying concepts, current visualization systems, their contributions and their shortcomings. The role of data visualization is to stimulate mental processes different from quantitative data analysis, such as the perception of spatial relationships or the discovery of patterns or anomalies while browsing through large data sets. Visualization often leads to an intuitive understanding of the meaning of data values and their relationships by sacrificing accuracy in interpreting the data values. In order to be accurate in the interpretation, data values need to be measured, computed on, and compared to theoretical or empirical models (quantitative analysis). If visualization software hampers quantitative analysis (which happens with some commercial visualization products), its use is greatly diminished for astrophysical data analysis. The software system STAR (Scientific Toolkit for Astrophysical Research) was developed as a prototype during the course of the project to better understand the pragmatic concerns raised in the project. STAR led to a better understanding on the importance of collaboration between astrophysicists and computer scientists. Twenty-one examples of the use of visualization for astrophysical data are included with this report. Sixteen publications related to efforts performed during or initiated through work on this project are listed at the end of this report.

Brugel, E. W.↗

DOE OSTI2016

A National Framework for Energy Audit Ordinances

A handful of U.S. cities have begun to incorporate energy audits into their building energy performance policies. Cities are beginning to recognize an opportunity to use several information tools to bring to real estate markets both motivation to improve efficiency and actionable pointers on how to improve. Care is necessary to combine such tools as operational ratings, energy audits, asset ratings, and building retro-commissioning in an effective policy regime that maximizes market impact. In this paper, the authors focus on energy audits and consider both the needs of the policies' implementers in local governments and the emerging standards and federal tools to improve data collection and practitioner engagement. Over the past two years, we have compared several related data formats such as New York City's existing audit reporting spreadsheet, ASHRAE guidance on building energy auditing, and the DOE Building Energy Asset Score, to identify a possible set of required and optional fields for energy audit reporting programs. Doing so revealed tensions between the ease of data collection and the value of more detailed information, which had implications for the effort and qualifications needed to complete the energy audit. The resulting list of data fields is now feeding back into the regulatory process in several cities currently working on implementing or developing audit policies. Using complementary policies and standardized tools for data transmission, the next generation of policies and programs will be tailored to local building stock and can more effectively target improvement opportunities through each building's life.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2024

h5bench: A unified benchmark suite for evaluating HDF5 I/O performance on pre‐exascale platforms

Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

ftmsRanalysis: An R package for exploratory data analysis and interactive visualization of FT-MS data

The high-resolution and mass accuracy of Fourier transform mass spectrometry (FT-MS) has made it an increasingly popular technique for discerning the composition of soil, plant and aquatic samples containing complex mixtures of proteins, carbohydrates, lipids, lignins, hydrocarbons, phytochemicals and other compounds. Thus, there is a growing demand for informatics tools to analyze FT-MS data that will aid investigators seeking to understand the availability of carbon compounds to biotic and abiotic oxidation and to compare fundamental chemical properties of complex samples across groups. We present ftmsRanalysis, an R package which provides an extensive collection of data formatting and processing, filtering, visualization, and sample and group comparison functionalities. The package provides a suite of plotting methods and enables expedient, flexible and interactive visualization of complex datasets through functions which link to a powerful and interactive visualization user interface, Trelliscope. Example analysis using FT-MS data from a soil microbiology study demonstrates the core functionality of the package and highlights the capabilities for producing interactive visualizations.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1976

Cubic spline function interpolation in atmosphere models for the software development laboratory: Formulation and data

A tabulation of selected altitude-correlated values of pressure, density, speed of sound, and coefficient of viscosity for each of six models of the atmosphere is presented in block data format. Interpolation for the desired atmospheric parameters is performed by using cubic spline functions. The recursive relations necessary to compute the cubic spline function coefficients are derived and implemented in subroutine form. Three companion subprograms, which form the preprocessor and processor, are also presented. These subprograms, together with the data element, compose the spline fit atmosphere package. Detailed FLOWGM flow charts and FORTRAN listings of the atmosphere package are presented in the appendix.

Kirkpatrick, J. C.↗

NASA NTRS1994

GSOSTATS Database: USAF Synchronous Satellite Catalog Data Conversion Software. User's Guide and Software Maintenance Manual, Version 2.1

The United States Air Force (USAF) provides NASA Lewis Research Center with monthly reports containing the Synchronous Satellite Catalog and the associated Two Line Mean Element Sets. The USAF Synchronous Satellite Catalog supplies satellite orbital parameters collected by an automated monitoring system and provided to Lewis Research Center as text files on magnetic tape. Software was developed to facilitate automated formatting, data normalization, cross-referencing, and error correction of Synchronous Satellite Catalog files before loading into the NASA Geosynchronous Satellite Orbital Statistics Database System (GSOSTATS). This document contains the User's Guide and Software Maintenance Manual with information necessary for installation, initialization, start-up, operation, error recovery, and termination of the software application. It also contains implementation details, modification aids, and software source code adaptations for use in future revisions.

Mallasch, Paul G.↗

DOE OSTI2024

Power System Waveform Datasets for Machine Learning

The desire for increased visibility across the electricity grid will necessarily increase the deployment of sensing and measurement devices and associated data management needs to unprecedented levels. For the existing sensing and measurement infrastructure, there remains a great amount of “value” yet to be extracted through advanced data management and analytics. Availability of more data will not, by itself, lead to changes in grid visibility, security, and resiliency. To create the predictive and prescriptive environment required to enable new markets and transactions for customer revenue and a reliable grid, the data must be collected, organized, evaluated, and analyzed using sophisticated algorithms to provide actionable information allowing operators and customers to reliably manage an increasingly complex grid. Progress in artificial intelligence (AI) has been largely driven by large, publicly available datasets that can be used to train AI algorithms such as MNIST, a database of handwritten images of digits, and ImageNet, an image database of everyday objects. These types of publicly available databases of real-world training datasets have been largely credited for advancement of image processing, computer vision, and deep learning algorithms that these use cases deploy. However, in the power systems industry to date, there are few databases with proper event labeling, and data access to a publicly available collection of power system event waveforms that will allow users to interact with grid signature data. Publicly available datasets of power system event waveforms, such as the DOE/EPRI dataset, often lack critical metadata or contain limited examples of each event type, and data formats vary widely across these datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

Quillinan, et al 2018 DOE Geothermal Technology Office REE Report for NEWTS Database and Case Studies

Produced water data processed into the NEWTS data format for easy input into aqueous chemistry modeling software, including oil & gas and coal bed methane produced waters, and geothermal waters. Includes information on rare earth elements (REEs) and critical minerals (CMs). Case studies are included to demonstrate solved streams using aqueous chemistry software. An input template is provided for modeling stream data in OLI Studio. Original data from: Quillinan, Scott, Nye, Charles, Engle, Mark, Bartos, Timothy T., Neupane, Ghanashyam, Brant, Jonathan, Bagdonas, Davin, McLing, Travis, McLaughlin, J. Fred, Phillips, Erin, Hallberg, Laura L., Shahabadi, Mahdi, and Johnson, Matthew. Assessing rare earth element concentrations in geothermal and oil and gas produced waters: A potential domestic source of strategic mineral commodities (Final Report). United States: N. p., 2018. Web. doi:10.2172/1509037.

Aqueous Chemistry↗

DOE OSTI2020

TripleGraph

RDF triplestores are great tools for online graph analytic processing (i.e., graph pattern query processing), but they do not provide graph mining capabilities (e.g., PageRank, connected-component analysis, node eccentricity, etc.). The software title “TripleGraph” is a graph analysis toolkit, which uses an RDF triplestore as its backend for creating, manipulating, mining, and programming with large scale property graphs. It allows users to run various graph mining algorithms easily. User can import edgelist-formatted (homogeneous graph) or JSON-formatted graph (property graph) into the RDF triplestore using the provided tool and perform various analysis such as (1) Node/edge retrieval and manipulation, (2) Pathfinding between two given nodes, (3) Running graph mining algorithms (PageRank/Personalized PageRank, Single Source Shortest Path/Multi-Source Shortest Path, Connected Component, Node Eccentricity, Peer Pressure Clustering). It supports standard graph data format and works with a standard SPARQL endpoint like Jena Fuseki. It allows users to perform online graph analytic processing and graph mining on the same platform (a triplestore).

Sangkeun, MattLee↗

NASA NTRS1978

Tracking performance of unbalanced QPSK demodulators. I - Biphase Costas loop with passive arm filters

Unbalanced quadriphase-shift-keying (QPSK) is an attractive means for transmitting two digital data streams which in general have different average powers, data rates, and data formats. Previous analyses of the tracking performance of Costas loop demodulators of unbalanced QPSK have accounted only for the filtering effect produced by the loop's two arm filters on the equivalent additive noise perturbing the loop. When the bandwidth of these filters is selected on the basis of the order of the data rate, as is typical of optimum Costas loop design, the filtering degradations of the data modulations themselves and the cross-modulation noise produced by their multiplication in the loop often cannot be neglected. The purpose of this paper is to incorporate these additional filtering effects into the analysis. Many of the results obtained herein are in the form of closed-form expressions which can easily be evaluated numerically for design and performance prediction purposes.

Simon, M. K.↗

NASA NTRS2015

Explore GPM IMERG and Other Global Precipitation Products with GES DISC GIOVANNI

New features and capabilities in the newly released GIOVANNI allow exploring GPM IMERG (Integrated Multi-satelliE Retrievals for GPM) Early, Late and Final Run global half-hourly and monthly precipitation products as well as other precipitation products distributed by the GES DISC such as TRMM Multi-Satellite Precipitation Analysis (TMPA), MERRA (Modern Era Retrospective-Analysis for Research and Applications), NLDAS (North American Land Data Assimilation Systems), GLDAS (Global Land Data Assimilation Systems), etc. GIOVANNI is a web-based tool developed by the GES DISC (Goddard Earth Sciences and Data Information Services Center) to visualize and analyze Earth science data without having to download data and software. The new interface in GIOVANNI allows searching and filtering precipitation products from different NASA missions and projects and expands the capabilities to inter-compare different precipitation products in one interface. Knowing differences in precipitation products is important to identify issues in retrieval algorithms, biases, uncertainties, etc. Due to different formats, data structures, units and so on, it is not easy to inter-compare precipitation products. Newly added features and capabilities (unit conversion, regridding, etc.) in GIOVANNI make inter-comparisons possible. In this presentation, we will describe these new features and capabilities along with examples.

Giovanni↗

NASA NTRS2018

ATD-2 TTP Traffic Management Messages

This document is the fifth of five documents that describe the TFDM Terminal Publication (TTP) implementation within ATD-2 including data message formats and data schemas.

ATD-2↗

NASA NTRSDate not supplied

The Satellite Flythrough and Reconstruction Softwares at CCMC

The next-generation of model data visualization to be offered at the Community Coordinated Modeling Center (CCMC) will be based on Kamodo, an open source python package. To increase the usefulness of our services, we are also developing new capabilities based on this software, called the satellite flythrough and the reconstruction tools, to be offered both as packages for offline analysis and through an online interface (coming soon). The satellite flythrough software ‘flies’ a satellite trajectory, whether real or imaginary, through either model data hosted at CCMC or on a personal machine. This service greatly simplifies the complexity of users’ access to model data, abstracting away the time-consuming details of model data formats and interpolation. We demonstrate execution times of a few seconds to a few minutes for several example flythroughs of a trajectory stretching over a few days, depending on the parameters chosen. We also demonstrate a reconstruction tool built on top of the satellite flythrough software, for use with mission planning and model-data comparisons. This tool, based on reconstructions provided for the GDC Science and Technology Definition Team, converts multiple, simultaneous satellite flythroughs into two-dimensional reconstructions. The reconstruction tool provides a software capability for satellite constellations to determine how many satellites are needed and in what configuration to resolve the desired features in the model data. Both tools are currently available through GitHub for a selection of CCMC-hosted ITM models. Finally, we present initial results from work in progress and plans for future work, including an expansion of the reconstruction tool to provide 3D reconstruction capabilities and a line-of-sight calculation tool.

software, python↗

NASA NTRS2008

Lunar Applications in Reconfigurable Computing

NASA s Constellation Program is developing a lunar surface outpost in which reconfigurable computing will play a significant role. Reconfigurable systems provide a number of benefits over conventional software-based implementations including performance and power efficiency, while the use of standardized reconfigurable hardware provides opportunities to reduce logistical overhead. The current vision for the lunar surface architecture includes habitation, mobility, and communications systems, each of which greatly benefit from reconfigurable hardware in applications including video processing, natural feature recognition, data formatting, IP offload processing, and embedded control systems. In deploying reprogrammable hardware, considerations similar to those of software systems must be managed. There needs to be a mechanism for discovery enabling applications to locate and utilize the available resources. Also, application interfaces are needed to provide for both configuring the resources as well as transferring data between the application and the reconfigurable hardware. Each of these topics are explored in the context of deploying reconfigurable resources as an integral aspect of the lunar exploration architecture.

Somervill, Kevin↗

DOE OSTI2025

ExaCA v2.0: A versatile, scalable, and performance portable cellular automata application for additive manufacturing solidification

The previously established ExaCA software for performance portable alloy grain structure simulation has been updated to better represent the solidification behavior during complex alloy processing conditions, such as those encountered during metal additive manufacturing (AM), and for improved performance and scalability. Here, an extension to the time–temperature history input data format and the core ExaCA algorithm to include an arbitrary number of melting and solidification events yielded improved prediction of texture for various melt pool geometries, expanding the range of AM-relevant conditions that can be accurately simulated. Improved heat transport process simulation coupling, including the creation of large raster datasets from single track time–temperature history data and in-memory coupling with the new, performance portable finite difference code Finch, were also demonstrated in example studies on the effect of multilayer AM microstructure predictions on hatch spacing and cell size, respectively. Additional new features are detailed and demonstrated, including the ability to perform simulations using various interfacial response function forms, execute simulations on state-of-the-art hardware, improved usability through post-processing versatility, and improved strong and weak scaling performance. The performance, physics, and versatility improvements demonstrated here will further enable large-scale studies on AM process–microstructure relationships that were not previously possible. Furthermore, the usability improvements and ability to run coupled AM process–microstructure simulations using the Finch-ExaCA workflow will facilitate broader use of this open-source software by the computational materials community.

36 MATERIALS SCIENCE↗