Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “File System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Low-Cost, User-Friendly, Rapid Analysis of Dynamic Data System Established

An issue of primary importance to the development of new jet and certain other airbreathing combined-cycle powered aircraft is the advancement of airframe-integrated propulsion technologies. Namely, engine inlets and their systems and subsystems are required to capture, convert, and deliver the atmospheric airflow demanded by such engines across their operating envelope in a form that can be used to provide efficient, stable thrust. This must be done while also minimizing aircraft drag and weight. Revolutionary inlet designs aided by new technologies are needed to enable new missions. An unwanted byproduct of pursuing these inlet technologies is increased time-variant airflow distortion. Such distortions reduce propulsion system stability, performance, operability, and life. To countermand these limitations and fully evaluate the resulting configurations, best practices dictate that this distortion be experimentally measured at large scale and analyzed. The required measurements consist of those made by an array of high-response pressure transducers located in the flow field at the aerodynamic interface plane (AIP) between the inlet and engine. Although the acquisition of the necessary pitot-pressure time histories is relatively straight-forward, until recent years, the analysis has proved to be very time-consuming, tedious, and expensive. To transform the analysis of these data into a tractable and timely proposition, researchers at the NASA Glenn Research Center created and established the Rapid Analysis of Dynamic Data (RADD) system. The system provides complete, near real-time analysis of time-varying inlet airflow distortion datasets with report quality output. This fully digital approach employs Institute of Electrical and Electronics Engineers (IEEE) binary data file format standardization to establish data-acquisition-system-independent processing on low cost personal computers. Features include invalid instrumentation code-out, logging, and multiple replacement schemes as needed for each channel of instrumentation. The AIP pressure distribution can be interpolated to simulate measurements by alternate AIP probe arrays, if desired. In addition, the RADD system provides for the application of filters that can be used to focus the analysis on the frequency range of interest.

Arend, David J.↗
NASA NTRS2019

Forming Aggregations using Virtual Sharding: Lessons Learned from Simple Scalable Storage (S3)

Data aggregation is the ability to combine separate datasets to form a single new logical dataset provides users with a powerful abstraction. The advantage of an aggregate dataset is that the users are freed from having to understand, and incorporate into their workflow, knowledge about the (ad hoc) organization of the constituent datasets. However, aggregating large numbers of files can be computationally complex with data server systems performing many repetitive operations. As part of the authors work on subsetting data stored on Amazon Web Service (AWS) Simple Storage Service (S3), we developed technology to read portions of otherwise monolithic data files. This enables the formation of virtual shards for user in subsetting data stored in HDF5 (hierarchical data format, version 5) files. This same tool can be used to form aggregations that combine data stored in many HDF5 files when those files are stored on S3. The nature of the virtual sharding and the algorithm that exploits it for subsetting is such that it can also be used for aggregation with the need for many of the repetitive operations required by the per file aggregation techniques. We will present timing information that demonstrates the flexibility of this approach. However, the lessons learned is that while this is a useful result in and of itself, these very same techniques can be applied in other contexts where data are stored in services and on media other than S3. For example, this same technique can be applied to data stored on spinning disk. Pushing the envelope for S3 forced a reexamination of our data access techniques which lead to unexpected positive benefits.

Gallagher, James↗
DOE OSTI2024

Generation and validation of comprehensive synthetic weather histories using auto-regressive moving-average models

As energy system design moves to more complex methods of optimization including machine learning there is a significant need for more weather data than is available. One method to solve this is using synthetic data models such as the auto-regressive moving-average (ARMA) model which has been frequently utilized to create such data. This paper looks at extending the ARMA algorithm to generate solar components through the use of clearsky detrending, maintaining vector relationships and by leveraging physical relationships. The method for the creation of entirely synthetic weather data files including key weather variables for energy system analysis is presented. Furthermore, a detailed comparison of energy system simulations utilizing both real and synthetic data is made using NREL’s System Advisor Model. Whilst good agreement is made for the solar variables, and other weather variables, ARMA methods often fail to capture the standard deviation and skew of annual weather distributions. Vector-ARMA is shown to maintain correlations between variables and thus generate data sets that perform similarly in energy system design. Here, it is finally shown that the ARMA method fails to preserve day-today correlations in weather variables and thus over-predicts optimal energy storage by 21% for a residential solar application.

42 ENGINEERING↗
NASA NTRS1992

The challenge of a data storage hierarchy

A discussion of Mesa Archival Systems' data archiving system is presented. This data archiving system is strictly a software system that is implemented on a mainframe and manages the data into permanent file storage. Emphasis is placed on the fact that any kind of client system on the network can be connected through the Unix interface of the data archiving system.

Ruderman, Michael↗
NASA NTRS1981

Command system output bit verification

An automatic test was developed to test the ability of the deep space station (DSS) command subsystem and exciter to generate and radiate, from the exciter, the correct idle bit sequence for a given flight project or to store and radiate received command data elements and files without alteration. This test, called the command system output bit verification test, is an extension of the command system performance test (SPT) and can be selected as an SPT option. The test compares the bit stream radiated from the DSS exciter with reference sequences generated by the SPT software program. The command subsystem and exciter are verified when the bit stream and reference sequences are identical. It is a key element of the acceptance testing conducted on the command processor assembly (CPA) operational program (DMC-0584-OP-G) prior to its transfer from development to operations.

Odd, C. W.↗
DOE OSTI2024

Pan-Arctic daily streamflow at 0.5-degree resolution

Data was derived using daily runoff from models participating in the Coupled Model Intercomparison Project, version 6 (CMIP6) that have been routed through the Model for Scale Adaptive River Transport (MOSART) routing model to represent streamflow at individual streamflow gage locations. Data from 750 streamflow gage locations are represented, which are indexed by gage id according to the metadata.csv file. Streamflow gage location sites were based on gage records downloaded from the United States Geological Survey (USGS), National Water Data (HYDAT) Archive from Canada, and the State Hydrological Institute of Russia. Data are available from 11 Earth System Models (ESMs) in total at a daily timestep with a duration of 1920-2099 (1850-2014 for E3SM only). The spatial domain of the modeled data is 0.5 degree resolution. Earth System Models and Units of mean daily streamflow are presented in cubic meters per second (cms). Data files consist of 11 Earth System Model mean daily streamflow output per gage location in .csv format, 1 metadata file in .csv format and 1 Readme file in .docx format (13 files in total). These data were used to benchmark CMIP6 modeled representations of streamflow against gage records.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2018

RESTful CFDP: Managing GDS Complexity with Microservices

NASA's Advanced Multi-Mission Operations System (AMMOS) is currently adding capability to support the CCSDS File Delivery Protocol (CFDP). This feature is being added as part of the AMMOS Mission Data Processing and Control System (AMPCS). In order to address the system’s increasing complexity, AMPCS has recently been re-architected to break down its monolithic applications into smaller, individually deployable microservices. The CFDP capability is the first new AMPCS feature to leverage this new architecture. The CFDP microservice provides a web-based Representational State Transfer (REST) application programming interface (API) for complete monitor and control of its operations, and this enables it to be decoupled from other AMPCS microservices. This also results in better scalability for redundancy and load balancing. AMPCS's CFDP microservice is designed to support generic CFDP operations, agnostic to AMPCS's legacy concept of Downlink Products. An optional runtime plug-in allows the CFDP microservice to simulate CFDP artifacts as Downlink Products. Applying the microservices software architecture pattern both in the latest release of AMPCS and in providing the new CFDP capability has resulted in a more flexible system with improved extensibility and maintainability. System complexity has also become more manageable.

Choi, Joshua S.↗
DOE OSTI2020

Characterization and identification of HPC applications at leadership computing facility

High Performance Computing (HPC) is an important method for scientific discovery via large-scale simulation, data analysis, or artificial intelligence. Leadership-class supercomputers are expensive, but essential to run large HPC applications. The Petascale era of supercomputers began in 2008, with the first machines achieving performance in excess of one petaflops, and with the advent of new supercomputers in 2021 (e.g., Aurora, Frontier), the Exascale era will soon begin. However, the high theoretical computing capability (i.e., peak FLOPS) of a machine is not the only meaningful target when designing a supercomputer, as the resources demand of applications varies. A deep understanding of the characterization of applications that run on a leadership supercomputer is one of the most important ways for planning its design, development and operation. In order to improve our understanding of HPC applications, user demands and resource usage characteristics, we perform correlative analysis of various logs for different subsystems of a leadership supercomputer. This analysis reveals surprising, sometimes counter-intuitive patterns, which, in some cases, conflicts with existing assumptions, and have important implications for future system designs as well as supercomputer operations. For example, our analysis shows that while the applications spend significant time on MPI, most applications spend very little time on file I/O. Combined analysis of hardware event logs and task failure logs show that the probability of a hardware FATAL event causing task failure is low. Combined analysis of control system logs and file I/O logs reveals that pure POSIX I/O is used more widely than higher level parallel I/O. Based on holistic insights of the application gained through combined and co-analysis of multiple logs from different perspectives and general intuition, we engineer features to "fingerprint" HPC applications. We use t-SNE (a machine learning technique for dimensionality reduction) to validate the explainability of our features and finally train machine learning models to identify HPC applications or group those with similar characteristic. To the best of our knowledge, this is the first work that combines logs on file I/O, computing, and inter-node communication for insightful analysis of HPC applications in production.

Liu, Zhengchun↗
NASA NTRS1989

A 3D visualization system for molecular structures

The properties of molecules derive in part from their structures. Because of the importance of understanding molecular structures various methodologies, ranging from first principles to empirical technique, were developed for computing the structure of molecules. For large molecules such as polymer model compounds, the structural information is difficult to comprehend by examining tabulated data. Therefore, a molecular graphics display system, called MOLDS, was developed to help interpret the data. MOLDS is a menu-driven program developed to run on the LADC SNS computer systems. This program can read a data file generated by the modeling programs or data can be entered using the keyboard. MOLDS has the following capabilities: draws the 3-D representation of a molecule using stick, ball and ball, or space filled model from Cartesian coordinates, draws different perspective views of the molecule; rotates the molecule on the X, Y, Z axis or about some arbitrary line in space, zooms in on a small area of the molecule in order to obtain a better view of a specific region; and makes hard copy representation of molecules on a graphic printer. In addition, MOLDS can be easily updated and readily adapted to run on most computer systems.

Green, Terry J.↗
DOE OSTI2020

Jefferson Laboratory C100 Superconducting Radio-Frequency Cavity Fault Data, 2020

The dataset was created to train machine learning models for the task of identifying the (1) cavity and (2) fault type from C100-type cryomodules at the Thomas Jefferson National Accelerator Facility (Jefferson Lab), thereby replacing the time-consuming efforts of a subject matter expert. Superconducting radio-frequency (SRF) cavity trips represent a significant source of accelerator downtime. Real-time – rather than post-mortem – identification of the offending cavity and classification of the fault type would give control room operators valuable feedback for corrective action planning. The anticipated benefit is increased beam-on-target time for users and provides performance metrics that can be used to improve future cavity designs. A series of 17 RF signals are recorded for each of the 8 cavities in a C100 cryomodule every time a cavity trips. These time-series signals are written to file using a specially designed data acquisition system. The dataset represents fault events recorded during Continuous Electron Beam Accelerator Facility (CEBAF) beam operations between January 18, 2019 and March 9, 2020. The following filtering steps were applied to collected data; (1) only 4 of the 17 signals per cavity are retained (GMES, GASK, CRFP, DETA2) (2) only events with data from each of the eight cavities in the cryomodule are kept, (3) only events that were sampled at 5 kHz were kept, (4) events from cryomodule 0L04 were neglected, (5) events occurring between February 4, 3PM and February 5, 12PM were neglected. As a result of preprocessing, the dataset is comprised of 2,375 unique events. The full dataset is comprised of three files: features.csv, cavity_labels.csv, fault_labels.csv. Each instance in faults.csv includes a timestamp (“date_time”), a label for the cryomodule which experienced the trip (“zone_label”), and 192 features (“feature_1”, “feature_2”... “feature_192”). The features correspond to 6 autoregressive features for each of 4 signals per cavity for each of the 8 cavities (6 × 4 signals/cavity × 8 cavities/cryomodule = 192). To deal with the large variation of signal amplitudes, time-series standardization via the z-score (standard score) function was applied prior to computing the features. For each instance, there is an associated label for the (1) cavity which faulted first (cavity_labels.csv) and (2) the type of fault that caused the trip (fault_labels.csv). The cavity identification can take values of [0, 1, 2, 3, 4, 5, 6, 7, 8] and the fault type can take values of [‘Microphonics’, ‘Quench_100ms’, ‘Controls_Fault’, ‘E_Quench’, ‘Quench_3ms’, ‘Single_Cav_Turn_Off’ , ‘Heat_Riser_Choke’, ‘Multi_Cav_Turn_Off’].

43 PARTICLE ACCELERATORS↗
NASA NTRS1978

Modifications to the accuracy assessment analysis routine SPATL to produce an output file

The SPATL is an analysis program in the Accuracy Assessment Software System which makes comparisons between ground truth information and dot labeling for an individual segment. In order to facilitate the aggregation cf this information, SPATL was modified to produce a disk output file containing the necessary information about each segment.

Carnes, J. G.↗
NASA NTRS1991

Analysis of IRAS solar system dust data

Data in the Infrared Astronomical Satellite (IRAS) Zodiacal History File were analyzed to extract dust band locations and peak brightness measurements from approximately 1,000 individual IRAS scans. The study had three goals. One was to show that the prominent solar system dust bands are associated with Hirayama asteroid families and thus that collisions between asteroids account for a significant fraction of the particles in the zodiacal cloud. Recent work suggests that while the Hirayama families are a major source of the dust in the bands, there may also be contributions from two or three smaller, more recently recognized asteroid families. A second goal was to show that there is evidence in the IRAS dust data for the transport of particles from asteroid belt to the Earth by Poynting-Robertson light drag and thus account for the fact that asteroid particles are collected in the Earth's stratosphere. Results of the study will confirm the location of the dust bands within the inner asteroid belt, and show conclusively that the material seen by IRAS is now spread over a wide range of distances from the sun. The third goal was to construct a model of the background zodiacal cloud that satisfies the proper dynamical constraints. Figures are provided to show the scans processed to remove zodiacal background and Galactic signals, and the resulting polynomial fits to the 25 micron scan. The latter provided objective estimates of band widths, peak locations, and peak fluxes. Modelling and analysis of the resulting band data has been presented at several conferences and is the subject of a number of forthcoming papers.

Dermott, S. F.↗
DOE OSTI2024

Documentation for Python Automation in CYME for DER Hosting Capacity Analysis of Different Feeder Configurations

This file provides the documentation for several Python scripts that were developed by Sandia National Laboratories to enhance the automation and customization capabilities for performing various distribution system planning and analysis tasks in CYME. Specifically, these scripts (.py files detailed in Figure 1) enable the user to evaluate different distribution system configurations and the resulting impacts on hosting capacity results and other metrics. In general, these scripts—and the accompanying documentation—provide the foundation upon which future customized tools can be created. For example, the scripts show how to extract and modify parameters of various circuit components, set up and run analyses using built-in CYME tools (iteratively), and export reports for further evaluations and comparisons. Thus, the capabilities and syntaxes used in the scripts can be adapted and leveraged for countless other objectives.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Cross-calibration of Fuji TR image plate and RAR 2492 x-ray film to determine the response of a DITABIS Super Micron image plate scanner

Fuji TR image plate is frequently used as a replacement detector medium for x-ray imaging and spectroscopy diagnostics at NIF, Omega, and Z facilities. However, the familiar Fuji BAS line of image plate scanners is no longer supported by the industry, and so a replacement scanning system is needed. While the General Electric Typhoon line of scanners could replace the Fuji systems, the shift away from photo stimulated luminescence units to 16-bit grayscale Tag Image File Format (TIFF) leaves a discontinuity when comparing data collected from both systems. For the purposes of quantitative spectroscopy, a known unit of intensity applied to the grayscale values of the TIFF is needed. The DITABIS Super Micron image plate scanning system was tested and shown to potentially rival the resolution and dynamic range of Kodak RAR 2492 x-ray film. However, the absolute sensitivity of the scanner is unknown. In this work, a methodology to cross calibrate Fuji TR image plate and the absolutely calibrated Kodak RAR 2492 x-ray film is presented. Details of the experimental configurations used are included. An energy dependent scale factor to convert Fuji TR IP scanned on a DITABIS Super Micron scanner from 16-bit grayscale TIFF to intensity units (i.e., photons per square micron) is discussed.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1994

NSSDC provides network access to key data via NDADS

The National Space Science Data Center (NSSDC) is making a growing fraction of its most customer-desirable data electronically accessible via both the local and wide area networks. NSSDC is witnessing a great increase in its data dissemination owing to this network accessibility. To provide its customers the best data accessibility, the NSSDC makes data available from a nearline, mass storage system, the NSSDC Data Archive and Dissemination Service (NDADS). The NDADS, the initial version was made available in January 1992, is a customized system of hardware and software that provides users access to the nearline data via ANONYMOUS FTP, an e-mail interface (ARMS), and a C-based software library. In January 1992, the NDADS registered 416 requests for 1,957 files. By December of 1994, NDADS had been populated with 800 gigabytes of electronically accessible data and had registered 1458 requests for 20,887 files. In this report we describe the NDADS system, both hardware and software. Later in the report, we discuss some of the lessons that were learned as a result of operating NDADS, particularly in the area of ingest and dissemination.

Behnke, Jeanne↗
NASA NTRS2021

Power Autonomy Research and Development Environment (PARDE) User’s Guide Version 0.1.2

This document is a user's guide for the Power Autonomy Research and Development Environment (PARDE) software package. PARDE is a version of NASA's Autonomous Power Control (APC) software that can be used to evaluate fault management and automatic power system reconfiguration algorithms in a relevant system without having to fully develop all the supporting software. Software items included are a set of C++ class source files representing simplified fault management and reconfiguration logic, a power system simulation representing a notional architecture for NASA's Gateway vehicle, a web-based graphical user interface for running and testing the simulation and APC, a Docker-based automatic setup script for a development environment, and a user's guide.

autonomous power control↗
DOE OSTI2016

A Data Quality Filter for PMU Measurements: Description, Experience, and Examples

Networks of phasor measurement units (PMUs) continue to grow, and along with them, the amount of data available for analysis. With so much data, it is impractical to identify and remove poor quality data manually. The data quality filter described in this paper was developed for use with the Data Integrity and Situation Awareness Tool (DISAT), which analyzes PMU data to identify anomalous system behavior. The filter operates based only on the information included in the data files, without supervisory control and data acquisition (SCADA) data, state estimator values, or system topology information. Measurements are compared to preselected thresholds to determine if they are reliable. Along with the filter's description, examples of data quality issues from application of the filter to nine months of archived PMU data are provided. The paper is intended to aid the reader in recognizing and properly addressing data quality issues in PMU data.

Phasor Measurement Unit↗
DOE OSTI2022

FY22 Progress Report: SRNL Analysis of ICCWR LCM and WAMS Data for Corrosion and Cracking

Algorithms for machine learning and data analysis for the 3013 Surveillance Program are being developed in an ongoing collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). To detect the presence of corrosion and cracking, data is collected from large binary files generated by a Laser Confocal Microscope (LCM) or Wide Area 3D Measurement System (WAMS). Software is being developed to use the physical attributes in the data files (e.g., height, color, and grayscale values; all as functions of a location in a plane projection) to detect the presence of surface corrosion and cracking. A user-friendly Matlab Graphical User Interface (GUI) that reads data from either LCM or WAMS files was developed to integrate input data with software developed for processing and evaluation. The GUI can selectively download binary data, interrogate data attributes, label data for training ML algorithms, flag significant features, execute Machine Learning (ML) algorithms, output parameters from trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. Surface defects can be called out by setting user-specified thresholds, feature based analysis or machine learning algorithms. Enhancements to data labeling capability have been developed to address this essential precursor to application of ML routines. Efficient labeling is particularly important in view of the very large volume of data required to train ML algorithms.

3013 Corrosion↗