Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,549 records · Page 86

Datacasting V3.0

Datacasting V3.0 provides an RSSbased feed mechanism for publishing the availability of Earth science data records in real time. It also provides a utility for subscribing to these feeds and sifting through all the items in an automatic manner to identify and download the data records that are required for a specific application. Datacasting is a method by which multiple data providers can publish the availability of new Earth science data and users download those files that meet a predefined need; for example, to only download data files related to a specific earthquake or region on the globe. Datacasting is a server-client architecture. The server-side software is used by data providers to create and publish the metadata about recently available data according to the Datacasting RSS (Really Simple Syndication) specification. The client software subscribes to the Datacasting RSS and other RSS-based feeds. By configuring filters associated with feeds, data consumers can use the client to identify and automatically download files that meet a specific need. On the client side, a Datacasting feed reader monitors the server for new feeds. The feed reader will be tuned by the user, via a graphical user interface (GUI), to examine the content of the feeds and initiate a data pull after some criteria are satisfied. The criteria might be, for example, to download sea surface temperature data for a particular region that has cloud cover less than 50% and during daylight hours. After the granule is downloaded to the client, the user will have the ability to visualize the data in the GUI. Based on the popular concept of podcasting, which gives listeners the capability to download only those MP3 files that match their preference, Earth science Datacasting will give users a method to download only the Earth science data files that are required for a particular application.

Bingham, Andrew W.↗

System Testing of Ground Cooling System Components

This internship focused primarily upon software unit testing of Ground Cooling System (GCS) components, one of the three types of tests (unit, integrated, and COTS/regression) utilized in software verification. Unit tests are used to test the software of necessary components before it is implemented into the hardware. A unit test determines that the control data, usage procedures, and operating procedures of a particular component are tested to determine if the program is fit for use. Three different files are used to make and complete an efficient unit test. These files include the following: Model Test file (.mdl), Simulink SystemTest (.test), and autotest (.m). The Model Test file includes the component that is being tested with the appropriate Discrete Physical Interface (DPI) for testing. The Simulink SystemTest is a program used to test all of the requirements of the component. The autotest tests that the component passes Model Advisor and System Testing, and puts the results into proper files. Once unit testing is completed on the GCS components they can then be implemented into the GCS Schematic and the software of the GCS model as a whole can be tested using integrated testing. Unit testing is a critical part of software verification; it allows for the testing of more basic components before a model of higher fidelity is tested, making the process of testing flow in an orderly manner.

Unit Test↗

Data Recovery Effort of Nimbus Era Observations by the NASA GES DISC

NASA launched seven Nimbus meteorological satellites in the 1960s and 70s. These satellites carried instruments for making observations of the Earth in the visible, infrared, ultraviolet, and microwave wavelengths. The original data archive consisted of a combination of magnetic tapes and various film media. As these media are well past their expected end of life, the valuable data they contain are now being migrated to the GES DISC modern online archive. The process involves recovering the digital data files from the tapes as well as scanning images of the data from film strips. This presentation will address the status and challenges of recovering the Nimbus data. The old data products were written on now obsolete hardware systems and outdated file formats. They lack any metadata standards and each product is often written in its own proprietary file structure. This requires creating metadata by reading the contents of the old data files. The job is tedious and laborious, as documentation may be incomplete, data files and tapes are sometimes corrupted, or were improperly copied at the time they were created.

Nimbus↗

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗

Internship in Augmented and Virtual Reality - Rapid Model Import Tool

The integration of virtual and augmented reality, sometimes called mixed reality, is an emerging technology which will likely skyrocket overnight much in the way smartphones did a decade ago. Kennedy Space Center's Augmented and Virtual Reality (AVR) Lab is developing a Rapid Model Import Tool (RMIT) to create a quick and efficient way to bring NASA's complex engineering 3D models into virtual and augmented environments. The long-term objective is to create a tool that will ultimately benefit KSC engineers. Its various uses within NASA can potentially span from astronaut training, to marketing, to public outreach, to name a few. Unity is a prolific cross-platform game engine that allows users to build high quality 2D and 3D games for desktop, mobile, web, and game console platforms. It is perhaps also the most widely used software for virtual reality game development. At the AVR lab, we are looking at alternative uses of Unity to build tools for NASA engineers to perform design, development, testing, and training on spacecraft, rocket delivery systems, ground support equipment, and facilities at KSC. As an intern for the RMIT project, I am charged with the task of performing research on Unity-compatible file types to develop an efficient, affordable, preservative process to bring models from CATIA 3D engineering software into the Unity environment. With a tool called the NASA Enterprise Visualization Application (NEVA), developed by the Boeing Design Visualization group at KSC, we are able to easily convert CATIA's design models to. DAE (also known as COLLADA) and .OBJ file formats. I first reduce the polygon count of the model within CATIA itself, make any necessary tweaks to reduce the model further, and then export using NEVA. The .OBJ or. DAE files that I am left with are then converted by another intern to a Unity-compatible file format using a custom Python script. I have generated extensive documentation of this process in a NEVA User Guide. By the end of this semester, we will have built a solid framework for RMIT based on a thorough understanding of virtual reality specifications and file requirements, allowing future software development teams to go forward with development on the custom tool.

Leap Motion↗

More Efficient Precipitation Rate Data Acquisition with the STORM Swath-Based Analysis Tool

A multitude of locations exist for acquiring satellite-derived precipitation rate data. Because the volume of data exceeds what can reasonably be analyzed manually, online tools have been developed to help researchers make more informed data acquisition decisions. Some interfaces provide sample images generated from prioritized variables to enable efficient visual inspection, while others include the ability to subset geographically to reduce the volume of the data to the area of interest. Building upon such value-added services is the STORM (the NASA Precipitation Processing System (PPS) data product ordering system) Swath-Based Analysis Tool (https://storm.pps.eosdis.nasa.gov/storm/Analysis.jsp). With it, users can compare overflights of multiple precipitation satellites simultaneously and from this make determinations about which swaths will best serve their research. They can preview these overflights with two distinct visualization tools. Finally, they can then download geographically subset swaths that have precipitation rate characteristics matching their research needs. A user starts by making up to six selections. They then select their date range of interest going back to 1997. Finally, they select a geographical region either by inputting latitude-longitude corners of a rectangle or drawing that rectangle on an interactive map. The system processes each file, determining aggregate statistics about the precipitation rates within the selected region as observed by each instrument. These statistical values are added to a time series graph as a "time remaining" counter is continuously updated. The chart is fully interactive, with detailed values displayed upon mouseover, and numerous options available upon clicking each point. Since the user is likely to need the relevant files for their own research, they can generate an order from their request. All files are geographically subset to the region of interest, and the user can use the statistical values to limit which files are added to the order. This flexibility helps to ensure researchers are only downloading the files they need for their work, making for a more efficient process both from a time-consumption and a storage perspective. This paper will introduce the Swath-Based Analysis Tool and explore its numerous features. It will discuss several use cases for precipitation scientists.

remote sensing↗

The Advantages of Synergy-Quantitative Earth Science Data Visualization and Analysis with Giovanni, Panoply, and Excel

The NASA Giovanni data analysis system provides a multitude of basic analysis capabilities for numerous Earth science data products which are available in the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) archive, as well as for additional selected data products provided by other NASA Distributed Active Archive Center (DAAC) archives. In Giovanni, users can easily generate time-averaged data maps, area-averaged time-series, Latitude-Time and Longitude-Time Hovmoeller diagrams, correlation maps, accumulation maps, and map animations (22 analysis options are available in total). While ASCII text output is available for time-series plots, it is not included as an option for data maps. In order to provide a quantitative, easy-to-use numerical output in ASCII text form, the NetCDF file output from a Giovanni visualization is downloaded and then opened with the free NASA visualization software package Panoply. Panoply provides the capability of translating the Giovanni file into comma-separated-variable (CSV) output. Panoply also provides additional visualization options, including the facile calculation of difference maps and quasi-anomaly maps using Giovanni output files. The CSV files from Panoply can then be imported into an Excel spreadsheet, where an Excel macro converts the CSV files. The output consists of latitude-longitude-data value triads in text form for maps, and either longitude-time-data value or latitude-time-data value triads in text form for Hovmoeller diagrams. This presentation will explicate the basic procedure for the conversion, and then provide several examples where the procedure is applied to Giovanni output from different analysis options.

Earth science↗

New Features of the NEQAIR Radiation Code

The longest-lived code for predicting shock layer radiation, NEQAIR, is now in its 5th decade of service. Substantial changes to the code have been made over the previous decade, the most recent report of which was at the 5th Workshop on Radiation in High Temperature Gases in 2014, for the version referred to as NEQAIR14. This paper will review some of the improvements made to the NEQAIR code since then, which is now at v15.2. Some of these features are discussed briefly below. NEQAIR15 and subsequent versions have enabled parallel evaluation of multiple lines of sight. This is accomplished by utilizing the HDF5 file format and placing multiple lines into a single file, LOS.h5, which is used for both input and output. This approach enables straightforward parallel execution both over the number of lines of sight and the number of points per line. For large problems, runtime reduces linearly with the number of nodes deployed since each line is processed independently by a subset of MPI ranks. Three applications of the multi-line solver are discussed. The first has to do with performing loosely coupled radiation-flowfield solutions. In this case the computed absorption and emission coefficients are used to evaluate the total energy absorbed or emitted at each point, allowing evaluation of the volumetric source term in the flowfield. The second computation is for obtaining heat flux from nonuniform flows, which require integration over spherical co-ordinates. These are of particular interest for evaluating radiation on the vehicle backshell. This 3D option improves the angular integration scheme and allows adaptive line selection that together reduce the number of lines required by about an order of magnitude. The final application is for remote observation, which is essentially the 3D integration problem over a small solid angle. For all three of these computations, data can be stored in the HDF5 file which allows a NEQAIR run to be restarted when it times out, or to add atmospheric absorption or instrument scan functions. An additional level of parallelism is enabled in NEQAIR15.2 using GPU routines. The GPU parallelism has realized up to 8x speed-up when running on a single core but diminishes as CPU parallelism is increased. For running multi-line simulations, it may be easier to reserve a large number of CPU nodes than to obtain the number of GPU nodes required for similar performance. A GUI, known as NEQTPY, allows for reading and creating input files, running NEQAIR, and displaying results. A significant feature of NEQTPY is the ability to perform spectral fits to data. The fits can operate on a single line spectrum (radiance vs. wavelength) or a 3D input file with multiple columns of data. Other new features include improved constants, additional species, more detailed non-Boltzmann modelling, advanced user controls, the ability to read and calculate spectra from HITRAN datafiles, photodissociation and photoionization cross-sections. A “fast” automatic grid option may reduce the size and time of spectral calculations while still maintaining good accuracy for total heat flux.

Brett A Cruden↗

ASDC’s Python-Based Metadata Extraction Pipeline for Suborbital Campaigns

The FAIRness of data products, especially findability and accessibility depend on rich metadata which, when extracted, can allow for proper curation. Over the past few years, the Atmospheric Science Data Center (ASDC) suborbital science support team has developed a metadata extraction pipeline to ensure the required metadata can be retrieved systematically, effectively, and efficiently to ensure the data can be used by a broad community. The development of a pipeline has presented many, but necessary, challenges to support archival and distribution of ASDC’s 30+ suborbital missions. Though sufficient metadata is provided by instrument scientists, the metadata may not be readily machine actionable due to different formats and templates. Further complicating metadata extraction, our team has found that the nature of metadata can be quite diverse given the difference in measurement types, instruments, and measurement platforms. A metadata extraction pipeline has been developed to provide an efficient, plugin-in based, method for adding new parsers, a configuration system that lets non-developers customize how files are processed, and a system for identifying and logging metadata quality issues to ensure they are readily found and addressed. The metadata extraction pipeline identifies critical pieces of metadata that are needed to promote data FAIRness, including location, file revision, measurement start/end datetime and can be easily modified to extract further information (such as variables). Given the wide-ranging datasets, the pipeline has been modified to accommodate multiple file formats, including multiple versions of ICARTT (International Consortium for Atmospheric Research on Transport and Transformation), HDF (Hierarchical Data Format), netCDF (network Common Data Form), and multiple versions of the Ames File Format. The pipeline also supports building metadata for file formats that cannot have metadata easily extracted from them, such as PDF (Portable Document Format) and GIF (Graphics Interchange Format). The pipeline has allowed our team to maintain a consistent flow of data and metadata to archival and distribution services, ensuring the ASDC meets the needs of the suborbital science community. This presentation will highlight the ASDC’s suborbital metadata extraction pipeline, its development, how it’s been modified to support data FAIRness, and plans for maintaining the pipeline and adding new features.

Abraham Porter↗

AIDE - Advanced Intrusion Detection Environment

Would you like to know when someone has dropped an undesirable executable binary on our system? What about something less malicious such as a software installation by a user? What about the user who decides to install a newer version of mod_perl or PHP on your web server without letting you know beforehand? Or even something as simple as when an undocumented config file change is made by another member of the admin group? Do you even want to know about all the changes that happen on a daily basis on your server? The purpose of an intrusion detection system (IDS) is to detect unauthorized, possibly malicious activity. The purpose of a host-based IDS, or file integrity checker, is check for unauthorized changes to key system files, binaries, libraries, and directories on the system. AIDE is an Open Source file and directory integrity checker. AIDE will let you know when a file or directory has been added, deleted, modified. It is included with the Red Hat Enterprise 6. It is available for other Linux distros. This is a case study describing the process of configuring AIDE on an out of the box RHEL6 installation. Its goal is to illustrate the thinking and the process by which a useful AIDE configuration is built.

AIDE↗

Sandia Advanced MEMS Design Tools, Version 2.2.5

The Sandia National Laboratories Advanced MEMS Design Tools, Version 2.2.5, is a collection of menus, prototype drawings, and executables that provide significant productivity enhancements when using AutoCAD to design MEMS components. This release is designed for AutoCAD 2000i, 2002, or 2004 and is supported under Windows NT 4.0, Windows 2000, or XP. SUMMiT V (Sandia Ultra planar Multi level MEMS Technology) is a 5 level surface micromachine fabrication technology, which customers internal and external to Sandia can access to fabricate prototype MEMS devices. This CD contains an integrated set of electronic files that: a) Describe the SUMMiT V fabrication process b) Facilitate the process of designing MEMS with the SUMMiT process (prototype file, Design Rule Checker, Standard Parts Library) New features in this version: AutoCAD 2004 support has been added. SafeExplode ? a new feature that explodes blocks without affecting polylines (avoids exploding polylines into objects that are ignored by the DRC and Visualization tools). Layer control menu ? a pull-down menu for selecting layers to isolate, freeze, or thaw. Updated tools: A check has been added to catch invalid block names. DRC features: Added username/password validation, added a method to update the user?s password. SNL_DRC_WIDTH ? a value to control the width of the DRC error lines. SNL_BIAS_VALUE ? a value use to offset selected geometry SNL_PROCESS_NAME ? a value to specify the process name Documentation changes: The documentation has been updated to include the new features. While there exist some files on the CD that are used in conjunction with software package AutoCAD, these files are not intended for use independent of the CD. Note that the customer must purchase his/her own copy of AutoCAD to use with these files.

Yarberry, Victor↗

Checkpoint-Restart in User Space

CRUISE implements a user-space file system that stores data in main memory and transparently spills over to other storage, like local flash memory or the parallel file system, as needed. CRUISE also exposes file contents fo remote direct memory access, allowing external tools to copy files to the parallel file system in the background with reduced CPU interruption.

Source record↗

ThermoPhonon

ThermoPhonon is a stand-alone code, which can be integrated into other software packages. Typically, it is used together with a density functional theory (DFT) code (such as VASP, Wien2k, AbInit, SIESTA) and a phonon code (such as Phonopy or Phon). The workflow is the following. Molecular dynamics (MD) in a supercell at a given temperature T is performed using another code. After sufficient equilibration, the output in the form of atomic positions and forces for a large number of selected MD steps is recorded into a file. If needed, one can modify this file by applying additional constraints, such as enforced crystal symmetry or subtracted motion of the center of mass. ThermoPhonon reads the file with atomic positions and forces and writes a new file with the force constants. Force constants can be used by another code (such as Phonopy or Phon) to produce phonon spectrum for plotting, in the assumption of known equilibrium atomic positions provided in a separate file.

Zarkevich, Nikolai↗

Sandia Advanced MEMS Design Tools v. 3.0

This is a major revision to the Sandia Advanced MEMS Design Tools. It replaces all previous versions. New features in this version: Revised to support AutoCAD 2014 and 2015 This CD contains an integrated set of electronic files that: a) Describe the SUMMiT V fabrication process b) Provide enabling educational information (including pictures, videos, technical information) c) Facilitate the process of designing MEMS with the SUMMiT process (prototype file, Design Rule Checker, Standard Parts Library) d) Facilitate the process of having MEMS fabricated at Sandia National Laboratories e) Facilitate the process of having post-fabrication services performed. While there exists some files on the CD that are used in conjunction with software package AutoCAD, these files are not intended for use independent of the CD. Note that the customer must purchase his/her own copy of AutoCAD to use with these files.

Yarberry, Victor R.↗

RatCAP Time to Digital Converter and Signal Processing module (TSPM)

The TSPM receives and processes ASIC signals and transmits processed data over to the PC using an Ethernet cable. The data is given a location and a time stamp. This is the heart of the device as it gathers and stamps the timing and location of events on each of the ASICs. The five files for the TSPM are needed to manufacture Pet scanners that are based on the RatCAP (Rat Conscious Animal PET). They include a TSPM schematic, a raw data file to build the RatCAP TSPM, an output file that along with the assay file is used by an assembly house to build the RatCAP TSPM, an assay file that provides the part list and XY location for the components that go on the RatCAP TSPM, firmware that includes the source code to program the FPGA, and a realized program on the TSPM based on the firmware.

Fried, Jack↗

ArrayBridge: Interweaving declarative array processing with high-performance computing

Scientists are increasingly turning to datacenter-scale computers to produce and analyze massive arrays. Despite decades of database research that extols the virtues of declarative query processing, scientists still write, debug and parallelize imperative HPC kernels even for the most mundane queries. This impedance mismatch has been partly attributed to the cumbersome data loading process; in response, the database community has proposed in situ mechanisms to access data in scientific file formats. Scientists, however, desire more than a passive access method that reads arrays from files. This paper describes ArrayBridge, a bi-directional array view mechanism for scientific file formats, that aims to make declarative array manipulations interoperable with imperative file-centric analyses. Our prototype implementation of ArrayBridge uses HDF5 as the underlying array storage library and seamlessly integrates into the SciDB open-source array database system. In addition to fast querying over external array objects, ArrayBridge produces arrays in the HDF5 file format just as easily as it can read from it. ArrayBridge also supports time travel queries from imperative kernels through the unmodified HDF5 API, and automatically deduplicates between array versions for space efficiency. Our extensive performance evaluation in NERSC, a large-scale scientific computing facility, shows that ArrayBridge exhibits statistically indistinguishable performance and I/O scalability to the native SciDB storage engine.

97 MATHEMATICS AND COMPUTING↗

Detecting Outliers in Network Transfers with Feature Extraction

Reliable file transfers are essential for successful science computations and experiments that need large complex data files to be moved across long distances. Networks use protocols such as TCP and UDP to support file transfers, and their performance degrades under packet losses and duplications, thereby adversely affecting science applications. Hence, scientists and engineers often monitor network statistics to find and repair network problems that cause such degradations.Outliers or anomalies can be detected using statistical and machine learning methods that highlight abnormal behaviors. However, without knowing what is normal, it is very difficult and subjective to determine the anomalies. In this paper, we investigate statistical and machine learning approaches to extract unknown or unsupervised features from TCP and perfSONAR data sets, with the aim to identify anomalies such as packet loss, duplication and retransmission sequencing that affect file transfer performance. Simple statistical and feature extraction methods (e.g. PCA) have shown that detecting anomalies is simpler if labeled data sets of normal behavior are available. While perfSONAR logs record losses,they do not detect duplications and reordering. We show that PCA applied to TCP statistics is able to determine abnormal behavior in the above cases. In particular, our results show that simple feature extraction techniques provide building blocks for more advanced network monitoring tools that can quickly flag anomalies using simple measurements, thereby helping to build confidence in file transfer reliability.

Rao, Nageswara S.↗

AutoSIM

AutoSIM enables several instances of an executable to run in parallel on a directory of input files and consolidate results, at scale on the world's fastest supercomputers. The software accomplishes this by balancing compression, transmission, and computation of input files, executables, and output files to allow scalable parallel runs of any executable. The software breaks-up simulations into a configurable number of "churns" or sequential instances of executable invocations on a configurable number of cores/node. It sends the executable, supporting files, and appropriate number of input files to each node for independent execution, and then transfers the executable results to a shared disk for post-processing analysis. This MPI-based code has been demonstrated using EnergyPlus, DOE's flagship whole-building simulation engine, but was architected to allow scalable runtime of any software for any application (e.g. sensitivity analysis and uncertainty quantification). The code has also been demonstrated to scale on the Titan supercomputer using 131,072 processors to run 524,288 EnergyPlus annual simulations, and write 45TB of data to disk in 68 mins 08 secs.

New, Joshua R↗