Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Parallel File System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly.

Lynnes, Christopher

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly. Reviewed by Mark McInerney ESDIS Deputy Project Manager.

Lynnes, Christopher

Mountain building processes in the Central Andes

False color composite images of the Thematic Mapper (TM) bands 5, 4, and 2 were examined to make visual interpretations of geological features. The use of the roam mode of image display with the International Imaging Systems (IIS) System 600 image processing package running on the IIS Model 75 was very useful. Several areas in which good comparisons with ground data existed, were examined in detail. Parallel to the visual approach, image processing methods are being developed which allow the complete use of the seven TM bands. The data was organized into easily accessible files and a visual cataloging of the quads (quarter TM scenes) with preliminary registration with the best available charts for the region. The catalog has proved to be a valuable tool for the rapid scanning of quads for a specific investigation. Integration of the data into a complete approach to the problems of uplift, deformation, and magnetism in relation to the Nazca-South American plate interaction is at an initial stage.

Bloom, A. L.

Orchestrator Telemetry Processing Pipeline

Orchestrator is a software application infrastructure for telemetry monitoring, logging, processing, and distribution. The architecture has been applied to support operations of a variety of planetary rovers. Built in Java with the Eclipse Rich Client Platform, Orchestrator can run on most commonly used operating systems. The pipeline supports configurable parallel processing that can significantly reduce the time needed to process a large volume of data products. Processors in the pipeline implement a simple Java interface and declare their required input from upstream processors. Orchestrator is programmatically constructed by specifying a list of Java processor classes that are initiated at runtime to form the pipeline. Input dependencies are checked at runtime. Fault tolerance can be configured to attempt continuation of processing in the event of an error or failed input dependency if possible, or to abort further processing when an error is detected. This innovation also provides support for Java Message Service broadcasts of telemetry objects to clients and provides a file system and relational database logging of telemetry. Orchestrator supports remote monitoring and control of the pipeline using browser-based JMX controls and provides several integration paths for pre-compiled legacy data processors. At the time of this reporting, the Orchestrator architecture has been used by four NASA customers to build telemetry pipelines to support field operations. Example applications include high-volume stereo image capture and processing, simultaneous data monitoring and logging from multiple vehicles. Example telemetry processors used in field test operations support include vehicle position, attitude, articulation, GPS location, power, and stereo images.

Powell, Mark

Visualization and Tracking of Parallel CFD Simulations

We describe a system for interactive visualization and tracking of a 3-D unsteady computational fluid dynamics (CFD) simulation on a parallel computer. CM/AVS, a distributed, parallel implementation of a visualization environment (AVS) runs on the CM-5 parallel supercomputer. A CFD solver is run as a CM/AVS module on the CM-5. Data communication between the solver, other parallel visualization modules, and a graphics workstation, which is running AVS, are handled by CM/AVS. Partitioning of the visualization task, between CM-5 and the workstation, can be done interactively in the visual programming environment provided by AVS. Flow solver parameters can also be altered by programmable interactive widgets. This system partially removes the requirement of storing large solution files at frequent time steps, a characteristic of the traditional 'simulate (yields) store (yields) visualize' post-processing approach.

Vaziri, Arsi

Cable Tester Box

Cables are very important electrical devices that carry power and signals across multiple instruments. Any fault in a cable can easily result in a catastrophic outcome. Therefore, verifying that all cables are built to spec is a very important part of Electrical Integration Procedures. Currently, there are two methods used in lab for verifying cable connectivity. (1) Using a Break-Out Box and an ohmmeter this method is time-consuming but effective for custom cables and (2) Commercial Automated Cable Tester Boxes this method is fast, but to test custom cables often requires pre-programmed configuration files, and cables used on spacecraft are often uniquely designed for specific purposes. The idea is to develop a semi-automatic continuity tester that reduces human effort in cable testing, speeds up the electrical integration process, and ensures system safety. The JPL-Cable Tester Box is developed to check every single possible electrical connection in a cable in parallel. This system indicates connectivity through LED (light emitting diode) circuits. Users can choose to test any pin/shell (test node) with a single push of a button, and any other nodes that are shorted to the test node, even if they are in the same connector, will light up with the test node. The JPL-Cable Tester Boxes offers the following advantages: 1. Easy to use: The architecture is simple enough that it only takes 5 minutes for anyone to learn how operate the Cable Tester Box. No pre-programming and calibration are required, since this box only checks continuity. 2. Fast: The cable tester box checks all the possible electrical connections in parallel at a push of a button. If a cable normally takes half an hour to test, using the Cable Tester Box will improve the speed to as little as 60 seconds to complete. 3. Versatile: Multiple cable tester boxes can be used together. As long as all the boxes share the same electrical potential, any number of connectors can be tested together.

Lee, Jason H.

Exploring the use of I/O nodes for computation in a MIMD multiprocessor

As parallel systems move into the production scientific-computing world, the emphasis will be on cost-effective solutions that provide high throughput for a mix of applications. Cost effective solutions demand that a system make effective use of all of its resources. Many MIMD multiprocessors today, however, distinguish between 'compute' and 'I/O' nodes, the latter having attached disks and being dedicated to running the file-system server. This static division of responsibilities simplifies system management but does not necessarily lead to the best performance in workloads that need a different balance of computation and I/O. Of course, computational processes sharing a node with a file-system service may receive less CPU time, network bandwidth, and memory bandwidth than they would on a computation-only node. In this paper we begin to examine this issue experimentally. We found that high performance I/O does not necessarily require substantial CPU time, leaving plenty of time for application computation. There were some complex file-system requests, however, which left little CPU time available to the application. (The impact on network and memory bandwidth still needs to be determined.) For applications (or users) that cannot tolerate an occasional interruption, we recommend that they continue to use only compute nodes. For tolerant applications needing more cycles than those provided by the compute nodes, we recommend that they take full advantage of both compute and I/O nodes for computation, and that operating systems should make this possible.

Kotz, David

Spaceflight optical disk recorder development

Mass memory systems based on rewriteable optical disk media are expected to play an important role in meeting the data system requirements for future NASA spaceflight missions. NASA has established a program to develop a high performance (high rate, large capacity) optical disk recorder focused on use aboard unmanned Earth orbiting platforms. An expandable, adaptable system concept is proposed based on disk drive modules and a modular controller. Drive performance goals are 10 gigabyte capacity, 300 megabit/s transfer rate, 10 exp -12 corrected bit error rate, and 150 millisec access time. This performance is achieved by writing eight data tracks in parallel on both sides of a 14 in. optical disk using two independent heads. System goals are 160 gigabyte capacity, 1.2 gigabits/s data rate with concurrent I/O, 250 millisec access time, and two to five year operating life on orbit. The system can be configured to meet various applications. This versatility is provided by the controller. The controller provides command processing, multiple drive synchronization, data buffering, basic file management, error processing, and status reporting. Technology developments, design concepts, current status including a computer model of the system and a Controller breadboard, and future plans for the Drive and Controller are presented.

Jurczyk, Stephen G.

PRISMA: PARALLEL REFINEMENT AND INTEGRATION SYSTEM FOR MULTI-AZIMUTHAL ANALYSIS

The Parallel Refinement and Integration System for Multi-azimuthal Analysis (PRISMA, version 1.1.0) is a Python application for processing X-ray diffraction (XRD) image data. PRISMA wraps GSAS-II to perform azimuthally-binned peak refinement, computes per-frame strain and d-spacing from those fits, and provides three PyQt5 graphical interfaces: (1) a Recipe Builder for selecting GSAS-II control (.imctrl) files, optional mask (.immask) files or threshold-ased masking, reference and experiment image sets, peaks, zimuthal range and bin size, and an optional ceria-based auto-calibration; (2) a Batch Processor that uses Dask on local workstations and pure MPI (mpi4py.futures.MPICommExecutor) on HPC to distribute GSAS-II refinement across cores or compute nodes and write results to a 4-dimensional (peaks x frames x azimuths x measurements) Zarr dataset; and (3) a Data Analyzer that renders heatmaps of fit parameters, strain, frame-to-frame deltas, and percent-change-vs-reference, and exports user-defined subsections to CSV or Excel. The peak-refinement algorithm is deterministic. Benchmark on ALCF Crux: a 20,000-image set, single-peak fit in frame mode with 44 azimuthal bins on 128 nodes x 128 workers, 48 seconds total wall time.

Lorenzo Martin, Maria De La Cinta [Argonne Nationa

Increased accuracy of multiphysics simulations through flexible execution, transient algorithms, and modular physics

The MOOSE framework is a foundational capability used by the NEAMS program to create over 15 different simulation tools for advanced nuclear reactors. Due to MOOSE’s broad use, improvements to the framework in support of modeling and simulation goals are critical to the program. Such improvements can take many forms, including optimization, improved user experience, streamlined application programming interfaces (APIs), parallelism, and new capabilities. The work described in this report was conducted in direct support of the simulation tools and has already been deployed. The capabilities were implemented in the same order as they are covered in this report: multiple time integrators in the same input file, initial design of framework Components, an input file Application block, extension of NetGen to 3D geometries in MOOSE, and deployment of executors in the multi-system paradigm. These five additions are fundamental capabilities that will be leveraged by many NEAMS applications.

97 MATHEMATICS AND COMPUTING

A scalable framework for efficient coupling of thermal and microstructural simulations in additive manufacturing

Predicting microstructure evolution in metal additive manufacturing (AM) is important for process optimization, but spatiotemporal scale disparities between thermal transport and microstructure evolution create significant challenges for efficient data transfer between simulation codes. To address this, we present Stork, a scalable framework for coupling thermal and microstructural simulations. Stork uses a sparse data representation to identify and store active solidification sub-volumes, enabling highly parallel quad-linear interpolation from coarse thermal grids to fine microstructure grids without large intermediate storage. We demonstrate the framework by coupling the semi-analytic heat transfer code 3DThesis with the time-parallel cellular automata code Toucan. This approach achieves over two orders of magnitude reduction in data generation time and file size compared to prior workflows. Numerical studies show that quad-linear interpolation preserves grain morphology and crystallographic texture in laser powder bed fusion (LPBF) simulations for coarsening ratios up to 16. Overall, Stork provides a scalable pathway for high-throughput, component-scale AM simulations on modern high-performance computing systems.

36 MATERIALS SCIENCE

Data reduction, management, and analysis software for CID

In an overview of the Data Reduction System, three major steps are examined. First, the raw data tapes were selected from the onboard recorders. These tapes should provide the best quality data for the data reduction software system. These tapes contained 352 channels of data, plus the monitor channels recorded in 8 bit Pulsed Coded Modulation (PCM) words. The next step consists of transcribing the PCM tapes from 8 bit serial digital data to 8 bit parallel digital data. This puts the data in the correct format for processing. The transcription process was accomplished here at LaRC in the Central Data Transportation Facility (CDTF). The last step in this 3 step process is to process the data through the reduction system developed for the Impact Dynamic Research Facility in the early part of 1980. Processing system criteria, system interface routines, and engineering units program that reads digitized data from tapes, and file management programs are discussed.

Davis, C. W.

Modeling and new equipment definition for the vibration isolation box equipment system

Our MSAD-funded research project is to provide numerical modeling support for the VIBES (Vibration Isolation Box Experiment System) which is an IML2 flight experiment being built by the Japanese research team of Dr. H. Azuma of the Japanese National Aerospace Laboratory. During this reporting period, the following have been accomplished: A semi-consistent mass finite element projection algorithm for 2D and 3D Boussinesq flows has been implemented on Sun, HP And Cray Platforms. The algorithm has better phase speed accuracy than similar finite difference or lumped mass finite element algorithms, an attribute which is essential for addressing realistic g-jitter effects as well as convectively-dominated transient systems. The projection algorithm has been benchmarked against solutions generated via the commercial code FIDAP. The algorithm appears to be accurate as well as computationally efficient. Optimization and potential parallelization studies are underway. Our implementation to date has focused on execution of the basic algorithm with at most a concern for vectorization. The initial time-varying gravity Boussinesq flow simulation is being set up. The mesh is being designed and the input file is being generated. Some preliminary 'small mesh' cases will be attempted on our HP9000/735 while our request to MSAD for supercomputing resources is being addressed. The Japanese research team for VIBES was visited, the current set up and status of the physical experiment was obtained and ongoing E-Mail communication link was established.

Sani, Robert L.

Distributed Finite Element Analysis Using a Transputer Network

The principal objective of this research effort was to demonstrate the extraordinarily cost effective acceleration of finite element structural analysis problems using a transputer-based parallel processing network. This objective was accomplished in the form of a commercially viable parallel processing workstation. The workstation is a desktop size, low-maintenance computing unit capable of supercomputer performance yet costs two orders of magnitude less. To achieve the principal research objective, a transputer based structural analysis workstation termed XPFEM was implemented with linear static structural analysis capabilities resembling commercially available NASTRAN. Finite element model files, generated using the on-line preprocessing module or external preprocessing packages, are downloaded to a network of 32 transputers for accelerated solution. The system currently executes at about one third Cray X-MP24 speed but additional acceleration appears likely. For the NASA selected demonstration problem of a Space Shuttle main engine turbine blade model with about 1500 nodes and 4500 independent degrees of freedom, the Cray X-MP24 required 23.9 seconds to obtain a solution while the transputer network, operated from an IBM PC-AT compatible host computer, required 71.7 seconds. Consequently, the $80,000 transputer network demonstrated a cost-performance ratio about 60 times better than the $15,000,000 Cray X-MP24 system.

Watson, James

Cytometer on a Chip

A cytometer now under development exploits spatial sorting of sampled cells on a microarray chip followed by use of grating-coupled surface-plasmon-resonance imaging (GCSPRI) to detect the sorted cells. This cytometer on a chip is a prototype of contemplated future miniature cytometers that would be suitable for rapidly identifying pathogens and other cells of interest in both field and laboratory applications and that would be attractive as alternatives to conventional flow cytometers. The basic principle of operation of a conventional flow cytometer requires fluorescent labeling of sampled cells, stringent optical alignment of a laser beam with a narrow orifice, and flow of the cells through the orifice, which is subject to clogging. In contrast, the principle of operation of the present cytometer on a chip does not require fluorescent labeling of cells, stringent optical alignment, or flow through a narrow orifice. The basic principle of operation of the cytometer on a chip also reduces the complexity, mass, and power of the associated laser and detection systems, relative to those needed in conventional flow cytometry. Instead of making cells flow in single file through a narrow flow orifice for sequential interrogation as in conventional flow cytometry, a liquid containing suspended sampled cells is made to flow over the front surface of a microarray chip on which there are many capture spots. Each capture spot is coated with a thin (.50-nm) layer of gold that is, in turn, coated with antibodies that bind to cell-surface molecules characteristic of the cell species of interest. The multiplicity of capture spots makes it possible to perform rapid, massively parallel analysis of a large cell population. The binding of cells to each capture spot gives rise to a minute change in the index of refraction at the surface of the chip. This change in the index of refraction is what is sensed in GCSPRI, as described briefly below. The identities of the various species in a sample of cells is spatially encoded in the chip by the pattern of capture spots. The number of cells of a particular species is determined from the magnitude of the GCSPRI signal from that spot. GCSPRI as used here can be summarized as follows: The cytometer chip is fabricated with a diffraction grating on its front surface. The chip is illuminated with a light emitting diode (LED) from the front. By proper choice of grating parameters and of the wavelength and the angle of incidence of a laser beam, laser light can be made to be coupled into an electromagnetic mode that resonates with surface plasmons and thus couples light into surface plasmons. Coupling of light into a surface plasmon at a given location reduces the amount of incident light reflected from that location. A change in the index of refraction at the surface of a capture spot gives rise to a change in the resonance condition. Depending on the specific design, the change in the index of refraction could manifest itself as a brightening or darkening, a change in the wavelength needed to excite the plasmon at a given angle of incidence, or a change in the angle of incidence needed to excite the plasmon at a given wavelength. Whereas a multiwavelength laser system with multichannel detection would be needed to detect multiple species in conventional flow cytometry, it suffices to use an LED and a single detector channel in the GCSPRI approach: this contributes significantly to reductions in cost, complexity, size, mass, and power. GCSPRI cytometer chips could be made of plastic and could be mass-produced cheaply by use of molding and other methods adopted from the manufacture of digital video disks. These methods are amenable to a high degree of miniaturization: such additional features as fluidic channels, reaction chambers, and fluid-coupling ports could readily be incorporated into the chips, without incurring substantial additional costs.

Fernandez, Salvador M.

Cytometer on a Chip

A cytometer now under development exploits spatial sorting of sampled cells on a microarray chip followed by use of grating-coupled surface-plasmon-resonance imaging (GCSPRI) to detect the sorted cells. This cytometer on a chip is a prototype of contemplated future miniature cytometers that would be suitable for rapidly identifying pathogens and other cells of interest in both field and laboratory applications and that would be attractive as alternatives to conventional flow cytometers. The basic principle of operation of a conventional flow cytometer requires fluorescent labeling of sampled cells, stringent optical alignment of a laser beam with a narrow orifice, and flow of the cells through the orifice, which is subject to clogging. In contrast, the principle of operation of the present cytometer on a chip does not require fluorescent labeling of cells, stringent optical alignment, or flow through a narrow orifice. The basic principle of operation of the cytometer on a chip also reduces the complexity, mass, and power of the associated laser and detection systems, relative to those needed in conventional flow cytometry. Instead of making cells flow in single file through a narrow flow orifice for sequential interrogation as in conventional flow cytometry, a liquid containing suspended sampled cells is made to flow over the front surface of a microarray chip on which there are many capture spots. Each capture spot is coated with a thin (approximately 50-nm) layer of gold that is, in turn, coated with antibodies that bind to cell-surface molecules characteristic of one the cell species of interest. The multiplicity of capture spots makes it possible to perform rapid, massively parallel analysis of a large cell population. The binding of cells to each capture spot gives rise to a minute change in the index of refraction at the surface of the chip. This change in the index of refraction is what is sensed in GCSPRI, as described briefly below. The identities of the various species in a sample of cells is spatially encoded in the chip by the pattern of capture spots. The number of cells of a particular species is determined from the magnitude of the GCSPRI signal from that spot. GCSPRI as used here can be summarized as follows: The cytometer chip is fabricated with a diffraction grating on its front surface. The chip is illuminated with a light emitting diode (LED) from the front. By proper choice of grating parameters and of the wavelength and the angle of incidence of a laser beam, laser light can be made to be coupled into an electromagnetic mode that resonates with surface plasmons and thus couples light into surface plasmons. Coupling of light into a surface plasmon at a given location reduces the amount of incident light reflected from that location. A change in the index of refraction at the surface of a capture spot gives rise to a change in the resonance condition. Depending on the specific design, the change in the index of refraction could manifest itself as a brightening or darkening, a change in the wavelength needed to excite the plasmon at a given angle of incidence, or a change in the angle of incidence needed to excite the plasmon at a given wavelength. Whereas a multiwavelength laser system with multichannel detection would be needed to detect multiple species in conventional flow cytometry, it suffices to use an LED and a single detector channel in the GCSPRI approach: this contributes significantly to reductions in cost, complexity, size, mass, and power. GCSPRI cytometer chips could be made of plastic and could be mass-produced cheaply by use of molding and other methods adopted from the manufacture of digital video disks. These methods are amenable to a high degree of miniaturization: such additional features as fluidic channels, reaction chambers, and fluid-coupling ports could readily be incorporated into the chips, without incurring substantial additional costs.

Fernandez, Salvador M.

Data for Roebuck et al. (2025), "Differences in dissolved organic matter composition between rivers and estuaries is conserved across freshwater and saltwater coastal regions"

Dissolved organic matter (DOM) in coastal surface waters influences local water quality and is an important component of biogeochemical cycling in coastal systems, but the processes that alter DOM composition along lower reaches of rivers and estuarine waters are poorly understood. Roebuck et al. (2025) leveraged a spatially distributed community sampling effort in coastal ecosystems across two regions to identify broad spatial drivers of surface water DOM composition and identify transferable trends between saltwater and freshwater coastal systems. Samples were collected by community members from 47 locations within the mid-Atlantic and Great Lakes coastal regions.This dataset includes:* A selection of commonly reported absorbance and fluorescence peaks normalized to dissolved organic carbon concentrations* Parallel factor output from the EC1 fluorescence datasets* A selection of commonly reported absorbance and fluorescence peaks * Spectral indices output from matlab script for absorbance and fluorescence datasets* CO2sys calculations of pH changes under varying temperatures and a constant salinity, DIC, and alkalinity concentrationAll data files are plain-text CSV (comma separated value) and no special software is required to read them.

54 ENVIRONMENTAL SCIENCES

BOREAS HYD-9 Hourly and Daily Rainfall Maps for the Southern Study Area

The Boreal Ecosystem-Atmosphere Study (BOREAS) Hydrology (HYD)-9 team collected data on precipitation and streamflow over portions of the Northern Study Area (NSA) and Southern Study Area (SSA). This data set contains Cartesian maps of rain accumulation for one-hour and daily periods during the summer of 1994 over the SSA only (not the full view of the radar). A parallel set of one-hour maps for the whole radar view has been prepared and is available upon request from the HYD-09 personnel. An incidental benefit of the areal selection was the elimination of some of the less accurate data, because for various reasons the radar rain estimates degrade considerably outside a range of about 100 km. The data are available in tabular ASCII files. The HYD-09 hourly and daily radar rainfall maps for the SSA are available from the Earth Observing System Data and Information System (EOSDIS) Oak Ridge National Laboratory (ORNL) Distributed Active Archive Center (DAAC). The data files are available on a CD-ROM (see document number 20010000884).

Eley, F. Joe