Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Data transfer for STAR grid jobs

The Solenoidal Tracker at RHIC (STAR) is a multipurpose experiment at the Relativistic Heavy Ion Collider (RHIC) with the primary goal to study the formation and properties of the quark-gluon plasma. STAR is an international collaboration of member institutions and laboratories from around the world. Yearly data-taking period produces PBytes of raw data collected by the experiment. STAR primarily uses its dedicated facility at BNL to process this data, but has routinely leveraged distributed systems, both high throughput (HTC) and high performance (HPC) computing clusters, to significantly augment the processing capacity available to the experiment. The ability to automate the efficient transfer of large data sets on reliable, scalable, and secure infrastructure is critical for any large-scale distributed processing campaign. For more than a decade, STAR computing has relied upon GridFTP with its x509-based authentication to build such data transfer systems and integrate them into its larger production workflow. The end of support by the community for both GridFTP and the x509 standard requires STAR to investigate other approaches to meet its distributed processing needs. In this study we investigate two multi-purpose data distribution systems, Globus.org and XRootD, as alternatives to GridFTP. We compare both their performance and the ease by which each service is integrated into the type of secure and automated data transfer systems STAR has previously built using GridFTP. The presented approach and study may be applicable to other distributed data processing use cases beyond STAR.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Disk-Based System for Producing and Distributing Science Products from MODIS

Since beginning operations in 1999, the MODIS Adaptive Processing System (MODAPS) has evolved to take advantage of trends in information technology, such as the falling cost of computing cycles and disk storage and the availability of high quality open-source software (Linux, Apache and Perl), to achieve substantial gains in processing and distribution capacity and throughput while driving down the cost of system operations.

Masuoka, Edward↗

Evolution of DUNE’s Production System

The DUNE experiment will start running in 2029 and record 30 PB/year of raw waveforms from Liquid Argon TPCs and photon detectors. The size of individual readouts can range from 100 MB to a typical 8 GB full readout of the detector, and even 100 TB for extended readouts from supernova candidates. These data then need to be cataloged, stored and distributed for processing worldwide. This massive amount of data and a heterogeneous computing environment necessitates a powerful and robust distributed computing infrastructure. In the process of building up that infrastructure, DUNE’s production system has recently undergone an overhaul, in which it has integrated 1) a new workflow management system (justIN) 2) a new data catalog (MetaCat) and 3) a state-of-the-art data management system (Rucio). Simulations of DUNE’s Far Detector and its prototypes ProtoDUNE Horizontal Drift (ProtoDUNE-HD) and ProtoDUNE Vertical Drift (ProtoDUNE-VD), as well as data from ProtoDUNE-HD serve as the first tests of this infrastructure.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

AIRSAR Web-Based Data Processing

The AIRSAR automated, Web-based data processing and distribution system is an integrated, end-to-end synthetic aperture radar (SAR) processing system. Designed to function under limited resources and rigorous demands, AIRSAR eliminates operational errors and provides for paperless archiving. Also, it provides a yearly tune-up of the processor on flight missions, as well as quality assurance with new radar modes and anomalous data compensation. The software fully integrates a Web-based SAR data-user request subsystem, a data processing system to automatically generate co-registered multi-frequency images from both polarimetric and interferometric data collection modes in 80/40/20 MHz bandwidth, an automated verification quality assurance subsystem, and an automatic data distribution system for use in the remote-sensor community. Features include Survey Automation Processing in which the software can automatically generate a quick-look image from an entire 90-GB SAR raw data 32-MB/s tape overnight without operator intervention. Also, the software allows product ordering and distribution via a Web-based user request system. To make AIRSAR more user friendly, it has been designed to let users search by entering the desired mission flight line (Missions Searching), or to search for any mission flight line by entering the desired latitude and longitude (Map Searching). For precision image automation processing, the software generates the products according to each data processing request stored in the database via a Queue management system. Users are able to have automatic generation of coregistered multi-frequency images as the software generates polarimetric and/or interferometric SAR data processing in ground and/or slant projection according to user processing requests for one of the 12 radar modes.

Chu, Anhua↗

Control structures for high speed processors

A special processor was designed to function as a Reed Solomon decoder with throughput data rate in the Mhz range. This data rate is significantly greater than is possible with conventional digital architectures. To achieve this rate, the processor design includes sequential, pipelined, distributed, and parallel processing. The processor was designed using a high level language register transfer language. The RTL can be used to describe how the different processes are implemented by the hardware. One problem of special interest was the development of dependent processes which are analogous to software subroutines. For greater flexibility, the RTL control structure was implemented in ROM. The special purpose hardware required approximately 1000 SSI and MSI components. The data rate throughput is 2.5 megabits/second. This data rate is achieved through the use of pipelined and distributed processing. This data rate can be compared with 800 kilobits/second in a recently proposed very large scale integration design of a Reed Solomon encoder.

Maki, G. K.↗

Restoring Historic Forest Disturbance Frequency Would Partially Mitigate Droughts in the Central Sierra Nevada Mountains

Forest thinning and prescribed fire are expected to improve the climate resilience and water security of forests in the western U.S., but few studies have directly modeled the hydrological effects of multi-decadal landscape-scale forest disturbance. By updating a distributed process-based hydrological model (DHSVM) with vegetation maps from a distributed forest ecosystem model (LANDIS-II), we simulate the water resource impacts of forest management scenarios targeting partial or full restoration of the pre-colonial disturbance return interval in the central Sierra Nevada mountains. In a fully restored disturbance regime that includes fire, thinning, and insect mortality, reservoir inflow increases by 4%–9% total and 8%–14% in dry years. At sub-watershed scales (10–100 km2), thinning dense forests can increase streamflow by >20% in dry years. In a thinner forest, increased understory transpiration compensates for decreased overstory transpiration. Consequentially, 73% of streamflow gains are attributable to decreased overstory rain and snow interception loss. Thinner forests can increase headwater peak flows, but reservoir-scale peak flows are almost exclusively influenced by climate. Uncertainty in future precipitation causes high uncertainty in future water yield, but the additional water yield attributable to forest disturbance is about five times less sensitive to annual precipitation uncertainty. This partial decoupling of the streamflow disturbance response from annual precipitation makes disturbance especially valuable for water supply during dry years. Our study can increase confidence in the water resource benefits of restoring historic forest disturbance frequencies in the central Sierra Nevada mountains, and our modeling framework is widely applicable to other forested mountain landscapes.

Boardman, Eli N. [University of Nevada, Reno, NV (↗

Science data processing in the Mission to Planet Earth era

The data-processing systems developed to support NASA's Mission to Planet Earth (MPE) are described in an overview of the system architectures. The MPE program is discussed in terms of flight segments and data types to examine the required elements of the information systems. The data-processing segments are described for the Atmospheric Laboratory for Applications and Science, the Tropical Rainfall Measuring Mission, the Earth Observing System, and the Geostationary Earth Observatory. The data and information systems for the separate segments provide some redundant processing services. Distributed active archive centers are employed for each segment composed of three subelements: (1) a product-generation system, (2) a data archive and distribution system, and (3) an information management system. The data processing design for the MPE is expected to facilitate the understanding of the entire earth system on a global scale.

Goodman, H. M.↗

EOSDIS Data for Oceans Research

The JPL Physical Oceanography Distributed Active Archive Center (DAAC) is one of nine centers processing data for the Earth Observation System. As the physical oceanography data center, it archives, processes, and distributes data to researchers from various satellites. Data holdings can be searched through the Global Change Master Directory, and an advanced information management system is now in prototype.

physical oceanography EOS↗

Investigation into Cloud Computing for More Robust Automated Bulk Image Geoprocessing

Geospatial resource assessments frequently require timely geospatial data processing that involves large multivariate remote sensing data sets. In particular, for disasters, response requires rapid access to large data volumes, substantial storage space and high performance processing capability. The processing and distribution of this data into usable information products requires a processing pipeline that can efficiently manage the required storage, computing utilities, and data handling requirements. In recent years, with the availability of cloud computing technology, cloud processing platforms have made available a powerful new computing infrastructure resource that can meet this need. To assess the utility of this resource, this project investigates cloud computing platforms for bulk, automated geoprocessing capabilities with respect to data handling and application development requirements. This presentation is of work being conducted by Applied Sciences Program Office at NASA-Stennis Space Center. A prototypical set of image manipulation and transformation processes that incorporate sample Unmanned Airborne System data were developed to create value-added products and tested for implementation on the "cloud". This project outlines the steps involved in creating and testing of open source software developed process code on a local prototype platform, and then transitioning this code with associated environment requirements into an analogous, but memory and processor enhanced cloud platform. A data processing cloud was used to store both standard digital camera panchromatic and multi-band image data, which were subsequently subjected to standard image processing functions such as NDVI (Normalized Difference Vegetation Index), NDMI (Normalized Difference Moisture Index), band stacking, reprojection, and other similar type data processes. Cloud infrastructure service providers were evaluated by taking these locally tested processing functions, and then applying them to a given cloud-enabled infrastructure to assesses and compare environment setup options and enabled technologies. This project reviews findings that were observed when cloud platforms were evaluated for bulk geoprocessing capabilities based on data handling and application development requirements.

Brown, Richard B.↗

Opportunities for Integrating Electric and Gas Planning

Growing demand for electricity in buildings is creating new opportunities and challenges for electric and natural gas planning. To date, electric and gas planning processes have largely been siloed, and only a few jurisdictions have started to examine how these two processes could be linked together. The report explores the current state of electric and gas system planning, and the benefits and challenges of integrating these planning processes. We frame the characteristics of an integrated electric and gas distribution planning process and identify indicators of progress toward the characteristics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

The Seasat low rate data processing system

The development of the algorithms for data processing and distribution, the circuitry, and the performance of the Seasat low rate data processing system are reviewed. The system controls data from the radar altimeter, scatterometer, microwave radiometer, and the visible and IR radiometer for independent transmission of each instrument's readings. The downlink operates at 25 kb/sec, and a yearlong program of geophysical evaluation proceeded shortly after launch, allowing on-line engineering evaluation and alteration of the control algorithms in the system. Some data is preformatted for immediate distribution and storage in archival quality. A catalog and abstracts are provided to users allowing a RAM search from remote terminals for historical conditions. Procedures for verifying and altering the algorithms are detailed.

Brown, J. W.↗

Catalyzing deep decarbonization with federated battery diagnosis and prognosis for better data management in energy storage systems

Industrial data analytics methods play a central role in improving energy storage performance and efficiency, impacting the future of electrified transportation and renewable electricity generation. However, significant challenges hinder the large-scale deployment of batteries. Conventional methods rely on centralized collection and processing of fleet-level data, leading to database size issues and privacy concerns due to potential data breaches. To enable scalable deployment of battery management systems, this article proposes a federated battery diagnosis and prognosis model, which distributes the processing of battery standard current-voltage-time-usage data in a privacy-preserving manner. Instead of transferring the raw data, this approach communicates only the locally processed parameters, thus reducing communication load and preserving data confidentiality. The federated model offers a paradigm shift in battery health management through privacy-preserving distributed methods for battery data processing and lifetime prediction, ensuring the reliable and sustainable deployment of lithium-ion batteries in a rapidly evolving world.

asset health management↗

Switching Device-Cognizant Sequential Distribution System Restoration

This paper presents an optimization framework for sequential reconfiguration using an assortment of switching devices and repair process in distribution system restoration. Compared to existing studies, this paper considers types, capabilities and operational limits of different switching devices, making it applicable in practice. We develop a novel multi-phase method to find the optimal sequential operation of various switching devices and repair faulted areas. We consider circuit breakers, reclosers, sectionalizers, load breaker switches, and fuses. The switching operation problem is decomposed into two mixed-integer linear programming (MILP) subproblems. The first subproblem determines the optimal network topology and estimates the number of steps to reach that topology, while the second subproblem generates a sequence of switching operations to coordinate the switches. For repairing the faults, we design an MILP model that dispatches repair crews to clear faults and replace melted fuses. After clearing a fault, we update the topology of the network by generating a new sequence of switching operations, and the process continues until all faults are cleared. To improve the computational efficiency, a network reduction algorithm is developed to group line sections, such that only switchable sections are present in the reduced network. The proposed method is validated on the IEEE 123-bus and 8500-bus systems.

distribution system↗

Distribution Development for the RDX Regional Model at Los Alamos National Laboratory - 20385

Representing uncertainty in model inputs often means finding a balance between uncertainty and physical reality. Developing wide distributions may seem conservative in principle, but this approach may lead to unrealistic model results. Characterizing the current state of knowledge of stochastic inputs presents many challenges, especially if the data available are limited or have limited relevance to the site. Relationships among these inputs may also be important to represent but are typically complex or difficult to define. Often special adjustments must be made to account for reduced credibility in particular data. If parameters are strongly related to one another, a correlation structure may be developed for input to the model. Other techniques such as regression models may be used to incorporate relationships between the information available and the desired parameters. The process of developing distributions must consider details of the model in terms of what the distribution is meant to represent. This paper uses the example of a probabilistic fate and transport model for hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX) in the regional aquifer at Los Alamos National Laboratory (LANL). For many parameters, a single draw is applied to all space and time over which the model is run, for a single iteration. This simplification is often made for many reasons, and can often be beneficial, but also adds additional complexity in the distribution development process. Defining the distributional goals as they relate to the modeling process is an important step, which should take place prior to evaluation of the data. Usually, the distributions developed are meant to characterize the average value of the parameter over the spatial and temporal domain of the model. Distribution development requires consideration of many sources of information on the parameter where available, ideally from multiple references. Examples of different sources include data from different references but also from different conditions, measurement methods, or experimental types. Depending on these conditions and the reliability or relevance of particular references, different sources of data may each contribute valuable information but have varying relevance to the site. In these cases, weighting data unequally is a useful way to incorporate this information. As an example, aqueous dispersivity data are available for a variety of rock types. Only a few values are available for the desired rock type, and this is not enough to develop a distribution. Therefore, dispersivity values from other rock types are included in distribution development but are down-weighted such that the best data have the most influence on the distribution developed. In another case, K{sub d} distributions in the model are meant to represent a known composition of multiple soil types. Data from these materials are weighted accordingly to develop a distribution for the weighted average K{sub d} across soil types. Other cases include varying reliability of different sources, and weighting data according to the confidence in these sources. In some cases, input parameters are correlated with one another. An example is advective porosity, which is positively correlated with total porosity and must be less than total porosity. Paired data with both parameters must exist to discern the relationship between the two parameters and if it is necessary to build a correlation structure into the model. In general, correlated parameters may be represented in the model by a multivariate distribution, or perhaps more desirably, capturing the correlation within the developed distributions which may be treated as independent from one another. In the example of porosity, this can be done by transforming advective porosity into a proportion of total porosity which may be drawn independently from the distribution of total porosity. This paper explores how complex data and correlations can be incorporated to meet the distributional goals of the model, using the RDX regional model as a detailed example. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Stability analysis of a liquid fuel annular combustion chamber

The problems of combustion instability in an annular combustion chamber are investigated. A modified Galerkin method was used to produce a set of modal amplitude equations from the general nonlinear partial differential acoustic wave equation. From these modal amplitude equations, the two variable perturbation method was used to develop a set of approximate equations of a given order of magnitude. These equations were modeled to show the effects of velocity sensitive combustion instabilities by evaluating the effects of certain parameters in the given set of equations. By evaluating these effects, parameters which cause instabilities to occur in the combustion chamber can be ascertained. It is assumed that in the annular combustion chamber, the liquid propellants are injected uniformly across the injector face, the combustion processes are distributed throughout the combustion chamber, and that no time delay occurs in the combustion processes.

Mcdonald, G. H.↗

Conference on Real-Time Computer Applications in Nuclear, Particle and Plasma Physics, 6th, Williamsburg, VA, May 15-19, 1989, Proceedings

Papers on real-time computer applications in nuclear, particle, and plasma physics are presented, covering topics such as expert systems tactics in testing FASTBUS segment interconnect modules, trigger control in a high energy physcis experiment, the FASTBUS read-out system for the Aleph time projection chamber, a multiprocessor data acquisition systems, DAQ software architecture for Aleph, a VME multiprocessor system for plasma control at the JT-60 upgrade, and a multiasking, multisinked, multiprocessor data acquisition front end. Other topics include real-time data reduction using a microVAX processor, a transputer based coprocessor for VEDAS, simulation of a macropipelined multi-CPU event processor for use in FASTBUS, a distributed VME control system for the LISA superconducting Linac, a distributed system for laboratory process automation, and a distributed system for laboratory process automation. Additional topics include a structure macro assembler for the event handler, a data acquisition and control system for Thomson scattering on ATF, remote procedure execution software for distributed systems, and a PC-based graphic display real-time particle beam uniformity.

Pordes, Ruth↗

Parallel volume ray-casting for unstructured-grid data on distributed-memory architectures

As computing technology continues to advance, computational modeling of scientific and engineering problems produces data of increasing complexity: large in size and unstructured in shape. Volume visualization of such data is a challenging problem. This paper proposes a distributed parallel solution that makes ray-casting volume rendering of unstructured-grid data practical. Both the data and the rendering process are distributed among processors. At each processor, ray-casting of local data is performed independent of the other processors. The global image composing processes, which require inter-processor communication, are overlapped with the local ray-casting processes to achieve maximum parallel efficiency. This algorithm differs from previous ones in four ways: it is completely distributed, less view-dependent, reasonably scalable, and flexible. Without using dynamic load balancing, test results on the Intel Paragon using from two to 128 processors show, on average, about 60% parallel efficiency.

Ma, Kwan-Liu↗