Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Permanent Source - Distributed Acoustic Sensing (PESO-DAS) v1

The SOV/DAS software program focuses on seismic monitoring and data processing of permanent seismic sources Surface Orbital Vibrators (SOVs) and fiber-optics sensing Distributed Acoustic Sensing (DAS). Its key features include automated data processing of the continuous seismic monitoring data acquired with DAS, and output of processed shot gathers and QC plots. The software supports timelapse seismic for long-term reservoir monitoring, and is used for monitoring of geological carbon storage sites, geothermal reservoirs, and oil and gas, and in general subsurface resource management.

dos Santos Maia Correa, Julia [Lawrence Berkeley N↗

Sensing Electrical Networks Securely & Economically (SENSE)

The growing adoption of distributed energy resources (DERs) like battery energy storage systems and roof top solar/PV and the rapid penetration of electric vehicles (EVs), the electric grid is undergoing a major transformation with elevated stress on legacy grid assets. Despite a lot of expenditure to address these challenges, both in dollars and manpower, utilities have not been able to receive the value that was promised. The gains have been most visible at the transmission and substation level, especially where the main objective was improving operational and economic efficiency for the utility. Improving visibility and control at a few select points enhances the existing and established paradigm of centralized command and control. With changing load patterns, load types and the overall transition to an “active grid”, the centralized control and coordination paradigm gets challenged. To address the challenges, a new architecture and mechanism is needed, one that supports decentralized control and decision making, extracting value streams at the grid edge, particularly as the changes are fueled by transitions occurring in the distribution system. To address this, a communications and data processing platform, “GAMMA” was developed and demonstrated through the project. At the heart of the platform, are distributed, intelligent edge nodes with sensing and compute capabilities, that can record and analyze information locally. They are embedded in sensors and actuators specific to different distribution system applications. Phase 1 of the project focused on developing novel sensor technology that can be used for monitoring utility pole top distribution transformers. The sensors were designed with the objective of being low-cost, communicating with the GAMMA cloud using novel “delay-tolerant” networking using Bluetooth and a secure mobile application. They were non-intrusive in nature so that they can be installed quickly in the field, resulting in overall low cost of deployment and operations. Following the successful completion of Phase 1, the team manufactured 100 units for a field demonstration in Phase 2. The field demonstration was carried out on two real feeder systems with the local utility partner. In total, 100 sensors were installed and operated over a period of 6 months in the state of Georgia. The platform is operational end to end, with the cloud infrastructure deployed on a distributed, serverless environment that can serve multiple data streams, an analytics engine and a portal to securely view the data from multiple assets. The data collected through the GAMMA Mobile Phone app showcased the viability of the novel delay tolerant networking architecture, and the data processing algorithms developed through the course of the project, were successful in extracting important information about the overall network, improving the utility’s visibility and situational awareness in the distribution feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Confocal (micro)-XRF for 3D anlaysis of elements distribution in hot environmental particles

Studies on the fate and transport of radioactive contaminates in the environment are often constrained by a lack of knowledge on the elemental distribution and general behavior of particulate bound radionuclides contained in hot particles. A number of hot particles were previously isolated from soil samples collected at former U.S. nuclear test sites in the Marshall Islands and characterized using non-destructive techniques [1]. The present investigation at HASYLAB is a part of larger research program at ITU regarding the characterization of environmental radioactive particles different locations and source-terms. Radioactive particles in the environment are formed under a number of different release scenarios and, as such, their physicochemical properties may provide a basis for identifying source-term specific contamination regimes. Consequently, studies on hot particles are not only important in terms of studying the elemental composition and geochemical behavior of hot particles but may also lead to advances in assessing the long-term impacts of radioactive contamination on the environment. Six particles isolated from soil samples collected at the Marshall Islands were studied. The element distribution in the particles was determined by confocal {micro}-XRF analysis using the ANKA FLUO beam line. The CRL (compound refractive lens) was used to focus the exciting beam and the polycapillary half lens to collimate the detector. The dimensions of confocal spot were measured by 'knife edge scanning' method with thin gold structure placed at Si wafer. The values of 3.1 x 1.4 x 18.4 {micro}m were achieved if defined as FWHMs of measured L?intensity profiles and when the19.1 keV exciting radiation was used. The collected XRF spectra were analyzed offline with AXIL [2] software to obtain net intensities of element characteristic lines.Further data processing and reconstruction of element distribution was done with the software 'R' [3] dedicated for statistical calculations. In figure 1 the distributions of Pu, Fe and Ti obtained for one of the studied hot particles are presented. The strongest signal was recorded for plutonium; the signals from iron and titanium are respectively 14 and 38 times less. It means that Pu is the most abundant of the observed elements. However, since the light elements are not detectable with the applied measurement conditions, it cannot be definitely stated if plutonium is the main element present in the sample. The isosurfaces are calculated at 20 % of maximum intensity for each element. Please note that the isosurfaces on the drawing are transparent. Changes in the spatial distribution of Pu, Fe, and Ti within the particle are shown in Fig. 2a, 2b, and 2c. Distinct elemental patterns are clearly visible at the higher concentration levels. The distributions of Cr, Cu, and Pb were also reconstructed but the results are not presented here. As it is shown in Fig. 1, the correlation between elements is good at low concentrations but the maxima of concentrations are not strongly correlated (see Fig. 2.). In general, the particle is inhomogeneous in terms of its elemental composition. Similar inhomogeneities were found for other particles with Pu identified as a major element in three of the six particles examined.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Expert system decision support for low-cost launch vehicle operations

Progress in assessing the feasibility, benefits, and risks associated with AI expert systems applied to low cost expendable launch vehicle systems is described. Part one identified potential application areas in vehicle operations and on-board functions, assessed measures of cost benefit, and identified key technologies to aid in the implementation of decision support systems in this environment. Part two of the program began the development of prototypes to demonstrate real-time vehicle checkout with controller and diagnostic/analysis intelligent systems and to gather true measures of cost savings vs. conventional software, verification and validation requirements, and maintainability improvement. The main objective of the expert advanced development projects was to provide a robust intelligent system for control/analysis that must be performed within a specified real-time window in order to meet the demands of the given application. The efforts to develop the two prototypes are described. Prime emphasis was on a controller expert system to show real-time performance in a cryogenic propellant loading application and safety validation implementation of this system experimentally, using commercial-off-the-shelf software tools and object oriented programming techniques. This smart ground support equipment prototype is based in C with imbedded expert system rules written in the CLIPS protocol. The relational database, ORACLE, provides non-real-time data support. The second demonstration develops the vehicle/ground intelligent automation concept, from phase one, to show cooperation between multiple expert systems. This automated test conductor (ATC) prototype utilizes a knowledge-bus approach for intelligent information processing by use of virtual sensors and blackboards to solve complex problems. It incorporates distributed processing of real-time data and object-oriented techniques for command, configuration control, and auto-code generation.

Szatkowski, G. P.↗

An analysis of bulk data movement patterns in large-scale scientific collaborations

Large-scale research efforts such as LHC experiments, ITER, and climate modelling are built upon large, globally distributed collaborations. For reasons of scalability and agility and to make effective use of existing computing resources, data processing and analysis for these projects is based on distributed computing models. Such projects thus depend on predictable and efficient bulk data movement between collaboration sites. However, the available computing and networking resources to different collaboration sites vary greatly. Large collaboration sites (such as Fermilab, CERN) have created data centres comprising hundreds, and even thousands, of computation nodes to develop massively scaled, highly distributed cluster-computing platforms. These sites are usually well connected to outside worlds with high-speed networks with bandwidth greater than 10Gbps. On the other hand, some small collaboration sites have limited computing resources or poor networking connectivity. Therefore, the bulk data movements across collaboration sites vary greatly. Fermilab is the US-CMS Tier-1 Centre and the main data centre for a few other large-scale research collaborations. Scientific traffic (e.g., CMS) dominates the traffic volumes in both inbound and outbound directions of Fermilab off-site traffic. Fermilab has deployed a Flow-based network traffic collection and analysis system to monitor and analyze the status and patterns of bulk data movement between the Laboratory and its collaboration sites. In this paper, we discuss the current status and patterns of bulk data movement between Fermilab and its collaboration sites.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Data-Driven Robust RVFLNs Modeling of a Blast Furnace Iron-Making Process Using Cauchy Distribution Weighted M-Estimation

Optimal operation of a practical blast furnace (BF) ironmaking process depends largely on a good measurement of molten iron quality (MIQ) indices. However, measuring the MIQ online is not feasible using the available techniques. In this paper, a novel data-driven robust modeling is proposed for online estimation of MIQ using improved random vector functional-link networks (RVFLNs). Since the output weights of traditional RVFLNs are obtained by the least squares approach, a robustness problem may occur when the training dataset is contaminated with outliers. This affects the modeling accuracy of RVFLNs. To solve this problem, a Cauchy distribution weighted M-estimation based robust RFVLNs is proposed. Since the weights of different outlier data are properly determined by the Cauchy distribution, their corresponding contribution on modeling can be properly distinguished. Thus robust and better modeling results can be achieved. Moreover, given that the BF is a complex nonlinear system with numerous coupling variables, the data-driven canonical correlation analysis is employed to identify the most influential components from multitudinous factors that affect the MIQ indices to reduce the model dimension. Finally, experiments using industrial data and comparative studies have demonstrated that the obtained model produces a better modeling and estimating accuracy and stronger robustness than other modeling methods.

42 ENGINEERING↗

SU-C-207A-06: On-Line Beam Range Verification with Multiple Scanning Particle Beams: Initial Feasibility Study with Simulations

Purpose: To investigate the feasibility and requirement for intra-fraction on-line multiple scanning particle beam range verifications (BRVs) with in-situ PET imaging, which is beyond the current single-beam BRV with extra factors that will affect the BR measurement accuracy, such as beam diameter, separation between beams, and different image counts at different BRV positions. Methods: We simulated a 110-MeV proton beam with 5-mm diameter irradiating a uniform PMMA phantom by GATE simulation, which generated nuclear interaction-induced positrons. In this preliminary study, we simply duplicated these positrons and placed them next to the initial protons to approximately mimic the two spatially separated positron distributions produced by two beams parallel to each other but with different beam ranges. These positrons were then imaged by a PET (∼2-mm resolution, 10% sensitivity, 320×320×128 mm^3 FOV) with different acquisition times. We calculated the positron activity ranges (ARs) from reconstructed PET images and compared them with the corresponding ARs of original positron distributions. Results: Without further image data processing and correction, the preliminary study show the errors between the measured and original ARs varied from 0.2 mm to 2.3 mm as center-to-center separations and range differences were in the range of 8–12 mm and 2–8 mm respectively, indicating the accuracy of AR measurement strongly depends on the beam separations and range differences. In addition, it is feasible to achieve ≤ 1.0-mm accuracy for both beams with 1-min PET acquisition and 12 mm beam separation. Conclusion: This study shows that the overlap between the positron distributions from multiple scanning beams can significantly impact the accuracy of BRVs of distributed particle beams and need to be further addressed beyond the established method of single-beam BRV, but it also indicates the feasibility to achieve accurate on-line multi-beam BRV with further improved method.

60 APPLIED LIFE SCIENCES↗

Computational technique for plasma parameters determination using Langmuir probe data

In the present work, we consider a new numerical method for processing the experimental information on the electron energy distribution function obtained with a Langmuir probe in a low-pressure plasma. This method offers the possibility to establish the temperature and concentration of the electrons for different forms of the distribution function. Some specific difficulties of the previous methods used to do such estimations are surpassed using the method proposed in this work.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Next Generation Workload Management System For Big Data on Heterogeneous Distributed Computing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS and ALICE are the largest collaborations ever assembled in the sciences and are at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, both experiments rely on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System (WMS) for managing the workflow for all data processing on hundreds of data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. The scale is demonstrated by the following numbers: PanDA manages O(10 2 ) sites, O(10 5 ) cores, O(10 8 ) jobs per year, O(10 3 ) users, and ATLAS data volume is O(10 17 ) bytes. In 2013 we started an ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF). The project titled 'Next Generation Workload Management and Analysis System for Big Data' (BigPanDA) is funded by DOE ASCR and HEP. Extending PanDA to clouds and LCF presents new challenges in managing heterogeneity and supporting workflow. The BigPanDA project is underway to setup and tailor PanDA at the Oak Ridge Leadership Computing Facility (OLCF) and at the National Research Center "Kurchatov Institute" together with ALICE distributed computing and ORNL computing professionals. Our approach to integration of HPC platforms at the OLCF and elsewhere is to reuse, as much as possible, existing components of the PanDA system. Finally, we will present our current accomplishments with running the PanDA WMS at OLCF and other supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications.

97 MATHEMATICS AND COMPUTING↗

Semi-automatic process partitioning for parallel computation

On current multiprocessor architectures one must carefully distribute data in memory in order to achieve high performance. Process partitioning is the operation of rewriting an algorithm as a collection of tasks, each operating primarily on its own portion of the data, to carry out the computation in parallel. A semi-automatic approach to process partitioning is considered in which the compiler, guided by advice from the user, automatically transforms programs into such an interacting task system. This approach is illustrated with a picture processing example written in BLAZE, which is transformed into a task system maximizing locality of memory reference.

Koelbel, Charles↗

PanDA: Production and Distributed Analysis System

The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at the LHC data processing scale. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year. PanDA’s flexibility and scalability make it suitable for the High Energy Physics community and wider science domains at the Exascale. Beyond High Energy Physics, PanDA’s relevance extends to other big data sciences, as evidenced by its adoption in the Vera C. Rubin Observatory and the sPHENIX experiment. As the significance of advanced workflows continues to grow, PanDA has transformed into a comprehensive ecosystem, effectively tackling challenges associated with emerging workflows and evolving computing technologies. The paper discusses PanDA’s prominent role in the scientific landscape, detailing its architecture, functionality, deployment strategies, project management approaches, results, and evolution into an ecosystem.

97 MATHEMATICS AND COMPUTING↗

Independent Orbiter Assessment (IOA): Assessment of the backup flight system FMEA/CIL

The results of the Independent Orbiter Assessment (IOA) of the Failure Modes and Effects Analysis (FMEA) and Critical Items List (CIL) are presented. The IOA effort first completed an analysis of the Backup Flight System (BFS) hardware, generating draft failure modes and Potential Critical Items. To preserve independence, this analysis was accomplished without reliance upon the results contained within the NASA FMEA/CIL documentation. The IOA results were then compared to the proposed NASA Post 51-L FMEA/CIL baseline. A resolution of each discrepancy from the comparison is provided through additional analysis as required. This report documents the results of that comparison for the Orbiter BFS hardware. The IOA product for the BFS analysis consisted of 29 failure mode worksheets that resulted in 21 Potential Critical Items (PCI) being identified. This product was originally compared with the proposed NASA BFS baseline and subsequently compared with the applicable Data Processing System (DPS), Electrical Power Distribution and Control (EPD and C), and Displays and Controls NASA CIL items. The comparisons determined if there were any results which had been found by the IOA but were not in the NASA baseline. The original assessment determined there were numerous failure modes and potential critical items in the IOA analysis that were not contained in the NASA BFS baseline. Conversely, the NASA baseline contained three FMEAs (IMU, ADTA, and Air Data Probe) for CIL items that were not identified in the IOA product.

Prust, E. E.↗

Real-time flight test data distribution and display

Enhancements to the real-time processing and display systems of the NASA Western Aeronautical Test Range are described. Display processing has been moved out of the telemetry and radar acquisition processing systems super-minicomputers into user/client interactive graphic workstations. Real-time data is provided to the workstations by way of Ethernet. Future enhancement plans include use of fiber optic cable to replace the Ethernet.

Nesel, Michael C.↗

The Role of Metadata Standards in EOSDIS Search and Retrieval Applications

Metadata standards play a critical role in data search and retrieval systems. Metadata tie software to data so the data can be processed, stored, searched, retrieved and distributed. Without metadata these actions are not possible. The process of populating metadata to describe science data is an important service to the end user community so that a user who is unfamiliar with the data, can easily find and learn about a particular dataset before an order decision is made. Once a good set of standards are in place, the accuracy with which data search can be performed depends on the degree to which metadata standards are adhered during product definition. NASA's Earth Observing System Data and Information System (EOSDIS) provides examples of how metadata standards are used in data search and retrieval.

Pfister, Robin↗

Software techniques for a distributed real-time processing system

The paper describes software techniques developed for the Unified Data System (UDS), a distributed processor network for control and data handling onboard a planetary spacecraft. These techniques include a structured language for specifying the programs contained in each module, and a small executive program in each module which performs scheduling and implements the module task.

Lesh, F.↗

Distributive On-line Processing, Visualization and Analysis System for Gridded Remote Sensing Data

The ability to use data stored in the current Earth Observing System (EOS) archives for studying regional or global phenomena is highly dependent on having a detailed understanding of the data's internal structure and physical implementation. Gaining this understanding and applying it to data reduction is a time- consuming task that must be undertaken before the core investigation can begin. This is an especially difficult challenge when science objectives require users to deal with large multi-sensor data sets that are usually of different formats, structures, and resolutions, for example, when preparing data for input into modeling systems. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has taken a major step towards meeting this challenge by developing an infrastructure with a Web interface that allows users to perform interactive analysis online without downloading any data, the GES-DISC Interactive Online Visualization and Analysis Infrastructure or "Giovanni." Giovanni provides interactive, online, analysis tools for data users to facilitate their research. There have been several instances of this interface created to serve TRMM users, Aerosol scientists, Ocean Color and Agriculture applications users. The first generation of these tools support gridded data only. The user selects geophysical parameters, area of interest, time period; and the system generates an output on screen in a matter of seconds. The currently available output options are: Area plot averaged or accumulated over any available data period for any rectangular area; Time plot time series averaged over any rectangular area; Time plots image view of any longitude-time and latitude-time cross sections; ASCII output for all plot types; Image animation for area plot. In the future, we will add correlation plots, GIS-compatible outputs, etc. This allow user to focus on data content (i.e. science parameters) and eliminate the need for expensive learning, development and processing tasks that are redundantly incurred by an archive's user community. The current implementation utilizes the GrADS-DODS Server (GDS), a stable, secure data server that provides subsetting and analysis services across the Internet for any GrADS-readable dataset. The subsetting capability allows users to retrieve a specified temporal and/or spatial subdomain from a large dataset, eliminating the need to download everything simply to access a small relevant portion of a dataset. The analysis capability allows users to retrieve the results of an operation applied to one or more datasets on the server. In our case, we use this approach to read pre-processed binary files and/or to read and extract the needed parts from HDF or HDF-EOS files. These subsets then serve as inputs into GrADS processing and analysis scripts. It can be used in a wide variety of Earth science applications: climate and weather events study and monitoring; modeling. It can be easily configured for new applications.

Leptoukh, G.↗

Improving Data Discovery, Analysis, and Visualizations With Cloud-Based User Services

The Global Hydrometeorology Resource Center (GHRC) Distributed Active Archive Center (DAAC) is one of 12 DAACs managed by the United States National Aeronautics and Space Administration (NASA) Earth Science Data and Information System (ESDIS) project [1]. GHRC and the other DAACs are designed to process, archive, document, and distribute NASA Earth-observing data, ranging from satellite missions to field campaigns [2]. A major goal of the DAACs is to enable science with these data. Science enabling can be difficult as datasets can be very large, use multiple formats, come from numerous platforms, and require three-dimensional visualization. GHRC is using its expertise with cloud-based technologies to develop open source and open science tools to empower users to explore, coincidentally visualize, and analyze multiple datasets. Being open source, the user community can develop visualizations for their own datasets. This presentation will expand on this objective and highlight the capabilities available to the international community now.

GHRC↗

Estimating the Distribution of Colored Dissolved Organic Matter During the Southern Ocean Gas Exchange Experiment Using Four-Dimensional Variational Data Assimilation

We diagnosed the effect of various physical processes on the distribution of mixed-layer colored dissolved organic matter (CDOM) and a sulfur hexauoride (SF6) tracer during the Southern Ocean Gas Exchange Experiment (SO GasEx). The biochemical upper ocean state estimate uses in situ and satellite biochemical and physical data in the study region, including CDOM (absorption coefcient and spectral slope), SF6, hydrography, and sea level anomaly. Modules for photobleaching of CDOM and surface transport of SF6 were coupled with an ocean circulation model for this purpose. The observed spatial and temporal variations in CDOM were captured by the state estimate without including any new biological source term for CDOM, assuming it to be negligible over the 26 days of the state estimate. Thermocline entrainment and photobleaching acted to diminish the mixed-layer CDOM with time scales of 18 and 16 days, respectively. Lateral advection of CDOM played a dominant role and increased the mixed-layer CDOM with a time scale of 12 days, whereas lateral diffusion of CDOM was negligible. A Lagrangian view on the CDOM variability was demonstrated by using the SF6 as a weighting function to integrate the CDOM elds. This and similar data assimilation methods can be used to provide reasonable estimates of optical properties, and other physical parameters over the short-term duration of a research cruise, and help in the tracking of tracer releases in large-scale oceanographic experiments, and in oceanographic process studies.

Del Castillo, C. E.↗