A REALIZATION OF AN AUTOMATED DATA FLOW FOR DATA COLLECTING, PROCESSING, STORING AND RETRIEVING: GEONET
Explore the source record for details and available documents.
Friedsam, H.↗
SEARCH · Engineering Papers
Results for “data flow”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Representing object oriented specifications and designs with extended data flow notations
The issue of using extended data flow notations to document object oriented designs and specifications is discussed. Extended data flow notations, for the purposes here, refer to notations that are based on the rules of Yourdon/DeMarco data flow analysis. The extensions include additional notation for representing real-time systems as well as some proposed extensions specific to object oriented development. Some advantages of data flow notations are stated. How data flow diagrams are used to represent software objects are investigated. Some problem areas with regard to using data flow notations for object oriented development are noted. Some initial solutions to these problems are proposed.
Buser, Jon Franklin↗
Software Tool Integrating Data Flow Diagrams and Petri Nets
Data Flow Diagram - Petri Net (DFPN) is a software tool for analyzing other software to be developed. The full name of this program reflects its design, which combines the benefit of data-flow diagrams (which are typically favored by software analysts) with the power and precision of Petri-net models, without requiring specialized Petri-net training. (A Petri net is a particular type of directed graph, a description of which would exceed the scope of this article.) DFPN assists a software analyst in drawing and specifying a data-flow diagram, then translates the diagram into a Petri net, then enables graphical tracing of execution paths through the Petri net for verification, by the end user, of the properties of the software to be developed. In comparison with prior means of verifying the properties of software to be developed, DFPN makes verification by the end user more nearly certain, thereby making it easier to identify and correct misconceptions earlier in the development process, when correction is less expensive. After the verification by the end user, DFPN generates a printable system specification in the form of descriptions of processes and data.
Thronesbery, Carroll↗
Mentat: An object-oriented macro data flow system
Mentat, an object-oriented macro data flow system designed to facilitate parallelism in distributed systems, is presented. The macro data flow model is a model of computation similar to the data flow model with two principal differences: the computational complexity of the actors is much greater than in traditional data flow systems, and there are persistent actors that maintain state information between executions. Mentat is a system that combines the object-oriented programming paradigm and the macro data flow model of computation. Mentat programs use a dynamic structure called a future list to represent the future of computations.
Grimshaw, Andrew S.↗
Processing Particle Data Flows with SmartNICs
Many distributed applications implement complex data flows and need a flexible mechanism for routing data between producers and consumers. Recent advances in programmable network interface cards, or SmartNICs, represent an opportunity to offload data-flow tasks into the network fabric, thereby freeing the hosts to perform other work. System architects in this space face multiple questions about the best way to leverage SmartNICs as processing elements in data flows. In this paper, we advocate the use of Apache Arrow as a foundation for implementing data-flow tasks on SmartNICs. We report on our experiences adapting a partitioning algorithm for particle data to Apache Arrow and measure the on-card processing performance for the BlueField-2 SmartNIC. Our experiments confirm that the BlueField-2’s (de)compression hardware can have a significant impact on in-transit workflows where data must be unpacked, processed, and repacked.
97 MATHEMATICS AND COMPUTING↗
Performance study of a data flow architecture
Teams of scientists studied data flow concepts, static data flow machine architecture, and the VAL language. Each team mapped its application onto the machine and coded it in VAL. The principal findings of the study were: (1) Five of the seven applications used the full power of the target machine. The galactic simulation and multigrid fluid flow teams found that a significantly smaller version of the machine (16 processing elements) would suffice. (2) A number of machine design parameters including processing element (PE) function unit numbers, array memory size and bandwidth, and routing network capability were found to be crucial for optimal machine performance. (3) The study participants readily acquired VAL programming skills. (4) Participants learned that application-based performance evaluation is a sound method of evaluating new computer architectures, even those that are not fully specified. During the course of the study, participants developed models for using computers to solve numerical problems and for evaluating new architectures. These models form the bases for future evaluation studies.
Adams, George↗
Iterative solution of large, sparse linear systems on a static data flow architecture - Performance studies
The applicability of static data flow architectures to the iterative solution of sparse linear systems of equations is investigated. An analytic performance model of a static data flow computation is developed. This model includes both spatial parallelism, concurrent execution in multiple PE's, and pipelining, the streaming of data from array memories through the PE's. The performance model is used to analyze a row partitioned iterative algorithm for solving sparse linear systems of algebraic equations. Based on this analysis, design parameters for the static data flow architecture as a function of matrix sparsity and dimension are proposed.
Reed, D. A.↗
A formal definition of data flow graph models
In this paper, a new model for parallel computations and parallel computer systems that is based on data flow principles is presented. Uninterpreted data flow graphs can be used to model computer systems including data driven and parallel processors. A data flow graph is defined to be a bipartite graph with actors and links as the two vertex classes. Actors can be considered similar to transitions in Petri nets, and links similar to places. The nondeterministic nature of uninterpreted data flow graphs necessitates the derivation of liveness conditions.
Kavi, Krishna M.↗
Blood flow imaging by optimal matching of computational fluid dynamics to 4D-flow data
Three-dimensional, time-resolved blood flow measurement (4D-flow) is a powerful research and clinical tool, but improved resolution and scan times are needed. Therefore, this study aims to (1) present a postprocessing framework for optimization-driven simulation-based flow imaging, called 4D-flow High-resolution Imaging with a priori Knowledge Incorporating the Navier-Stokes equations and the discontinuous Galerkin method (4D-flow HIKING), (2) investigate the framework in synthetic tests, (3) perform phantom validation using laser particle imaging velocimetry, and (4) demonstrate the use of the framework in vivo. An optimizing computational fluid dynamics solver including adjoint-based optimization was developed to fit computational fluid dynamics solutions to 4D-flow data. Synthetic tests were performed in 2D, and phantom validation was performed with pulsatile flow. Reference velocity data were acquired using particle imaging velocimetry, and 4D-flow data were acquired at 1.5 T. In vivo testing was performed on intracranial arteries in a healthy volunteer at 7 T, with 2D flow as the reference. Results Synthetic tests showed low error (0.4%-0.7%). Phantom validation showed improved agreement with laser particle imaging velocimetry compared with input 4D-flow in the horizontal (mean -0.05 vs -1.11 cm/s, P < .001; SD 1.86 vs 4.26 cm/s, P < .001) and vertical directions (mean 0.05 vs -0.04 cm/s, P = .29; SD 1.36 vs 3.95 cm/s, P < .001). In vivo data show a reduction in flow rate error from 14% to 3.5%. Phantom and in vivo results from 4D-flow HIKING show promise for future applications with higher resolution, shorter scan times, and accurate quantification of physiological parameters.
4D-flow MRI↗
A Large-Grain Mapping Approach for Multiprocessor Systems Through Data Flow Model Ph.D. Thesis
A large-grain level mapping method is presented of numerical oriented applications onto multiprocessor systems. The method is based on the large-grain data flow representation of the input application and it assumes a general interconnection topology of the multiprocessor system. The large-grain data flow model was used because such representation best exhibits inherited parallelism in many important applications, e.g., CFD models based on partial differential equations can be presented in large-grain data flow format, very effectively. A generalized interconnection topology of the multiprocessor architecture is considered, including such architectural issues as interprocessor communication cost, with the aim to identify the 'best matching' between the application and the multiprocessor structure. The objective is to minimize the total execution time of the input algorithm running on the target system. The mapping strategy consists of the following: (1) large-grain data flow graph generation from the input application using compilation techniques; (2) data flow graph partitioning into basic computation blocks; and (3) physical mapping onto the target multiprocessor using a priority allocation scheme for the computation blocks.
Kim, Hwa-Soo↗
Data flow modeling techniques
There have been a number of simulation packages developed for the purpose of designing, testing and validating computer systems, digital systems and software systems. Complex analytical tools based on Markov and semi-Markov processes have been designed to estimate the reliability and performance of simulated systems. Petri nets have received wide acceptance for modeling complex and highly parallel computers. In this research data flow models for computer systems are investigated. Data flow models can be used to simulate both software and hardware in a uniform manner. Data flow simulation techniques provide the computer systems designer with a CAD environment which enables highly parallel complex systems to be defined, evaluated at all levels and finally implemented in either hardware or software. Inherent in data flow concept is the hierarchical handling of complex systems. In this paper we will describe how data flow can be used to model computer system.
Kavi, K. M.↗
Adapting high-level language programs for parallel processing using data flow
EASY-FLOW, a very high-level data flow language, is introduced for the purpose of adapting programs written in a conventional high-level language to a parallel environment. The level of parallelism provided is of the large-grained variety in which parallel activities take place between subprograms or processes. A program written in EASY-FLOW is a set of subprogram calls as units, structured by iteration, branching, and distribution constructs. A data flow graph may be deduced from an EASY-FLOW program.
Standley, Hilda M.↗
Creating an Interprocedural Analyst-Oriented Data Flow Representation for Binary Analysts (CIAO)
National security missions require understanding third-party software binaries, a key element of which is reasoning about how data flows through a program. However, vulnerability analysts protecting software lack adequate tools for understanding data flow in binaries. To reduce the human time burden for these analysts, we used human factors methods in a rolling discovery process to derive user-centric visual representation requirements. We encountered three main challenges: analysis projects span weeks, analysis goals significantly affect approaches and required knowledge, and analyst tools, techniques, conventions, and prioritization are based on personal preference. To address these challenges, we initially focused our human factors methods on an attack surface characterization task. We generalized our results using a two-stage modified sorting task, creating requirements for a data flow visualization. We implemented these requirements partially in manual static visualizations, which we informally evaluated, and partially in automatically generated interactive visualizations, which have yet to be integrated into workflows for evaluation. Our observations and results indicate that 1) this data flow visualization has the potential to enable novel code navigation, information presentation, and information sharing, and 2) it is an excellent time to pursue research applying human factors methods to binary analysis workflows.
42 ENGINEERING↗
Sap flow data from San Lorenzo, Panama (PA-SLZ), 2019
This data package contains sap flow data from the NGEE Tropics site in San Lorenzo, Panama (PA-SLZ) from 22 January 2019 to 31 December 2019. Sap flow measurements quantify plant-level transpiration, which varies across spatial and temporal scales. It is a key trait in understanding plant hydraulic functioning. Here sap flow was measured using ‘Granier-type’ thermal dissipation probes. Raw data downloaded from each data logger are in the 'Raw data' folder as CSV files, along with the Campbell data logger programs which can be read when opened with a text editor. The data was processed using the AquaFlux package in R. R code for preparing the data for use in AquaFlux is in the 'R code for pre-processing data folder'; it is saved as a text file that can be copy and pasted into R. The processed data for each data logger is in the 'Processed data' folder as CSV files. Documentation for the Granier sensors and AquaFlux package is in the ‘Manuals’ folder as PDF files.
54 ENVIRONMENTAL SCIENCES↗
Detector architecture and data flow of the CMS HGCAL Phase-2 Endcap Calorimeter Upgrade
The CMS High Granularity Calorimeter (HGCAL) is the Phase-2 endcap calorimeter upgrade for the High-Luminosity Large Hadron Collider, designed to provide precise spatial, energy, and timing measurements in a high-radiation, high-pileup environment. This work presents an overview of the HGCAL assembly process, detector architecture, and end-to-end data flow. The data flow goes from signal generation in silicon sensors and scintillator tiles through front-end electronics, optical transmission, and back-end data acquisition. The study highlights how the integrated detector, electronics, and readout systems enable efficient event reconstruction and support the physics objectives of the HL-LHC.
Osei Boakye, Prince [Grambling State U.]↗
A Network Management System for Handling Scientific Data Flows
Large scientific data transfers often occur at high rates causing increased burstiness in Internet traffic. To limit the adverse effects of these high-rate large-sized flows, which are referred to as $\alpha $ flows, on delay-sensitive audio/video flows, a network management system called Alpha Flow Traffic Engineering System (AFTES) is proposed for intra-domain traffic engineering. An offline approach is used in which AFTES analyzes NetFlow records collected by routers, extracts source–destination address prefixes of $\alpha $ flows, and uses these prefixes to configure firewall filters at ingress routers of a provider’s network to redirect future $\alpha $ flows to traffic-engineered paths and isolated queues. The effectiveness of this scheme was evaluated through an analysis of 7 months of NetFlow data obtained from an ESnet router. For this data set, 91 % of bytes generated by $\alpha $ flows during high-rate intervals would have been directed had AFTES been deployed. The negative aspect of using address prefixes in firewall filters, i.e., the redirection of $\beta $ flows to $\alpha $-flow paths/queues, was also quantified.
97 MATHEMATICS AND COMPUTING↗
A mathematical model of asynchronous data flow in parallel computers *
Abstract We present a simplified model of data flow on processors in a high-performance computing framework involving computations necessitating inter-processor communications. From this ordinary differential model, we take its asymptotic limit, resulting in a model which treats the computer as a continuum of processors and data flow as an Eulerian fluid governed by a conservation law. We derive a Hamilton–Jacobi equation associated with this conservation law for which the existence and uniqueness of solutions can be proven. We then present the results of numerical experiments for both discrete and continuum models; these show a qualitative agreement between the two and the effect of variations in the computing environment’s processing capabilities on the progress of the modelled computation.
Barnard, Richard C.↗