Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Movement”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

pnnl/ZebrafishBMD

ZebrafishBMD: A software tool to perform benchmark dose calculations using toxicity screening data of high-throughput zebrafish assays. High-content imaging of larval zebrafish behavior can be used as a screening approach to rapidly evaluate the relative potential for chemicals to cause toxicity. However, most statistical methods applied to these data transform movement values to incidence-based “hits” and calculate lowest effect levels (LELs), which loses individual fish resolution of behavior and defies hazard ranking due to reliance on applied dose levels. We developed a parallelizable workflow to calculate benchmark dose (BMD) values from dynamic, high-content zebrafish behavior data that scales for high throughput chemical screening. To capture the zebrafish movement response from light to dark stimulus, we summarized time-dependent data using both area under the curve and the immediate change at the transition point into two novel metrics that characterized abnormal behavior as a function of chemical concentration. The BMD workflow was applied to calculate BMD10 values of 1,060 ToxCast chemicals for 24 zebrafish endpoints, including behavior, mortality and morphology. ZebrafishBMD implements the BMD workflow and provides Matlab scripts to read the data (stored in a specific structure) from a .mat file, to calculate the behavioral endpoint values from movement versus time data, to construct concentrationresponse curves, to perform curve-fitting, and to calculate BMD10 values.

Thomas, Dennis↗
DOE OSTI2015

Method and system for managing power grid data

A system and method of managing time-series data for smart grids is disclosed. Data is collected from a plurality of sensors. An index is modified for a newly created block. A one disk operation per read or write is performed. The one disk operation per read includes accessing and looking up the index to locate the data without movement of an arm of the disk, and obtaining the data. The one disk operation per write includes searching the disk for free space, calculating an offset, modifying the index, and writing the data contiguously into a block of the disk the index points to.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2015

Pooling the resources of the CMS Tier-1 sites

The CMS experiment at the LHC relies on 7 Tier-1 centres of the WLCG to perform the majority of its bulk processing activity, and to archive its data. During the first run of the LHC, these two functions were tightly coupled as each Tier-1 was constrained to process only the data archived on its hierarchical storage. This lack of flexibility in the assignment of processing workflows occasionally resulted in uneven resource utilisation and in an increased latency in the delivery of the results to the physics community.The long shutdown of the LHC in 2013-2014 was an opportunity to revisit this mode of operations, disentangling the processing and archive functionalities of the Tier-1 centres. The storage services at the Tier-1s were redeployed breaking the traditional hierarchical model: each site now provides a large disk storage to host input and output data for processing, and an independent tape storage used exclusively for archiving. Movement of data between the tape and disk endpoints is not automated, but triggered externally through the WLCG transfer management systems.With this new setup, CMS operations actively controls at any time which data is available on disk for processing and which data should be sent to archive. Thanks to the high-bandwidth connectivity guaranteed by the LHCOPN, input data can be freely transferred between disk endpoints as needed to take advantage of free CPU, turning the Tier-1s into a large pool of shared resources. The output data can be validated before archiving them permanently, and temporary data formats can be produced without wasting valuable tape resources. Lastly, the data hosted on disk at Tier-1s can now be made available also for user analysis since there is no risk any longer of triggering chaotic staging from tape.In this contribution, we describe the technical solutions adopted for the new disk and tape endpoints at the sites, and we report on the commissioning and scale testing of the service. We detail the procedures implemented by CMS computing operations to actively manage data on disk at Tier-1 sites, and we give examples of the benefits brought to CMS workflows by the additional flexibility of the new system.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS2006

Lunar Relay Satellite Network for Space Exploration: Architecture, Technologies and Challenges

NASA is planning a series of short and long duration human and robotic missions to explore the Moon and then Mars. A key objective of these missions is to grow, through a series of launches, a system of systems infrastructure with the capability for safe and sustainable autonomous operations at minimum cost while maximizing the exploration capabilities and science return. An incremental implementation process will enable a buildup of the communication, navigation, networking, computing, and informatics architectures to support human exploration missions in the vicinities and on the surfaces of the Moon and Mars. These architectures will support all space and surface nodes, including other orbiters, lander vehicles, humans in spacesuits, robots, rovers, human habitats, and pressurized vehicles. This paper describes the integration of an innovative MAC and networking technology with an equally innovative position-dependent, data routing, network technology. The MAC technology provides the relay spacecraft with the capability to autonomously discover neighbor spacecraft and surface nodes, establish variable-rate links and communicate simultaneously with multiple in-space and surface clients at varying and rapidly changing distances while making optimum use of the available power. The networking technology uses attitude sensors, a time synchronization protocol and occasional orbit-corrections to maintain awareness of its instantaneous position and attitude in space as well as the orbital or surface location of its communication clients. A position-dependent data routing capability is used in the communication relay satellites to handle the movement of data among any of multiple clients (including Earth) that may be simultaneously in view; and if not in view, the relay will temporarily store the data from a client source and download it when the destination client comes into view. The integration of the MAC and data routing networking technologies would enable a relay satellite system to provide end-to-end communication services for robotic and human missions in the vicinity, or on the surface of the Moon with a minimum of Earth-based operational support.

Bhasin, Kul B.↗
NASA NTRS2022

Non-Repudiation for Drone-Related Data

Concepts for the management of Uncrewed Aircraft Systems (UAS) at scale rely on the exchange of data amongst multiple stakeholders. Even as these concepts vary across nations and industries, the movement of data between entities is a common theme. While there is universal agreement on the necessity of appropriate cybersecurity measures to address data communication, there has been minimal focus on the feasibility of implementing non-repudiation solutions for UAS systems. This means that data exchanged in support of UAS operations are open to “attack” via parties that may deny sending or receiving certain data, which can weaken the effectiveness and acceptability of these systems. This paper highlights the current and future need for non-repudiation, supported by references to multiple international organizations, and an approach to implementing non-repudiation leveraging open standards.

drone↗
DOE OSTI2017

November 2016 West Hackberry Subsidence Report

Subsidence monitoring is a critical component to understanding the cavern integrity of salt storage caverns. This report looks at historical and recent data from two of the three West Hackberry dome cavern operators. DOE SPR and LA Storage are coordinating subsidence surveys to create a comprehensive understanding of ground movement above the dome. Data from annual level and rod surveys, GPS, and tiltmeter data show the sites are experiencing typical ground movement. The highest subsidence rate is seen in the middle of the DOE SPR site at just under one inch per year with less ground movement around the edge of the site. A GPS and tiltmeter instrument in the northeast areas of the DOE SPR site has not seen any trend change since the devices were installed in 2013. Comparison between recent ground movement data and historical trends suggest that there is no reason to believe that any DOE SPR or LA Storage caverns have been structurally compromised.

58 GEOSCIENCES↗
NASA NTRS2014

Understanding the Effects of Spaceflight on Head-trunk Coordination during Walking and Obstacle Avoidance

Prolonged exposure to spaceflight conditions results in a battery of physiological changes, some of which contribute to sensorimotor and neurovestibular deficits. Upon return to Earth, functional performance changes are tested using the Functional Task Test (FTT), which includes an obstacle course to observe post‐flight balance and postural stability, specifically during turning. The goal of this study was to quantify changes in movement strategies during turning events by observing the latency between head‐and‐trunk coordinated movements. It was hypothesized that subjects experiencing neurovestibular adaptations would exhibit head‐to‐trunk locking ('en bloc' movement) during turning, exhibited by a decrease in latency between head and trunk movement. FTT data samples were collected from ISS missions. Samples were analyzed three times pre‐exposure, immediately postexposure (1 day post) and 2‐to‐3 times during recovery from the microgravity environment. Two 3D inertial measurements units (XSens MTx) were attached to subjects, one on the head and one on the upper back. This study focused primarily on the yaw movements about the subject's center of rotation. Time differences (latency) between head and trunk movement were calculated at two points on the obstacle course: the first turn to enter the obstacle course (approximately 90⁰ turn) and averaged across a slalom obstacle portion, consisting of three turns (approximately three 90⁰ turns). Preliminary analysis of the data shows a trend toward decreasing head‐to‐trunk movement latency during postflight ambulation in slalom turning after reintroduction to Earth gravity in ISS astronauts. It is clear that changes in movement strategies are adopted during exposure to the microgravity environment and upon reintroduction to a gravity environment. Most ISS subjects exhibit symptoms of neurovestibular changes ('en bloc head and trunk movement) which may impact their ability to perform post‐flight functional tasks.

Madansingh, S.↗
NASA NTRS1996

Analysis of the access patterns at GSFC distributed active archive center

The Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC) has been operational for more than two years. Its mission is to support existing and pre Earth Observing System (EOS) Earth science datasets, facilitate the scientific research, and test Earth Observing System Data and Information System (EOSDIS) concepts. Over 550,000 files and documents have been archived, and more than six Terabytes have been distributed to the scientific community. Information about user request and file access patterns, and their impact on system loading, is needed to optimize current operations and to plan for future archives. To facilitate the management of daily activities, the GSFC DAAC has developed a data base system to track correspondence, requests, ingestion and distribution. In addition, several log files which record transactions on Unitree are maintained and periodically examined. This study identifies some of the users' requests and file access patterns at the GSFC DAAC during 1995. The analysis is limited to the subset of orders for which the data files are under the control of the Hierarchical Storage Management (HSM) Unitree. The results show that most of the data volume ordered was for two data products. The volume was also mostly made up of level 3 and 4 data and most of the volume was distributed on 8 mm and 4 mm tapes. In addition, most of the volume ordered was for deliveries in North America although there was a significant world-wide use. There was a wide range of request sizes in terms of volume and number of files ordered. On an average 78.6 files were ordered per request. Using the data managed by Unitree, several caching algorithms have been evaluated for both hit rate and the overhead ('cost') associated with the movement of data from near-line devices to disks. The algorithm called LRU/2 bin was found to be the best for this workload, but the STbin algorithm also worked well.

Johnson, Theodore↗
NASA NTRSDate not supplied

Non-Repudiation for Drone-Related Data

Concepts for the management of Uncrewed Aircraft Systems (UAS) at scale rely on the exchange of data amongst multiple stakeholders. Even as these concepts vary from State to State and company to company as of today, the movement of data between different entities is a common theme. While there is universal agreement on the necessity of appropriate cybersecurity applied to the various systems involved in communicating these data, there has been little focus on a feasible implementation of non-repudiation in these systems. This paper highlights the current and future need for non-repudiation, supported by references to multiple international organizations, and an approach to implementing non-repudiation leveraging open standards.

Unmanned Aviation Systems Traffic Management↗
DOE OSTI2017

A Unified Data-Driven Approach for Programming In Situ Analysis and Visualization: An Interim Report of Sandia Sub-Team Contributions

As we look ahead to next generation high performance computing platforms, the placement and movement of data is becoming the key-limiting factor on both performance and energy efficiency. Furthermore, the increased quantities of data the systems are capable of generating, in conjunction with the insufficient rate of improvements in the supporting I/0 infrastructure, is forcing applications away from the off-line post-processing of data towards techniques based on in ,situ analysis and visualization. Together, these challenges are shaping how we will both design and develop effective, performant and energy-efficient software. In particular, the challenges highlight the need for data and data-centric operations to be fundamental in the reasoning about, and optimization of, scientific workflows on extreme-scale architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Non-invasive measurement of proppant pack deformation

In this study, we describe a method to non-invasively study the movement of proppant packs at the sub-fracture scale by applying three-dimensional digital image correlation techniques to X-ray tomography data. Proppant movement is tracked in a fractured core of Marcellus shale placed under a series of increasing confining pressures up to 10,000 psi. The analysis reveals the sudden failure of a region of the proppant pack, accompanied by the large-scale rearrangement of grains across the entire fracture surface. The failure of the pack coincides with the appearance of vortex-like grain motions similar to features observed in biaxial compression of two dimensional granular assemblies.

02 PETROLEUM↗
NASA NTRS1986

Performance measurements of a data link protocol

The end-to-end performance of a program implementing IEEE Std 802.2 logical link control protocol, which provides virtual circuit and datagram service and which constitutes the top sublayer of the data link layer of the reference model for Local Area Networks is studied. The measurements of relative execution times of various functions required in transferring data, over virtual circuits or as datagrams, showed that most of the program execution time is spent in a few relatively low-level functions. One of these functions performs block movement of data in memory, and two other functions provide access to various queues. Based on this result, two alternative versions of the program which move data efficiently were written and measured. In this paper the results of the study are reported and their impact on designing hardware and software systems for a network interface are discussed.

Kanakia, Hemant↗
NASA NTRS1987

A system for routing arbitrary directed graphs on SIMD architectures

There are many problems which can be described in terms of directed graphs that contain a large number of vertices where simple computations occur using data from connecting vertices. A method is given for parallelizing such problems on an SIMD machine model that is bit-serial and uses only nearest neighbor connections for communication. Each vertex of the graph will be assigned to a processor in the machine. Algorithms are given that will be used to implement movement of data along the arcs of the graph. This architecture and algorithms define a system that is relatively simple to build and can do graph processing. All arcs can be transversed in parallel in time O(T), where T is empirically proportional to the diameter of the interconnection network times the average degree of the graph. Modifying or adding a new arc takes the same time as parallel traversal.

Tomboulian, Sherryl↗
NASA NTRS1988

Overview and extensions of a system for routing directed graphs on SIMD architectures

Many problems can be described in terms of directed graphs that contain a large number of vertices where simple computations occur using data from adjacent vertices. A method is given for parallelizing such problems on an SIMD machine model that uses only nearest neighbor connections for communication, and has no facility for local indirect addressing. Each vertex of the graph will be assigned to a processor in the machine. Rules for a labeling are introduced that support the use of a simple algorithm for movement of data along the edges of the graph. Additional algorithms are defined for addition and deletion of edges. Modifying or adding a new edge takes the same time as parallel traversal. This combination of architecture and algorithms defines a system that is relatively simple to build and can do fast graph processing. All edges can be traversed in parallel in time O(T), where T is empirically proportional to the average path length in the embedding times the average degree of the graph. Additionally, researchers present an extension to the above method which allows for enhanced performance by allowing some broadcasting capabilities.

Tomboulian, Sherryl↗
NASA NTRS2011

Implementing Strategic Planning Capabilities Within the Mars Relay Operations Service

Since the Mars Exploration Rovers (MER), Spirit and Opportunity, began their travels across the Martian surface in January of 2004, orbiting spacecraft such as the Mars 2001 Odyssey orbiter have relayed the majority of their collected scientific and operational data to and from Earth. From the beginning of those missions, it was evident that using orbiters to relay data to and from the surface of Mars was a vastly more efficient communications strategy in terms of power consumption and bandwidth compared to direct-to-Earth means. However, the coordination between the various spacecraft, which are largely managed independently and on differing commanding timelines, has always proven to be a challenge. Until recently, the ground operators of all these spacecraft have coordinated the movement of data through this network using a collection of ad hoc human interfaces and various, independent software tools. The Mars Relay Operations Service (MaROS) has been developed to manage the evolving needs of the Mars relay network, and specifically to standardize and integrate the relay planning and coordination data into a centralized infrastructure. This paper explores the journey of developing the MaROS system, from inception to delivery and acceptance by the Mars mission users.

tactical planning↗
NASA NTRS2014

Understanding the Effects of Spaceflight on Head-trunk Coordination During Walking and Obstacle Avoidance

Prolonged exposure to spaceflight conditions results in a battery of physiological changes, some of which contribute to sensorimotor and neurovestibular deficits. Upon return to Earth, functional performance changes are tested using the Functional Task Test (FTT), which includes an obstacle course to observe post-flight balance and postural stability, specifically during turning. The goal of this study was to quantify changes in movement strategies during turning events by observing the latency between head-and-trunk coordinated movements. It was hypothesized that subjects experiencing neurovestibular adaptations would exhibit head-to-trunk locking ('en bloc' movement) during turning, exhibited by a decrease in latency between head and trunk movement. FTT data samples were collected from 13 ISS astronauts and 26 male 70-day head down tilt bed rest subjects, including bed rest controls (10 BRC) and bed rest exercisers (16 BRE). Samples were analyzed three times pre-exposure, immediately post-exposure (0 or 1 day post) and 2-to-3 times during recovery from the unloading environment. Two 3D inertial measurements units (XSens MTx) were attached to subjects, one on the head and one on the upper back. This study focused primarily on the yaw movements about the subject's center of rotation. Time differences (latency) between head and trunk movement were averaged across a slalom obstacle portion, consisting of three turns (approximately three 60⁰ turns). All participants were grouped as 'decreaser' or 'increaser,' relating to their change in head-to-trunk movement latency between pre- and post- environmental adaptation measures. Space flight unloading (ISS) showed a bimodal response between the 'increaser' and 'decreaser' group, while both bed rest control (BRC) and bed rest exercise (BRE) populations showed increased preference towards a 'decreaser' categorization, displaying greater head-trunk locking. It is clear that changes in movement strategies are adopted during exposure to an unloading environment. These results further the understanding of vestibular-somatosensory convergence and support the use of bed rest as an exclusionary model to better understand sensorimotor changes in space flight.

Madansingh, S.↗
DOE OSTI2023

Characterizing the performance of node-aware strategies for irregular point-to-point communication on heterogeneous architectures

Supercomputer architectures are trending toward higher computational throughput due to the inclusion of heterogeneous compute nodes. These multi-GPU nodes increase on-node computational efficiency, while also increasing the amount of data to be communicated and the number of potential data flow paths. In this work, we characterize the performance of irregular point-to-point communication with MPI on heterogeneous compute environments through performance modeling, demonstrating the limitations of standard communication strategies for both device-aware and staging-through-host communication techniques. Presented models suggest staging communicated data through host processes then using node-aware communication strategies for high inter-node message counts. Notably, the models also predict that node-aware communication utilizing all available CPU cores to communicate inter-node data leads to the most performant strategy when communicating with a high number of nodes. Furthermore, model validation is provided via a case study of irregular point-to-point communication patterns in distributed sparse matrix–vector products. Importantly, we include a discussion on the implications model predictions have on communication strategy design for emerging supercomputer architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Machine learning-based analysis of COVID-19 pandemic impact on US research networks

Here in this study we explore how fallout from the changing public health policy around COVID-19 has changed how researchers access and process their science experiments. Using a combination of techniques from statistical analysis and machine learning, we conduct a retrospective analysis of historical network data for a period around the stay-at-home orders that took place in March 2020. Our analysis takes data from the entire ESnet infrastructure to explore DOE high-performance computing (HPC) resources at OLCF, ALCF, and NERSC, as well as User sites such as PNNL and JLAB. We look at detecting and quantifying changes in site activity using a combination of t-Distributed Stochastic Neighbor Embedding (t-SNE) and decision tree analysis. Our findings bring insights into the working patterns and impact on data volume movements, particularly during late-night hours and weekends.

97 MATHEMATICS AND COMPUTING↗