Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

Structure Slicing: Extending Logical Regions with Fields, In: SC '14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Applications on modern supercomputers are increasingly limited by the cost of data movement, but mainstream programming systems have few abstractions for describing the structure of a program's data. Consequently, the burden of managing data movement, placement, and layout currently falls primarily upon the programmer. To address this problem we previously proposed a data model based on logical regions and described Legion, a programming system incorporating logical regions. In this paper, we present structure slicing, which incorporates fields into the logical region data model. We show that structure slicing enables Legion to automatically infer task parallelism from field non-interference, decouple the specification of data usage from layout, and reduce the overall amount of data moved. We demonstrate that structure slicing enables both strong and weak scaling of three Legion applications including S3D, a production combustion simulation that uses logical regions with thousands of fields, with speedups of up to 3.68X over a vectorized CPU-only Fortran implementation and 1.88X over an independently hand-tuned OpenACC code.

Computer Science↗

slaclab/dmConda

This repo contains scripts and information for building packages, creating and managing conda environments for PCDS Data Management.

Shankar, Murali↗

Report on HEP/ASCR Data Summit

Representatives from the HEP and ASCR communities met at Germantown on April 2-3, 2013, to discuss issues in carrying out science with large datasets and associated data-intensive computing tasks. The authors of this report acknowledge the important contributions made by all of the ASCR and HEP participants at the data summit and thank them for their efforts. During the meeting, HEP scientists made a series of presentations highlighting how they as a community interact with data, what tools they have developed in doing so, and the challenges to be faced moving forward. From the presentations and discussions, it became clear that the emerging data-intensive challenges span all HEP frontiers. Furthermore, since HEP is building fewer machines and experiments, there is increased emphasis in further optimizing the extraction of science results from the data by staying informed about the latest technologies and taking advantage of those that are particularly relevant. The joint summit proved to be very useful, as it not only familiarized the ASCR team with HEP science, but it also made apparent to the HEP group that there are significant common issues across the three HEP research frontiers that, in the past, have traditionally been addressed separately within each domain. A number of potential areas of common interest across the ASCR and HEP communities were identified. They cover the full range from pure R&D to focused assistance in the development and optimization of specific applications. It was generally agreed that HEP data management tools and techniques are already state of the art, and meet current needs. The community has extensive experience with management and analysis of large data sets. Properly planned joint HEP-ASCR research could incorporate transformational advances in developing tools for future applications. Both communities expressed the need to have viable models for maintenance of such tools and software, once developed.

97 MATHEMATICS AND COMPUTING↗

INTEGRATED WORKFLOW MANAGEMENT FOR PARTICLE ACCELERATOR SIMULATION

Supercomputing systems are used for a wide range of computationally demanding tasks in many fields of science and engineering. They play a key role in numerical simulation, in which mathematical models are computed in order to simulate the behavior of physical systems. Scientists and engineers that use supercomputers for numerical simulation often have their productivity limited by the need to manually organize and manage extremely large amounts of data that are often produced and consumed by the software programs run on these systems. Recognizing these limitations, Kitware Inc. (Clifton Park, NY) and SLAC National Accelerator Laboratory (Menlo Park, CA) are developing an advanced software platform that can reduce the cognitive overhead required by knowledge workers when using supercomputers for numerical simulation. Phase I of the project is complete and includes the development of new capabilities for organizing simulation project files, improvements to the user interface and overall usability, and deployment of a “middle tier” server to sit between user desktop machines and supercomputers to offload much of the data management workload. The project also developed prototype software for executing sequences of numerical simulations, and a prototype for migrating supercomputing software to cloud-based computing systems to provide a potential alternative to supercomputers with different logistical and price-to-performance tradeoffs.

Tourtellott, John↗

Status of FUDGE [Slides]

This presentation discusses the nuclear data management code FUDGE (For Updating Data and Generating Evaluations). While FUDGE is designed to support GNDS, the presentation also states that ENDF-6 and ENDL data files are also supported but must first be translated into GNDS; translators are included with FUDGE. It also states that FUDGE supports plotting, manipulating, checking for physical content, resonance reconstruction, Doppler broadening, etc. FUDGE also supports processing for Monte Carlo and deterministic transport. FUDGE is open source, and the latest public release was FUDGE-4.2.3 with support for GNDS-1.9. In summation, FUDGE capabilities include translating older formats into GNDS, translating GNDS back to ENDF-6, and visualizing, modifying, checking, and processing GNDS data. It also states that a new version of FUDGE is coming soon. The plan is to support GNDS-2.0 specification, but it may be released sooner if v2.0 is delayed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Developing a Cyclotron Radiation Emission Spectroscopy Detection System

Cyclotron Radiation Emission Spectroscopy is a new technique for ultra-precise spectroscopy of low-energy electrons. This project aimed at developing capabilities that would be useful to a future CRES experiment. The recent results from the Project 8 and He6-CRES collaborations indicate that CRES has a promising future, but will need advancements to continue developing. In this report we cover the four tasks that comprised the project: data acquisition, data management, updating the raw-data format, a cloud computing.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

New directions in space electronics

Five new directions in space electronics are defined to yield improvements over present means: automated operations aimed at a tenfold reduction in mission support costs, precision pointing and control, efficient data acquisition to permit the tenfold increase in information collection needed for global coverage, real-time data management, and low-cost data distribution to allow a thousand-fold increase in information availability and space-systems effectiveness. Technological developments and resources for each of the new directions are derived from space-technology and space-theme workshops. One to two order of magnitude improvements in instrument pointing stability will be necessary for Shuttle and planetary payloads of the 1980s. Low-cost user terminals will involve technological addressing both near-term and long-term applications. NASA has reoriented its research and technology-base programs to support the goals represented by the new directions in space electronics. Numerous charts and schematic diagrams supplement the text.

Kurzhals, P. R.↗

Magnetic Field Satellite (Magsat) data processing system specifications

The software specifications for the MAGSAT data processing system (MDPS) are presented. The MDPS is divided functionally into preprocessing of primary input data, data management, chronicle processing, and postprocessing. Data organization and validity, and checks of spacecraft and instrumentation are dicussed. Output products of the MDPS, including various plots and data tapes, are described. Formats for important tapes are presented. Dicussions and mathematical formulations for coordinate transformations and field model coefficients are included.

Berman, D.↗

Space Station Technology, 1983

This publication is a compilation of the panel summaries presented in the following areas: systems/operations technology; crew and life support; EVA; crew and life support: ECLSS; attitude, control, and stabilization; human capabilities; auxillary propulsion; fluid management; communications; structures and mechanisms; data management; power; and thermal control. The objective of the workshop was to aid the Space Station Technology Steering Committee in defining and implementing a technology development program to support the establishment of a permanent human presence in space. This compilation will provide the participants and their organizations with the information presented at this workshop in a referenceable format. This information will establish a stepping stone for users of space station technology to develop new technology and plan future tasks.

Wright, R. L.↗

Problems in merging Earth sensing satellite data sets

Satellite remote sensing systems provide a tremendous source of data flow to the Earth science community. These systems provide scientists with data of types and on a scale previously unattainable. Looking forward to the capabilities of Space Station and the Earth Observing System (EOS), the full realization of the potential of satellite remote sensing will be handicapped by inadequate information systems. There is a growing emphasis in Earth science research to ask questions which are multidisciplinary in nature and global in scale. Many of these research projects emphasize the interactions of the land surface, the atmosphere, and the oceans through various physical mechanisms. Conducting this research requires large and complex data sets and teams of multidisciplinary scientists, often working at remote locations. A review of the problems of merging these large volumes of data into spatially referenced and manageable data sets is presented.

Smith, Paul H.↗

A reduced version of the NMC DERF 2 data set

The National Meterological Center (NMC) Dynamical Extended Range Forecast (DERF 2) data represents a major computational effort to better ascertain the potential for extended range forecasts and to develop a strategy for performing operational extended range forecasts using dynamical models. A major stumbling block for using this data has been the sheer volume of data that must be processed to perform even simple calculations. The product of the data reduction described is a manageable data set that fits comfortably on five magnetic tapes or on one compact disc. The document outlines the data reduction process of the second phase of DERF data. It contains the description of the fields and the resolution of both the original and final fields. In order to assist the users of this data set, maps of selected fields, using both the original truncation at rhomboidal 30 and the truncation of the final data at triangular 20, are displayed.

Schubert, Siegfried↗

Data Base On Cables And Connectors

Report describes Connector Adapter Cable Information Data Base (CONNAID) computer program, managing data base containing necessary information concerning electrical connectors, breakout boxes, adapter cables, backshells, and pertinent torque specifications for engineering project.

Bowen, Arlen R.↗

Life Sciences Data Archive (LSDA)

In the early days of spaceflight, space life sciences data were been collected and stored in numerous databases, formats, media-types and geographical locations. While serving the needs of individual research teams, these data were largely unknown/unavailable to the scientific community at large. As a result, the Space Act of 1958 and the Science Data Management Policy mandated that research data collected by the National Aeronautics and Space Administration be made available to the science community at large. The Biomedical Informatics and Health Care Systems Branch of the Space Life Sciences Directorate at JSC and the Data Archive Project at ARC, with funding from the Human Research Program through the Exploration Medical Capability Element, are fulfilling these requirements through the systematic population of the Life Sciences Data Archive. This program constitutes a formal system for the acquisition, archival and distribution of data for Life Sciences-sponsored experiments and investigations. The general goal of the archive is to acquire, preserve, and distribute these data using a variety of media which are accessible and responsive to inquiries from the science communities.

Fitts, M.↗

A Large-Scale Design Integration Approach Developed in Conjunction with the Ares Launch Vehicle Program

This paper presents a method for performing large-scale design integration, taking a classical 2D drawing envelope and interface approach and applying it to modern three dimensional computer aided design (3D CAD) systems. Today, the paradigm often used when performing design integration with 3D models involves a digital mockup of an overall vehicle, in the form of a massive, fully detailed, CAD assembly; therefore, adding unnecessary burden and overhead to design and product data management processes. While fully detailed data may yield a broad depth of design detail, pertinent integration features are often obscured under the excessive amounts of information, making them difficult to discern. In contrast, the envelope and interface method results in a reduction in both the amount and complexity of information necessary for design integration while yielding significant savings in time and effort when applied to today's complex design integration projects. This approach, combining classical and modern methods, proved advantageous during the complex design integration activities of the Ares I vehicle. Downstream processes, benefiting from this approach by reducing development and design cycle time, include: Creation of analysis models for the Aerodynamic discipline; Vehicle to ground interface development; Documentation development for the vehicle assembly.

Redmon, John W.↗

Tagging Components in the Test Complex Area

Why tag components differently? - Various methods are in place to tag different components, but not every component is tracked; - Variance in method can make it difficult to obtain component information and identify discrepancies; -Time and resources are spent on component localization and verification. Previous work of RA10 interns: - Researching more efficient asset tracking methods; - Investigating cost effective ways to localize components; - Past interns identified using QR (Quick Response) codes as a potential solution. Link QR codes to Windchill (PLM - Product Lifecycle Management system) DDMS (Design and Data Management System) after QR code is printed/etched and physically tagging it as quickly as possible

Gomez, Sarah↗

Design and Construction of a NASA Airborne and Field Investigation Inventory

NASA conducts airborne and field investigations that produce a wealth of valuable research data. Unfortunately, this data is often scattered across individual scientist hard drives or NASA Distributed Active Archive Centers and it can be difficult to locate and retrieve. Although satellite data has been successfully consolidated by tools such as EarthData Search, airborne and field investigation data present unique challenges stemming from the variability of temporal, spatial, platform, and instrument metadata. To address these difficulties with data retrieval and metadata variability, the Interagency Implementation and Concepts Team established an Airborne Data Management Group to improve airborne data search, understanding, access, and use. Surveys have been conducted of end users in order to build query lists that will drive the augmentation and standardization of existing metadata. Detailed metadata was then laboriously compiled from present and historic airborne and field investigations to build a database that will enable intelligent data search and retrieval. The inventory structure and function will be described and demonstrated. The purpose of this presentation is to bring awareness to this effort, to highlight and describe the issues and complications in development, and to increase user interest prior to public release in 2020.

Davis, Carson↗

Closing the Gap between FAIR Data Repositories and Hierarchical Data Formats

Many in the scientific community, particularly in publicly funded research, are pushing to adhere to more accessible data standards to maximize the findability, accessibility, interoperability, and reusability (FAIR) of scientific data, especially with the growing prevalence of machine learning augmented research. Online FAIR data repositories, such as the Open Science Framework (OSF), help facilitate the adoption of these standards by providing frameworks for storage, access, search, APIs, and other features that create organized hubs of scientific data. However, the wider acceptance of such repositories is hindered by the lack of support of hierarchical data formats, such as Technical Data Management Streaming (TDMS) and Hierarchical Data Format 5 (HDF5), that many researchers rely on to organize their datasets. Various tools and strategies should be used to allow hierarchical data formats, FAIR data repositories, and scientific organizations to work more seamlessly together. A pilot project at Los Alamos National Laboratory (LANL) addresses the disconnect between them by integrating the OSF FAIR data repository with hierarchical data renderers, extending support for additional file types in their framework. The multifaceted interactive renderer displays a tree of metadata alongside a table and plot of the data channels in the file. This allows users to quickly and efficiently load large and complex data files directly in the OSF webapp. Users who are browsing files can quickly and intuitively see the files in the way they or their colleagues structured the hierarchical form and immediately grasp their contents. This solution helps bridge the gap between hierarchical data storage techniques and FAIR data repositories, making both of them more viable options for scientific institutions like LANL which have been put off by the lack of integration between them.

97 MATHEMATICS AND COMPUTING↗

Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations

Cloud computing has the potential to bring high performance computing capabilities to the average science researcher. However, in order to take full advantage of cloud capabilities, the science data used in the analysis must often be reorganized. This typically involves sharding the data across multiple nodes to enable relatively fine-grained parallelism. This can be either via cloud-based file systems or cloud-enabled databases such as Cassandra, Rasdaman or SciDB. Since storing an extra copy of data leads to increased cost and data management complexity, NASA is interested in determining the benefits and costs of various cloud analytics methods for real Earth Observation cases. Accordingly, NASA's Earth Science Technology Office and Earth Science Data and Information Systems project have teamed with cloud analytics practitioners to run a benchmark comparison on cloud analytics methods using the same input data and analysis algorithms. We have particularly looked at analysis algorithms that work over long time series, because these are particularly intractable for many Earth Observation datasets which typically store data with one or just a few time steps per file. This post will present side-by-side cost and performance results for several common Earth observation analysis operations.

science data management↗