Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data storage data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

On the Use of GPUs in Realizing Cost-Effective Distributed RAID

The exponential growth in user and application data entails new means for providing fault tolerance and protection against data loss. High Performance Computing (HPC) storage systems, which are at the forefront of handling the data deluge, typically employ hardware RAID at the backend. However, such solutions are costly, do not ensure end-to-end data integrity, and can become a bottleneck during data reconstruction. In this paper, we design an innovative solution to achieve a flexible, fault-tolerant, and high-performance RAID-6 solution for a parallel file system (PFS). Our system utilizes low-cost, strategically placed GPUs - both on the client and server sides - to accelerate parity computation. In contrast to hardware-based approaches, we provide full control over the size, length and location of a RAID array on a per file basis, end-to-end data integrity checking, and parallelization of RAID array reconstruction. We have deployed our system in conjunction with the widely-used Lustre PFS, and show that our approach is feasible and imposes acceptable overhead.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

New architectural paradigms for multi-petabyte distributed storage systems

In the not too distant future, programs such as NASA's Earth Observing System, NSF/ARPA/NASA's Digital Libraries Initiative and Intelligence Community's (NSA, CIA, NRO, etc.) mass storage system upgrades will all require multi-petabyte (petabyte: 1015 bytes of bitfile data) (or larger) distributed storage solutions. None of these requirements, as currently defined, will meet their objectives utilizing either today's architectural paradigms or storage solutions. Radically new approaches will be required to not only store and manage veritable 'mountain ranges of data', but to make the cost of ownership affordable, much less practical in today's (and certainly the future's) austere budget environment! Within this paper we will explore new architectural paradigms and project systems performance benefits and dollars per petabyte of information stored. We will discuss essential 'top down' approaches to achieving an overall systems level performance capability sufficient to meet the challenges of these major programs.

Lee, Richard R.↗

MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or collective communication to fetch remote data, with optimal performance depending on (i) dataset characteristics, (ii) training scale, and (iii) interconnection network. Empirical analysis shows collective communication excels with larger mini-batch sizes and/or fewer processes, whereas one-sided communication outperforms at larger scales. We propose MDLoader, a hybrid in-memory data loader for distributed graph neural network training. MDLoader features a model-driven performance estimator that dynamically selects between one-sided and collective communication at the beginning of training using Tree of Parzen Estimators (TPE). Evaluations on NERSC Perlmutter and OLCF Summit show MDLoader outperforms single-backend loaders by up to 2.83 × and predicts the suitable communication method with 96.3% (Perlmutter) and 94.3% (Summit) success rate.

Bae, Jonghyun↗

Py MILab: Capturing, Analyzing and Storing Test Data

Integrated Computational Materials Engineering (ICME) has recently received widespread attention due to its promises in reducing dependence on physical testing for engineering design by relying on simulation, reducing both time and cost to market for various applications. ICME however requires validated multiscale material models, which is heavily dependent on available test data with full material and test pedigree, including material processing, test and measurement equipment, raw data collection, and analysis methodology and results. Populating searchable information management systems with such rich data sets is often burdensome for data producers, resulting in a lack of findable data for modelers to validate and verify their models. To overcome these cultural barriers to ICME, NASA has developed of various database-integration toolsets that perform both data management activities within the organization’s best practices with additional functionality that relieves the effort of the data producer and promotes adoption of information management system. One such tool currently under development is Py MILab, an automatic framework for automatic capturing, analysis, maintenance, and storage of material test data. Py MILab uses a modular approach for capturing raw data, analyzing the data, and storing the data in a database, interfaced by neutral file structures, to promote plug-and-play capabilities for various analysis types. TMAnalysis is a Python-based tool that performs automatic data reduction and analysis of uniaxial thermomechanical test data. The TMAnalysis toolset can be implemented within the Analysis module of Py MILab, and thus requires a populated neutral file form the Raw Data Module of Py MILab and outputs a Analysis neutral file compatible with the Database Module of Py MILab. TMAnalysis is able to perform automatic segmentation of multistage tests and perform data analysis and reduction, including determination of point-wise properties in tension, compression, and shear, analysis of stress relaxation tests, creep analysis and zone identification, and combination of these stage types for tests with complex loading histories. The TMAnalysis code is accompanied with a graphical user interface (GUI) that allows users to easily analyze test data in bulk, verify the automatic, consistent analysis performed by the backend code, and edit stage segmentation if necessary before producing the output neutral files, ensuring data is properly analyzed and maintained with full traceability.

Data management↗

Medical-Information-Management System

Medical Information Management System (MIMS) computer program interactive, general-purpose software system for storage and retrieval of information. Offers immediate assistance where manipulation of large data bases required. User quickly and efficiently extracts, displays, and analyzes data. Used in management of medical data and handling all aspects of data related to care of patients. Other applications include management of data on occupational safety in public and private sectors, handling judicial information, systemizing purchasing and procurement systems, and analyses of cost structures of organizations. Written in Microsoft FORTRAN 77.

Alterescu, Sidney↗

Spinoff 2011

Topics include: Bioreactors Drive Advances in Tissue Engineering; Tooling Techniques Enhance Medical Imaging; Ventilator Technologies Sustain Critically Injured Patients; Protein Innovations Advance Drug Treatments, Skin Care; Mass Analyzers Facilitate Research on Addiction; Frameworks Coordinate Scientific Data Management; Cameras Improve Navigation for Pilots, Drivers; Integrated Design Tools Reduce Risk, Cost; Advisory Systems Save Time, Fuel for Airlines; Modeling Programs Increase Aircraft Design Safety; Fly-by-Wire Systems Enable Safer, More Efficient Flight; Modified Fittings Enhance Industrial Safety; Simulation Tools Model Icing for Aircraft Design; Information Systems Coordinate Emergency Management; Imaging Systems Provide Maps for U.S. Soldiers; High-Pressure Systems Suppress Fires in Seconds; Alloy-Enhanced Fans Maintain Fresh Air in Tunnels; Control Algorithms Charge Batteries Faster; Software Programs Derive Measurements from Photographs; Retrofits Convert Gas Vehicles into Hybrids; NASA Missions Inspire Online Video Games; Monitors Track Vital Signs for Fitness and Safety; Thermal Components Boost Performance of HVAC Systems; World Wind Tools Reveal Environmental Change; Analyzers Measure Greenhouse Gasses, Airborne Pollutants; Remediation Technologies Eliminate Contaminants; Receivers Gather Data for Climate, Weather Prediction; Coating Processes Boost Performance of Solar Cells; Analyzers Provide Water Security in Space and on Earth; Catalyst Substrates Remove Contaminants, Produce Fuel; Rocket Engine Innovations Advance Clean Energy; Technologies Render Views of Earth for Virtual Navigation; Content Platforms Meet Data Storage, Retrieval Needs; Tools Ensure Reliability of Critical Software; Electronic Handbooks Simplify Process Management; Software Innovations Speed Scientific Computing; Controller Chips Preserve Microprocessor Function; Nanotube Production Devices Expand Research Capabilities; Custom Machines Advance Composite Manufacturing; Polyimide Foams Offer Superior Insulation; Beam Steering Devices Reduce Payload Weight; Models Support Energy-Saving Microwave Technologies; Materials Advance Chemical Propulsion Technology; and High-Temperature Coatings Offer Energy Savings.

Source record↗

Automation of Data Collection Methods for Online Monitoring of Nuclear Power Plants

Maintenance activities in the United States nuclear power industry are mainly centered on preventive and corrective maintenance. Predictive maintenance, which is a variable time basis maintenance strategy that is dependent on the condition of the equipment, is less utilized but potentially more effective. In the current maintenance practices, the nuclear power plants rely on manual processes to supply information to a human decision making process. The plants have several processes that generate information that is not typically used beyond the intended target for collecting that information. This information is a source of data that is manually collected by labor intensive tasks. The data is therefore expensive to collect, yet limited in use. The industry has realized the opportunity present in reducing the labor intensive tasks by automating them and increasing the uses of the data collected by either developing advanced predictive maintenance methods using data-driven decision making that could span from trending to machine learning and artificial intelligence. This shift in maintenance strategy may result in high cost saving impact. In addition to reducing labor requirements and frequency of maintenance activities, this shift in strategy will reduce the materials portion of the operations and maintenance cost as well. The aim of this specific effort is to focus on the automation of data collection process. It is part of a series of planned efforts to target multiple elements of migrating the current preventive maintenance approach in nuclear power plants to a data-driven approach. These elements include data collection, management, analysis, visualization, in addition to value analysis and change enabling. The other five elements of migration were not considered because they are planned to be explicitly studied in multiple parallel efforts. Challenges including regulations requirements or limitations are planned in a future effort that focuses on change enabling. The risk and benefit/cost of the approach shift is planned in another effort targeted towards value. The storage, communication, computational power, data architecture, and protection are part of the information infrastructure and are planned in dedicated data management effort. The best means to present the data for assisted human decision making is planned in a data visualization effort. The data interpretation and patterns recognition for machine decision making is planned through multiple efforts in data analytics over the following years. This work identified seventeen data sources collection methods in nuclear power plants that could be automated for an integrated data platform that enables comprehensive and informed decision making. For each of the identified seventeen data sources, the base, modern, and state of the art states were described. The current state of data collection used by most utilities was identified as the "base state". The current achievable state was identified as the "modern state", a state that has either been achieved by at least some US utilities or in other industries. The future state that requires research and development to reach was identified as the "state of the art" state. The described states used an out-of-the-box thinking approach. This effort found that there is a hug potential for task automation and data fidelity improvements by evolving the current industry data collection methods. The majority of the processes analyzed were found to be manual and labor intensive and fell in the base state. As a result, the modern state often targeted reducing the manual logging efforts by using electronic and semi-intelligent means or complementing the work with sensors and mobile or fixed technologies that can reduce the demand of labor activities. The state of the art state involved applications advanced methods of data correlation, data mining, and machine learning along with increasing the spatial distribution and fidelity of data, coupling multiple data sources, and enabling the use of sophisticated mobile devices and machine comprehension methods.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Community Energy Operations and Planning System: Concept, Use cases, Metrics, and Benefits

Community and city leaders are interested in achieving sustainability goals, providing resilient energy infrastructure, and improving economic competitiveness. Community-level data acquisition and analysis can provide energy and associated benefits that are not possible at the single building level. However, there is a lack of organizational structure, common semantic data models, interoperable systems, and methods to support data-driven decision-making for community-scale energy supply and demand systems. We explored the need and opportunity for a Community Energy Operations and Planning System (Community EOPS), a potential data exchange platform. We conducted “customer discovery” interviews, and reviewed literature, public tools, and technology platforms to identify key energy data “users” and use cases in communities. The key users of the Community EOPS could be developers of mixed-use districts, corporate, defense and university campus energy managers, and city managers of cities that own their energy utility. The value could be for community planning and reporting (for energy data-integrated land use planning and community infrastructure investments in microgrids, storage, district heating and cooling), energy efficiency (leveraging optimizations for community scale energy supply and demand), flexible load management (grid-edge load management to offset, shift, and flatten loads for multiple buildings and EV fleets), cost savings and revenue generation (participating in grid services), and social benefits such as energy resilience, equity, and awareness. We developed a conceptual Community EOPS architecture with recommendations for streamlined and prioritized data acquisition, sharing, and integration driven by prioritized use cases, common metrics, and actionable visualizations that can provide value to a community’s users.

Singh, Reshma↗

Expert systems and imaging - NASA's start-up work in intelligent image management

The problem of image management for NASA's Space Shuttle program and the future role of expert systems are considered. The difficulties involved in the storage and retrieval of the estimated 100,000 color photographs used for verification of essential installation, test, and repair procedures for Space Shuttle prelaunch are discussed. It is noted that the collection, classification, storage, and retrieval of high-quality pictoral images is possible through the use of pixels and an expert systems-assisted relational data base management system. The development of a query by pictorial example process is discussed. The NASA Intelligent Interactive Visual Data Base Management Systems project is detailed. It aims to conduct a systems analysis of a large-scale traditional image processing operation, collect data on critical human performance factors in the areas of image creation, processing, classification, storage, and retreival, and validate the attributes identified through the creation and testing of prototype models.

Ragusa, James M.↗

Leveraging public AI tools to explore systems biology resources in mathematical modeling

Predictive mathematical modeling is an essential part of systems biology and is interconnected with information management. Systems biology information is often stored in specialized formats to facilitate data storage and analysis. These formats are not designed for easy human readability and thus require specialized software to visualize and interpret results. Therefore, comprehending modeling and underlying networks and pathways is contingent on mastering systems biology tools, which is particularly challenging for users with no or little background in data science or system biology. To address this challenge, we investigated the usage of public Artificial Intelligence (AI) tools in exploring systems biology resources in mathematical modeling. We tested public AI’s understanding of mathematics in models, related systems biology data, and the complexity of model structures. Our approach can enhance the accessibility of systems biology for non-system biologists and help them understand systems biology without a deep learning curve.

59 BASIC BIOLOGICAL SCIENCES↗

Migration to WebDAV in Belle II Experiment

The usage of WebDAV protocol has become more and more popular within the physics experiments using grid middleware in the last decade, and today it represents a valid alternative to the GridFTP currently supported at best-effort level after the retirement of Globus Toolkit. Belle II experiment established the adoption of WebDAV protocol as the main protocol for data access and third-party-copy transfers, without relying on Storage Resource Manager interface (SRM). The migration process, carried on with continuous and gradual steps, has required a large effort to guarantee a smooth transition maintaining the production infrastructure fully operational. In this contribution we show the transition process, the tool of support developed to monitor step by step the status of third-party-copy support with WebDAV protocol by storages of the collaboration tested in both case pull and push, the strategy adopted to configure DIRAC and the solutions put in place for the corner cases. Finally, we will present some statistics of utilization and we will analyse the achieved results.

97 MATHEMATICS AND COMPUTING↗

Leveraging Computational Storage Devices in Campaign Storage [Slides]

Computational storage provides new ways of accelerating data-intensive applications. In-drive data management schemes matter (O_DIRECT, clustered index). Layer violation: “cheating” one filesystem may be possible; cheating multiple layers of filesystems is hard (FS internal load balancing, fail over, compression, concurrency control). The future directions include block-based acceleration to object-based acceleration.

97 MATHEMATICS AND COMPUTING↗

AAFE man-made noise experiment project. Volume 3: Appendices

Management and operational considerations involved in the project to measure man-made electromagnetic noise at earth orbital altitudes are discussed. The subjects considered are: (1) launch and orbit of the Scout D vehicles, (2) experiment management, (3) receiver scanning considerations, (4) data handling, and (5) threshold measurements. The storage requirements for a high resolution, complete data storage library are defined. Mathematical models of signal detection probability are developed.

Source record↗

SPAR: Structural-performance analysis and redesign

System of processor programs performs stress, buckling, and vibrational analysis of large linear finite element systems in excess of 50,000 degrees of freedom, while minimizing processing cost, execution time, central memory storage, and secondary data storage requirements. Programs use sparse matrix solution techniques and other computational and data management procedures.

Whetstone, W. D.↗

A Robust, Low-Cost Virtual Archive for Science Data

Despite their expense tape silos are still often the only affordable option for petabytescale science data archives, particularly when other factors such as data reliability, floor space, power and cooling load are accounted for. However, the complexity, management software, hardware reliability and access latency of tape silos make online data storage ever more attractive. Drastic reductions in low-cost mass-market PC disk drivers help to make this more affordable (approx. 1$/GB), but are challenging to scale to the petabyte range and of questionable reliability for archival use, On the other hand, if much of the science archive could be "virtualized", i.e., produced on demand when requested by users, we would need store only a fraction of the data online, perhaps bringing an online-only system into in affordable range. Radiance data from the satellite-borne Moderate Resolution Imaging Spectroradiometer (MODIS) instrument provides a good opportunity for such a virtual archive: the raw data amount to 140 GB/day, but these are small relative to the 550 GB/day making up the radiance products. These data are routinely processed as inputs for geophysical parameter products and then archived on tape at the Goddard Earth Sciences Distributed Active Archive (GES DAAC) for distributing to users. Virtualizing them would be an immediate and signifcant reduction in the amount of data being stored in the tape archives and provide more customizable products. A prototype of such a virtual archive is being developed to prove the concept and develop ways of incorporating the robustness that a science data archive requires.

Lynnes, Christopher↗

An information management system for a spent nuclear fuel interim storage facility.

We describe an integrated information management system for an independent spent fuel dry-storage installation (ISFSI) that can provide for (1) secure and authenticated data collection, (2) data analysis, (3) dissemination of information to appropriate stakeholders via a secure network, and (4) increased public confidence and support of the facility licensing and operation through increased transparency. This information management system is part of a collaborative project between Sandia National Laboratories, Taiwan Power Co., and the Fuel Cycle Materials Administration of Taiwan's Atomic Energy Council, which is investigating how to implement this concept.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Immutable Secure Data Exchange and Storage for Urban Air Mobility Environments

Urban air mobility (UAM) is a concept that proposes to develop short-range aerial vehicles to overcome increasing surface congestion. Within the UAM environment, UAM operators work collaboratively to manage aerial vehicles in the urban environment. Providers of Services for UAM (PSU), UAM operators, and Supplemental Data Service Providers (SDSP) provide services to support flight operations within the UAM environment. The growth in the development of UAM systems, and the associated data exchange and service interactions will be at risk due to numerous types of cybersecurity attacks. To address these challenges, this research focuses on the secure data exchange and storage of this decentralized UAM environment. The intent of this research is to leverage a permissioned blockchain approach to address cybersecurity threats that may impact a UAM environment.

Urban Air Mobility↗

Immutable Secure Data Exchange and Storage for Urban Air Mobility Environments

Urban air mobility (UAM) is a concept that proposes to develop short-range aerial vehicles to overcome increasing surface congestion. Within the UAM environment, UAM operators work collaboratively to manage aerial vehicles in the urban environment. Providers of Services for UAM (PSU), UAM operators, and Supplemental Data Service Providers (SDSP) provide services to support flight operations within the UAM environment. The growth in the development of UAM systems, and the associated data exchange and service interactions will be at risk due to numerous types of cybersecurity attacks. To address these challenges, this research focuses on the secure data exchange and storage of this decentralized UAM environment. The intent of this research is to leverage a permissioned blockchain approach to address cybersecurity threats that may impact a UAM environment.

Urban Air Mobility↗