Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data sharing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71

Identifying location of data granules in global virtual address space

An approach is disclosed that identifies a home node of a data granule. The process is performed by an information handling system (a local node) that retrieves a global virtual address directory. The global virtual address directory maps shared virtual addresses to a number nodes that includes the local node with one of the nodes being the home node. The shared virtual addresses correspond to a plurality of memory addresses that are stored in a shared virtual memory that is shared amongst the plurality of nodes. The approach receives a selected shared virtual address, retrieves, from the global virtual address directory, the home node associated with the selected shared virtual address, and accesses the data granule corresponding to the selected shared virtual address from the home node.

Johns, Charles R.↗

Beyond Fair: Engagement, Data Usability, and Open Community Productivity through the NASA Open Science Data Repository

The FAIR principle (findable, accessible, interoperable, and reusable) governs the storage and sharing of NASA space biology and health data[1]. These guiding principles maximize reuse of data and the reproducibility of scientific findings. The NASA Open Science Data Repository (OSDR; an expansion of NASA GeneLab) was built on the FAIR principles and houses over 500 studies and close to 1000 datasets from decades of space life sciences experiments. OSDR embodies the FAIR principles through data governance that includes mediated, embargoed, and fully open access data. The FAIR data governance principles were recently proposed to be expanded to encompass a FAIREST framework for assessing research data repositories (FAIR + Engagement, Social connections, and Trust)[2]. FAIREST emphasizes the importance of data repositories engaging with the scientific community and gaining the trust of researchers regarding data quality. Trust also refers to the TRUST principles developed for assessment of digital repositories: Transparency, Responsibility, User Focus, Sustainability, Technology[3]. We present the “Open Science for Life in Space” Analysis Working Groups (AWGs) as evidence regarding the power of engagement, social connections, and trust which has enhanced OSDR’s capabilities and productivity. AWG members engage in two main activities. One, members provide feedback on OSDR scientific standards for data ingestion, curation, and reuse (study, subject and assay metadata; processing pipelines; dataset formats and uniformed structures for machine-readability). Two, AWG members collaborate to mine-reuse OSDR data to conduct scientific analysis. With nearly 800 active members, the AWGs have resulted in 32 publications re-using OSDR data and contributed many papers in two major special issues in Cell (2020) and Nature (2024). AWGs also serve as networking groups, facilitate social connections between researchers at all levels of experience, and also have a social online ‘Forum’ used to keep members informed on projects and opportunities. This community-centric, productive, and trustworthy data culture has resulted in a broader effect with international space agencies, academics, and the commercial space sector wanting to submit their data to OSDR. Ten studies of Inspiration 4 data were recently publicly released by OSDR, as were some JAXA human data. Coming up soon in OSDR are data submissions from the European Space Agency, Virgin Galactic PIs, and SpaceX Polaris Dawn. A major benefit of OSDR is the array of standardized and uniformly formatted data (which was developed through AWG member consensus), from which visualization tools, analysis tools, and machine learning models can be built or trained. This talk will cover the Multi-Study Visualization Tool, the Environmental Data Application, RadLab, and a UCSF-NSF funded knowledge graph biomedical health discovery tool ‘SPOKE’ currently being integrated with OSDR. OSDR also provides training programs in bioinformatics and machine learning to improve the scientific community’s awareness of data availability and to boost their ability to perform data analysis. The increasing engagement of the scientific community and the public with technologies powered by artificial intelligence (AI) heightens the need for data analysis to be transparent. The AI for Life in Space initiative leverages the data products provided in OSDR to train AI models, with an emphasis on explainable and trustworthy AI, which would not be possible without FAIR data and metadata. Overall, here we will demonstrate the importance for NASA life sciences data repositories to adhere to the FAIREST framework, by providing examples and success stories from different aspects of OSDR.

data↗

The Test Analysis Retrieval System (TARS): Meeting the challenges of the network's test processes

The Networks Systems Test Section (GSFC 531.4) is responsible for managing a variety of engineering and operational tests used to assess the status of the Network elements relative to readiness certification for new and ongoing mission support and for performance trending. To conduct analysis of data collected during these tests, to disseminate and share the information, and to catalog and create reports based on the analysis is currently a cumbersome and inefficient task due primarily to the manual handling of paper products and the inability to easily exchange information between the various Networks elements. The Test Analysis and Retrieval System (TARS) is being implemented to promote concise data analysis, intelligible reporting of test results, to minimize test duplication by fostering a broad sharing of test data, and perhaps most importantly, to provide significantly improved response to the Network's internal and external customers. This paper outlines the intended application, architecture, and benefits of the TARS.

Stelmaszek, Robert L.↗

Move to Talk, Talk to Move: Tightly Integrated Communication and Control for Coordinated Swarms of Small Spacecraft

The Move to Talk, Talk to Move: Tightly Integrated Communication and Control for Coordinated Swarms of Small Spacecraft project will build on existing research on collaborative autonomy of multi-agent systems and design techniques that will enable coordinated communication and control of spacecraft. The success of many space exploration and science missions hinges on real-time monitoring of time-varying and/or geographically distributed phenomena. This monitoring can be achieved using a swarm of small spacecraft, which collect data about the environment and share information within the swarm of spacecraft. Current space exploration missions typically issue commands to control each spacecraft individually from Earth, and the data gathered by each spacecraft is also transmitted to Earth separately via X-band communication over the Deep Space Network (DSN). This approach is expensive, slow, and unreliable. Many coordinated tasks amongst a swarm of autonomous agents (or, specifically, small spacecraft) rely on communication. Existing control, estimation, and decision algorithms often assume that mostly reliable communications are available; however, this is often not the case in actual environments and thus is a barrier to operating swarms of small spacecraft.

Qi Han↗

A Vision for Coupling Operation of US Fusion Facilities with HPC Systems and the Implications for Workflows and Data Management

The operation of large US Department of Energy (DOE) research facilities, like the DIII-D National Fusion Facility, results in the collection of complex multi-dimensional scientific datasets, both experimental and model-generated. In the future, it is envisioned that integrated data analysis coupled with large-scale high performance computing (HPC) simulations will be used to improve experimental planning and operation. Practically, massive data sets from these simulations provide the physics basis for generation of both reduced semi-analytic and machine-learning-based models. Storage of both HPC simulation datasets (generated from US DOE leadership computing facilities) and experimental datasets presents significant challenges. In this paper, we present a vision for a DOE-wide data management workflow that integrates US DOE fusion facilities with leadership computing facilities. Data persistence and long-term availability beyond the length of allocated projects is essential, particularly for verification and recalibration of artificial intelligence and machine learning (AI/ML) models. Because these data sets are often generated and shared among hundreds of users across multiple leadership computing facility centers, they would benefit from cross-platform accessibility, persistent identifiers (e.g. DOI, or digital object identifier), and provenance tracking. Here, the ability to handle different data access patterns suggests that a combination of low cost, high latency (e.g. for storing ML training sets) and high cost, low latency systems (e.g. for real-time, integrated machine control feedback) may be needed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Overview of the Upper Atmosphere Research Satellite: Observations from 1991 to 2002

The Upper Atmosphere Research Satellite (UARS) was launched in September 1991 by the Space Shuttle Discovery. Seven of the original ten instruments aboard the UARS are functional and six instruments regularly take measurements. The UARS is in a stable observing configuration, in spite of experiencing several anomalies which have impacted the data tape storage and power available. Power sharing and "real-time" operations using the Tracking and Data Relay Satellites have reduced the data collection over the past five years. Although the UARS measurements have not been continuous, the UARS instruments HALOE, SUSIM, SOLSTICE, HRDI, WINDII, and PEM have provided important observations over an entire solar cycle. The UARS HALOE instrument measures ozone and other constituents important in understanding ozone variations. The UARS SUSIM and SOLSTICE instruments observe ultraviolet light between 120 and 420 nm, which influence middle atmospheric constituent fluctuations. The UARS HRDI and WINDII instruments measure winds in the stratosphere, mesosphere, and thermosphere, which move constituents among the various atmospheric regions. The UARS PEM instrument provides observations of input precipitating charged particles, including both electrons and protons. This paper will provide an overview about the UARS mission and its relevant atmospheric measurements.

Jackman, Charles H.↗

Malware detection and analysis

Embodiments of the invention describe systems and methods for malicious software detection and analysis. A binary executable comprising obfuscated malware on a host device may be received, and incident data indicating a time when the binary executable was received and identifying processes operating on the host device may be recorded. The binary executable is analyzed via a scalable plurality of execution environments, including one or more non-virtual execution environments and one or more virtual execution environments, to generate runtime data and deobfuscation data attributable to the binary executable. At least some of the runtime data and deobfuscation data attributable to the binary executable is stored in a shared database, while at least some of the incident data is stored in a private, non-shared database.

97 MATHEMATICS AND COMPUTING↗

A data model of the Climate and Forecast metadata conventions (CF-1.6) with a software implementation (cf-python v2.1)

Abstract. The CF (Climate and Forecast) metadata conventions are designed to promote the creation, processing, and sharing of climate and forecasting data using Network Common Data Form (netCDF) files and libraries. The CF conventions provide a description of the physical meaning of data and of their spatial and temporal properties, but they depend on the netCDF file encoding which can currently only be fully understood and interpreted by someone familiar with the rules and relationships specified in the conventions documentation. To aid in development of CF-compliant software and to capture with a minimal set of elements all of the information contained in the CF conventions, we propose a formal data model for CF which is independent of netCDF and describes all possible CF-compliant data. Because such data will often be analysed and visualised using software based on other data models, we compare our CF data model with the ISO 19123 coverage model, the Open Geospatial Consortium CF netCDF standard, and the Unidata Common Data Model. To demonstrate that this CF data model can in fact be implemented, we present cf-python, a Python software library that conforms to the model and can manipulate any CF-compliant dataset.

54 ENVIRONMENTAL SCIENCES↗

Generic, Extensible, Configurable Push-Pull Framework for Large-Scale Science Missions

The push-pull framework was developed in hopes that an infrastructure would be created that could literally connect to any given remote site, and (given a set of restrictions) download files from that remote site based on those restrictions. The Cataloging and Archiving Service (CAS) has recently been re-architected and re-factored in its canonical services, including file management, workflow management, and resource management. Additionally, a generic CAS Crawling Framework was built based on motivation from Apache s open-source search engine project called Nutch. Nutch is an Apache effort to provide search engine services (akin to Google), including crawling, parsing, content analysis, and indexing. It has produced several stable software releases, and is currently used in production services at companies such as Yahoo, and at NASA's Planetary Data System. The CAS Crawling Framework supports many of the Nutch Crawler's generic services, including metadata extraction, crawling, and ingestion. However, one service that was not ported over from Nutch is a generic protocol layer service that allows the Nutch crawler to obtain content using protocol plug-ins that download content using implementations of remote protocols, such as HTTP, FTP, WinNT file system, HTTPS, etc. Such a generic protocol layer would greatly aid in the CAS Crawling Framework, as the layer would allow the framework to generically obtain content (i.e., data products) from remote sites using protocols such as FTP and others. Augmented with this capability, the Orbiting Carbon Observatory (OCO) and NPP (NPOESS Preparatory Project) Sounder PEATE (Product Evaluation and Analysis Tools Elements) would be provided with an infrastructure to support generic FTP-based pull access to remote data products, obviating the need for any specialized software outside of the context of their existing process control systems. This extensible configurable framework was created in Java, and allows the use of different underlying communication middleware (at present, both XMLRPC, and RMI). In addition, the framework is entirely suitable in a multi-mission environment and is supporting both NPP Sounder PEATE and the OCO Mission. Both systems involve tasks such as high-throughput job processing, terabyte-scale data management, and science computing facilities. NPP Sounder PEATE is already using the push-pull framework to accept hundreds of gigabytes of IASI (infrared atmospheric sounding interferometer) data, and is in preparation to accept CRIMS (Cross-track Infrared Microwave Sounding Suite) data. OCO will leverage the framework to download MODIS, CloudSat, and other ancillary data products for use in the high-performance Level 2 Science Algorithm. The National Cancer Institute is also evaluating the framework for use in sharing and disseminating cancer research data through its Early Detection Research Network (EDRN).

Foster, Brian M.↗

Operation of the Australian Store.Synchrotron for macromolecular crystallography

The Store.Synchrotron service, a fully functional, cloud computing-based solution to raw X-ray data archiving and dissemination at the Australian Synchrotron, is described. The Store.Synchrotron service, a fully functional, cloud computing-based solution to raw X-ray data archiving and dissemination at the Australian Synchrotron, is described. The service automatically receives and archives raw diffraction data, related metadata and preliminary results of automated data-processing workflows. Data are able to be shared with collaborators and opened to the public. In the nine months since its deployment in August 2013, the service has handled over 22.4 TB of raw data (∼1.7 million diffraction images). Several real examples from the Australian crystallographic community are described that illustrate the advantages of the approach, which include real-time online data access and fully redundant, secure storage. Discoveries in biological sciences increasingly require multidisciplinary approaches. With this in mind, Store.Synchrotron has been developed as a component within a greater service that can combine data from other instruments at the Australian Synchrotron, as well as instruments at the Australian neutron source ANSTO. It is therefore envisaged that this will serve as a model implementation of raw data archiving and dissemination within the structural biology research community.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Flexible Data-Aware Scheduling for Workflows over an In-Memory Object Store

This paper explores novel techniques for improving the performance of many-task workflows based on the Swift scripting language. We propose novel programmer options for automated distributed data placement and task scheduling. These options trigger a data placement mechanism used for distributing intermediate workflow data over the servers of Hercules, a distributed key-value store that can be used to cache file system data. We demonstrate that these new mechanisms can significantly improve the aggregated throughput of many-task workflows with up to 86x, reduce the contention on the shared file system, exploit the data locality, and trade off locality and load balance.

Duro, Francisco Rodrigo↗

The composition, heating value and renewable share of the energy content of mixed municipal solid waste in Finland

Highlights: • New experimental data of mixed MSW properties in a Finnish case region. • The share of renewable energy of mixed MSW. • The results were compared with earlier international studies. • The average share of renewable energy was 30% and the average LHVar 19 MJ/kg. • Well operating source separation decreases the renewable energy content of MSW. - Abstract: For the estimation of greenhouse gas emissions from waste incineration it is essential to know the share of the renewable energy content of the combusted waste. The composition and heating value information is generally available, but the renewable energy share or heating values of different fractions of waste have rarely been determined. In this study, data from Finnish studies concerning the composition and energy content of mixed MSW were collected, new experimental data on the compositions, heating values and renewable share of energy were presented and the results were compared to the estimations concluded from earlier international studies. In the town of Lappeenranta in south-eastern Finland, the share of renewable energy ranged between 25% and 34% in the energy content tests implemented for two sample trucks. The heating values of the waste and fractions of plastic waste were high in the samples compared to the earlier studies in Finland. These high values were caused by good source separation and led to a low share of renewable energy content in the waste. The results showed that in mixed municipal solid waste the renewable share of the energy content can be significantly lower than the general assumptions (50–60%) when the source separation of organic waste, paper and cardboard is carried out successfully. The number of samples was however small for making extensive conclusions on the results concerning the heating values and renewable share of energy and additional research is needed for this purpose.

09 BIOMASS FUELS↗

Machine Learning for Materials Scientists: An Introductory Guide toward Best Practices

This Methods/Protocols article is intended for materials scientists interested in performing machine learning-centered research. Herein, we cover broad guidelines and best practices regarding the obtaining and treatment of data, feature engineering, model training, validation, evaluation and comparison, popular repositories for materials data and benchmarking data sets, model and architecture sharing, and finally publication. In addition, we include interactive Jupyter notebooks with example Python code to demonstrate some of the concepts, workflows, and best practices discussed. Overall, the data-driven methods and machine learning workflows and considerations are presented in a simple way, allowing interested readers to more intelligently guide their machine learning research using the suggested references, best practices, and their own materials domain expertise.

36 MATERIALS SCIENCE↗

Automation of Data Traffic Control on DSM Architecture

The design of distributed shared memory (DSM) computers liberates users from the duty to distribute data across processors and allows for the incremental development of parallel programs using, for example, OpenMP or Java threads. DSM architecture greatly simplifies the development of parallel programs having good performance on a few processors. However, to achieve a good program scalability on DSM computers requires that the user understand data flow in the application and use various techniques to avoid data traffic congestions. In this paper we discuss a number of such techniques, including data blocking, data placement, data transposition and page size control and evaluate their efficiency on the NAS (NASA Advanced Supercomputing) Parallel Benchmarks. We also present a tool which automates the detection of constructs causing data congestions in Fortran array oriented codes and advises the user on code transformations for improving data traffic in the application.

Frumkin, Michael↗

An MPA-IO interface to HPSS

This paper describes an implementation of the proposed MPI-IO (Message Passing Interface - Input/Output) standard for parallel I/O. Our system uses third-party transfer to move data over an external network between the processors where it is used and the I/O devices where it resides. Data travels directly from source to destination, without the need for shuffling it among processors or funneling it through a central node. Our distributed server model lets multiple compute nodes share the burden of coordinating data transfers. The system is built on the High Performance Storage System (HPSS), and a prototype version runs on a Meiko CS-2 parallel computer.

Jones, Terry↗

An MPI-IO Interface to HPSS

This paper describes an implementation of the proposed MPI-IO standard for parallel I/O. Our system uses third-party transfer to move data over an external network between the processors where it is used and the I/O devices where it resides. Data travels directly from source to destination, without the need for shuffling it among processors or funneling it through a central node. Our distributed server model lets multiple compute nodes share the burden of coordinating data transfers. The system is built on the High Performance Storage System (HPSS), and a prototype version runs on a Meiko CS-2 parallel computer.

Parallel Processing↗

Implementing EVM Data Analysis Adding Value from a NASA Project Manager's Perspective

Data Analysis is one of the keys to an effective Earned Value Management (EVM) Process. Project Managers (PM) must continually evaluate data in assessing the health of their projects. Good analysis of data can assist PMs in making better decisions in managing projects. To better support our P Ms, National Aeronautics and Space Administration (NASA) - Marshall Space Flight Center (MSFC) recently renewed its emphasis on sound EVM data analysis practices and processes, During this presentation we will discuss the approach that MSFC followed in implementing better data analysis across its Center. We will address our approach to effectively equip and support our projects in applying a sound data analysis process. In addition, the PM for the Space Station Biological Research Project will share her experiences of how effective data analysis can benefit a PM in the decision making process. The PM will discuss how the emphasis on data analysis has helped create a solid method for assessing the project s performance. Using data analysis successfully can be an effective and efficient tool in today s environment with increasing workloads and downsizing workforces

Counts, Stacy↗

Open Chemistry, JupyterLab, REST, and quantum chemistry

Quantum chemistry must evolve if it wants to fully leverage the benefits of the internet age, where the worldwide web offers a vast tapestry of tools that enable users to communicate and interact with complex data at the speed and convenience of a button press. The Open Chemistry project has developed an open-source framework that offers an end-to-end solution for producing, sharing, and visualizing quantum chemical data interactively on the web using an array of modern tools and approaches. These tools build on some of the best open-source community projects such as Jupyter for interactive online notebooks, coupled with 3D accelerated visualization, state-of-the-art computational chemistry codes including NWChem and Psi4, and emerging machine learning and data mining tools such as ChemML and ANI. They offer flexible formats to import and export data, along with approaches to compare computational and experimental data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗