Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

EMASS (tm): An expandable solution for NASA space data storage needs

The data acquisition, distribution, processing, and archiving requirements of NASA and other U.S. Government data centers present significant data management challenges that must be met in the 1990's. The Earth Observing System (EOS) project alone is expected to generate daily data volumes greater than 2 Terabytes (2(10)(exp 12) Bytes). As the scientific community makes use of this data their work product will result in larger, increasingly complex data sets to be further exploited and managed. The challenge for data storage systems is to satisfy the initial data management requirements with cost effective solutions that provide for planned growth. This paper describes the expandable architecture of the E-Systems Modular Automated Storage System (EMASS (TM)), a mass storage system which is designed to support NASA's data capture, storage, distribution, and management requirements into the 21st century.

Peterson, Anthony L.↗

Exascale Storage Systems the SIRIUS Way

As the exascale computing age emerges, data related issues are becoming critical factors that determine how and where we do computing. Popular approaches used by traditional I/O solution and storage libraries become increasingly bottlenecked due to their assumptions about data movement, re-organization, and storage. While, new technologies, such as "burst buffers", can help address some of the short-term performance issues, it is essential that we reexamine the underlying storage and I/O infrastructure to effectively support requirements and challenges at exascale and beyond. In this paper we present a new approach to the exascale Storage System and I/O (SSIO), which is based on allowing users to inject application knowledge into the system and leverage this knowledge to better manage, store, and access large data volumes so as to minimize the time to scientific insights. Central to our approach is the distinction between the data, metadata, and the knowledge contained therein, transferred from the user to the system by describing "utility" of data as it ages.

97 MATHEMATICS AND COMPUTING↗

High Performance Computing Facility Operational Assessment, FY 2010 Oak Ridge Leadership Computing Facility

Oak Ridge National Laboratory's (ORNL's) Cray XT5 supercomputer, Jaguar, kicked off the era of petascale scientific computing in 2008 with applications that sustained more than a thousand trillion floating point calculations per second - or 1 petaflop. Jaguar continues to grow even more powerful as it helps researchers broaden the boundaries of knowledge in virtually every domain of computational science, including weather and climate, nuclear energy, geosciences, combustion, bioenergy, fusion, and materials science. Their insights promise to broaden our knowledge in areas that are vitally important to the Department of Energy (DOE) and the nation as a whole, particularly energy assurance and climate change. The science of the 21st century, however, will demand further revolutions in computing, supercomputers capable of a million trillion calculations a second - 1 exaflop - and beyond. These systems will allow investigators to continue attacking global challenges through modeling and simulation and to unravel longstanding scientific questions. Creating such systems will also require new approaches to daunting challenges. High-performance systems of the future will need to be codesigned for scientific and engineering applications with best-in-class communications networks and data-management infrastructures and teams of skilled researchers able to take full advantage of these new resources. The Oak Ridge Leadership Computing Facility (OLCF) provides the nation's most powerful open resource for capability computing, with a sustainable path that will maintain and extend national leadership for DOE's Office of Science (SC). The OLCF has engaged a world-class team to support petascale science and to take a dramatic step forward, fielding new capabilities for high-end science. This report highlights the successful delivery and operation of a petascale system and shows how the OLCF fosters application development teams, developing cutting-edge tools and resources for next-generation systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

AIAA/NASA International Symposium on Space Information Systems, 2nd, Pasadena, CA, Sept. 17-19, 1990, Proceedings. Vols. 1 & 2

These proceedings overview major space information system projects and lessons learned from current missions. Other topics include the science information system requirements for the 1990s, an information systems design approach for major programs, the technology needs and projections, the standards for space data information systems, the artificial intelligence technology and applications, international interoperability, and spacecraft data systems and architectures advanced communications. Other topics include the software engineering technology and applications, the multimission multidiscipline information system architectures, the distributed planning and scheduling systems and operations, and the computer and information systems architectures. Paper presented include prospects for scientific data analysis systems for solar-terrestrial physics in the 1990s, the Columbus data management system, data storage technologies for the future, the German aerospace research establishment, and launching artificial intelligence in NASA ground systems.

Tavenner, Leslie A.↗

NASA GES DISC Earth Science Data Support

It is the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) mission statement to facilitate data access and evaluation, as well as scientific exploration and discovery. Recently, GES DISC has been evolving and improving our data management and services in order to promote GES DISC data to be easily discovered, improve usage and made more interoperable with common tools. As a result, we will present a brief review of our recent data services at the GES DISC including our new science-data driven website, subsetting and resampling services across multitude of satellite processing levels, visualization services, and how we utilize social media tools to interact with user communities.

data support↗

Radiosurgery of Glomus Jugulare Tumors: A Meta-Analysis

Purpose: During the past two decades, radiosurgery has arisen as a promising approach to the management of glomus jugulare. In the present study, we report on a systematic review and meta-analysis of the available published data on the radiosurgical management of glomus jugulare tumors. Methods and Materials: To identify eligible studies, systematic searches of all glomus jugulare tumors treated with radiosurgery were conducted in major scientific publication databases. The data search yielded 19 studies, which were included in the meta-analysis. The data from 335 glomus jugulare patients were extracted. The fixed effects pooled proportions were calculated from the data when Cochrane's statistic was statistically insignificant and the inconsistency among studies was <25%. Bias was assessed using the Egger funnel plot test. Results: Across all studies, 97% of patients achieved tumor control, and 95% of patients achieved clinical control. Eight studies reported a mean or median follow-up time of >36 months. In these studies, 95% of patients achieved clinical control and 96% achieved tumor control. The gamma knife, linear accelerator, and CyberKnife technologies all exhibited high rates of tumor and clinical control. Conclusions: The present study reports the results of a meta-analysis for the radiosurgical management of glomus jugulare. Because of its high effectiveness, we suggest considering radiosurgery for the primary management of glomus jugulare tumors.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Thermal Management for FPGA Nodes in HPC Systems

The integration of FPGAs into large-scale computing systems is gaining attention. In these systems, real-time data handling for networking, tasks for scientific computing, and machine learning can be executed with customized datapaths on reconfigurable fabric within heterogeneous compute nodes. At the same time, thermal management, particularly battling the cooling cost and guaranteeing the reliability, is a continuing concern. The introduction of new heterogeneous components into HPC nodes only adds further complexities to thermal modeling and management. The thermal behavior of multi-FPGA systems deployed within large compute clusters is less explored. Here, we first show that the thermal behaviors of different FPGAs of the same generation can vary due to their physical locations in a rack and process variation, even though they are running the same tasks. We present a machine learning–based model to capture the thermal behavior of each individual FPGA in the cluster. We then propose two thermal management strategies guided by our thermal model. First, we mitigate thermal variation and hotspots across the cluster by proactive thermal-aware task placement. Under the tested system and benchmarks, we achieve up to 26.4° C and on average 13.3° C system temperature reduction with no performance penalty. Second, we utilize this thermal model to guide HLS parameter tuning at the task design stage to achieve improved thermal response after deployment.

97 MATHEMATICS AND COMPUTING↗

Modular Subsurface Sensors and Integrated Software for Advanced Subsurface Characterization and Monitoring using Unoccupied Vehicles

The advent and subsequent proliferation of autonomous airborne, waterborne, and groundbased vehicles (i.e., “drones”) promises to broadly transform the geosciences and associated industries, including fossil energy exploration and development, mineral resource exploration and development, water-resource management, and environmental remediation. For geophysical characterization and monitoring, the prospect of programming highly repeatable and low-cost drone missions for subsurface imaging will allow for deployments in hazardous and previously inaccessible areas. Coupled with autonomous workflows for data processing, management, and visualization, drone-based geophysical characterization and monitoring will enable unprecedented, real-time insight into diverse subsurface properties and processes of scientific and engineering importance. Toward this end, the objectives of this Lab Directed Research and Development (LDRD) project were to develop new (1) instrumentation for dronebased electromagnetic induction (EMI) geophysical imaging, including separated transmitter and receivers and associated electronics, (2) software for real-time data telemetry, processing, management, and visualization. Although EMI has been previously deployed using unoccupied aerial systems (UASs), these applications failed to capitalize on the game-changing capabilities of drone platforms. Whereas drone-based data acquisition allows for collection of rich, three-dimensional (3D) multi-offset/multi-angle configurations between transmitters and receivers, past efforts have relied on conventional instrumentation that was designed for ground-based data collection with the transmitter and a single receiver housed in the same unit; nor did these previous applications demonstrate real-time delivery of results to support rapid management decisions in the field. In this 1-year project, we (1) designed and constructed new lightweight independent transmitter and receiver antenna platforms that communicate with a laptop computer; (2) developed software to control data acquisition, manage/transfer data, and visualize data as its collected; and (3) demonstrated the operation of the new hardware and software systems in a ground-based field test. Our work entails major technological advances for EMI and established a foundation on which to build a new drone-based, real-time geophysical EMI imaging capability to support diverse challenges facing the nation.

47 OTHER INSTRUMENTATION↗

PanDA: Production and Distributed Analysis System

The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at the LHC data processing scale. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year. PanDA’s flexibility and scalability make it suitable for the High Energy Physics community and wider science domains at the Exascale. Beyond High Energy Physics, PanDA’s relevance extends to other big data sciences, as evidenced by its adoption in the Vera C. Rubin Observatory and the sPHENIX experiment. As the significance of advanced workflows continues to grow, PanDA has transformed into a comprehensive ecosystem, effectively tackling challenges associated with emerging workflows and evolving computing technologies. The paper discusses PanDA’s prominent role in the scientific landscape, detailing its architecture, functionality, deployment strategies, project management approaches, results, and evolution into an ecosystem.

97 MATHEMATICS AND COMPUTING↗

NASA Exhibits

A series of NASA presentations for the Supercomputing 2001 conference are summarized. The topics include: (1) Mars Surveyor Landing Sites "Collaboratory"; (2) Parallel and Distributed CFD for Unsteady Flows with Moving Overset Grids; (3) IP Multicast for Seamless Support of Remote Science; (4) Consolidated Supercomputing Management Office; (5) Growler: A Component-Based Framework for Distributed/Collaborative Scientific Visualization and Computational Steering; (6) Data Mining on the Information Power Grid (IPG); (7) Debugging on the IPG; (8) Debakey Heart Assist Device: (9) Unsteady Turbopump for Reusable Launch Vehicle; (10) Exploratory Computing Environments Component Framework; (11) OVERSET Computational Fluid Dynamics Tools; (12) Control and Observation in Distributed Environments; (13) Multi-Level Parallelism Scaling on NASA's Origin 1024 CPU System; (14) Computing, Information, & Communications Technology; (15) NAS Grid Benchmarks; (16) IPG: A Large-Scale Distributed Computing and Data Management System; and (17) ILab: Parameter Study Creation and Submission on the IPG.

Deardorff, Glenn↗

Conquering Data Chaos: Research Data Management with Kubernetes

Managing massive volumes of data and effectively making it accessible to researchers poses significant challenges and is a barrier to scientific discovery. In many cases, critical data is locked up in unwieldy file formats or one-off databases and is too large to effectively process on a single machine. This talk explores the role of Kubernetes, an open-source container orchestration platform, in addressing research data management challenges. I will discuss how we are using a set of publicly available open-source and home-grown tools in the National Renewable Energy Lab (NREL) Data, Analysis, and Visualization (DAV) group to help researchers overcome data-related bottlenecks. The talk will begin by providing an overview of the data challenges faced in research data management, including data storage, processing, and analysis. I will highlight Kubernetes' ability to handle large-scale data by leveraging containerization and distributed computing, including distributed storage. Kubernetes allows researchers to encapsulate data processing infrastructure and workflows into portable containers, enabling reproducibility and ease of deployment. Kubernetes can then schedule and manage the resource allocation of these containers to enable efficient utilization of limited computing resources, leading to more efficient data processing and analysis. I will discuss some limitations of traditional, siloed approaches to dealing with data and emphasize the need for solutions which foster collaboration. I will highlight how we are using Kubernetes at NREL to facilitate data sharing and cooperation among research teams. Kubernetes' flexible architecture enables the deployment of shared computing environments, such as Apache Superset, where researchers can seamlessly access and analyze shared datasets. Providing the ability to have one research team easily consume data generated by another, utilizing Kubernetes' as a central data platform, is one of the major wins we've encountered by adopting the platform. Finally, I will showcase real-world use cases from NREL where we have used Kubernetes to solve some persistent data challenges involving large volumes of sensor and monitoring data. I will discuss the challenges we encountered when creating our cluster and making it available as a production-ready resource. I will also discuss the specific suite of tools, including Postgres and Apache Druid for columnar and timeseries data, and Redpanda Kafka for streaming data we have deployed in our infrastructure, and the process that went into the selection of these tools.

collaborative environment↗

Collaboration Portal for Petascale Simulations

The emergence of leadership class computing is creating a tsunami of data from petascale simulations. Results are typically analyzed by dozens of scientists. In order for the scientist to digest the vast amount of data being produced from the simulations and auxiliary programs, it is critical to automate the effort to manage, analyze, visualize, and share this data. One aspect of this is leveraging of their collective knowledge and experiences through a scientific social network. This can be archived through a combination of parallel back-end services, provenance capturing, and an easy to use front-end tool. 'eSimMon', is one such tool we developed as part of the Scientific Discovery through Advanced Computing (SciDAC) program. In this paper we describe eSimMon, discuss its ease of use, its efficiency, and its ability to accelerate scientific discovery through advanced computing.

97 MATHEMATICS AND COMPUTING↗

Monitoring Coastal Marshes for Persistent Saltwater Intrusion

Primary goal: Provide resource managers with remote sensing products that support ecosystem forecasting models requiring salinity and inundation data. Work supports the habitat-switching modules in the Coastal Louisiana Ecosystem Assessment and Restoration (CLEAR) model, which provides scientific evaluation for restoration management (Visser et al., 2008). Ongoing work to validate flooding with radar (NWRC/USGS) and enhance persistence estimates through "fusion" of MODIS and Landsat time series (ROSES A.28 Gulf of Mexico). Additional work will also investigate relationship between saltwater dielectric constant and radar returns (Radarsat) (ROSES A.28 Gulf of Mexico).

Kalcic, Maria↗

Blast from the Past: ASDC Curation for NASA Suborbital Legacy Missions to Promote Data Discovery and Accessibility

NASA has an extensive history of conducting suborbital field campaigns to further advances in atmospheric sciences. Beginning with the Chemical Instrument Test and Evaluation (CITE) conducted in 1983-1984, NASA has completed many suborbital campaigns over the past three decades. Since the early 2010s, suborbital missions are typically assigned to a NASA Distributed Active Archive Center (DAAC) prior to the mission for long-term archival and distribution. Efforts are being made by NASA’s Earth Science Data and Information System (ESDIS) Project and the Airborne Data Management Group (ADMG) to assign legacy missions to DAACs for permanent archival and distribution, so that these valuable datasets remain to be available to the scientific community. NASA’s Atmospheric Science Data Center (ASDC) has been named the assigned DAAC for nearly 20 atmospheric composition legacy missions, including missions conducted as part of the Global Tropospheric Experiment (GTE) and expects to be named the assigned DAAC for more of these missions over the next few years. The primary goal of the ASDC is to provide access to the datasets as they are currently formatted to the broad user community and enhance their findability and accessibility. However, data reporting standards have evolved significantly since 1983 and the datasets span a wide variety of file formats, including text, Ames, GTE, and ICARTT (International Consortium for Atmospheric Research on Transport and Transformation), and the amount of metadata and relevant information included in the files also varies greatly and can not be readily extracted without subject matter knowledge. This has caused challenges for the ASDC’s suborbital metadata extraction pipeline in ensuring that accurate and necessary metadata is being provided for the missions by all the ASDC’s existing search mechanisms. To make the data more findable and accessible, the ASDC has begun researching ways to further enhance the datasets, including distributing value-added products (i.e. consistent file format such as ICARTT or netCDF), adding standard names from the ESDIS Standards Coordination Office (ESCO)-approved Atmospheric Composition Variable Standard Names Convention (ACVSNC), and creating outreach materials such as ArcGIS StoryMaps, User Guides, and Micro Articles, providing overviews of the missions and what type of data was collected during the missions. These efforts also help support NASA’s Open-Source Science by enhancing the FAIRness of the legacy data products. This presentation will review the ASDC’s ongoing efforts, progress made, and future plans for legacy missions.

Megan Buzanowicz↗

A customizable data management framework for high-repetition-rate high-energy-density science

The high-energy-density (HED) physics community is moving toward a new paradigm of high-repetition-rate (HRR) operation. To fully leverage the scientific power of HRR HED facilities, all of the components of each subsystem (laser, targetry, and performance diagnostics) must be connected and synchronized in a reliable and robust manner while the data acquired are tagged and archived in real time. To this end, GA has begun developing a generalized NoSQL-database framework, the MongoDB repository for information and archiving. An organizational strategy has been developed that shifts HED data organization from a shot-based to a diagnostic-based approach in order to increase archival and retrieval efficiency that lends itself to optimization applications. This work is a first step in pushing HRR HED science toward data management solutions that emphasize machine actionability and aim to stimulate community engagement to define data standards in HED science.

Instruments & Instrumentation↗

Biodefense Knowledge Management System

After the Amerithrax attacks, the Department of Homeland Security (DHS) recognized the need to link technical scientific information with law enforcement and national security information in the fight against bioterrorism. The Biodefense Knowledge Center (BKC) was formed to provide timely analysis of technical biodefense-related topics to support DHS and its customers in the interagency. The Biodefense Knowledge Management System (BKMS) was created as the backbone for information management within the BKC. It is a scalable, web-based knowledge management system for organizing disparate data sources. By combining information from 35+ data sources, the BKMS provides a searchable repository for technical information including scientific reports, journal articles, news articles, research funding databases, patent databases, and genomic datasets. The current system contains over 58 million documents and 450 million genomic records, and operates at several classification levels.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Implementation of data citations and persistent identifiers at the ORNL DAAC

A requirement of data archives is that data holdings can be easily discovered, accessed, and used. One approach to improving data discovery and access is through data citations coupled with Digital Object Identifiers (DOI). The Oak Ridge National Laboratory Distributed Active Archive Center (ORNL DAAC) since 1998 has issued data citations that have been accepted and used in peer-reviewed journals. Citation elements established by the ORNL DAAC are similar to those used for journal articles (authors, year, product title, and information to locate), and beginning in 2007 included a DOI that is persistent, actionable, specific, and complete. The approach used at the ORNL DAAC also allows for referring to subsets of the data, by including within the citation the temporal and spatial extent, and parameters used. Data citations allow readers to find data and reproduce the results of the research article, and also use those data to test new hypotheses, design new sample collections, or construct or evaluate models. The ORNL DAAC uses a manual method to compile data citations and has developed a database that links research articles and their use of specific ORNL DAAC data products. Automation of the data citation compilation process, as is the case for articles, will enable data citations to become a more common practice. In addition to enhancing discovery and access of the data used in a research article, the citation gives credit to data generators, data centers and their funders, and, through citation indices, determine the scientific impact of a data set.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

The FEWSION for Community Resilience (F4R) Process: Building Local Technical and Social Capacity for Critical Supply Chain Resilience

Local business leaders, policy makers, elected officials, city planners, emergency managers, and private citizens are responsible for, and deeply affected by, the performance of critical supply chains and related infrastructures. At the center of critical supply chains is the food-energy-water nexus (FEW); a nexus that is key to a community’s wellbeing, resilience, and sustainability. In the 21st century, managing a local FEW nexus requires accurate data describing the function and structure of a community’s supply chains. However, data is not enough; we need data-informed conversation and technical and social capacity building among local stakeholders to utilize the data effectively. There are some resources available at the mesoscale and for food, energy, or water, but many communities lack the data and tools needed to understand connections and bridge the gaps between these scales and systems. As a result, we currently lack the capacity to manage these systems in small and medium sized communities where the vast majority of people, decisions, and problems reside. This study develops and validates a participatory citizen science process for FEW nexus capacity building and data-driven problem solving in small communities at the grassroots level. The FEWSION for Community Resilience (F4R) process applies a Public Participation in Scientific Research (PPSR) framework to map supply chain data for a community’s FEW nexus, to identify the social network that manages the nexus, and then to generate a data-informed conversation among stakeholders. F4R was piloted and co-developed with participants over a 2-year study, using a design-based research process to make evidence-based adjustments as needed. Results show that the F4R model was successful at improving volunteers’ awareness about nexus and supply chain issues, at creating a network of connections and communication with stakeholders across state, regional, and local organizations, and in facilitating data-informed discussion about improvements to the system. In this paper we describe the design and implementation of F4R and discuss four recommendations for the successful application of the F4R model in other communities: 1) embed opportunities for co-created PPSR, 2) build social capital, 3) integrate active learning strategies with user-friendly digital tools, and 4) adopt existing materials and structure.

54 ENVIRONMENTAL SCIENCES↗