Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Data Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

The Goddard High Resolution Spectrograph Scientific Support Contract

In 1988, Computer Sciences Corporation (CSC) was selected as the Goddard High Resolution Spectrograph (GHRS) Scientific Support Contractor (SSC). This was to have been a few months before the launch of NASA's first Great Observatory, the Hubble Space Telescope (HST). As one of five scientific instruments on HST, the GHRS was designed to obtain spectra in the 1050-3300 A ultraviolet wavelength region with a resolving power, lambda/Delta(lambda) , of up to 100,000 and relative photometric accuracy to 1%. It was built by Ball AeroSpace Systems Group under the guidance of the GHRS Investigation Definition Team (IDT), comprised of 16 scientists from the US and Canada. After launch, the IDT was to perform the initial instrument calibration and execute a broad scientific program during a five-year Guaranteed Time Observation (GTO) period. After a year's delay, the launch of HST occurred in April 1990, and CSC participated in the in-orbit calibration and first four years of GTO observations with the IDT. The HST primary mirror suffered from spherical aberration, which reduced the spatial and spectral resolution of Large Science Aperture (LSA) observations and decreased the throughput of the Small Science Aperture (SSA) by a factor of two. Periodic problems with the Side 1 carrousel electronics and anomalies with the low-voltage power supply finally resulted in a suspension of the use of Side 1 less than two years after launch. At the outset, the GHRS SSC task involved work in four areas: 1) to manage and operate the GHRS Data Analysis Facility (DAF); 2) to support the second Servicing Mission Observatory Verification (SMOV) program, as well as perform system engineering analysis of the GHRS as nesessary; 3) to assist the GHRS IDT with their scientific research programs, particularly the GSFC members of the team, and 4) to provide administrative and logistic support for GHRS public information and educational activities.

Source record↗

The OSS-1/STS-3 mission

Integration, test, flight operations, program and science management, and experiment results from the OSS-1/STS-3 flight are reviewed. The experimental pallet, manufactured by ESA members, carried instrumentation for astronomy, space plasma studies, physics, planetology, solar physics, and life sciences. The ability of the equipment to operate in a worst-case Orbiter thermal environment was tested, as was the plasma environment around the Orbiter. The solar UV spectral irradiance monitor microprocessor was found to reset concurrently with start-up of the Shuttle hydraulic pumps. Degradations were also observed in one of the three solar flare X ray polarimeter modules, the thermal cannister experiment microprocessor, and the scientific data recorder no. 2. The Orbiter was determined to perform well during firing of an electron beam. A contamination monitor recorded matter accretion to be dependent on the solar orientation of the Orbiter. Finally, the distribution of the responsibilities and tasks for mission management are reviewed.

Kissin, K.↗

SDS: A Framework for Scientific Data Services

Large-scale scientific applications typically write their data to parallel file systems with organizations designed to achieve fast write speeds. Analysis tasks frequently read the data in a pattern that is different from the write pattern, and therefore experience poor I/O performance. In this paper, we introduce a prototype framework for bridging the performance gap between write and read stages of data access from parallel file systems. We call this framework Scientific Data Services, or SDS for short. This initial implementation of SDS focuses on reorganizing previously written files into data layouts that benefit read patterns, and transparently directs read calls to the reorganized data. SDS follows a client-server architecture. The SDS Server manages partial or full replicas of reorganized datasets and serves SDS Clients' requests for data. The current version of the SDS client library supports HDF5 programming interface for reading data. The client library intercepts HDF5 calls and transparently redirects them to the reorganized data. The SDS client library also provides a querying interface for reading part of the data based on user-specified selective criteria. We describe the design and implementation of the SDS client-server architecture, and evaluate the response time of the SDS Server and the performance benefits of SDS.

59 BASIC BIOLOGICAL SCIENCES↗

Crosscut report: Exascale Requirements Reviews, March 9–10, 2017 – Tysons Corner, Virginia. An Office of Science review sponsored by: Advanced Scientific Computing Research, Basic Energy Sciences, Biological and Environmental Research, Fusion Energy Sciences, High Energy Physics, Nuclear Physics

The mission of the U.S. Department of Energy Office of Science (DOE SC) is the delivery of scientific discoveries and major scientific tools to transform our understanding of nature and to advance the energy, economic, and national security missions of the United States. To achieve these goals in today’s world requires investments in not only the traditional scientific endeavors of theory and experiment, but also in computational science and the facilities that support large-scale simulation and data analysis. The Advanced Scientific Computing Research (ASCR) program addresses these challenges in the Office of Science. ASCR’s mission is to discover, develop, and deploy computational and networking capabilities to analyze, model, simulate, and predict complex phenomena important to DOE. ASCR supports research in computational science, three high-performance computing (HPC) facilities — the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory and Leadership Computing Facilities at Argonne (ALCF) and Oak Ridge (OLCF) National Laboratories — and the Energy Sciences Network (ESnet) at Berkeley Lab. ASCR is guided by science needs as it develops research programs, computers, and networks at the leading edge of technologies. As we approach the era of exascale computing, technology changes are creating challenges for science programs in SC for those who need to use high performance computing and data systems effectively. Numerous significant modifications to today’s tools and techniques will be needed to realize the full potential of emerging computing systems and other novel computing architectures. To assess these needs and challenges, ASCR held a series of Exascale Requirements Reviews in 2015–2017, one with each of the six SC program offices,1 and a subsequent Crosscut Review that sought to integrate the findings from each. Participants at the reviews were drawn from the communities of leading domain scientists, experts in computer science and applied mathematics, ASCR facility staff, and DOE program managers in ASCR and the respective program offices. The purpose of these reviews was to identify mission-critical scientific problems within the DOE Office of Science (including experimental facilities) and determine the requirements for the exascale ecosystem that would be needed to address those challenges. The exascale ecosystem includes exascale computing systems, high-end data capabilities, efficient software at scale, libraries, tools, and other capabilities. This effort will contribute to the development of a strategic roadmap for ASCR compute and data facility investments and will help the ASCR Facility Division establish partnerships with Office of Science stakeholders. It will also inform the Office of Science research needs and agenda. The results of the six reviews have been published in reports available on the web at http://exascaleage.org/. This report presents a summary of the individual reports and of common and crosscutting findings, and it identifies opportunities for productive collaborations among the DOE SC program offices.

97 MATHEMATICS AND COMPUTING↗

Towards a Scalable and Adaptive Application Support Platform for Large-Scale Distributed E-Sciences in High-Performance Network Environments

The advent of large-scale collaborative scientific applications has demonstrated the potential for broad scientific communities to pool globally distributed resources to produce unprecedented data acquisition, movement, and analysis. System resources including supercomputers, data repositories, computing facilities, network infrastructures, storage systems, and display devices have been increasingly deployed at national laboratories and academic institutes. These resources are typically shared by large communities of users over Internet or dedicated networks and hence exhibit an inherent dynamic nature in their availability, accessibility, capacity, and stability. Scientific applications using either experimental facilities or computation-based simulations with various physical, chemical, climatic, and biological models feature diverse scientific workflows as simple as linear pipelines or as complex as a directed acyclic graphs, which must be executed and supported over wide-area networks with massively distributed resources. Application users oftentimes need to manually configure their computing tasks over networks in an ad hoc manner, hence significantly limiting the productivity of scientists and constraining the utilization of resources. The success of these large-scale distributed applications requires a highly adaptive and massively scalable workflow platform that provides automated and optimized computing and networking services. This project is to design and develop a generic Scientific Workflow Automation and Management Platform (SWAMP), which contains a web-based user interface specially tailored for a target application, a set of user libraries, and several easy-to-use computing and networking toolkits for application scientists to conveniently assemble, execute, monitor, and control complex computing workflows in heterogeneous high-performance network environments. SWAMP will enable the automation and management of the entire process of scientific workflows with the convenience of a few mouse clicks while hiding the implementation and technical details from end users. Particularly, we will consider two types of applications with distinct performance requirements: data-centric and service-centric applications. For data-centric applications, the main workflow task involves large-volume data generation, catalog, storage, and movement typically from supercomputers or experimental facilities to a team of geographically distributed users; while for service-centric applications, the main focus of workflow is on data archiving, preprocessing, filtering, synthesis, visualization, and other application-specific analysis. We will conduct a comprehensive comparison of existing workflow systems and choose the best suited one with open-source code, a flexible system structure, and a large user base as the starting point for our development. Based on the chosen system, we will develop and integrate new components including a black box design of computing modules, performance monitoring and prediction, and workflow optimization and reconfiguration, which are missing from existing workflow systems. A modular design for separating specification, execution, and monitoring aspects will be adopted to establish a common generic infrastructure suited for a wide spectrum of science applications. We will further design and develop efficient workflow mapping and scheduling algorithms to optimize the workflow performance in terms of minimum end-to-end delay, maximum frame rate, and highest reliability. We will develop and demonstrate the SWAMP system in a local environment, the grid network, and the 100Gpbs Advanced Network Initiative (ANI) testbed. The demonstration will target scientific applications in climate modeling and high energy physics and the functions to be demonstrated include workflow deployment, execution, steering, and reconfiguration. Throughout the project period, we will work closely with the science communities in the fields of climate modeling and high energy physics including Spallation Neutron Source (SNS) and Large Hadron Collider (LHC) projects to mature the system for production use.

97 MATHEMATICS AND COMPUTING↗

Time-Series Forecast Modeling on High-Bandwidth Network Measurements

With the increasing number of geographically distributed scientific collaborations and the growing sizes of scientific data, it has become challenging for users to achieve the best possible network performance on a shared network. In this paper, we have developed a model to forecast expected bandwidth utilization on high-bandwidth wide area networks. The forecast model can improve the efficiency of the resource utilization and scheduling of data movements on high-bandwidth networks to accommodate ever increasing data volume for large-scale scientific data applications. A univariate time-series forecast model is developed with the Seasonal decomposition of Time series by Loess (STL) and the AutoRegressive Integrated Moving Average (ARIMA) on Simple Network Management Protocol (SNMP) path utilization measurement data. Compared with the traditional approach such as Box-Jenkins methodology to train the ARIMA model, our forecast model reduces computation time up to 92.6 %. It also shows resilience against abrupt network usage changes. Finally, our forecast model conducts the large number of multi-step forecast, and the forecast errors are within the mean absolute deviation (MAD) of the monitored measurements.

97 MATHEMATICS AND COMPUTING↗

Cyber Dumpster Diving: Creating New Software Systems for Less

Modern scientific software is daunting in its breadth and complexity. From massively parallel simulations running on the world’s largest supercomputers to visualizations and user support environments that manage ever-growing, complex data collections, the challenges for software engineers are plentiful indeed.

software architecture↗

An interactive environment for the analysis of large Earth observation and model data sets

Envision is an interactive environment that provides researchers in the earth sciences convenient ways to manage, browse, and visualize large observed or model data sets. Its main features are support for the netCDF and HDF file formats, an easy to use X/Motif user interface, a client-server configuration, and portability to many UNIX workstations. The Envision package also provides new ways to view and change metadata in a set of data files. It permits a scientist to conveniently and efficiently manage large data sets consisting of many data files. It also provides links to popular visualization tools so that data can be quickly browsed. Envision is a public domain package, freely available to the scientific community. Envision software (binaries and source code) and documentation can be obtained from either of these servers: ftp://vista.atmos.uiuc.edu/pub/envision/ and ftp://csrp.tamu.edu/pub/envision/. Detailed descriptions of Envision capabilities and operations can be found in the User's Guide and Reference Manuals distributed with Envision software.

Bowman, Kenneth P.↗

Envision: An interactive system for the management and visualization of large geophysical data sets

Envision is a software project at the University of Illinois and Texas A&M, funded by NASA's Applied Information Systems Research Project. It provides researchers in the geophysical sciences convenient ways to manage, browse, and visualize large observed or model data sets. Envision integrates data management, analysis, and visualization of geophysical data in an interactive environment. It employs commonly used standards in data formats, operating systems, networking, and graphics. It also attempts, wherever possible, to integrate with existing scientific visualization and analysis software. Envision has an easy-to-use graphical interface, distributed process components, and an extensible design. It is a public domain package, freely available to the scientific community.

Searight, K. R.↗

NASA DEVELOP’s Communication Strategy for Sharing Applied Science Outcomes

NASA’s DEVELOP Program conducts rapid feasibility projects that enable the future workforce and current decision makers to collaborate and build capacity to use Earth science data to enhance environmental management. The program communicates its results and applications to a broad audience through multiple methods: social media, website blog posts, print materials, short project videos, an “Ambassador Corps,” scientific and policy conference presentations, community engagement activities, end-of-project presentations, project “hand-offs,” traditional publications (both gray literature and peer-reviewed), and a project archive gallery. This presentation will describe the various methods employed by DEVELOP to communicate the program’s scientific outputs, target audiences, general statistics, community response, and lessons learned.

Capacity Building↗

Characterizing the Performance of Executing Many-tasks on Summit

Many scientific workloads are comprised of many tasks, where each task is an independent simulation or analysis of data. The execution of millions of tasks on heterogeneous HPC platforms requires scalable dynamic resource management and multi-level scheduling. RADICAL-Pilot (RP) -- an implementation of the Pilot abstraction, addresses these challenges and serves as an effective runtime system to execute workloads comprised of many tasks. In this paper, we characterize the performance of executing many tasks using RP when interfaced with JSM and PRRTE on Summit: RP is responsible for resource management and task scheduling on acquired resource; JSM or PRRTE enact the placement of launching of scheduled tasks. Our experiments provide lower bounds on the performance of RP when integrated with JSM and PRRTE. Specifically, for workloads comprised of homogeneous single-core, 15 minutes-long tasks we find that: PRRTE scales better than JSM for > O(1000) tasks; PRRTE overheads are negligible; and PRRTE supports optimizations that lower the impact of overheads and enable resource utilization of 63% when executing O(16K), 1-core tasks over 404 compute nodes.

Matteo, Turilli↗

SULI Project Report: Mu2e otsdaq Documentation and Analytics

This report outlines the history of the CLFV puzzle, gives an overview of the Mu2e Experiment, and details the results of the DOE (Department of Energy)-funded SULI (Science Undergraduate Laboratory Internship) research completed over the summer 2024 on site at Fermilab. The project focused on the otsdaq software responsible for data acquisition. The project included drafting updated documentation for new users, including new procedures relevant for the Alma- Linux 9 operating system update, and the UPS package managing system replacement with Spack (scientific software package manager). A new art module (ReadTriggerCounts) to track trigger rates was created, with compatibility to be displayed via a Grafana server. The author also assisted in calorimeter cable installation in the detector hall.

Saarinen, Talia↗

Human Specimen Repository: Sample Collection for the Future [#133-000445]

In early 2014, an executive memo was released directing the improvement of the management of and access to scientific collections funded by government resources. In the spirit of this memo, NASA’s Human Research Program (HRP) funded a project to review existing residual samples in long-term storage for viability, identification, and scientific significance. These samples were to be collected in a centralized location and a detailed inventory was to be conducted to create the Human Sample Repository (HSR). HSR would work to review data from over 27,000 samples, from flight research projects dating back to 1998. The samples themselves would be reviewed, re-labeled, and organized into easy to retrieve, long-term storage. In addition to the older residual samples, HSR would expand to include pristine samples from all consenting crew who have flown on International Space Station (ISS) missions. All samples inventoried and managed by HSR are intended for the same purpose; to support future research projects which will utilize advanced technologies or techniques, by providing one-of-a-kind sample collections that can not be replicated. With this mission in mind, HSR will continue to collect residual samples from HRP funded projects and pristine samples from consenting crew, to further build this unique collection and increase the diversity of flight samples available to the scientific community.

Human Specimen Repository↗

Paraview-MCP

This project provides a streamlined way for users to interact with and control powerful scientific visualization software (ParaView) through a conversational interface. By developing an automated "Model Context Protocol" (MCP) server with a Python-based ParaView manager, the system allows users to seamlessly load and visualize complex datasets, explore visualization options with AI assistance, and optimize visualization output in a close loop. This is achieved by issuing intuitive, natural-language commands. The result is a user-friendly interface that integrates high-level conversation and scriptable data visualization, making scientific visualization tools more accessible to a broad audience.

Liu, Shusen [Lawrence Livermore National Laborator↗

Hot Springs and Geysers: Exploring Historical and Modern Impacts of Geothermal Energy Production on Associated Natural Surface Systems and Standardizing Management Practices

Surface thermal features, most notably hot springs and geysers are increasingly being recognized for their importance to ecosystems, indigenous cultures, and in some cases agriculture, recreation, and tourism. Geothermal project development poses a potential risk to these natural features but current regulatory requirements for assessing and managing these risks during exploration, permitting and monitoring are somewhat inconsistent and unpredictable across different geothermal fields. This has resulted in uncertainty and increases in exploration risk for geothermal energy developers that have led to costly project delays, cancellations, or hesitation to commit. Varying regulatory requirements may also influence public perception, fostering confusion, distrust and ultimately opposition to geothermal projects, further contributing to project delays or cancellations. At a time when there is an increasing urgency for reliable baseload clean energy, geothermal is a net-zero, renewable solution that additionally provides access to more equitable and environmentally just clean power. Continued integration of geothermal energy into the national energy roadmap can be facilitated through consistent and predictable permitting, providing regulators the framework they need, developers a clear path forward, and transparency that the public deserves. This project, currently in its beginning phases, seeks to address this important issue by providing a technical basis from which to build a preliminary protocol for assessing and managing potential impacts from new or existing geothermal energy projects to surface thermal features and their associated ecosystems. Development of this preliminary protocol will be informed by (1) a literature review of well-documented case studies in the western U.S. and New Zealand to understand the range of conditions that exemplify geothermal-surface thermal systems; (2) development of generic illustrative conceptual-numerical models to quantify, understand, and predict the first-order controls (e.g., pressure and permeability) on surface flows; and (3) additional independent and scientifically rigorous evaluations of geothermal-surface thermal system case studies from the Basin and Range Province that incorporate publicly available data as well as data provided by industry through data-sharing agreements. Learning from the successes of the process used to develop the Induced Seismicity Management Protocol (ISMP), we ultimately aim to use these initial efforts as a springboard for establishing a surface thermal feature management working group that will work collaboratively to finalize the protocol as well as co-create recommended best practices for implementation. We envision that the working group will primarily be composed of representatives from regulatory entities, government agencies, Tribes, academia, national laboratories, and industry, and will include early and regular engagement with community organizations and environmental groups. This will help ensure broad acceptance and implementation of the protocol, which will facilitate a more consistent, predictable, and standardized regulatory process, and help to ensure that geothermal energy continues to provide a reliable source of clean energy, and a pathway to achieving greater energy equity in the U.S.

Best Practices↗

A galactic approach to neutron scattering science

Neutron scattering science is leading to significant advances in our understanding of materials and will be key to solving many of the challenges that society is facing today. Improvements in scientific instruments are actually making it more difficult to analyze and interpret the results of experiments due to the vast increases in the volume and complexity of data being produced and the associated computational requirements for processing that data. New approaches to enable scientists to leverage computational resources are required, and Oak Ridge National Laboratory (ORNL) has been at the forefront of developing these technologies. We recently completed the design and initial implementation of a neutrons data interpretation platform that allows seamless access to the computational resources provided by ORNL. For the first time, we have demonstrated that this platform can be used for advanced data analysis of correlated quantum materials by utilizing the world's most powerful computer system, Frontier. In particular, we have shown the end-to-end execution of the DCA++ code to determine the dynamic magnetic spin susceptibility χ(q, ω) for a single-band Hubbard model with Coulomb repulsion U/t = 8 in units of the nearest-neighbor hopping amplitude t and an electron density of n = 0.65. The following work describes the architecture, design, and implementation of the platform and how we constructed a correlated quantum materials analysis workflow to demonstrate the viability of this system to produce scientific results.

97 MATHEMATICS AND COMPUTING↗

The crustal dynamics intelligent user interface anthology

The National Space Science Data Center (NSSDC) has initiated an Intelligent Data Management (IDM) research effort which has, as one of its components, the development of an Intelligent User Interface (IUI). The intent of the IUI is to develop a friendly and intelligent user interface service based on expert systems and natural language processing technologies. The purpose of such a service is to support the large number of potential scientific and engineering users that have need of space and land-related research and technical data, but have little or no experience in query languages or understanding of the information content or architecture of the databases of interest. This document presents the design concepts, development approach and evaluation of the performance of a prototype IUI system for the Crustal Dynamics Project Database, which was developed using a microcomputer-based expert system tool (M. 1), the natural language query processor THEMIS, and the graphics software system GSS. The IUI design is based on a multiple view representation of a database from both the user and database perspective, with intelligent processes to translate between the views.

Short, Nicholas M., Jr.↗

The National Center for Radioecology (NCoRE): A Network of Excellence for Environmental and Human Radiation Risk Reduction - 13365

Radioecology in the United States can be traced back to the early 1950's when small research programs were established to address the fate and effects of radionuclides released in the environment from activities at nuclear facilities. These programs focused primarily on local environmental effects, but global radioactive fallout from nuclear weapons testing and the potential for larger scale local releases of radioisotopes resulted in major concerns about the threat, not only to humans, but to other species and to ecosystems that support all life. These concerns were shared by other countries and it was quickly recognized that a multi-disciplinary approach would be required to address and understand the implications of anthropogenic radioactivity in the environment. The management, clean-up and long-term monitoring of legacy wastes at Department of Energy (DOE), Department of Defense (DOD), and Nuclear Regulatory Commission (NRC)-regulated facilities continues to be of concern as long as nuclear operations continue. Research conducted through radioecology programs provides the credible scientific data needed for decision-making purposes. The current status of radioecology programs in the United States are: fragmented with little coordination to identify national strategies and direct programs; suffering from a steadily decreasing funding base; soon to be hampered by closure of key infrastructure; hampered by aging and retiring workforce (loss of technical expertise); and in need of training of young scientists to ensure continuation of the science (no formal graduate education program in radioecology remaining in the U.S.). With these concerns in mind, the Savannah River National Laboratory (SRNL) took the lead to establish the National Center for Radioecology (NCoRE) as a network of excellence of the remaining radioecology expertise in the United States. As part of the NCoRE mission, scientists at SRNL are working with six key partner universities to re-establish a graduate education training program for radioecology. Recently, NCoRE hosted a workshop to identify the immediate needs for science-driven discoveries, tool development and the generation of scientific data to support the legislative decision-making process for remediation strategies, long-term monitoring of radiologically- contaminated sites and protection of human health and the environment. Some of the immediate strategic research needs were identified in the fields of functional genomics for determining low-dose effects, improved low-level dosimetry, and mixed (radiological and chemical) contaminant studies. Longer term strategic research and tool development areas included development of radioecology case study sites, comprehensive decision-making tools, consequence response actions, and optimized scenario based ecosystem modeling. A summary of the NCoRE workshop findings related to waste management needs and priority areas will be presented in this paper. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗