Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Correlative visualization techniques for multidimensional data

Critical to the understanding of data is the ability to provide pictorial or visual representation of those data, particularly in support of correlative data analysis. Despite the advancement of visualization techniques for scientific data over the last several years, there are still significant problems in bringing today's hardware and software technology into the hands of the typical scientist. For example, there are other computer science domains outside of computer graphics that are required to make visualization effective such as data management. Well-defined, flexible mechanisms for data access and management must be combined with rendering algorithms, data transformation, etc. to form a generic visualization pipeline. A generalized approach to data visualization is critical for the correlative analysis of distinct, complex, multidimensional data sets in the space and Earth sciences. Different classes of data representation techniques must be used within such a framework, which can range from simple, static two- and three-dimensional line plots to animation, surface rendering, and volumetric imaging. Static examples of actual data analyses will illustrate the importance of an effective pipeline in data visualization system.

Treinish, Lloyd A.↗

HDF5-Version 1.12.0

The HDF5 library and file format implement the HDF5 data model for storing and managing data. This implementation supports an unlimited variety of datatypes and is designed for flexible and efficient I/O and for high volume and complex data. It is highly portable and extensible, allowing applications to evolve in their use of HDF5. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format and other implementations.

The HDF Group, None↗

Jackson State University's Center for Spatial Data Research and Applications: New facilities and new paradigms

Jackson State University recently established the Center for Spatial Data Research and Applications, a Geographical Information System (GIS) and remote sensing laboratory. Taking advantage of new technologies and new directions in the spatial (geographic) sciences, JSU is building a Center of Excellence in Spatial Data Management. New opportunities for research, applications, and employment are emerging. GIS requires fundamental shifts and new demands in traditional computer science and geographic training. The Center is not merely another computer lab but is one setting the pace in a new applied frontier. GIS and its associated technologies are discussed. The Center's facilities are described. An ARC/INFO GIS runs on a Vax mainframe, with numerous workstations. Image processing packages include ELAS, LIPS, VICAR, and ERDAS. A host of hardware and software peripheral are used in support. Numerous projects are underway, such as the construction of a Gulf of Mexico environmental data base, development of AI in image processing, a land use dynamics study of metropolitan Jackson, and others. A new academic interdisciplinary program in Spatial Data Management is under development, combining courses in Geography and Computer Science. The broad range of JSU's GIS and remote sensing activities is addressed. The impacts on changing paradigms in the university and in the professional world conclude the discussion.

Davis, Bruce E.↗

Requirements for company-wide management

Computing system requirements were developed for company-wide management of information and computer programs in an engineering data processing environment. The requirements are essential to the successful implementation of a computer-based engineering data management system; they exceed the capabilities provided by the commercially available data base management systems. These requirements were derived from a study entitled The Design Process, which was prepared by design engineers experienced in development of aerospace products.

Southall, J. W.↗

DC-magnetic field vector measurement

A magnetometer experiment was designed to determine the local magnetic field by measuring the total of the Earth's magnetic field and that of an unknown spacecraft. The measured field vector components are available to all onboard experiments via the Spacelab command and data management system. The experiment consists of two parts, an electronic box and the magnetic field sensor. The sensor includes three independent measuring flux-gate magnetometers, each measuring one component. The physical background is the nonlinearity of the B-H curve of a ferrite material. Two coils wound around a ferrite rod are necessary. One of them, a tank coil, pumps the ferrite rod at approximately 20 kilohertz. As a consequence of the nonlinearity, many harmonics can be produced. The second coil (i.e., the detection coil) resonates to the first harmonic. If an unknown dc or low-frequency magnetic field exists, the amplitude of the first harmonic is a measure for the unknown magnetic field. The voltages detected by the sensors are to be digitized and transferred to the command and data management system.

Schmidt, R.↗

A Grassroots Network and Community Roadmap for Interconnected Autonomous Science Laboratories for Accelerated Discovery

Scientific discovery is being revolutionized by AI and autonomous systems, yet current autonomous laboratories remain isolated islands unable to collaborate across institutions. We present the Autonomous Interconnected Science Lab Ecosystem (AISLE), a grassroots network transforming fragmented capabilities into a unified system that shorten the path from ideation to innovation to impact and accelerates discovery from decades to months. AISLE addresses five critical dimensions: (1) cross-institutional equipment orchestration, (2) intelligent data management with FAIR compliance, (3) AI-agent driven orchestration grounded in scientific principles, (4) interoperable agent communication interfaces, and (5) AI/ML-integrated scientific education. By connecting autonomous agents across institutional boundaries, autonomous science can unlock research spaces inaccessible to traditional approaches while democratizing cutting-edge technologies. This paradigm shift toward collaborative autonomous science promises breakthroughs in sustainable energy, materials development, and public health.

Ferreira da Silva, Rafael [Oak Ridge National Labo↗

Biomedical and Human Factors Requirements for a Manned Earth Orbiting Station

The primary objective of this study is to determine which biomedical and human factors measurements must be made aboard a space station to assure adequate evaluation of the astronaut's health and performance during prolonged space flights. The study has employed, where possible, a medical and engineering systems analysis to define the pertinent life sciences and space station design parameters and their influence on a measurement program. The major areas requiring evaluation in meeting the study objectives include a definition of the space environment, man's response to the environment, selection of measurement and data management techniques, experimental program, space station design requirements, and a trade-off analysis with final recommendations. The space environment factors that are believed to have a significant effect on man were evaluated. This includes those factors characteristic of the space environment (e. g. weightlessness, radiation) as well as those created within the space station (e. g. toxic contaminants, capsule atmosphere). After establishing the general features of the environment, an appraisal was made of the anticipated response of the astronaut to each of these factors. For thoroughness, the major organ systems and functions of the body were delineated, and a determination was made of their anticipated response to each of the environmental categories. A judgment was then made on the medical significance or importance of each response, which enabled a determination of which physiological and psychological effects should be monitored. Concurrently, an extensive list of measurement techniques and methods of data management was evaluated for applicability to the space station program. The various space station configurations and design parameters were defined in terms of the biomedical and human factors requirements to provide the measurements program. Research design of experimental programs for various station configurations, mission durations, and crew sizes were prepared, and, finally, a trade-off analysis of the critical variables in the station planning was completed with recommendations to enhance the confidence in the measurement program.

EARTH ORBIT↗

High-resolution Hindcasts for U.S. Wave Energy Resource Characterization

The marine and hydrokinetic (MHK) industry is at an early stage of development and has the potential to play a significant role in diversifying the U.S. energy portfolio and reducing the U.S. carbon footprint. Wave energy is the largest among all the U.S. MHK energy resources, which include wave energy, ocean current, tidal-instream, ocean thermal energy conversion, and river-instream. Wave resource characterization is an essential step for regional wave energy assessments, Wave Energy Converter (WEC) project development, site selection and WEC design. The present paper provides an overview of a joint modelling effort by the Pacific Northwest National Laboratory and Sandia National Laboratories on high-resolution wave hindcasts to support the U.S. Department of Energy’s Water Power Technologies Office’s program of wave resource characterization, assessment and classifications in all US coastal regions. Topics covered include the modelling approach, model input requirements, model validation strategies, high performance computing resource requirements, model outputs and data management strategies. Examples of model setup and validation for different regions are provided along with application to development of classification systems, and analysis of regional wave climates. Lessons learned and technical challenges of the long-term, high-resolution regional wave hindcast are discussed.

Yang, Zhaoqing↗

Software Requirements for a Consequence Management Sample Data Simulator for Training and Drills

This document describes the requirements for a software tool that will enable FRMAC to simulate large sets of sample result data that is based realistically on simulated radionuclide deposition grids from NARAC. The user of this tool would be scientists involved in exercise and drill planning or part of the simulation cell of an exercise controller team. A key requirement is that this tool must be able to be run with a reasonable amount of training and job aids by any person within the Assessment, Laboratory Analysis, or Monitoring and Sampling divisions of the FRMAC to support any level of exercise from the small IPX to the national level full scale exercise. This tool should be relatively lean and stand-alone so that the user can run it in the field with limited IT resources. This document will describe the desired architecture, design characteristics, order of operations, and algorithms that can be given to a software development team to assist them in project scoping, costing, and eventually, development.

61 RADIATION PROTECTION AND DOSIMETRY↗

Management and Storage of Scientific Data

Scientific discoveries rely heavily on efficient access, search, and management of massive data sets. Data management technologies have, for decades, provided foundational capabilities for scientific computing. Just as storage, input/output (I/O), and data management have been fundamental to simulation-based science for many years, so too are capable data-management technologies key to the success of today’s scientific workflows utilizing data intensive and machine learning (ML) techniques. The Department of Energy, Office of Science, Advanced Scientific Computing Research (ASCR) program has invested broadly in data-management research focused on high-performance computing (HPC) systems, from parallel file systems that store data to application software that makes these systems more productive. Still, advances in technology combined with growing diversity of supported science strongly motivate continued investment in this area. In January 2022, ASCR convened a workshop to identify priority research directions in the area of data management for high-performance and scientific computing. Attendees were challenged to identify promising approaches that would support the breadth of the DOE mission, including the explosion of artificial intelligence (AI) uses and the growing needs of experimental and observational science. Technological and science drivers were identified and considered as they relate to key aspects of data management such as interfaces, architectural design, and FAIR principles (Findable, Accessible, Interoperable, and Reusable). The thoughts of the workshop participants were distilled into a set of four priority research directions with the potential for high impact on DOE science. These research directions are summarized in the following pages.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

A Survey on Sustainable Software Ecosystems to Support Experimental and Observational Science at Oak Ridge National Laboratory

In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. The study was conducted among 38 ORNL participants between June and July of 2021 and followed the required guidelines for human subjects training. We plan to use the collected information to help guide a vision for sustainable, community-based, and reusable scientific software ecosystems that need to adapt effectively to: i) the evolving landscape of heterogeneous hardware in the next generation of instruments and computing (e.g. edge, distributed, accelerators), and ii) data management requirements for data-driven science using artificial intelligence.

Bernholdt, David↗

Experiential findings for sustainable software ecosystems to support experimental and observational science

In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. Here, the study was conducted among 38 ORNL participants between June and July of 2021 and followed the required guidelines for human subjects training. We plan to use the collected information to help guide a vision for sustainable, community-based, and reusable scientific software ecosystems that need to adapt effectively to: (i) the evolving landscape of heterogeneous hardware in the next generation of instruments and computing (e.g. edge, distributed, accelerators), and (ii) data management requirements for data-driven science using artificial intelligence.

97 MATHEMATICS AND COMPUTING↗

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗

Data from Managing Flowering Time in Miscanthus and Sugarcane to Facilitate Intra- and Intergeneric Crosses

Miscanthus is a close relative of saccharum and a potentially valuable genetic resource for improving sugarcane. Differences in flowering time within and between miscanthus and saccharum hinders intra- and interspecific hybridizations. A series of greenhouse experiments were conducted over three years to determine how to synchronize flowering time of saccharum and miscanthus genotypes. We found that day length was an important factor influencing when miscanthus and saccharum flowered. Sugarcane could be induced to flower in a central Illinois greenhouse using supplemental lighting to reduce the rate at which days shortened during the autumn and winter to 1 min d-1, which allowed us to synchronize the flowering of some sugarcane genotypes with Miscanthus genotypes primarily from low latitudes. In a complementary growth chamber experiment, we evaluated 33 miscanthus genotypes, including 28 M. sinensis , 2 M. floridulus , and 3 M. ×giganteus collected from 20.9° S to 44.9° N for response to three day lengths (10 h, 12.5 h, and 15 h). High latitude-adapted M. sinensis flowered mainly under 15 h days, but unexpectedly, short days resulted in short, stocky plants that did not flower; in some cases, flag leaves developed under short days but heading did not occur. In contrast, for M. sinensis and M. floridulus from low latitudes, shorter day lengths typically resulted in earlier flowering, and for some low latitude genotypes, 15 h days resulted in no flowering. However, the highest ratio of reproductive shoots to total number of culms was typically observed for 12.5 h or 15 h days. Latitude of origin was significantly associated with culm length, and the shorter the days, the stronger the relationship. Nearly all entries achieved maximal culm length under the 15 h treatment, but the nearer to the equator an accession originated, the less of a difference in culm length between the short-day treatments and the 15 h day treatment. Under short days, short culms for high-latitude accessions was achieved by different physiological mechanisms for M. sinensis genetic groups from the mainland in comparison to those from Japan; for mainland accessions, the mechanism was reduced internode length, whereas for Japanese accessions the phyllochron under short days was greater than under long days. Thus, for M. sinensis , short days typically hastened floral induction, consistent with the expectations for a facultative short-day plant. However, for high latitude accessions of M. sinensis , days less than 12.5 h also signaled that plants should prepare for winter by producing many short culms with limited elongation and development; moreover, this response was also epistatic to flowering. Thus, to flower M. sinensis that originates from high latitudes synchronously with sugarcane, the former needs day lengths >12.5 h (perhaps as high as 15 h), whereas that the latter needs day lengths <12.5 h.

Feedstock Production↗

Research Software Engineering Efforts for DataFed: FY2021 Developments

DataFed is a scientific data management system for big data providing simple and uniform data access, organization, discovery, and sharing within and across scientific facilities - with the goals of enhancing productivity and scientific reproducibility. We hope to aid in communicating development contributions to DataFed over the last year and any planning we can provide for the future developments in the next year

42 ENGINEERING↗

Blockchain Empowered Provenance Framework for Sensor Identity Management and Data Flow Security in Fossil-based Power Plants

The overall goals of this project are divided into 3 important phases. First, a peer-to-peer SCADA network is to be established where identity profile of sensors from fossil-power plants will be stored in an immutable manner in order to verify data flow integrity in real time. Second, it is aimed to develop a Blockchain-based provenance platform to audit equipment operations along with data and process integrity violations. In this phase, various operational rules will be incorporated in the ledger on behalf of the operating sensor so its abnormal use can be easily detected. In addition, it is aimed to achieve provenance when the rate of transactions is exceedingly high, thus enabling to design a scalable provenance scheme that can work in real time when number of data collecting points or sensor units increase in fossil-based power plants. Finally, a working testbed will be developed to integrate the Blockchain platform with sensor-based fossil power plant comprising of Remote Terminal Units (RTU) and SCADA devices. Using multiple Raspberry-PIs mimicking the role of RTUs that are interfaced with multiple sensors devices will be deployed to serve as a critical component of fossil power plant and the performance metrics such as throughput, latency, bootstrapping time, etc., will be evaluated on the testbed.

20 FOSSIL-FUELED POWER PLANTS↗