Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

A Grassroots Network and Community Roadmap for Interconnected Autonomous Science Laboratories for Accelerated Discovery

Scientific discovery is being revolutionized by AI and autonomous systems, yet current autonomous laboratories remain isolated islands unable to collaborate across institutions. We present the Autonomous Interconnected Science Lab Ecosystem (AISLE), a grassroots network transforming fragmented capabilities into a unified system that shorten the path from ideation to innovation to impact and accelerates discovery from decades to months. AISLE addresses five critical dimensions: (1) cross-institutional equipment orchestration, (2) intelligent data management with FAIR compliance, (3) AI-agent driven orchestration grounded in scientific principles, (4) interoperable agent communication interfaces, and (5) AI/ML-integrated scientific education. By connecting autonomous agents across institutional boundaries, autonomous science can unlock research spaces inaccessible to traditional approaches while democratizing cutting-edge technologies. This paradigm shift toward collaborative autonomous science promises breakthroughs in sustainable energy, materials development, and public health.

Ferreira da Silva, Rafael [Oak Ridge National Labo↗

Biomedical and Human Factors Requirements for a Manned Earth Orbiting Station

The primary objective of this study is to determine which biomedical and human factors measurements must be made aboard a space station to assure adequate evaluation of the astronaut's health and performance during prolonged space flights. The study has employed, where possible, a medical and engineering systems analysis to define the pertinent life sciences and space station design parameters and their influence on a measurement program. The major areas requiring evaluation in meeting the study objectives include a definition of the space environment, man's response to the environment, selection of measurement and data management techniques, experimental program, space station design requirements, and a trade-off analysis with final recommendations. The space environment factors that are believed to have a significant effect on man were evaluated. This includes those factors characteristic of the space environment (e. g. weightlessness, radiation) as well as those created within the space station (e. g. toxic contaminants, capsule atmosphere). After establishing the general features of the environment, an appraisal was made of the anticipated response of the astronaut to each of these factors. For thoroughness, the major organ systems and functions of the body were delineated, and a determination was made of their anticipated response to each of the environmental categories. A judgment was then made on the medical significance or importance of each response, which enabled a determination of which physiological and psychological effects should be monitored. Concurrently, an extensive list of measurement techniques and methods of data management was evaluated for applicability to the space station program. The various space station configurations and design parameters were defined in terms of the biomedical and human factors requirements to provide the measurements program. Research design of experimental programs for various station configurations, mission durations, and crew sizes were prepared, and, finally, a trade-off analysis of the critical variables in the station planning was completed with recommendations to enhance the confidence in the measurement program.

EARTH ORBIT↗

Target diagnostic control system implementation for the National Ignition Facility (invited)

The extreme physics of targets shocked by NIF's 192-beam laser is observed by a diverse suite of diagnostics. Many diagnostics are being developed by collaborators at other sites, but ad hoc controls could lead to unreliable and costly operations. A diagnostic control system (DCS) framework for both hardware and software facilitates development and eases integration. Each complex diagnostic typically uses an ensemble of electronic instruments attached to sensors, digitizers, cameras, and other devices. In the DCS architecture each instrument is interfaced to a low-cost WINDOWS XP processor and JAVA application. Each instrument is aggregated with others as needed in the supervisory system to form an integrated diagnostic. The JAVA framework provides data management, control services, and operator graphical user interface generation. DCS instruments are reusable by replication with reconfiguration for specific diagnostics in extensible markup language. Advantages include minimal application code, easy testing, and high reliability. Collaborators save costs by assembling diagnostics with existing DCS instruments. This talk discusses target diagnostic instrumentation used on NIF and presents the DCS architecture and framework.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

AGR-2 Data Qualification Interim Report

Projects for the very high temperature reactor (VHTR) Technology Development Office program provide data in support of Nuclear Regulatory Commission licensing of the VHTR. Fuel and materials to be used in the reactor are tested and characterized to quantify performance in high temperature and high fluence environments. The VHTR program established the NGNP Data Management and Analysis System (NDMAS) to manage and document VHTR data qualification, for storage of the data in a readily accessible electronic form, and to assist in the analysis and presentation of the data. This document gives the status of NDMAS processing and qualification of data associated with the initial reactor cycle (147A) of the second Advanced Gas Reactor (AGR-2) experiment which began on June 21, 2010. Because it is early in the AGR-2 experiment, data from only two AGR-2 data streams are reported on: Fuel Fabrication and Fuel Irradiation data. As of August 1, 2010, approximately 311,000 irradiation data records have been stored in NDMAS, and qualification tests are in progress. Preliminary information indicates that TC 2 in Capsule 2 failed prior to start of the experiment, and NDMAS testing has thus far identified only two invalid data values from the METSO data collection system Data from the Fission Product Monitoring System (FPMS) are not currently processed until after reactor cycle shutdown and have not yet been received. A description of the ATR operating conditions data associated with the AGR-2 experiment (e.g., power levels) are summarized in the AGR-1 data qualification report (INL/EXT-09-16460). Since ATR data are collected under ATR program data quality requirements (i.e., outside the VHTR program), the NGNP program and NDMAS do not take additional actions to qualify these data other than NDMAS capture testing. Data qualification of graphite characterization data collected under the Graphite Technology Development Project is reported in a separate status report (Hull 2010).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

High-resolution Hindcasts for U.S. Wave Energy Resource Characterization

The marine and hydrokinetic (MHK) industry is at an early stage of development and has the potential to play a significant role in diversifying the U.S. energy portfolio and reducing the U.S. carbon footprint. Wave energy is the largest among all the U.S. MHK energy resources, which include wave energy, ocean current, tidal-instream, ocean thermal energy conversion, and river-instream. Wave resource characterization is an essential step for regional wave energy assessments, Wave Energy Converter (WEC) project development, site selection and WEC design. The present paper provides an overview of a joint modelling effort by the Pacific Northwest National Laboratory and Sandia National Laboratories on high-resolution wave hindcasts to support the U.S. Department of Energy’s Water Power Technologies Office’s program of wave resource characterization, assessment and classifications in all US coastal regions. Topics covered include the modelling approach, model input requirements, model validation strategies, high performance computing resource requirements, model outputs and data management strategies. Examples of model setup and validation for different regions are provided along with application to development of classification systems, and analysis of regional wave climates. Lessons learned and technical challenges of the long-term, high-resolution regional wave hindcast are discussed.

Yang, Zhaoqing↗

Pansophy data as used to develop metrics and evaluate trends across SRF projects and facilities to further quality improvement initiatives

Pansophy, a JLAB SRF engineering data management system (EDMS), is composed of a collection of technolo-gies that together provide for the collection, management and analysis of data for the production and testing of cavities and cryomodules. From its inception in 2000, when data collection was a priority for such projects as SNS, CEBAF 12GeV upgrade, LCLS-II, and in the future the SNS-PPU, the focus has turned to data analysis and reporting on quality metrics and key performance indica-tors (KPIs). Reporting enhancements include monthly quality metrics, project specific KPIs, and trending across projects. With the use of Pareto Charts to help analyze vendor quality and non-conformance, timelines of pro-ject and facility metrics, project managers and subject matter experts (SME) are able to look for trends and pre-pare further quality improvement initiatives for their projects.

McDonald, M. G.↗

An integrated assessment of location-dependent scaling for microalgae biofuel production facilities

Successful development of a large-scale microalgae-based biofuels industry requires comprehensive analysis and understanding of the feedstock supply chain—from facility siting and design through processing and upgrading of the feedstock to a fuel product. The evolution from pilot-scale production facilities to energy-scale operations presents many multi-disciplinary challenges, including a sustainable supply of water and nutrients, operational and infrastructure logistics, and economic competitiveness with petroleum-based fuels. These challenges are partially addressed by applying the Integrated Assessment Framework (IAF) – an integrated multi-scale modeling, analysis, and data management suite – to address key issues in developing and operating an open-pond microalgae production facility. This is done by analyzing how variability and uncertainty over space and through time affect feedstock production rates, and determining the site-specific “optimum” facility scale to minimize capital and operational expenses. This approach explicitly and systematically assesses the interdependence of biofuel production potential, associated resource requirements, and production system design trade-offs. To provide a baseline analysis, the IAF was applied in this paper to a set of sites in the southeastern U.S. with the potential to cumulatively produce 5 billion gallons per year. Finally, the results indicate costs can be reduced by scaling downstream processing capabilities to fit site-specific growing conditions, available and economically viable resources, and specific microalgal strains.

09 BIOMASS FUELS↗

Software Requirements for a Consequence Management Sample Data Simulator for Training and Drills

This document describes the requirements for a software tool that will enable FRMAC to simulate large sets of sample result data that is based realistically on simulated radionuclide deposition grids from NARAC. The user of this tool would be scientists involved in exercise and drill planning or part of the simulation cell of an exercise controller team. A key requirement is that this tool must be able to be run with a reasonable amount of training and job aids by any person within the Assessment, Laboratory Analysis, or Monitoring and Sampling divisions of the FRMAC to support any level of exercise from the small IPX to the national level full scale exercise. This tool should be relatively lean and stand-alone so that the user can run it in the field with limited IT resources. This document will describe the desired architecture, design characteristics, order of operations, and algorithms that can be given to a software development team to assist them in project scoping, costing, and eventually, development.

61 RADIATION PROTECTION AND DOSIMETRY↗

Management and Storage of Scientific Data

Scientific discoveries rely heavily on efficient access, search, and management of massive data sets. Data management technologies have, for decades, provided foundational capabilities for scientific computing. Just as storage, input/output (I/O), and data management have been fundamental to simulation-based science for many years, so too are capable data-management technologies key to the success of today’s scientific workflows utilizing data intensive and machine learning (ML) techniques. The Department of Energy, Office of Science, Advanced Scientific Computing Research (ASCR) program has invested broadly in data-management research focused on high-performance computing (HPC) systems, from parallel file systems that store data to application software that makes these systems more productive. Still, advances in technology combined with growing diversity of supported science strongly motivate continued investment in this area. In January 2022, ASCR convened a workshop to identify priority research directions in the area of data management for high-performance and scientific computing. Attendees were challenged to identify promising approaches that would support the breadth of the DOE mission, including the explosion of artificial intelligence (AI) uses and the growing needs of experimental and observational science. Technological and science drivers were identified and considered as they relate to key aspects of data management such as interfaces, architectural design, and FAIR principles (Findable, Accessible, Interoperable, and Reusable). The thoughts of the workshop participants were distilled into a set of four priority research directions with the potential for high impact on DOE science. These research directions are summarized in the following pages.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Coupling the System Analysis Module with SAS4A/SASSYS-1

SAS4A/SASSYS-1 is a simulation tool used to perform deterministic analysis of anticipated events as well as design basis and beyond design basis accidents for advanced reactors, with an emphasis on sodium fast reactors. SAS4A/SASSYS-1 has been under development and in active use for nearly forty-five years, and is currently maintained by the U.S. Department of Energy under the Office of Advanced Reactor Technology. Although SAS4A/SASSYS-1 contains a very capable primary and intermediate system modeling component, PRIMAR-4, it also has some shortcomings: outdated data management and code structure makes extension of the PRIMAR-4 module somewhat difficult. The user input format for PRIMAR-4 also limits the number of volumes and segments that can be used to describe a given system. The System Analysis Module (SAM) is a fairly new code development effort being carried out under the U.S. DOE Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. SAM is being developed with advanced physical models, numerical methods, and software engineering practices; however, it is currently somewhat limited in the system components and phenomena that can be represented. For example, component models for electromagnetic pumps and multi-layer stratified volumes have not yet been developed. Nor is there support for a balance of plant model. Similarly, system-level phenomena such as control-rod driveline expansion and vessel elongation are not represented. This report documents fiscal year 2016 work that was carried out to couple the transient safety analysis capabilities of SAS4A/SASSYS-1 with the system modeling capabilities of SAM under the joint support of the ART and NEAMS programs. The coupling effort was successful and is demonstrated by evaluating an unprotected loss of flow transient for the Advanced Burner Test Reactor (ABTR) design. There are differences between the stand-alone SAS4A/SASSYS-1 simulations and the coupled SAS/SAM simulations, but these are mainly attributed to the limited maturity of the SAM development effort. The severe accident modeling capabilities in SAS4A/SASSYS-1 (sodium boiling, fuel melting and relocation) will continue to play a vital role for a long time. Therefore, the SAS4A/SASSYS-1 modernization effort should remain a high priority task under the ART program to ensure continued participation in domestic and international SFR safety collaborations and design optimizations. On the other hand, SAM provides an advanced system analysis tool, with improved numerical solution schemes, data management, code flexibility, and accuracy. SAM is still in early stages of development and will require continued support from NEAMS to fulfill its potential and to mature into a production tool for advanced reactor safety analysis. The effort to couple SAS4A/SASSYS-1 and SAM is the first step on the integration of these modeling capabilities.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Building Simulation Modelers are we big-data ready?

Recent advances in computing and sensor technologies have pushed the amount of data we collect or generate to limits previously unheard of. Sub-minute resolution data from dozens of channels is becoming increasingly common and is expected to increase with the prevalence of non-intrusive load monitoring. Experts are running larger building simulation experiments and are faced with an increasingly complex data set to analyze and derive meaningful insight. This paper focuses on the data management challenges that building modeling experts may face in data collected from a large array of sensors, or generated from running a large number of building energy/performance simulations. The paper highlights the technical difficulties that were encountered and overcome in order to run 3.5 million EnergyPlus simulations on supercomputers and generating over 200 TBs of simulation output. This extreme case involved development of technologies and insights that will be beneficial to modelers in the immediate future. The paper discusses different database technologies (including relational databases, columnar storage, and schema-less Hadoop) in order to contrast the advantages and disadvantages of employing each for storage of EnergyPlus output. Scalability, analysis requirements, and the adaptability of these database technologies are discussed. Additionally, unique attributes of EnergyPlus output are highlighted which make data-entry non-trivial for multiple simulations. Practical experience regarding cost-effective strategies for big-data storage is provided. The paper also discusses network performance issues when transferring large amounts of data across a network to different computing devices. Practical issues involving lag, bandwidth, and methods for synchronizing or transferring logical portions of the data are presented. A cornerstone of big-data is its use for analytics; data is useless unless information can be meaningfully derived from it. In addition to technical aspects of managing big data, the paper details design of experiments in anticipation of large volumes of data. The cost of re-reading output into an analysis program is elaborated and analysis techniques that perform analysis in-situ with the simulations as they are run are discussed. The paper concludes with an example and elaboration of the tipping point where it becomes more expensive to store the output than re-running a set of simulations.

Sanyal, Jibonananda↗

A Survey on Sustainable Software Ecosystems to Support Experimental and Observational Science at Oak Ridge National Laboratory

In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. The study was conducted among 38 ORNL participants between June and July of 2021 and followed the required guidelines for human subjects training. We plan to use the collected information to help guide a vision for sustainable, community-based, and reusable scientific software ecosystems that need to adapt effectively to: i) the evolving landscape of heterogeneous hardware in the next generation of instruments and computing (e.g. edge, distributed, accelerators), and ii) data management requirements for data-driven science using artificial intelligence.

Bernholdt, David↗

Experiential findings for sustainable software ecosystems to support experimental and observational science

In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. Here, the study was conducted among 38 ORNL participants between June and July of 2021 and followed the required guidelines for human subjects training. We plan to use the collected information to help guide a vision for sustainable, community-based, and reusable scientific software ecosystems that need to adapt effectively to: (i) the evolving landscape of heterogeneous hardware in the next generation of instruments and computing (e.g. edge, distributed, accelerators), and (ii) data management requirements for data-driven science using artificial intelligence.

97 MATHEMATICS AND COMPUTING↗

Virtual machine provisioning, code management, and data movement design for the Fermilab HEPCloud Facility

The Fermilab HEPCloud Facility Project has as its goal to extend the current Fermilab facility interface to provide transparent access to disparate resources including commercial and community clouds, grid federations, and HPC centers. This facility enables experiments to perform the full spectrum of computing tasks, including data-intensive simulation and reconstruction. We have evaluated the use of the commercial cloud to provide elasticity to respond to peaks of demand without overprovisioning local resources. Full scale data-intensive workflows have been successfully completed on Amazon Web Services for two High Energy Physics Experiments, CMS and NOνA, at the scale of 58000 simultaneous cores. This paper describes the significant improvements that were made to the virtual machine provisioning system, code caching system, and data movement system to accomplish this work. The virtual image provisioning and contextualization service was extended to multiple AWS regions, and to support experiment-specific data configurations. A prototype Decision Engine was written to determine the optimal availability zone and instance type to run on, minimizing cost and job interruptions. We have deployed a scalable on-demand caching service to deliver code and database information to jobs running on the commercial cloud. It uses the frontiersquid server and CERN VM File System (CVMFS) clients on EC2 instances and utilizes various services provided by AWS to build the infrastructure (stack). We discuss the architecture and load testing benchmarks on the squid servers. We also describe various approaches that were evaluated to transport experimental data to and from the cloud, and the optimal solutions that were used for the bulk of the data transport. Finally, we summarize lessons learned from this scale test, and our future plans to expand and improve the Fermilab HEP Cloud Facility.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

METAREP: JCVI metagenomics reports—an open source tool for high-performance comparative metagenomics

JCVI Metagenomics Reports (METAREP) is a Web 2.0 application designed to help scientists analyze and compare annotated metagenomics datasets. It utilizes Solr/Lucene, a high performance scalable search engine, to quickly query large data collections. Furthermore, users can use its SQL-like query syntax to filter and refine datasets. METAREP provides graphical summaries for top taxonomic and functional classifications as well as a GO, NCBI Taxonomy and KEGG Pathway Browser. Users can compare absolute and relative counts of multiple datasets at various functional and taxonomic levels. Advanced comparative features comprise statistical tests as well as multidimensional scaling, heatmap and hierarchical clustering plots. Summaries can be exported as tab-delimited files, publication quality plots in PDF format. A data management layer allows collaborative data analysis and result sharing.

59 BASIC BIOLOGICAL SCIENCES↗

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗

DataFed

There is a need to easily and uniformly capture, access, enrich, organize, search, and collaborate on working scientific data sets across geographically distributed experiment, computation, and data analytics facilities. DataFed provides a set of services and applications enabling simple and uniform cross-facility scientific data management with federated (distributed) data repositories. This solution greatly reduces effort required by scientist to organize and/or move data between facilities/institutes, Reduces likelihood of misidentification and subsequent misuse of data, and improves scientific reproducibility through deep, domain-specific context and provenance capture.

Stansberry, DaleV.↗