Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Advanced Workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

An advanced workflow for single-particle imaging with the limited data at an X-ray free-electron laser

An improved analysis for single-particle imaging (SPI) experiments, using the limited data, is presented here. Results are based on a study of bacteriophage PR772 performed at the Atomic, Molecular and Optical Science instrument at the Linac Coherent Light Source as part of the SPI initiative. Existing methods were modified to cope with the shortcomings of the experimental data: inaccessibility of information from half of the detector and a small fraction of single hits. The general SPI analysis workflow was upgraded with the expectation-maximization based classification of diffraction patterns and mode decomposition on the final virus-structure determination step. The presented processing pipeline allowed us to determine the 3D structure of bacteriophage PR772 without symmetry constraints with a spatial resolution of 6.9 nm. The obtained resolution was limited by the scattering intensity during the experiment and the relatively small number of single hits.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2014

Nationwide Buildings Energy Research enabled through an integrated Data Intensive Scientific Workflow and Advanced Analysis Environment

Modern workflow systems enable scientists to run ensemble simulations at unprecedented scales and levels of complexity, allowing them to study system sizes previously impossible to achieve, due to the inherent resource requirements needed for the modeling work. However as a result of these new capabilities the science teams suddenly also face unprecedented data volumes that they are unable to analyze with their existing tools and methodologies in a timely fashion. In this paper we will describe the ongoing development work to create an integrated data intensive scientific workflow and analysis environment that offers researchers the ability to easily create and execute complex simulation studies and provides them with different scalable methods to analyze the resulting data volumes. The integration of simulation and analysis environments is hereby not only a question of ease of use, but supports fundamental functions in the correlated analysis of simulation input, execution details and derived results for multi-variant, complex studies. To this end the team extended and integrated the existing capabilities of the Velo data management and analysis infrastructure, the MeDICi data intensive workflow system and RHIPE the R for Hadoop version of the well-known statistics package, as well as developing a new visual analytics interface for the result exploitation by multi-domain users. The capabilities of the new environment are demonstrated on a use case that focusses on the Pacific Northwest National Laboratory (PNNL) building energy team, showing how they were able to take their previously local scale simulations to a nationwide level by utilizing data intensive computing techniques not only for their modeling work, but also for the subsequent analysis of their modeling results. As part of the PNNL research initiative PRIMA (Platform for Regional Integrated Modeling and Analysis) the team performed an initial 3 year study of building energy demands for the US Eastern Interconnect domain, which they are now planning to extend to predict the demand for the complete century. The initial study raised their data demands from a few GBs to 400GB for the 3year study and expected tens of TBs for the full century.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Advanced Manufacturing Workflows for Tokamak Internal Components

The project aim was to create a new advanced manufacturing workflow for the design and creation of an internal component in a fusion device, enabling a low-cost, rapid design and fabrication cycle. As a result, two demonstration components were manufactured, and mechanical properties for two feedstock materials were tested in the context of fusion device conditions. This work shows a technological scoping toward future manufacturing of complex components in topologies relevant to magnetic fusion devices via additive manufacturing.

42 ENGINEERING↗
DOE OSTI2021

Emerging Frameworks for Advancing Scientific Workflows Research, Development, and Education

Lightning talks of the Workflows in Support of Large-Scale Science (WORKS) workshop are a venue where the workflow community (researchers, developers, and users) can discuss work in progress, emerging technologies and frameworks, and training and education materials. This paper summarizes the WORKS 2021 lightning talks, which cover four broad topics: (i) libEnsemble, a Python library to coordinate the concurrent evaluation of dynamic ensembles of calculations; (ii) Edu WRENCH, a set of online pedagogic modules that provides simulation-driven hands-on activity in the browser; (iii) VisDict, an envisioned visual dictionary framework that will translate terms, jargon, and concepts between research domains and workflow providers; and (iv) Pegasus Kickstart, a lightweight tool for capturing workflow tasks' performance, including performance metrics from Nvidia GPUs.

Casanova, Henri↗
DOE OSTI2025

Advanced multi-modal mass spectrometry imaging reveals functional differences of placental villous compartments at microscale resolution

The placenta is a complex and heterogeneous organ that links the mother and fetus, playing a crucial role in nourishing and protecting the fetus throughout pregnancy. Integrative spatial multi-omics approaches can provide a systems-level understanding of molecular changes underlying the mechanisms leading to the histological variations of the placenta during healthy pregnancy and pregnancy complications. Herein, we advance our metabolome-informed proteome imaging (MIPI) workflow to include lipidomic imaging, while also expanding the molecular coverage of metabolomic imaging by incorporating on-tissue chemical derivatization (OTCD). The improved MIPI workflow advances biomedical investigations by leveraging state-of-the-art molecular imaging technologies. Lipidome imaging identifies molecular differences between two morphologically distinct compartments of a placental villous functional unit, syncytiotrophoblast (STB) and villous core. Next, our advanced metabolome imaging maps villous functional units with enriched metabolomic activities related to steroid and lipid metabolism, outlining distinct molecular distributions across morphologically different villous compartments. Complementary proteome imaging on these villous functional units reveals a plethora of fatty acid- and steroid-related enzymes uniquely distributed in STB and villous core compartments. Integration across our advanced MIPI imaging modalities enables the reconstruction of active biological pathways of molecular synthesis and maternal-fetal signaling across morphologically distinct placental villous compartments with micrometer-scale resolution.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Software Framework for Federated Science Instruments

There is an unprecedented promise of enhanced capabilities for federations of leadership computing systems and experimental science facilities by leveraging software technologies for fast and efficient operations. These federations seek to unify different science instruments, both computing and experimental, to effectively support science users and operators to execute complex workflows. The FedScI project addresses the software challenges associated with the formation and operation of federated environments by leveraging recent advances in containerization of software and softwarization of hardware. We propose a software framework to streamline the federation usage by science users and it’s provisioning and operations by facility providers. A distinguishing element of our work is the support for improved interaction between experimental devices, such as beam-line instruments, and more traditional high-performance computing resources, including compute, network, storage systems. We present guiding principles for the software framework and highlight portions of a current prototype implementation. We describe our science use case involving neutron imaging beam-lines (SNAP/BL-3, Imaging/CG-1D) at the Spallation Neutron Source and High Flux Isotope Reactor facilities at Oak Ridge National Laboratory. Additionally, we detail plans for a more direct instrument interaction within a federated environment, which could enable more advanced workflows with feedback loops to shorten the time to science.

Naughton III, Thomas↗
DOE OSTI2024

Empowering Scientific Innovation Through An Integrated Research Infrastructure: The Role of the Advanced Computing Ecosystem

As the landscape of computational science evolves, the Department of Energy (DOE) is reimagining the roles of its large-scale computing facilities to meet emerging research challenges. The Integrated Research Infrastructure (IRI) program aims to transform how experiments are designed, conducted, and shared, with significant impacts on all stakeholders. In response, the Oak Ridge Leadership Computing Facility (OLCF) has established the Advanced Computing Ecosystem (ACE), a strategic framework to prepare its hardware, software, and experimental capabilities for the IRI era. ACE focuses on integrating novel compute environments, orchestrating advanced workflows, and developing foundational technologies, ensuring a seamless transition to IRI while accelerating scientific discovery. This paper outlines ACE's role in advancing OLCF's mission and its impact on the future of computational science.

Widener, Patrick↗
DOE OSTI2024

PanDA: Production and Distributed Analysis System

The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at the LHC data processing scale. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year. PanDA’s flexibility and scalability make it suitable for the High Energy Physics community and wider science domains at the Exascale. Beyond High Energy Physics, PanDA’s relevance extends to other big data sciences, as evidenced by its adoption in the Vera C. Rubin Observatory and the sPHENIX experiment. As the significance of advanced workflows continues to grow, PanDA has transformed into a comprehensive ecosystem, effectively tackling challenges associated with emerging workflows and evolving computing technologies. The paper discusses PanDA’s prominent role in the scientific landscape, detailing its architecture, functionality, deployment strategies, project management approaches, results, and evolution into an ecosystem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Navier: Dataflow Architecture for Computation Chemistry

Navier’s objectives were two evaluate the use of emerging technologies, especially dataflow accelerators, for high-performance computing (HPC) applications, specifically in the domain of chemistry, and to develop a prototype software stack to support such applications. Navier builds on capabilities previously developed by synergistic projects, such as PNNL Data Model Convergence (DMC) LDRD Hardware Advanced Workflows (HAW) and DuOMO, as well as DOE ARIAA. Throughout its 18 months, the Navier team developed new capabilities and artifacts at all levels of the HW/SW stack, provided a seamless way to integrate novel computing architectures (Sambanova SN10 and Xilinx Versal AI) into an existing software stack, developed chemistry workflows, data analytics tools, and HPC molecular dynamics workflows that leverage the developed stack and PNNL institutional investments in emerging architectures. Navier also explored the use of active learning to accelerate a computational chemistry workflow for organic molecules on PNNL Junction cluster (in collaboration with AMD/Xilinx). Navier developed tools, methodologies, and studies for hardware software co-design and (sparse) dataflow accelerators that are composable and can be used together or separately. These methodologies are now used in other projects, such as DOE AMAIS and HPDA. This report describes Navier’s achievement, the developed tools and methodologies, and the research findings and conclusions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)↗
DOE OSTI2015

Exploring Two Approaches for an End-to-End Scientific Analysis Workflow

The advance of the scientific discovery process is accomplished by the integration of independently-developed programs run on disparate computing facilities into coherent workflows usable by scientists who are not experts in computing. For such advancement, we need a system which scientists can use to formulate analysis workflows, to integrate new components to these workflows, and to execute different components on resources that are best suited to run those components. In addition, we need to monitor the status of the workflow as components get scheduled and executed, and to access the intermediate and final output for visual exploration and analysis. Finally, it is important for scientists to be able to share their workflows with collaborators. Moreover we have explored two approaches for such an analysis framework for the Large Synoptic Survey Telescope (LSST) Dark Energy Science Collaboration (DESC), the first one is based on the use and extension of Galaxy, a web-based portal for biomedical research, and the second one is based on a programming language, Python. In our paper, we present a brief description of the two approaches, describe the kinds of extensions to the Galaxy system we have found necessary in order to support the wide variety of scientific analysis in the cosmology community, and discuss how similar efforts might be of benefit to the HEP community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

58 GEOSCIENCES↗