Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Workflow management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

PATHA: Performance Analysis Tool for HPC Applications

Large science projects rely on complex workflows to analyze terabytes or petabytes of data. These jobs are often running over thousands of CPU cores and simultaneously performing data accesses, data movements, and computation. It is difficult to identify bottlenecks or to debug the performance issues in these large workflows. In order to address these challenges, we have developed Performance Analysis Tool for HPC Applications (PATHA) using the state-of-art open source big data processing tools. Our framework can ingest system logs to extract key performance measures, and apply the most sophisticated statistical tools and data mining methods on the performance data. Furthermore, it utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of PATHA, we conduct a case study on the workflows from an astronomy project known as the Palomar Transient Factory (PTF). This study processed 1.6 TB of system logs collected on the NERSC supercomputer Edison. Using PATHA, we were able to identify performance bottlenecks, which reside in three tasks of PTF workflow with the dependency on the density of celestial objects.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2018

dV/dt - Accelerating the Rate of Progress towards Extreme Scale Collaborative Science

This report introduces publications that report the results of a project that aimed to design a computational framework that enables computational experimentation at scale while supporting the model of “submit locally, compute globally”. The project focuses on estimating application resource needs, finding the appropriate computing resources, acquiring those resources,deploying the applications and data on the resources, managing applications and resources during run.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Use of Geriatric Assessment Tools in Selecting Therapies in Women Aged ≥70 Years With Hormone Receptor–Positive Early-Stage Breast Cancer: Preliminary Experience With a Quality Improvement Initiative

Purpose: To develop a multidisciplinary algorithmic approach to management of women aged ≥70 years with clinically staged T1N0 hormone receptor–positive breast cancer, including geriatric assessments predicting life expectancy and the likelihood of functional decline in the near future, in the context of a program-wide quality improvement initiative, to better select patients for therapeutic interventions. Methods and Materials: Two geriatric assessment tools, the Combined Lee-Schonberg Index and the Vulnerable Elderly Scale, were introduced into our clinical workflow to predict long-term mortality and likelihood of functional decline. Scores from these tools, along with patient preferences and clinical features, were incorporated into a preoperative algorithm addressing the use of sentinel lymph node biopsy (SLNB), and a postoperative algorithm addressing the use of adjuvant radiation therapy (RT). Results: The algorithms were approved for use in August 2015. Twenty-four patients were identified by in-clinic screening and have been managed using the algorithms as a guide. Mean patient age was 80 years (range, 71-89 years). Per the preoperative algorithm, consideration of omission of SLNB was an option in 11 of 24 patients (46%), and in total 18 of 24 (75%) opted against SLNB. Per the postsurgical algorithm, consideration of omission of adjuvant RT was an option for 19 of 24 patients (79%), and in total 17 of 24 (71%) opted to forego RT. Conclusion: Incorporation of simple geriatric assessments seems to have had a marked impact on decision making regarding both surgical and adjuvant therapies for women aged ≥70 years with early-stage hormone-positive breast cancer compared with historical patterns, with ≥71% omission of both SLNB and adjuvant RT in patients managed according to an institutional quality improvement initiative.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2016

Sci-Fri AM: Quality, Safety, and Professional Issues 06: An Evaluation of Incident Reporting and Learning using the Canadian National System for Incident Reporting – Radiation Treatment

We report on the development and clinical deployment of an in-house incident reporting and learning system that implements the taxonomy of the Canadian National System for Incident Reporting – Radiation Treatment (NSIR-RT). In producing our new system, we aimed to: Analyze actual incidents, as well as potentially dangerous latent conditions. Produce recommendations on the NSIR-RT taxonomy. Incorporate features to divide reporting responsibility among clinical staff and expedite incident categorization within the NSIR-RT framework. Share anonymized incident data with the national database. Our multistep incident reporting workflow is focused around an initial report and a detailed follow-up investigation. An investigator, chosen at the time of reporting, is tasked with performing the investigation. The investigation feature is connected to our electronic medical records database to allow automatic field population and quick reference of patient and treatment information. Additional features include a robust visualization suite, as well as the ability to flag incidents for discussion at monthly Risk Management meetings and task ameliorating actions to staff. Our system was deployed into clinical use in January 2016. Over the first three months of use, 45 valid incidents were reported; 31 of which were reported as actual incidents as opposed to near-misses or reportable circumstances. However, we suspect there is ambiguity within our centre in determining the appropriate event type, which may be arising from the taxonomy itself. Preliminary trending analysis aided in revealing workflow issues pertaining to storage of treatment accessories and treatment planning delays. Extensive analysis will be undertaken as more data are accrued.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

A deep learning-based workflow for fast prediction of 3D state variables in geological carbon storage: A dimension reduction approach

Deep learning (DL) models are extensively used as surrogate models for high-fidelity simulations of multiphase fluid flow in porous media at large scales, enabling fast forecasts of the spatial–temporal evolution of three-dimensional (3D) state variables in geological carbon storage (GCS). However, training these models in high-dimensional space remains computationally demanding and prone to overfitting because of limited training data. This paper presents a novel workflow to address these challenges by integrating dimension reduction (DR) methods. Here, the proposed workflow employed pre-trained DR models to extract the latent variables of geological models and state variables and utilized the multi-layer perceptron (MLP) for constructing mapping functions between the input and output variables in latent spaces. Subsequently, the pre-trained reconstruction models converted the MLP-predicted latent state variables to their original high-dimensional form. Furthermore, we proposed a novel strategy for the DR and reconstruction of 3D saturation fields to account for the unique data characteristics of sparsity, nonuniformity, and discontinuity. The proposed strategy applied PCA and inverse PCA for 2D average saturation fields and developed a DL-based 3D reconstruction model, leveraging three 2D average saturation fields as input to produce a 3D saturation field as output. The pre-training of DR and reconstruction models and training of MLP models were conducted on 84 Gulf of Mexico (GoM) simulations and evaluated on 12 testing simulations. Each simulation contained 720 monthly time steps, with the first 360 months as the injection period and the rest as the post-injection period. The proposed workflow, incorporating DR and DL models, accurately predicts the normalized 3D pressure fields, achieving mean square error (MSE) of 2.92 × 10 -7 compared to the ground truth obtained from a full-physics simulator. Furthermore, the proposed strategy outperformed PCA and convolutional autoencoder (CAE) models on 3D saturation fields, resulting in minor workflow prediction errors with an MSE of 2.93 × 10 -5 . The results suggest the proposed workflow provides sufficient predictive fidelity across temporal and spatial scales, and enables a speedup of 160 times compared to the full-physics simulator, facilitating improved decision-making and risk assessment for large-scale GCS management in real-time scenarios.

3D reconstruction model↗
DOE OSTI2019

A Limited-In-Scope Comparison of Subsidence Scenarios For 3D Vadose Zone PORFLOW Trench Models

This technical memorandum summarizes the results from two separate sequences of 3D PORFLOW vadose zone flow and contaminant transport models that were used to test the impact of different spatial distributions of subsidence boundary conditions. More specifically, the two simulation setups used a general conceptual model for an engineered trench (ET): one with a subsided region specified in the center of the trench and one with a subsided region located at the end of the trench. Both subsided regions had the same geometric specifications (e.g., length, width, orientation), but used water infiltration rate boundary conditions that appropriately accounted for surface runoff and lateral drainage from the adjacent upslope intact regions of the final closure cap. Because of the substantially higher water infiltration rate associated with the subsided region at the end of the trench, this geometry produced higher absolute peak fluxes to the water table for all six radionuclides that were simulated. However, the slower release rate of the centrally located subsided hole geometry produced a higher flux to the water table in the time period subsequent to the absolute peak. A third model was implemented as a control, where the water infiltration rate that corresponds to the centrally located subsided region was applied at the end of the trench to highlight the purely geometric effects. The results show that a higher water infiltration rate produces a sharper peak, while a lower infiltration rate leads to a broader peak. A centrally located hole impacts more of the waste zone, producing a higher peak when an equal infiltration rate is applied. These generalized qualitative differences, and their impacts on the overall modeling workflow (i.e., from vadose zone models to limits and dose calculations), should receive appropriate consideration during the development of conceptual models.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

32 examples of LLM applications in materials science and chemistry: towards automation, assistants, agents, and accelerated scientific discovery

Abstract Large language models (LLMs) are reshaping many aspects of materials science and chemistry research, enabling advances in molecular property prediction, materials design, scientific automation, knowledge extraction, and more. Recent developments demonstrate that the latest class of models are able to integrate structured and unstructured data, assist in hypothesis generation, and streamline research workflows. To explore the frontier of LLM capabilities across the research lifecycle, we review applications of LLMs through 32 total projects developed during the second annual LLM hackathon for applications in materials science and chemistry, a global hybrid event. These projects spanned seven key research areas: (1) molecular and material property prediction, (2) molecular and material design, (3) automation and novel interfaces, (4) scientific communication and education, (5) research data management and automation, (6) hypothesis generation and evaluation, and (7) knowledge extraction and reasoning from the scientific literature. Collectively, these applications illustrate how LLMs serve as versatile predictive models, platforms for rapid prototyping of domain-specific tools, and much more. In particular, improvements in both open source and proprietary LLM performance through the addition of reasoning, additional training data, and new techniques have expanded effectiveness, particularly in low-data environments and interdisciplinary research. As LLMs continue to improve, their integration into scientific workflows presents both new opportunities and new challenges, requiring ongoing exploration, continued refinement, and further research to address reliability, interpretability, and reproducibility.

Computer Science↗
DOE OSTI2018

Enforcing End-to-end I/O Policies for Scientific Workflows using Software-Defined Storage Resource Enclaves

Data-intensive knowledge discovery requires scientific applications to run concurrently with analytics and visualization codes executing in situ for timely output inspection and knowledge extraction. Consequently, I/O pipelines of scientific workflows can be long and complex because they comprise many stages of analytics across different layers of the I/O stack of high-performance computing systems. Performance limitations at any I/O layer or stage can cause an I/O bottleneck resulting in greater than expected end-to-end I/O latency. In this paper, we present the design and implementation of a novel data management infrastructure called Software-Defined Storage Resource Enclaves (SIREN) at system level to enforce end-to-end policies that dictate an I/O pipeline's performance. SIREN provides an I/O performance interface for users to specify the desired storage resources in the context of in-situ analytics. If suboptimal performance of analytics is caused by an I/O bottleneck when data are transferred between simulations and analytics, schedulers in different layers of the I/O stack automatically provide the guaranteed lower bounds on I/O throughput. Lastly, our experimental results demonstrate that SIREN provides performance isolation among scientific workflows sharing multiple storage servers across two I/O layers (burst buffer and parallel file systems) while maintaining high system scalability and resource utilization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

MURMoT: Design and Application of Microbial Uranium Reduction Monitoring Tools

The overarching project goal of the MURMoT project was the design of tools to elucidate the presence, abundance, dynamics, spatial distribution, and activity of metal- and radionuclide-transforming bacteria. To accomplish these objectives, an integrated approach that combined nucleic acid-based tools, proteomic workflows, uranium isotope measurements, and U(IV) speciation and structure analyses using the Advanced Photon Source (APS) at Argonne National Laboratory was developed.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2015

Data Intensive Scientific Workflows on a Federated Cloud: CRADA Final Report

The Fermilab Scientific Computing Division and the KISTI Global Science Experimental Data Hub Center have built a prototypical large-scale infrastructure to handle scientific workflows of stakeholders to run on multiple cloud resources. The demonstrations have been in the areas of (a) Data-Intensive Scientific Workflows on Federated Clouds, (b) Interoperability and Federation of Cloud Resources, and (c) Virtual Infrastructure Automation to enable On-Demand Services.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

HPC-FAIR: A Framework Managing Data and AI Models for Analyzing and Optimizing Scientific Applications

The increasing reliance on machine learning (ML) to analyze and optimize large-scale scientific applications on supercomputers faces a significant bottleneck: the lack of readily available, high-quality training datasets and the difficulty in reusing existing AI models. This project was motivated by the urgent need to address the “FAIR” principles (Findability, Accessibility, Interoperability, Reusability) for both training datasets and AI models in the high-performance computing (HPC) domain. The project developed HPC-FAIR, a high-performance computing data management framework designed to centralize HPC-related datasets and AI models within a unified hub. To ensure interoperability, the framework established a standardized representation and vocabulary (ontology) for both data and models. HPC-FAIR also implemented automated workflows to streamline data processing, model access, and benchmarking. Additionally, the project focused on optimizing data harnessing efficiency through advanced techniques like deep reuse and compression-based analytics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

PARETO UI 1.1.0 Release

PARETO is an open-source Python-based software package for oilfield produced water management and beneficiary reuse optimization. PARETO supports produced water industry by providing cost-effective water management solutions. This version introduced an updated User Interface (UI) which makes it easier to navigate and understand the solution for industry users. New Features: - Map files are added for visualization - Added output export function button - Water residual view added - Workflow was streamlined - File extension was expanded - Minor bugfix

AS↗
NASA NTRS2007

Sensor Webs with a Service-Oriented Architecture for On-demand Science Products

This paper describes the work being managed by the NASA Goddard Space Flight Center (GSFC) Information System Division (ISD) under a NASA Earth Science Technology Ofice (ESTO) Advanced Information System Technology (AIST) grant to develop a modular sensor web architecture which enables discovery of sensors and workflows that can create customized science via a high-level service-oriented architecture based on Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) web service standards. These capabilities serve as a prototype to a user-centric architecture for Global Earth Observing System of Systems (GEOSS). This work builds and extends previous sensor web efforts conducted at NASA/GSFC using the Earth Observing 1 (EO-1) satellite and other low-earth orbiting satellites.

Mandl, Daniel↗
NASA NTRS2019

Quantifying Water Storage Change and Land Subsidence Induced by Reservoir Impoundment Using GRACE, Landsat, and GPS Data

The construction of hydropower dams is a common strategy to support a country's increasing need for electricityand river water management for industry and agriculture. Although the hydrological and geophysical impacts ofwater relocation are usually assessed prior to impoundment, their accuracy is generally limited due to the lack ofin situ observations, especially in a remote area. This study presents a workflow to quantify the terrestrial waterstorage change (TWS) and land subsidence induced by a reservoir's water impoundment using multiple satelliteobservations (GRACE, Landsat), land surface models (CABLE, GLDAS, NCEP, ECMWF), and GPS data. The studysite is the Bakun Dam, located in Sarawak, Malaysia, which is the largest hydropower dam in Southeast Asia.Commencing operation in late 2010, the dam induced a change of water mass and lake surface area that wasclearly observed by GRACE and Landsat observations, respectively. During the 17-month impounding period(from August 2010 to December 2011), GRACE observed a dramatic increase of approximately 200mmequivalent water height, while Landsat detected an increased lake extent of around 600 km2. In this paper, aforward model is developed to determine the increased water surface level corresponding to GRACE observations,estimated to be about 120 m. In contrast to GRACE, the TWS derived from land surface models cannotcapture the increased TWS, due to the lack of reservoir routing algorithms in the models. In addition, the landsubsidence was calculated using the disk load model constructed based on the GRACE-derived lake level andLandsat-derived lake extent; the result is validated with the GPS data from BIN1 station, located at the westerncoast of Borneo. The commencement stage of the Bakun Dam induces the large-scale land subsidence, whichcauses the GPS-BIN1 station to subside by ~9 mm, and move toward the Bakun Lake by ~4 mm. Computation ofthe surface displacements directly from GRACE spherical harmonic coefficient data fails to capture the subsidencefeature, mainly due to the truncation error. Overall, this study demonstrates that evaluating GRACE inconjunction with Landsat, LSMs, and GPS data allows the exploitation of the gravity signal at a much smallerspatial scale than its intrinsic resolution. Benefiting from global coverage, the newly developed satellite-basedalgorithm is a valuable tool for assessing the impacts of reservoir operation on hydrological and geophysicalchanges from local to regional scales.

Tangdamrongsub, Natthachet↗
DOE OSTI2023

FAIR for AI: An interdisciplinary and international community building perspective

A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to include the software, tools, algorithms, and workflows that produce data. FAIR principles are now being adapted in the context of AI models and datasets. Here, we present the perspectives, vision, and experiences of researchers from different countries, disciplines, and backgrounds who are leading the definition and adoption of FAIR principles in their communities of practice, and discuss outcomes that may result from pursuing and incentivizing FAIR AI research. The material for this report builds on the FAIR for AI Workshop held at Argonne National Laboratory on June 7, 2022.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Sighten Final Technical Report DEEE0006690 Deploying an integrated and comprehensive solar financing software platform

Over the project, Sighten built a comprehensive software-as-a-service (Saas) platform to automate and streamline the residential solar financing workflow. Before the project period, significant time and money was spent by companies on front-end tools related to system design and proposal creation, but comparatively few resources were available to support the many back-end calculations and data management processes that underpin third party financing. Without a tool like Sighten, the solar financing processes involved passing information from the homeowner prospect into separate tools for system design, financing, and then later to reporting tools including Microsoft Excel, CRM software, in-house software, outside software, and offline, manual processes. Passing data between tools and attempting to connect disparate systems results in inefficiency and inaccuracy for the industry. Sighten was built to consolidate all financial and solar-related calculations in a single software platform. It significantly improves upon the accuracy of these calculations and exposes sophisticated new analysis tools resulting in a rigorous, efficient and cost-effective toolset for scaling residential solar. Widely deploying a platform like Sighten’s significantly and immediately impacts the residential solar space in several important ways: 1) standardizing and improving the quality of all quantitative calculations involved in the residential financing process, most notably project finance, system production and reporting calculations; 2) representing a true step change in terms of reporting and analysis capabilities by maintaining more accurate data and exposing sophisticated tools around simulation, tranching, and financial reporting, among others, to all stakeholders in the space; 3) allowing a broader group of developers/installers/finance companies to access the capital markets by providing an out-of-the-box toolset that handles the execution of running investor capital through a rooftop solar financing program. Standardizing and improving all calculations, improving data quality, and exposing new analysis tools previously unavailable affects investment in the residential space in several important ways: 1) lowering the cost of capital for existing capital providers by mitigating uncertainty and de-risking the solar asset class; 2) attracting new, lower cost investors to the solar asset class as reporting and data quality resemble standards of more mature asset classes; 3) increasing the prevalence of liquidity options for investors through back leverage, securitization, or secondary sale by providing the tools necessary for lenders, ratings agencies, etc. to properly understand a portfolio of residential solar assets. During the project period, Sighten successfully built and scaled a commercially ready tool for the residential solar market. The software solution built by Sighten has been deployed with key target customer segments identified in the award deliverables: solar installers, solar developers/channel managers, and solar financiers, including lenders. Each of these segments greatly benefits from the availability of the Sighten toolset.

14 SOLAR ENERGY↗
DOE OSTI2025

White Paper: Scalable Digital Twin Capabilities for Aging and Surveillance of Engineered Systems

This white paper presents a multi-year initiative to develop practical, secure, and scalable digital twin capabilities for engineered systems in aging and surveillance contexts—an approach pioneered at the National Nuclear Security Administration (NNSA) Lawrence Livermore National Laboratory (LLNL) that maps directly onto the needs and ambitions of the Navy for ship- and fleet-level digital twins. LLNL’s work in building part- and process-level digital twins for advanced manufacturing, with a vision to scale up to entire factory floors and, ultimately, enterprise-wide digital twins, offers an adaptable pathway for the Navy as it seeks to modernize lifecycle management, readiness, and predictive maintenance across ships and fleets. For our application, we integrate physics-based modeling with automated data ingestion, processing, and AI-driven calibration, creating hybrid models that are both interpretable and data responsive. We modernized legacy workflows, established centralized data infrastructure, automated experimental pipelines, and demonstrated end-to-end coupling of accelerated aging data with finite element simulations via optimization and surrogate modeling. The result is a generalizable framework that supports part-level digital twins today and lays the groundwork for future system-level twins suitable for Navy applications.

36 MATERIALS SCIENCE↗