Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,405 records · Page 78

Lightweight Software Process Improvement Using Productivity and Sustainability Improvement Planning (PSIP)

Productivity and Sustainability Improvement Planning (PSIP) is a lightweight, iterative workflow that allows software development teams to identify development bottlenecks and track progress to overcome them. In this paper, we present an overview of PSIP and how it compares to other software process improvement (SPI) methodologies, and provide two case studies that describe how the use of PSIP led to successful improvements in team effectiveness and efficiency.

Heroux, Michael A.↗

Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions

The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.

Durden, David J.↗

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗

Scaling SQL to the Supercomputer for Interactive Analysis of Simulation Data

AI and simulation workloads consume and generate large amounts of data that need to be searched, transformed and merged with other data. With the goal of treating data as a first-class citizen inside a traditionally compute-centric HPC environment, we explore how the use of accelerators and high-speed interconnects can speed up tasks which otherwise constitute bottlenecks in computational discovery workflows. BlazingSQL is SQL engine that runs natively on NVIDIA GPUs and supports internode communication for fast analytics on terabyte-scale tabular data sets. We show how a fast interconnect improves query performance if leveraged through the Unified Communication X (UCX) middleware. We envision that future computing platforms will integrate accelerated database query capabilities for immediate and interactive analysis of large simulation data.

Glaser, Jens↗

Building an Integrated Ecosystem of Computational and Observational Facilities to Accelerate Scientific Discovery

Future scientific discoveries will rely on flexible ecosystems that incorporate modern scientific instruments, high performance computing resources, parallel distributed data storage, and performant networks across multiple, independent facilities. In addition to connecting physical resources, such an ecosystem presents many challenges in logistics and accessibility, especially in orchestrating computations and experiments that span across leadership computing systems and experimental instruments. Past efforts have typically been application-specific or limited to interfaces for computing resources. This paper proposes a general framework for integrating computation resources and instrument operations, addressing challenges in code development/execution, data staging and collection, software stack, control mechanisms, resource authorization and governance, and hardware integration. We also describe a demonstration use case wherein a Bayesian optimization algorithm running on an edge computing resource guides a scanning probe microscope to autonomously and intelligently characterize a material sample. This science edge ecosystem framework will provide a blueprint for federating multi-institutional, disparate resources and orchestrating scientific workflows across them to enable next-generation discoveries.

Somnath, Suhas↗

Unveiling User Behavior on Summit Login Nodes as a User

We observe and analyze usage of the login nodes of the leadership class Summit supercomputer from the perspective of an ordinary user—not a system administrator—by periodically sampling user activities (job queues, running processes, etc.) for two full years (2020–2021). Our findings unveil key usage patterns that evidence misuse of the system, including gaming the policies, impairing I/O performance, and using login nodes as a sole computing resource. Our analysis highlights observed patterns for the execution of complex computations (workflows), which are key for processing large-scale applications.

Wilkinson, Sean↗

On the Feasibility of Simulation-Driven Portfolio Scheduling for Cyberinfrastructure Runtime Systems

Runtime systems that automate the execution of applications on distributed cyberinfrastructures need to make scheduling decisions. Researchers have proposed many scheduling algorithms, but most of them are designed based on analytical models and assumptions that may not hold in practice. The literature is thus rife with algorithms that have been evaluated only within the scope of their underlying assumptions but whose practical effectiveness is unclear. It is thus difficult for developers to decide which algorithm to implement in their runtime systems.To obviate the above difficulty, we propose an approach by which the runtime system executes, throughout application execution, simulations of this very execution. Each simulation is for a different algorithm in a scheduling algorithm portfolio, and the best algorithm is selected based on simulation results. The main objective of this work is to evaluate the feasibility and potential merit of this portfolio scheduling approach, even in the presence of simulation inaccuracy, when compared to the traditional one-algorithm approach. We perform this evaluation via a case study in the context of scientific workflows. Our main finding is that portfolio scheduling can outperform the best one-algorithm approach even in the presence of relatively large simulation inaccuracies.

Casanova, Henri↗

Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument Ecosystems

Science ecosystems are being built by federating computing systems and instruments located at geographically distributed sites over wide-area networks. These computing-instrument ecosystems are expected to support complex workflows that incorporate remote, automated AI-driven science experiments. Their realization, however, requires various designs to be explored and software components to be developed, in order to support the orchestration of distributed computations and experiments. It is often too expensive, infeasible, or disruptive for the entire ecosystem to be available during the typically long software development and testing periods. We propose a Virtual Infrastructure Twin (VIT) of the ecosystem that emulates its network and computing components, and incorporates its instrument software simulators. It provides a software environment nearly identical to the ecosystem to support early development and testing, and design space exploration. We present a brief overview of previous digital infrastructure twins that culminated in the VIT concept, including (i) the virtual science network environment for developing software-defined networking solutions, and (ii) the virtual federated science instrument environment for testing the federation software stack and remote instrument control software. We briefly describe VITs for Nion microscope steering and access to GPU systems.

Rao, Nageswara↗

Developing Ultrahigh-Resolution E3SM Land Model for GPU Systems

Designing and refactoring complex scientific code, such as the E3SM land model (ELM), for new computing architectures is challenging. This paper presents design strategies and technical approaches to develop a data-oriented, GPU-ready ELM model using compiler directives (OpenACC/OpenMP). We first analyze the datatypes and processes in the original ELM code. Then we present design considerations for ultrahigh-resolution ELM (uELM) development for massive GPU systems. These techniques include the global data-oriented simulation workflow, domain partition, code porting and data copy, memory reduction, parallel loop restructure and flattening, and race condition detection. We implemented the first version of uELM using OpenACC targeting the NVidia GPUs in the Summit supercomputer at Oak Ridge National Laboratory. During the implementation, we developed a software tool (named SPEL) to facilitate code generation, verification, and performance tuning using these techniques. The first uELM implementation for Nvidia GPUs on Summit delivered promising results: 1) over 98% of the ELM code was automatically generated and tuned by scripts. Most ELM modules had better computational performances than the original ELM code for CPUs. The GPU-ready uELM is more scalable than the CPU code on fully-loaded Summit nodes. Example profiling results from several modules are also presented to illustrate the performance improvements and race condition detection. The lessons learned and toolkit developed in the study are also suitable for further uELM deployment using OpenMP on the first US exascale computer, Frontier, equipped with AMD CPUs and GPUs.

Schwartz, Peter↗

Rethinking Programming Paradigms in the QC-HPC Context

Programming for today’s quantum computers is making significant strides toward modern workflows compatible with high performance computing (HPC), but fundamental challenges still remain in the integration of these vastly different technologies. Quantum computing (QC) programming languages share some common ground, as well as their emerging runtimes and algorithmic modalities. In this short paper, we explore avenues of refinement for the quantum processing unit (QPU) in the context of many-tasks management, asynchronous or otherwise, in order to understand the value it can play in linking QC with HPC. Through examples, we illustrate how its potential for scientific discovery might be realized.

Wong, Elaine↗

Visual Systems Mapping to Define and Compare Woody Biomass LCAs for Sustainable Systems

The challenge addressed in this research centres on the need to choose between several biomass sources and energy production processes, while supporting rural economies and resilience of forest systems. A key barrier to effective decision-making for strategies using biomass is the lack of standardized and transparent life cycle assessment (LCA) baselines. These baselines are critical for assessing the impacts of biomass strategies but often vary due to regional factors and chosen simplifying assumptions of the LCAs. However, omitting key variables can mean the LCA omits key feedback and balancing loops relevant to fully assessing impacts of the change or test scenario. To address these complexities, this project employs a systems engineering approach: visual systems mapping. This technique is used to define the boundaries and dynamic behaviours of LCA baselines, enhancing transparency. By examining five literature sources and their documented baseline scenarios, the systems mapping case-studies demonstrates an approach to documenting and archiving these baselines. Recommendations are that visual systems mapping should be used to document key assumptions, such as baselines, of LCAs. Further, where possible open data repositories should hold key information about LCA baselines and reproducible workflows (e.g., using open-source tools) should be used to improve transparency and comparability in LCAs. Given the consensus within the broader scientific community on the importance of replicable data practices, this research reinforces the need for standardized frameworks and systems engineering tools in LCAs. This research demonstrates a pathway to more transparent, standardized, and comparable LCAs, that may bolster decisions for biomass systems.

Davis, Maggie [ORNL] (ORCID:0000000181319328)↗

Omics-driven onboarding of the carotenoid producing red yeast Xanthophyllomyces dendrorhous CBS 6938

Transcriptomics is a powerful approach for functional genomics and systems biology, yet it can also be used for genetic part discovery. Here, we derive constitutive and light-regulated promoters directly from transcriptomics data of the basidiomycete red yeast Xanthophyllomyces dendrorhous CBS 6938 (anamorph Phaffia rhodozyma) and use these promoters with other genetic elements to create a modular synthetic biology parts collection for this organism. X. dendrorhous is currently the sole biotechnologically relevant yeast in the Tremellomycete class-it produces large amounts of astaxanthin, especially under oxidative stress and exposure to light. Thus, we performed transcriptomics on X. dendrorhous under different wavelengths of light (red, green, blue, and ultraviolet) and oxidative stress. Differential gene expression analysis (DGE) revealed that terpenoid biosynthesis was primarily upregulated by light through crtI, while oxidative stress upregulated several genes in the pathway. Further gene ontology (GO) analysis revealed a complex survival response to ultraviolet (UV) where X. dendrorhous upregulates aromatic amino acid and tetraterpenoid biosynthesis and downregulates central carbon metabolism and respiration. The DGE data was also used to identify 26 constitutive and regulated genes, and then, putative promoters for each of the 26 genes were derived from the genome. Simultaneously, a modular cloning system for X. dendrorhous was developed, including integration sites, terminators, selection markers, and reporters. Each of the 26 putative promoters were integrated into the genome and characterized by luciferase assay in the dark and under UV light. The putative constitutive promoters were constitutive in the synthetic genetic context, but so were many of the putative regulated promoters. Notably, one putative promoter, derived from a hypothetical gene, showed ninefold activation upon UV exposure. Thus, this study reveals metabolic pathway regulation and develops a genetic parts collection for X. dendrorhous from transcriptomic data. Therefore, this study demonstrates that combining systems biology and synthetic biology into an omics-to-parts workflow can simultaneously provide useful biological insight and genetic tools for nonconventional microbes, particularly those without a related model organism. This approach can enhance current efforts to engineer diverse microbes.

60 APPLIED LIFE SCIENCES↗

New approaches to secondary metabolite discovery from anaerobic gut microbes

The animal gut microbiome is a complex system of diverse, predominantly anaerobic microbiota with secondary metabolite potential. These metabolites likely play roles in shaping microbial community membership and influencing animal host health. As such, novel secondary metabolites from gut microbes hold significant biotechnological and therapeutic interest. Despite their potential, gut microbes are largely untapped for secondary metabolites, with gut fungi and obligate anaerobes being particularly under-explored. To advance understanding of these metabolites, culture-based and (meta)genome-based approaches are essential. Culture-based approaches enable isolation, cultivation, and direct study of gut microbes, and (meta)genome-based approaches utilize in silico tools to mine biosynthetic gene clusters (BGCs) from microbes that have not yet been successfully cultured. In this mini-review, we highlight recent innovations in this area, including anaerobic biofoundries like ExFAB, the NSF BioFoundry for Extreme & Exceptional Fungi, Archaea, and Bacteria. These facilities enable high-throughput workflows to study oxygen-sensitive microbes and biosynthetic machinery. Such recent advances promise to improve our understanding of the gut microbiome and its secondary metabolism.

59 BASIC BIOLOGICAL SCIENCES↗

Volcano infrasound: progress and future directions

Over the past two decades (2000–2020), volcano infrasound (acoustic waves with frequencies less than 20 Hz propagating in the atmosphere) has evolved from an area of academic research to a useful monitoring tool. As a result, infrasound is routinely used by volcano observatories around the world to detect, locate, and characterize volcanic activity. It is particularly useful in confirming subaerial activity and monitoring remote eruptions, and it has shown promise in forecasting paroxysmal activity at open-vent systems. Fundamental research on volcano infrasound is providing substantial new insights on eruption dynamics and volcanic processes and will continue to do so over the next decade. The increased availability of infrasound sensors will expand observations of varied eruption styles, and the associated increase in data volume will make machine learning workflows more feasible. More sophisticated modeling will be applied to examine infrasound source and propagation effects from local to global distances, leading to improved infrasound-derived estimates of eruption properties. Future work will use infrasound to detect, locate, and characterize moving flows, such as pyroclastic density currents, lahars, rockfalls, lava flows, and avalanches. Infrasound observations will be further integrated with other data streams, such as seismic, ground- and satellite-based thermal and visual imagery, geodetic, lightning, and gas data. The volcano infrasound community should continue efforts to make data and codes accessible and to improve diversity, equity, and inclusion in the field. In summary, the next decade of volcano infrasound research will continue to advance our understanding of complex volcano processes through increased data availability, sensor technologies, enhanced modeling capabilities, and novel data analysis methods that will improve hazard detection and mitigation.

58 GEOSCIENCES↗

Classical and quantum compression for edge computing: the ubiquitous data dimensionality reduction

Edge computing aims to address the challenges associated with communicating and transferring large amounts of data generated remotely to a data center in a timely and efficient manner. A central pillar of edge computing is local (i.e., at- or near-source) data processing capability so that data transfer to a data center for processing can be minimized. Data compression at the edge is therefore a natural component of edge workflows. Here we present a survey of data compression algorithms with a focus on edge computing. Not all compression algorithms can accommodate the data type heterogeneity, tight processing and communication time constraints, or energy efficiency requirement characteristics of edge computing. We discuss specific examples of compression algorithms that are being explored in the context of edge computing. We end our review with a brief survey of emerging quantum compression techniques that are of importance in quantum information processing, including the proposed concept of quantum edge computing.

97 MATHEMATICS AND COMPUTING↗

Deep-learning based artificial intelligence tool for melt pools and defect segmentation

Accelerating fabrication of additively manufactured components with precise microstructures is important for quality and qualification of built parts, as well as for a fundamental understanding of process improvement. Accomplishing this requires fast and robust characterization of melt pool geometries and structural defects in images. This paper proposes a pragmatic approach based on implementation of deep learning models and self-consistent workflow that enable systematic segmentation of defects and melt pools in optical images. Deep learning is based on an image-to-image translation–conditional generative adversarial neural network architecture. An artificial intelligence (AI) tool based on this deep learning model enables fast and incrementally more accurate predictions of the prevalent geometric features, including melt pool boundaries and printing-induced structural defects. We present statistical analysis of geometric features that is enabled by the AI tool, showing strong spatial correlation of defects and the melt pool boundaries. The correlations of widths and heights of melt pools with dataset processing parameters show the highest sensitivity to thermal influences resulting from laser passes in adjacent and subsequent layer passes. The presented models and tools are demonstrated on the aluminum alloy and datasets produced with different sets of processing parameters. However, they have universal quality and could easily be adapted to different material compositions. The method can be easily generalized to microstructural characterizations other than optical microscopy.

additive manufacturing↗

Poisson’s ratio of eTPU molded bead foams in compression via in situ synchrotron X-ray microtomography

In situ synchrotron X-ray microtomography was used to characterize the bulk deformation behavior by computing the Poisson's ratio of expanded thermoplastic polyurethane (eTPU) molded bead foams used in footwear midsole during compression. Quantitative data on morphological characteristics were obtained using an iterative image processing workflow. Image correlation on the 4D datasets using DVC was performed to calculate the volumetric and axial strain to estimate the Poisson ratio. Strain maps from DVC showed the influence of variability in ligament thickness distribution on the global mechanical behavior exhibited which dominated the response seen in these bead foams. Lastly, our results showed a strong correlation between Poisson ratio and distribution of ligament thickness in foams.

36 MATERIALS SCIENCE↗

Combining Deep Learning and scatterControl for High-Throughput X-ray CT Based Non-Destructive Characterization of Large-Scale Casted Metallic Components

X-ray computed tomography (XCT) is essential for nondestructive evaluation and quality control of large-scale metal components. XCT imaging, however, faces significant challenges from metal artifacts, particularly those caused by Compton scattering, which degrade image quality and obscure critical details. Hardware-based solutions (e.g. scatterControl) offer advancements by intercepting scattered photons and reducing artifacts, but they can be time-consuming and require additional processing. Here, we propose modifying and leveraging a novel deep learning (DL) framework, Simurgh, to enhance and accelerate scatter correction in XCT. By combining scatterControl with DL-based artifact removal, we demonstrate significant reduction in scan time while producing high-quality reconstructions. Through extensive evaluation on industrial XCT data, we show that our methods reduce scan time by up to more than 10 x while preserving flaw detectability. Quantitative analysis across multiple segmentation techniques confirms that Simurgh-based reconstructions consistently outperform traditional Feldkamp-Davis-Kress, model-based iterative reconstruction, and commercial DL models in both pixel-level and task-specific evaluations, enabling scalable, high-throughput XCT workflows for characterization of large scale components in applications such as casting and metal additive manufacturing.

Complex metal parts↗