Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Direct Nanoscale Characterization of Deep Levels in AgCuInGaSe 2 Using Electron Energy–Loss Spectroscopy in the Scanning Transmission Electron Microscope

A new experimental framework for the characterization of defects in semiconductors is demonstrated. Through the direct, energy-resolved correlation of three analytical techniques spanning six orders of magnitude in spatial resolution, a critical mid-bandgap electronic trap level (E V + 0.56 eV) within Ag 0.2 Cu 0.8 In 1–x Ga x Se 2 is traced to its nanoscale physical location and chemical source. This is achieved through a stepwise, site-specific correlated characterization workflow consisting of device-scale (≈1 mm 2 ) deep level transient spectroscopy (DLTS) to survey the traps present, scanning probe–based DLTS (scanning-DLTS) for mesoscale-resolved (hundreds of nanometers) mapping of the target trap state’s spatial distribution, and scanning transmission electron microscope based electron energy-loss spectroscopy (STEM-EELS) and X-ray energy-dispersive spectroscopy for nanoscale energy-, structure, and chemical-resolved investigation of the defect source. Furthermore, this first demonstration of the direct observation of sub-bandgap defect levels via STEM-EELS, combined with the DLTS methods, provides strong evidence that the longsuspected Cu In/Ga substitutional defects are indeed the most likely source of the E V + 0.56 eV trap state and serves as a key example of this approach for the fundamental identification of defects within semiconductors, in general.

14 SOLAR ENERGY↗

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

The use of convolutional neural networks (CNNs) for building extraction from remote sensing images has been widely studied and many public datasets have been made available for accelerating development of these CNN models. Yet adapting pretrained models at scale in real-world scenarios remains a challenging task. The main barrier is that certain new labels are still needed to compensate for domain shifting between the labeled data and new images that potentially cover new geographic locations or that are from a different sensor. In this article, we propose to add informatively labeled samples from a new image pool under the paradigm of active learning. To select the most useful samples based on model uncertainty, we first tackle the problem of uncalibrated uncertainty estimation due to distribution shifting by adapting feature extractors with boundary-based adversarial learning. Calibrated uncertainty is used as the query criterion in the active learning process, where the most uncertain samples are selected for annotation and included for model retraining. The proposed workflow was tested with three data pairs in which each workflow represents a scenario often encountered in real-world applications, including adapting pretrained models to new images collected with different sensors or to new geographic areas where appearances and types of buildings are very different. Compared to several baselines, including random sampling, temperature scaling (a well-known uncertainty calibration technique), different query strategies, and active domain adaptation methods, the proposed workflow shows that strategically querying a smaller set of samples for labeling achieves comparable or better building extraction performance. The proposed method reduces the number of labeled samples required to achieve sufficient model accuracy, thus significantly reducing hundreds of person-hours for labeled data creation. In addition, we include a few considerations when deploying this workflow in a GPU cluster that can be easily adapted to achieve operational building extraction model retraining.

97 MATHEMATICS AND COMPUTING↗

Automating Mid- and Long-Range Scheduling for NASA's Deep Space Network

NASA has recently deployed a new mid-range scheduling system for the antennas of the Deep Space Network (DSN), called Service Scheduling Software, or S(sup 3). This system is architected as a modern web application containing a central scheduling database integrated with a collaborative environment, exploiting the same technologies as social web applications but applied to a space operations context. This is highly relevant to the DSN domain since the network schedule of operations is developed in a peer-to-peer negotiation process among all users who utilize the DSN (representing 37 projects including international partners and ground-based science and calibration users). The initial implementation of S(sup 3) is complete and the system has been operational since July 2011. S(sup 3) has been used for negotiating schedules since April 2011, including the baseline schedules for three launching missions in late 2011. S(sup 3) supports a distributed scheduling model, in which changes can potentially be made by multiple users based on multiple schedule "workspaces" or versions of the schedule. This has led to several challenges in the design of the scheduling database, and of a change proposal workflow that allows users to concur with or to reject proposed schedule changes, and then counter-propose with alternative or additional suggested changes. This paper describes some key aspects of the S(sup 3) system and lessons learned from its operational deployment to date, focusing on the challenges of multi-user collaborative scheduling in a practical and mission-critical setting. We will also describe the ongoing project to extend S(sup 3) to encompass long-range planning, downtime analysis, and forecasting, as the next step in developing a single integrated DSN scheduling tool suite to cover all time ranges.

scheduling↗

A multi-dimensional parametric study of variability in multi-phase flow dynamics during geologic CO 2 sequestration accelerated with machine learning

Successful geologic CO 2 storage projects depend on numerical simulations to predict reservoir performance during site selection, injection verification, and post-injection monitoring phases of the project. These numerical simulations solve non-linear sets of coupled partial differential equations, while accounting for multi-phase fluid dynamics on the basis of constitutive equations that are embedded into the solution scheme. As a consequence, individual simulations often require tens to hundreds of hours to complete on high-performance computing clusters. Moreover, laboratory experiments reveal that parametric functions for capillary pressure and relative permeability exhibit substantial variability, even within the same rock type. This combination of computational expense and wide-ranging parametric variability means that there remains substantial uncertainty in the behavior of multi-phase CO 2 -water systems, particularly in the context of feedbacks between relative permeability and capillary pressure. To bridge this knowledge gap, here we develop a novel workflow that utilizes physics-based numerical simulation to train an artificial neural network (ANN) emulator for interrogating the multivariate parameter space that governs both capillary pressure and relative permeability. With this approach, the ANN is trained to emulate both fluid pressure distribution and CO 2 saturation, which are then interrogated quantitatively to generate parametric response surface mappings with high-fidelity resolution. Results from this study initially show that capillary entry pressure is the dominant control on both CO 2 plume geometry and fluid pressure propagation when considering the combined effects of capillary pressure and relative permeability, particularly when phase interference is low and residual CO 2 saturation is high. Moreover, the ANN emulator provides tremendous computational speed-up by computing 2691 individual simulations in several minutes; whereas, the same simulation ensemble would have required ~3 years of simulation time using only physics-based simulation methods (25,000 times speed up).

58 GEOSCIENCES↗

Large scale and low latency analysis facilities for the CMS experiment: Development and operational aspects

While a majority of CMS data analysis activities rely on the distributed computing infrastructure on the WLCG Grid, dedicated local computing facilities have been deployed to address particular requirements in terms of latency and scale. The CMS CERN Analysis Facility (CAF) was primarily designed to host a large variety of latency-critical workflows. These break down into alignment and calibration, detector commissioning and diagnosis, and high-interest physics analysis requiring fast turnaround. In order to reach the goal for fast turnaround tasks, the Workload Management group has designed a CRABServer based system to fit with two main needs: to provide a simple, familiar interface to the user (as used in the CRAB Analysis Tool[7]) and to allow an easy transition to the Tier-0 system. While the CRABServer component had been initially designed for Grid analysis by CMS end-users, with a few modifications it turned out to be also a very powerful service to manage and monitor local submissions on the CAF. Transition to Tier-0 has been guaranteed by the usage of the WMCore, a library developed by CMS to be the common core of workload management tools, for handing data driven workflow dependencies. This system is now being used with the first use cases, and important experience is being acquired. In addition to the CERN CAF facility, FNAL has CMS dedicated analysis resources at the FNAL LHC Physics Center (LPC). In the first few years of data collection FNAL has been able to accept a large fraction of CMS data. The remote centre is not well suited for the extremely low latency work expected of the CAF, but the presence of substantial analysis resources, a large resident community, and a large fraction of the data make the LPC a strong facility for resource intensive analysis. We present the building, commissioning and operation of these dedicated analysis facilities in the first year of LHC collisions, we also present the specific development to our software needed to allow for the use of these computing facilities in the special use cases of fast turnaround analyses.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Wang, Tianle [Brookhaven National Laboratory (BNL)↗

Operational Evolution of FTS3: A DevOps Driven Approach to Elastic Operations

The File Transfer Service (FTS3) is a distributed data movement service developed at CERN and widely used to transfer data across the Worldwide LHC Computing Grid (WLCG). At Fermilab, FTS3 supports data transfers for multiple experiments, including Intensity Frontier experiments such as DUNE, enabling reliable data movement between WebDAV endpoints in Europe and the Americas.​ At CHEP 2021, we reported on the initial containerized deployment of FTS3 on OKD, the community Kubernetes distribution of Red Hat OpenShift. In this work, we present the subsequent evolution of this deployment, focusing on new operational capabilities introduced to improve scalability, robustness, and long-term maintainability.​ We describe the adoption of more secure and reproducible container build workflows, the integration of DevOps-driven operational practices, and enhancements in monitoring and automation. A key new result is the introduction of horizontal scaling and elastic resource management, allowing FTS3 components to dynamically adapt to workload variations while maintaining service reliability. We also discuss improvements in fault tolerance and operational procedures derived from production experience.​ Finally, we summarize lessons learned from operating FTS3 as a Kubernetes-native service and outline how these developments have improved the resilience and efficiency of data movement operations at Fermilab.

Munoz Flores, Victor Leopoldo [Fermilab]↗

Towards PCC for Concurrent and Distributed Systems (Work in Progress)

We outline some conceptual challenges in extending the PCC paradigm to a concurrent and distributed setting, and sketch a generalized notion of module correctness based on viewing communication contracts as economic games. The model supports compositional reasoning about modular systems and is meant to apply not only to certification of executable code, but also of organizational workflows.

Henriksen, Anders S.↗

Application of unsupervised deep learning to image segmentation and in-situ contact angle measurements in a CO 2 -water-rock system

Rock surface wettability is a critical property that regulates multiphase flows in porous media, which can be quantified using the surface contact angle (CA). X-ray micro-computed tomography (μCT) provides an effective approach to in-situ measurements of surface CAs. However, the CA measurement accuracy depends significantly on the quality of CT image segmentation, which is the clustering of CT pixels into separate phases. Inspired by this, we developed a deep learning (DL)-based CA measurement workflow. Motivated by the recent tremendous progress in unsupervised learning techniques and aiming to avoid expensive manual data annotations, an unsupervised DL pipeline for CT image segmentation was proposed and implemented, which includes unsupervised model training and post-processing. The unsupervised model training was driven by a novel loss function constrained with feature similarity and spatial continuity and implemented by iterative forward and backward paths; the former clustered the pixel-wise feature vectors extracted by convolution neural networks, whereas the latter updated the parameters using gradient descent. An over-segmentation strategy was adopted for model training. The post-processing steps based on agglomerative hierarchical clustering (AHC) were implemented to further merge the over-segmented model output to the desired cluster number, which is intended to improve the efficiency of image segmentation. The developed unsupervised DL pipeline was compared with other commonly-used image segmentation methods using pixel-wise and physics-based evaluation metrics on a synthetic raw-image dataset, which had a known ground truth. The unsupervised DL pipeline showed the best performance. Next, the segmented images were input to an automatic CA measurement tool, and the results were validated by comparisons with manual measurements. The CA values from the manual and automatic measurements showed similar distributions and statistical properties. The automatic measurement demonstrated a wider spectrum because of the much larger number of measurement data points. The primary novelty of the unsupervised DL pipeline developed in this study lies in the novel loss function and the over-segmentation strategy associated with AHC post-processing. Finally, the workflow has been proven an efficient tool for pore-scale wettability characterization, which has a wide range of applications in fundamental studies of multiphase flows in natural porous media, which have critical implications to geological carbon sequestration, hydrocarbon energy recovery, and contaminant transport in groundwater.

42 ENGINEERING↗

Simulation-based characterization of the variability of earthquake risk to buildings in the near-field

Recent advancements in high performance computing platforms and computational workflow for regional-scale simulations are enabling unprecedented modeling of fault-to-structure earthquake processes. Regional simulations resolving ground motions at frequencies relevant to engineered systems are becoming computationally viable and provide a new capability to improve understanding of the geographical distribution and intensity of risk to buildings and critical infrastructure. As computational capabilities advance, it is essential to move beyond illustrative single rupture realizations for scenario earthquake events towards the development of a full suite of rupture realizations that appropriately characterize the range of risk to building systems. The work described in this article investigates the application of a suite of fault rupture realizations with the objective of assessing near-fault, site-specific seismic demand variability for building structures. A representative high-performance regional-scale computational model is utilized to execute ground motion and building response simulations based on 18 kinematic rupture realizations of an M7 strike-slip scenario earthquake. The fault rupture models for the scenario earthquake are created by systematically perturbing the hypocenter location and stochastically generating rupture parameters (slip, rise time, rake angle) to represent a breadth of ground motion intensities resulting from the spatial and temporal variabilities of an earthquake rupture process. The resulting seismic demand variability for three-story (short period) and forty-story (long period) steel moment-resisting frame buildings is characterized in terms of the median and distribution of peak inter-story drift ratio for a range of near-fault sites. The full suite of 18 fault rupture realizations and approximately 280,000 nonlinear dynamic building simulations indicate that the three-story building undergoes higher median seismic demand and significantly greater variability of demand at a given site than the forty-story building, which has important implications for the level of certainty in predicting building performance during an earthquake. The simulations performed provide deeper insight into the relationship between fault rupture parameterization and building response, which is essential information for developing a representative suite of rupture realizations for specific earthquake scenarios.

58 GEOSCIENCES↗

Semi-supervised Machine Learning Enables the Robust Detection of Multireference Character at Low Cost

Multireference (MR) diagnostics are common tools for identifying strongly correlated electronic structure that makes single-reference (SR) methods (e.g., density functional theory or DFT) insufficient for accurate property prediction. However, MR diagnostics typically require computationally demanding correlated wave function theory (WFT) calculations, and diagnostics often disagree or fail to predict MR effects on properties. To overcome these challenges, we introduce a semi-supervised machine learning (ML) approach with virtual adversarial training (VAT) of an MR classifier using 15 WFT and DFT MR diagnostics as inputs. In semi-supervised learning, only the most extreme SR or MR points are labeled, and the remaining point labels are learned. The resulting VAT model outperforms the alternatives, as quantified by the distinct property distributions of SR- and MR-classified molecules. Additionally, to reduce the cost of generating inputs to the VAT model, we leverage the VAT model’s robustness to noisy inputs by replacing WFT MR diagnostics with regression predictions in an MR decision engine workflow that preserves excellent performance. We demonstrate the transferability of our approach to larger molecules and those with distinct chemical composition from the training set. This MR decision engine demonstrates promise as a low-cost, high-accuracy approach to the automatic detection of strong correlation for predictive high-throughput screening.

36 MATERIALS SCIENCE↗

Control over Conformational Landscapes of Polypeptoids by Monomer Sequence Patterning

The ability to program chain conformation and structure through control over the monomer sequence of synthetic polymers has broad implications for next-generation material design. While related problems of protein-folding and de novo design have generated accurate predictions of 3D folded chain structures, generalization to synthetic polymers remains intractable due to the requirement of large structural databases and the intrinsically disordered nature of polymer building blocks. In this work, polypeptoids, a class of peptidomimetic synthetic polymers, are utilized to build a general workflow for the study of relationships between monomer sequence and dynamic 3D chain structure in solution. Furthermore, this work demonstrates how control over the monomer sequence can alter the conformational landscape of synthetic polymers to deviate dramatically from classical chain statistics. Specifically, the distribution of end-to-end distances, as measured by double electron-electron resonance spectroscopy in dilute solvent, is systematically skewed towards shorter distances with an increasing number of hydrophobes and further refined by hydrophobe arrangement in amphiphilic polypeptoid chains.

36 MATERIALS SCIENCE↗

Exploring Container Technologies for Large Scientific Libraries: Docker and Trilinos

The Trilinos Project produces, maintains and distributes a large collection of reusable, parallel scientific libraries. Docker provides container technologies that support compilation, packaging, distribution and execution of software on Linux, Mac OS and Windows systems, with emerging support for Cray platforms. In this short article we describe recent efforts to explore the potential for using Docker in a variety of settings to enhance several Trilinos Project workflows. The technical foundation for this article is presented in an Honors thesis of one of the authors.

97 MATHEMATICS AND COMPUTING↗

FELIX-2.0: New version of the finite element solver for the time dependent generator coordinate method with the Gaussian overlap approximation

The time-dependent generator coordinate method (TDGCM) is a powerful method to study the largeamplitude collective motion of quantum many-body systems such as atomic nuclei. Under the GaussianOverlap Approximation (GOA), the TDGCM leads to a local, time-dependent Schrödinger equation in amulti-dimensional collective space. In this paper, we present the version 2.0 of the code FELIX that solvesthe collective Schrödinger equation in a finite element basis. This new version features: (i) the ability tosolve a generalized TDGCM+GOA equation with a metric term in the collective Hamiltonian, (ii) supportfor new kinds of finite elements and different types of quadrature to compute the discretized Hamiltonianand overlap matrices, (iii) the possibility to leverage the spectral element scheme, (iv) an explicit Krylovapproximation of the time propagator for time integration instead of the implicit Crank–Nicolson methodimplemented in the first version, (v) an entirely redesigned workflow. We benchmark this release on ananalytic problem as well as on realistic two-dimensional calculations of the low-energy fission of240Puand256Fm. Low to moderate numerical precision calculations are most efficiently performed with simplexelements with a degree 2 polynomial basis. Higher precision calculations should instead use the spectralelement method with a degree 4 polynomial basis. We emphasize that in a realistic calculation of fissionmass distributions of240Pu, FELIX-2.0 is about 20 times faster than its previous release (within a numericalprecision of a few percents).

Regnier, D.↗

Biofoundries: Principles, Tools, and Applications

This chapter aims to provide a broad overview of biofoundries and introduces the principles, concepts, and case studies. We first outline the underlying principles of the Design-Build-Test-Learn (DBTL) framework and the role of automation, digital integration, and standardization. The chapter then explores core biofoundry technologies including robotic liquid handlers, high-throughput analytical instruments, and digital infrastructure for data management and workflow scheduling. Case studies spanning DNA assembly, protein engineering, metabolic engineering, and mammalian cell culture demonstrate the practical applications of the biofoundries. Economic and societal impacts are assessed alongside current limitations. We discuss the emerging trends including artificial intelligence integration and cloud-based distributed facilities to highlight its potential for biotechnology and the bioeconomy.

Singh, Nilmani↗

Optimizing the Weather Research and Forecasting Model with OpenMP Offload and Codee

Currently, the Weather Research and Forecasting model (WRF) utilizes shared memory (OpenMP) and distributed memory (MPI) parallelisms. To take advantage of GPU resources on the Perlmutter supercomputer at NERSC, we port parts of the computationally expensive routine Fast Spectral Bin Microphysics (FSBM) to NVIDIA GPUs using OpenMP device offloading directives. To facilitate this process, we explore a workflow for optimization which uses both runtime profilers and a static code inspection tool Codee to refactor the subroutine. We observe an 2.24x overall speedup for the CONUS-12km storm test case.

Wichitrnithed, Chayanon (Namo) [Odin Institute]↗

Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT)

For the past three years, a large analysis and visualization effort—funded by the Department of Energy’s Office of Biological and Environmental Research (BER), the National Aeronautics and Space Administration (NASA), and the National Oceanic and Atmospheric Administration (NOAA)—has brought together a wide variety of industry-standard scientific computing libraries and applications to create Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT) to serve the global climate simulation and observational research communities. To support interactive analysis and visualization, all components connect through a provenance application–programming interface to capture meaningful history and workflow. Components can be loosely coupled into the framework for fast integration or tightly coupled for greater system functionality and communication with other components. The overarching goal of UV-CDAT is to provide a new paradigm for access to and analysis of massive, distributed scientific data collections by leveraging distributed data architectures located throughout the world. The UV-CDAT framework addresses challenges in analysis and visualization and incorporates new opportunities, including parallelism for better efficiency, higher speed, and more accurate scientific inferences. Today, it provides more than 600 users access to more analysis and visualization products than any other single source.

54 ENVIRONMENTAL SCIENCES↗

SU-E-J-78: Adaptive Planning Workflow in a Pencil Beam Scanning Proton Therapy Center

Purpose: The susceptibility of proton therapy to changes in patient setup and anatomy necessitates an adaptive planning process. With the right planning tools and clinical workflow, an adaptive plan can be created in a timely manner without adding significant workload to the treatment planning staff. Methods: In our center, a weekly QA CT is performed on most patients to assess setup, anatomy change, and tumor response. The QA CT is fused to the treatment planning CT, the contours are transferred via deformable registration, and the plan dose is recalculated on the QA CT. A physicist assesses the dose distribution, and an adaptive plan is requested based on tumor coverage or OAR dose changes. After the physician confirms or alters the deformed contours, a dosimetrist develops an adaptive plan using our TPS adaptation module. The plan is assessed for robustness and is then reviewed by the physician. Patient QA is performed within three days following the first adapted treatment. Results: Of the patients who received QA CTs, 19% required at least one adaptive plan (18.5% H&N, 18.5% brain, 11.1% breast, 14.8% chestwall, 14.8% lung, 18.5% pelvis and 3.8% abdomen). Of these patients, 14% went on a break, while the remainder was treated with the previous plan during the re-planning process. Adaptive plans were performed based on tumor shrinkage, anatomy change or positioning uncertainties for 37.9%, 44.8%, and 17.3% of the patients, respectively. On average, 3 full days are required between the QA CT and the first adapted plan treatment. Conclusion: Adaptive planning is a crucial component of proton therapy and should be applied to any site when the QA CT shows significant deviation from the plan. With an efficient workflow, an adaptive plan can be applied without delaying patient treatment or burdening the dosimetry and medical physics team.

60 APPLIED LIFE SCIENCES↗