Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Detecting anomalous packets in network transfers: investigations using PCA, autoencoder and isolation forest in TCP

Large-scale scientific workflows rely heavily on high-performance file transfers. These transfers require strict quality parameters such as guaranteed bandwidth, no packet loss or data duplication. To have successful file transfers, methods such as predetermined thresholds and statistical analysis need to be done to determine abnormal patterns. Network administrators routinely monitor and analyze network data for diagnosing and alleviating these, making decisions based on their experience. However, as networks grow and become complex, monitoring large data files and quickly processing them, makes it improbable to identify errors and rectify these. Abnormal file transfers have been classified by simply setting alert thresholds, via tools such as PerfSonar and TCP statistics (Tstat). This paper investigates the feasibility of unsupervised feature extraction methods for identifying network anomaly patterns with three unsupervised classification methods—principal component analysis, autoencoder and isolation forest. Here, we collect file transfer statistics from two experiment sets—synthetic iPerf generated traffic and 1000 Genome workflow runs, with synthetically introduced anomalies. Our results show that while PCA and a simple autoencoder finds it difficult to detect clusters, the tree-variant isolation forest is able to identify anomalous packets by breaking down TCP traces into tree classes early.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

LandScan HD: a high-resolution gridded ambient population methodology for the world

Unwarned population distributions accounting for routine human activities are needed to address many global human security challenges, including disasters, conflict, and infrastructure demand. LandScan High Definition (LSHD) supports this need through gridded ambient population estimates that measure average human presence between daytime and nighttime at a high spatial resolution of 3 arcseconds (approximately 90 m). Although LSHD has traditionally been produced on a country-specific basis, advances in global foundational data and computational resources now enable scaling its methodology to the world. Combining aspects of top-down and bottom-up gridded population methods, LSHD allocates subnational population totals from authoritative statistics to built-up areas based on occupancy estimates for multiple facility types (e.g., residential, commercial) and then reaggregates these estimates to a global population grid. We scale this approach by organizing the LSHD data stack into a 1° resolution tileset of vector analytic features, enabling an efficient and repeatable workflow for all countries worldwide. Examining the Philippines as an output of the global LSHD baseline dataset, we contrast unwarned and residential (WorldPop) population distributions by (1) exploring a practical application of flood risk assessment and (2) evaluating their congruence with outcomes of collective human activities (subnational CO 2 emissions). Finally, we discuss plans to address current LSHD limitations through data/modeling and uncertainty quantification improvements and provide outlook for workflow automation and extending the model to social, demographic and economic population characteristics.

Building morphology↗
DOE OSTI2024

Oil-Pressure Based Apparatus for In-Situ High-Energy Synchrotron X-Ray Diffraction Studies During Biaxial Deformation

Background: Understanding biaxial loading response at the microstructural level is crucial in helping better design sheet manufacturing processes and calibrate/validate material deformation models. Objective: The objective of this work was to develop a low-cost testing apparatus to probe, with sufficient spatial resolution, the micro-mechanical response of a sheet material in-situ under biaxial loading conditions. Methods: The testing apparatus fabricated as a part of this study operates in a similar fashion to a standard bulge test and uses oil pressure to generate biaxial loading conditions. This biaxial testing apparatus was operated within a synchrotron beamline to characterize the mechanical response of a flash-processed steel sheet using in-situ high-energy X-ray diffraction (XRD) measurements. Further, the GSAS-II package was utilized to develop a workflow for the analysis of the large volume of diffraction data acquired. The workflow was then used to extract the peak position, width, and integrated intensity of the XRD peaks corresponding to the major body-centered cubic phase. Results: The equi-biaxial nature of the loading in the measured area was independently corroborated using experimental (XRD) and simulation (finite element analysis) methods. Furthermore, we discuss the evolution of elastic strain in the major body-centered cubic phase as a function of applied oil pressure and location on the steel sheet. Conclusions: A key advantage of the biaxial apparatus fabricated in this synchrotron study is demonstrated using the results obtained for the flash-processed steel sheet – i.e., mapping the lattice plane-dependent response to biaxial loading for a relatively large sample area in a spatially resolved manner.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Ad Hoc File Systems for High-Performance Computing

Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include nodelocal SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 “Challenges and Opportunities of User-Level File Systems for HPC” and discusses application scenarios as well as design strategies for ad hoc file systems using node-local storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Multimodal Defect Imaging of Pure Tungsten Components Fabricated via Electron Beam Powder Bed Fusion

The utilization of additive manufacturing (AM) techniques for refractory materials in high-temperature environments has significantly expanded because of the ability to fabricate geometrically complex components. Electron beam powder bed fusion (EB-PBF), which provides lower residual stress, a cleaner vacuum environment, and better efficiency for high melting point, is one of the best-suited AM methods to produce advanced refractory components. However, the property variation attributed to the heterogeneous microstructure and process-induced defects has hindered the widespread adoption of EB-PBF-produced material like tungsten. While numerous in-situ monitoring and defect detection methods have been demonstrated for EB-PBF, a workflow that compares and evaluates process-induced abnormalities from different imaging perspectives is still limited. This study examines a feature-embedded tungsten component manufactured via the EB-PBF process to demonstrate the defect detection capabilities of a multimodal defect imaging workflow. The predefined and process-induced defects are evaluated by harnessing various imaging techniques, including in-situ electron imaging, layerwise near-infrared (NIR) imaging, post-build high-energy x-ray computed tomography (CT), and conventional destructive metallography. The results highlight the strengths and limitations of distinctive defect imaging techniques concerning specific defect types, sizes, and conditions. It was found that electron imaging can provide more abnormal detection capabilities while maintaining a higher measuring accuracy, against the conventional metallography in this case study, compared with NIR and CT imaging techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Advances in Multimodal Characterization of Structural Materials

The myriad detectors and instruments now available for materials characterization provide researchers with an ever-growing suite of tools to probe material behavior. Progress in the development of instrumentation and workflows that enable the collection, and leverage the potential, of various data modalities have provided novel insights into material behavior. Using data across multiple length scales, or performing complementary analyses of in situ and ex situ data, can help reveal a more complete picture of dynamic processes or material structure. However, the accurate combination, or fusion, of these disparate data modalities presents new challenges. Differences in resolution, as well as the varying length scales at which physical phenomena are exploited to generate these data, necessitate novel approaches to accurately interpret and combine these data. Furthermore, the papers within this special topic focus on the collection and fusion of multimodal data to better understand structural materials. From new frameworks and workflows for data segmentation and analysis, process monitoring, enhancing simulations, or interrogating mechanical response, these papers reveal the potential benefits of utilizing multimodal data.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Enhancing occupant behavior representation for interoperability between building information modeling and building energy modeling

Building Performance Simulation (BPS) has been adopted as an essential tool for designing, operating, and retrofitting buildings to optimize energy efficiency throughout the building life cycle. The Green Building XML (gbXML) schema facilitates seamless data exchange between Building Information Modeling (BIM) and Building Energy Modeling (BEM) software tools. However, limited occupant behavior (OB) representation in BIM often leads to inconsistent and inaccurate energy simulation in BEM software. This paper presents 154 systematic enhancements to the existing occupant behavior XML (obXML) schema v1.3.4, initially developed for standardizing OB representation for BEM, to address existing limitations and improve interoperability with BIM models. The enhancements encompass improved integration with BIM models through extended building representations and system operations, expanded support for advanced OB models with additional environmental parameters and mathematical capabilities, and implementation of a standardized model documentation framework. To facilitate seamless data transformation between gbXML and obXML schemas, we developed a publicly available gb-obXML Schema Converter. Three case studies demonstrate the enhanced schema’s capabilities: representation of building information using a two-story office building model, documentation of a window operation behavior model, and validation of the schema converter’s functionality. The enhanced obXML schema v1.4 enables sophisticated modeling of occupant-building interactions while maintaining consistency with industry-standard BIM schemas. The standardized documentation framework facilitates reproducibility and knowledge sharing in the OB research community, while the schema converter automates the integration of building information into OB simulation workflows. These enhancements establish a foundation for more accurate building performance simulation by supporting sophisticated representation of occupant behavior within the BIM-to-BEM simulation workflows.

Chung, Jihoon↗
DOE OSTI2021

Performance of Julia for High Energy Physics Analyses

We argue that the Julia programming language is a compelling alternative to currently more common implementations in Python and C++ for common data analysis workflows in high energy physics. We compare the speed of implementations of different workflows in Julia with those in Python and C++. Furthermore, our studies show that the Julia implementations are competitive for tasks that are dominated by computational load rather than data access. For work that is dominated by data access, we demonstrate an application with concurrent file reading and parallel data processing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Analysis-Specific Fast Simulation at the LHC with Deep Learning

Abstract We present a fast-simulation application based on a deep neural network, designed to create large analysis-specific datasets. Taking as an example the generation of W + jet events produced in $$\sqrt{s}=$$ s = 13 TeV proton–proton collisions, we train a neural network to model detector resolution effects as a transfer function acting on an analysis-specific set of relevant features, computed at generation level, i.e., in absence of detector effects. Based on this model, we propose a novel fast-simulation workflow that starts from a large amount of generator-level events to deliver large analysis-specific samples. The adoption of this approach would result in about an order-of-magnitude reduction in computing and storage requirements for the collision simulation workflow. This strategy could help the high energy physics community to face the computing challenges of the future High-Luminosity LHC.

Chen, C.↗
DOE OSTI2024

PanDA: Production and Distributed Analysis System

The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at the LHC data processing scale. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year. PanDA’s flexibility and scalability make it suitable for the High Energy Physics community and wider science domains at the Exascale. Beyond High Energy Physics, PanDA’s relevance extends to other big data sciences, as evidenced by its adoption in the Vera C. Rubin Observatory and the sPHENIX experiment. As the significance of advanced workflows continues to grow, PanDA has transformed into a comprehensive ecosystem, effectively tackling challenges associated with emerging workflows and evolving computing technologies. The paper discusses PanDA’s prominent role in the scientific landscape, detailing its architecture, functionality, deployment strategies, project management approaches, results, and evolution into an ecosystem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Mode connectivity in the loss landscape of parameterized quantum circuits

Variational training of parameterized quantum circuits (PQCs) underpins many workflows employed on near-term noisy intermediate scale quantum (NISQ) devices. It is a hybrid quantum-classical approach that minimizes an associated cost function in order to train a parameterized ansatz. In this work we adapt the qualitative loss landscape characterization for neural networks introduced in Goodfellow et al. (2014); Li et al. (2017) and tests for connectivity used in Draxler et al. (2018) to study the loss landscape features in PQC training. We present results for PQCs trained on a simple regression task, using the bilayer circuit ansatz, which consists of alternating layers of parameterized rotation gates and entangling gates. Multiple circuits are trained with 3 different batch gradient optimizers: stochastic gradient descent, the quantum natural gradient, and Adam. We identify large features in the landscape that can lead to faster convergence in training workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A data-centric weak supervised learning for highway traffic incident detection

Using the data from loop detector sensors for near-real-time detection of traffic incidents on highways is crucial to averting major traffic congestion. While recent supervised machine learning methods offer solutions to incident detection by leveraging human-labeled incident data, the false alarm rate is often too high to be used in practice. Specifically, the inconsistency in the human labeling of the incidents significantly affects the performance of supervised learning models. To that end, we focus on a data-centric approach to improve the accuracy and reduce the false alarm rate of traffic incident detection on highways. We develop a weak supervised learning workflow to generate high-quality training labels for the incident data without the ground truth labels, and we use those generated labels in the supervised learning setup for final detection. This approach comprises three stages. First, we introduce a data preprocessing and curation pipeline that processes traffic sensor data to generate high-quality training data through leveraging labeling functions, which can be domain knowledge-related or simple heuristic rules. Second, we evaluate the training data generated by weak supervision using three supervised learning models-random forest, k-nearest neighbors, and a support vector machine ensemble-and long short-term memory classifiers. The results show that the accuracy of all of the models improves significantly after using the training data generated by weak supervision. Third, we develop an online real-time incident detection approach that leverages the model ensemble and the uncertainty quantification while detecting incidents. Finally, we show that our proposed weak supervised learning workflow achieves a high incident detection rate (0.90) and low false alarm rate (0.08).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Demonstration and analysis of volumetric additive manufacturing via sub-orbital spaceflight testing

Computed Axial Lithography (CAL) represents a significant advancement in the emerging field of Volumetric Additive Manufacturing (VAM). CAL addresses key limitations of traditional photopolymer additive manufacturing technologies, by eliminating the need for layering and support structures. Unlike conventional methods, CAL prints components by illuminating all points within a desired geometry simultaneously, using tomographic reconstruction to form the object in a single step. This unique approach eliminates the relative motion between the object and the precursor material, enabling faster printing speeds and reducing the waste associated with support structures. However, CAL parts require post-processing steps before they can be utilized. CAL's core attributes make it particularly suited for In-Space Manufacturing (ISM), due to its fast fabrication times, wide breadth of materials it can use, and minimized footprint. CAL has been successfully demonstrated in microgravity during parabolic flight experiments. However to fully validate and understand CAL's behaviour in microgravity, all manufacturing and post-processing steps must be integrated. In June 2024, we conducted SpaceCAL Mission 3, testing this entire workflow on a suborbital flight aboard Virgin Galactic's SpaceShipTwo. During ~140 s of microgravity, the system autonomously manufactured and post-processed four parts using PEGDA700 resin. Post-flight analysis showed that 2/4 parts were recognisable, while others were distorted due to bubble formation from residual water droplets, off-axis optical aberrations, and non-uniform solvent rinsing. Despite these limitations, this study represents the first integrated CAL workflow in space, providing an initial experimental demonstration and analysis for closed-loop in-space manufacturing.

Waddell, Taylor [Department of Mechanical Engineer↗
DOE OSTI2026

Future foundries: A convergent manufacturing platform

This article introduces the Future Foundries platform developed at Oak Ridge National Laboratory, a first-generation research system designed to demonstrate convergent manufacturing. Convergent manufacturing brings together additive, subtractive, and transformative processes in a digitally interconnected environment to enable end-to-end production workflows. By linking traditionally discrete steps, convergent platforms accelerate production, improve repeatability, and support high-mix, low-volume manufacturing. The Future Foundries platform exemplifies this vision in practice by combining four modular, vendor-agnostic process cells that include robotic WAAM, induction heating, optical metrology, and machining, coordinated through an automated pallet handler and a ROS 2-based digital thread. This architecture provides the flexibility and scalability needed for agile production in small and medium-sized manufacturing enterprises and for field deployable manufacturing. Two case studies illustrate the platform’s capabilities. The first presents an integrated workflow for fabricating, transforming, and repairing critical replacement components, showing how consolidated thermal, additive, inspection, and machining operations reduce manual part handling and streamline process flow. The second case study highlights coordinated multi-part production enabled by automated pallet logistics and multi-cell scheduling. Together, these examples showcase convergent manufacturing as a practical and scalable strategy for strengthening domestic casting and forging capacity, improving supply-chain resilience, and enabling rapid, adaptable production of mission-critical components.

Convergent manufacturing↗
DOE OSTI2023

Application of unsupervised deep learning to image segmentation and in-situ contact angle measurements in a CO 2 -water-rock system

Rock surface wettability is a critical property that regulates multiphase flows in porous media, which can be quantified using the surface contact angle (CA). X-ray micro-computed tomography (μCT) provides an effective approach to in-situ measurements of surface CAs. However, the CA measurement accuracy depends significantly on the quality of CT image segmentation, which is the clustering of CT pixels into separate phases. Inspired by this, we developed a deep learning (DL)-based CA measurement workflow. Motivated by the recent tremendous progress in unsupervised learning techniques and aiming to avoid expensive manual data annotations, an unsupervised DL pipeline for CT image segmentation was proposed and implemented, which includes unsupervised model training and post-processing. The unsupervised model training was driven by a novel loss function constrained with feature similarity and spatial continuity and implemented by iterative forward and backward paths; the former clustered the pixel-wise feature vectors extracted by convolution neural networks, whereas the latter updated the parameters using gradient descent. An over-segmentation strategy was adopted for model training. The post-processing steps based on agglomerative hierarchical clustering (AHC) were implemented to further merge the over-segmented model output to the desired cluster number, which is intended to improve the efficiency of image segmentation. The developed unsupervised DL pipeline was compared with other commonly-used image segmentation methods using pixel-wise and physics-based evaluation metrics on a synthetic raw-image dataset, which had a known ground truth. The unsupervised DL pipeline showed the best performance. Next, the segmented images were input to an automatic CA measurement tool, and the results were validated by comparisons with manual measurements. The CA values from the manual and automatic measurements showed similar distributions and statistical properties. The automatic measurement demonstrated a wider spectrum because of the much larger number of measurement data points. The primary novelty of the unsupervised DL pipeline developed in this study lies in the novel loss function and the over-segmentation strategy associated with AHC post-processing. Finally, the workflow has been proven an efficient tool for pore-scale wettability characterization, which has a wide range of applications in fundamental studies of multiphase flows in natural porous media, which have critical implications to geological carbon sequestration, hydrocarbon energy recovery, and contaminant transport in groundwater.

42 ENGINEERING↗
DOE OSTI2022

Prediction of local concentration fields in porous media with chemical reaction using a multi scale convolutional neural network

The study of solute transport in porous media is of interest in many chemical engineering systems. Some example applications include packed bed catalytic reactors, filtration devices, and batteries. The pore scale modeling of these systems is time consuming and may require large computing resources, for this reason computational fluid dynamics (CFD) simulations are not practical if a large number of simulations is required, like in multiscale modeling, where a model at a large scale calls for pore scale simulations. It has been shown that neural networks can be trained with a dataset of flow simulations and then predict fields orders of magnitude faster, and with less computational resources, in new domains. However, it is crucial to provide the neural network with an effective description of the domain and the undergoing operating conditions to be able to train models that generalize accurately in unseen samples. Therefore, research is needed to employ neural networks in new complex systems. The appropriate training of a network for predicting coupled flow and solute transport processes is an outstanding problem due to the complex interplay between geometry and operating conditions. In this work, we train a multi scale convolutional neural network (MSNet) with a diverse dataset of simulations of transport and chemical reaction in porous media to predict the local concentration fields in images of porous media. Our dataset contains a wide diversity of sphere pack arrangements under different operating conditions (Péclet and Reynolds numbers). Further, we train a robust model by employing different input descriptors that represent the medium and the different operating conditions of each system. Our trained model is able to provide nearly instantaneous predictions, compared to around twenty hours of the CFD workflow, with less than 3.5% error on new geometries and transport conditions. Thus the model could be easily integrated in a multiscale workflow where fast response is needed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Atlas of Transcription Factor Binding Sites from ENCODE DNase Hypersensitivity Data across 27 Tissue Types

Characterizing the tissue-specific binding sites of transcription factors (TFs) is essential to reconstruct gene regulatory networks and predict functions for non-coding genetic variation. DNase-seq footprinting enables the prediction of genome-wide binding sites for hundreds of TFs simultaneously. Despite the public availability of high-quality DNase-seq data from hundreds of samples, a comprehensive, up-to-date resource for the locations of genomic footprints is lacking. Here, we develop a scalable footprinting workflow using two state-of-the-art algorithms: Wellington and HINT. We apply our workflow to detect footprints in 192 ENCODE DNase-seq experiments and predict the genomic occupancy of 1,515 human TFs in 27 human tissues. We validate that these footprints overlap true-positive TF binding sites from ChIP-seq. We demonstrate that the locations, depth, and tissue specificity of footprints predict effects of genetic variants on gene expression and capture a substantial proportion of genetic risk for complex traits.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Ultrahigh performance LC/FT-MS non-targeted screening for biomass burning organic aerosol with MZmine2 and MFAssignR

In recent years, ultrahigh performance liquid chromatography Fourier transform mass spectrometry (LC/FT-MS) based non-targeted screening (NTS) methods have become increasingly popular for comprehensive analysis of complex organic mixtures. However, applying these methods for environmental complex mixture analysis is challenging due to the extreme complexity of natural samples and a lack of standard samples or surrogates for environmental complex mixtures. Furthermore, limited molecular markers in the databases and insufficient data processing software workflows make the application of these methods more challenging for environmental complex mixtures. In this work, we implement a new NTS data processing workflow to process data collected from ultrahigh performance liquid chromatography and Fourier transform Orbitrap Elite Mass Spectrometry (LC/FT-MS) by combining MZmine2 and MFAssignR, two opensource data processing tools and commercial Mesquite liquid smoke as a surrogate for biomass burning organic aerosol. MZmine2.53 data extraction followed MFAssignR molecular formula assignment offered noise free and highly accurate 1733 individual molecular formulas presented in liquid smoke with 4906 molecular species, including isomers. The results of this new approach were consistent with the results of direct infusion FT-MS analysis confirming its reliability. Over 90% of the molecular formulas presented in mesquite liquid smoke were matched with the molecular formulas of ambient biomass burning organic aerosol. This suggests the potential use of commercial liquid smoke as a surrogate for biomass burning organic aerosol research. Furthermore, the presented method significantly improves the identification of the molecular composition of biomass burning organic aerosol by successfully addressing some of the limitations related to the data analysis and giving a semi quantitative insight into the analysis.

54 ENVIRONMENTAL SCIENCES↗