SEARCH · Engineering Papers

Results for “workflow systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

All sources NASA NTRS DOE OSTI

DOE OSTI2024

PanDA: Production and Distributed Analysis System

The Production and Distributed Analysis (PanDA) system is a data-driven workload management system engineered to operate at the LHC data processing scale. The PanDA system provides a solution for scientific experiments to fully leverage their distributed heterogeneous resources, showcasing scalability, usability, flexibility, and robustness. The system has successfully proven itself through nearly two decades of steady operation in the ATLAS experiment, addressing the intricate requirements such as diverse resources distributed worldwide at about 200 sites, thousands of scientists analyzing the data remotely, the volume of processed data beyond the exabyte scale, dozens of scientific applications to support, and data processing over several billion hours of computing usage per year. PanDA’s flexibility and scalability make it suitable for the High Energy Physics community and wider science domains at the Exascale. Beyond High Energy Physics, PanDA’s relevance extends to other big data sciences, as evidenced by its adoption in the Vera C. Rubin Observatory and the sPHENIX experiment. As the significance of advanced workflows continues to grow, PanDA has transformed into a comprehensive ecosystem, effectively tackling challenges associated with emerging workflows and evolving computing technologies. The paper discusses PanDA’s prominent role in the scientific landscape, detailing its architecture, functionality, deployment strategies, project management approaches, results, and evolution into an ecosystem.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Invertible Temper Modeling using Normalizing Flows and the Effects of Structure Preserving Loss

Advanced manufacturing research and development is typically small-scale, owing to costly experiments associated with these novel processes. Deep learning techniques could help accelerate this development cycle but frequently struggle in small-data regimes like the advanced manufacturing space. While prior work has applied deep learning to modeling visually plausible advanced manufacturing microstructures, little work has been done on data-driven modeling of how microstructures are affected by heat treatment, or assessing the degree to which synthetic microstructures are able to support existing workflows. We propose to address this gap by using invertible neural networks (normalizing flows) to model the effects of heat treatment, e.g., tempering. The model is developed using scanning electron microscope imagery from samples produced using shear-assisted processing and extrusion (ShAPE) manufacturing. This approach not only produces visually and topologically plausible samples, but also captures information related to a sample’s material properties or experimental process parameters. We also demonstrate that topological data analysis, used in prior work to characterize microstructures, can also be used to stabilize model training, preserve structure, and improve downstream results. We assess directions for future work and identify our approach as an important step towards end-to-end deep learning system for accelerating advanced manufacturing research and development.

Howland, Sylvia↗

DOE OSTI2022

Modular automated bottom-up proteomic sample preparation for high-throughput applications

Manual proteomic sample preparation methods limit sample throughput and often lead to poor data quality when thousands of samples must be analyzed. Automated liquid handler systems are increasingly used to overcome these issues for many of the sample preparation steps. Here, we detail a step-by-step protocol to prepare samples for bottom-up proteomic analysis for Gram-negative bacterial and fungal cells. The full modular protocol consists of three optimized protocols to: (A) lyse Gram-negative bacteria and fungal cells; (B) quantify the amount of protein extracted; and (C) normalize the amount of protein and set up tryptic digestion. These protocols have been developed to facilitate rapid, low variance sample preparation of hundreds of samples, be easily implemented on widely-available Beckman-Coulter Biomek automated liquid handlers, and allow flexibility for future protocol development. By using this workflow 50 micrograms of protein from 96 samples can be prepared for tryptic digestion in under an hour. We validate these protocols by analyzing 47 Pseudomonas putida and Rhodosporidium toruloides samples and show that this modular workflow provides robust, reproducible proteomic samples for high-throughput applications. The expected results from these protocols are 94 peptide samples from Gram-negative bacterial and fungal cells prepared for bottom-up quantitative proteomic analysis without the need for desalting column cleanup and with protein relative quantity variance (CV%) below 15%.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Analyzing the Impact of Future Weather Data on Energy Consumption in Weatherization Assistant

This study supports the mission of the U.S. Department of Energy’s Weatherization Assistance Program (WAP), which aims to increase the energy efficiency of dwellings and reduce their total residential expenditures. Specifically, we examine how projected future climate conditions may affect residential building energy performance by integrating future weather data into the National Energy Audit Tool (NEAT). Since WAP evaluates the cost-effectiveness of retrofit measures over lifespans of up to 30 years, accounting for evolving climate conditions is increasingly important. To reflect future household energy demands, this study replaces historically based Typical Meteorological Year (TMY3) weather inputs with Future Typical Meteorological Year (fTMY) datasets derived from global climate model (GCM) projections. A simulation-based framework was established to enable NEAT analysis under future weather conditions. This workflow involves converting EPW-format weather files into JSON inputs compatible with NEAT and generating degree-hour metrics needed for load calculations. The fTMY dataset used in this study was developed by Oak Ridge National Laboratory through downscaling of six GCMs under different emission scenarios and covers the period from 2020 to 2100. In contrast, the TMY3 dataset is based on historical weather data from 1961 to 1990. Simulations were conducted for benchmark single-family prototype buildings across ASHRAE climate zones 1–7, which cover all regions of the U.S. except the subarctic Zone 8 in northern Alaska, evaluating both heating and cooling loads under TMY3 and fTMY conditions. Four foundation types were tested, while heating systems were standardized, as NEAT does not differentiate thermal energy load by HVAC system type in its load calculations. Results show that fTMY weather input consistently yield lower heating loads and higher cooling loads across most locations, aligning with expected climate warming trends. Notably, colder regions such as zones 6A, 6B, and 7 experience marked reductions in heating load, while warmer and transitional zones, such as 2A (Lufkin, TX) and 3C (San Francisco, CA), have substantial increases in cooling loads. Although this study does not directly assess the performance of retrofit measures under future climate conditions, it provides a critical foundation for doing so. By quantifying shifts in baseline (i.e., pre-retrofit case) energy loads between historical and future weather files, the study highlights the importance of integrating climate-responsive data into audit tools. These findings will inform future efforts to evaluate the long-term effectiveness and cost-effectiveness of weatherization measures under changing climate conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2020

Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions

The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.

Durden, David J.↗

DOE OSTI2024

Assessing horizontal gene transfer in the rhizosphere of Brachypodium distachyon using fabricated ecosystems (EcoFABs)

ABSTRACT Horizontal gene transfer (HGT) is a major process by which genes are transferred between microbes in the rhizosphere. However, examining HGT remains challenging due to the complexity of mimicking conditions within the rhizosphere. Fabricated ecosystems (EcoFABs) have been used to investigate several complex processes in plant-associated environments. Here we show that EcoFABs are efficient tools to examine and measure HGT frequency in the rhizosphere. We provide the first demonstration of gene transfer via a triparental conjugation system in the Brachypodium distachyon rhizosphere in an EcoFAB using Pseudomonas putida KT2440 as both donor and recipient bacterial strain with the donor containing a mobilizable and non-self-transmissible plasmid. We observed that the frequency of plasmid transfer in the rhizosphere is potentially dependent on the plant developmental stage and the composition and amount of root exudates. The frequency of plasmid transfer also increased with higher numbers of donor cells. We demonstrate the transfer of plasmid from P. putida to another B. distachyon root colonizer, Burkholderia sp. OAS925, showing HGT within a rhizosphere microbial community. Environmental stresses also influenced the rate and efficiency of HGT in the rhizosphere between different species and genera. This study provides a robust workflow to evaluate transfer of engineered plasmids in the rhizosphere when such plasmids are potentially introduced in a field or other plant-associated environments. IMPORTANCE We report the use of EcoFABs to investigate the HGT process in a rhizosphere environment. It highlights the potential of EcoFABs in recapitulating the dynamic rhizosphere conditions as well as their versatility in studying plant-microbe interactions. This study also emphasizes the importance of studying the parameters impacting the HGT frequency. Several factors such as plant developmental stages, nutrient conditions, number of donor cells, and environmental stresses influence gene transfer within the rhizosphere microbial community. This study paves the way for future investigations into understanding the fate and movement of engineered plasmids in a field environment.

Priya, Shweta↗

DOE OSTI2021

Toward a Seamless Integration of Computing, Experimental, and Observational Science Facilities: A Blueprint to Accelerate Discovery

The Department of Energy, Office of Science operates world-leading facilities for experimental, observational, and computational science. DOE supercomputing facilities will reach performance at the scale of ExaFLOPs in the coming years, enabling new vistas of scale and precision for large scale simulations and data analysis. Experimental scientific facilities are undergoing similar upgrades that will lead to higher data rates and correspondingly larger computational demands, and will increase the need for near-real-time processing and resilient support for more complex workflows. A transformation of science is underway, with workloads at supercomputing facilities increasingly driven by this explosion of data from instruments and experimental facilities, as well as the accelerating use of Artificial Intelligence (AI) as a tool for scientific discovery. A seamless integration of computing, networking, instruments, and experimental facilities is required to support these emerging workloads and open up a new frontier of U.S. leadership in scientific discovery. We propose to accomplish this by providing frictionless access to the ASCR supercomputing facilities. We describe our vision of combining the power of ASCR supercomputers and networking infrastructure into an integrated scalable fabric, available to end user scientists via interfaces that aim to automate and simplify access to high performance computing systems. This will enable unprecedented computational science capabilities for experimental and observational facilities, and will create new opportunities to combine large simulations and modeling with experimental facility data analysis. This blueprint for creating an integrated network of computational and experimental facilities will provide an enriched discovery environment and open doors for new scientific communities to access the DOE’s world-leading computing and networking capabilities.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2004

Grid Enabled Geospatial Catalogue Web Service

Geospatial Catalogue Web Service is a vital service for sharing and interoperating volumes of distributed heterogeneous geospatial resources, such as data, services, applications, and their replicas over the web. Based on the Grid technology and the Open Geospatial Consortium (0GC) s Catalogue Service - Web Information Model, this paper proposes a new information model for Geospatial Catalogue Web Service, named as GCWS which can securely provides Grid-based publishing, managing and querying geospatial data and services, and the transparent access to the replica data and related services under the Grid environment. This information model integrates the information model of the Grid Replica Location Service (RLS)/Monitoring & Discovery Service (MDS) with the information model of OGC Catalogue Service (CSW), and refers to the geospatial data metadata standards from IS0 19115, FGDC and NASA EOS Core System and service metadata standards from IS0 191 19 to extend itself for expressing geospatial resources. Using GCWS, any valid geospatial user, who belongs to an authorized Virtual Organization (VO), can securely publish and manage geospatial resources, especially query on-demand data in the virtual community and get back it through the data-related services which provide functions such as subsetting, reformatting, reprojection etc. This work facilitates the geospatial resources sharing and interoperating under the Grid environment, and implements geospatial resources Grid enabled and Grid technologies geospatial enabled. It 2!so makes researcher to focus on science, 2nd not cn issues with computing ability, data locztic~, processir,g and management. GCWS also is a key component for workflow-based virtual geospatial data producing.

Chen, Ai-Jun↗

DOE OSTI2023

Laser Driven Hydrothermal Processing (LDHP) for Rapid Dissolution of Glassy Nuclear Debris

Discovered in 2009 on the SLAB laser system at LLNL, Laser-Driven Hydrothermal Processing (LDHP) both spalls and transiently dissolves a wetted surface (e.g., rock, concrete, paint) when that surface is struck by a laser beam pulse under specific conditions. Previous projects have explored the application of LDHP to expediting sample dissolution for post-detonation nuclear forensics. Early results were very promising, indicating that this method could decrease material dissolution time by a factor of 10 or more. However, the large and complicated lasers previously used are impractical for this application. Under the R^3 venture we aimed to benchmark the LDHP capabilities of turn-key commercial off-the-shelf (COTS) lasers, and to establish how this process could be integrated into an analytical workflow for nuclear forensic analysis. We demonstrated that geological rock standards could be quickly processed into fine powder by LDHP followed by complete acid digestion in 15 minutes, without elemental fractionation that would skew analytical results. We determined that the COTS laser can be used for LDHP that could potentially reduce dissolution time for solid nuclear debris samples in post detonation nuclear forensics. One downside to this approach, however, is that the laser interactions are different with every material, so extensive method validation would be required for all likely material compositions. The LDHP processing rates of several materials were tested and determined to be favorable in the beginning with significant slowdown due to particulate scattering. This could be mitigated either by stopping and centrifuging the sample at specific intervals, or through design of a continuous solvent flow through mechanism.

36 MATERIALS SCIENCE↗

DOE OSTI2021

A New Method for High Resolution Surface Change Detection: Data Collection and Validation of Measurements from UAS at the Nevada National Security Site, Nevada, USA

The use of uncrewed aerial systems (UAS) increases the opportunities for detecting surface changes in remote areas and in challenging terrain. Detecting surface topographic changes offers an important constraint for understanding earthquake damage, groundwater depletion, effects of mining, and other events. For these purposes, changes on the order of 5–10 cm are readily detected, but sometimes it is necessary to detect smaller changes. An example is the surface changes that result from underground explosions, which can be as small as 3 cm. Previous studies that described change detection methodologies were generally not aimed at detecting sub-5-cm changes. Additionally, studies focused on high-fidelity accuracy were either computationally modeled or did not fully provide the necessary examples to highlight the usability of these workflows. Detecting changes at this threshold may be critical in certain applications, such as global security research and monitoring for high-consequence natural hazards, including landslides. Here we provide a detailed description of the methodology we used to detect 2–3 cm changes in an important applied research setting—surface changes related to underground explosions. This methodology improves the accuracy of change detection data collection and analysis through the optimization of pre-field planning, surveying, flight operations, and post-processing the collected data, all of which are critical to obtaining the highest output data resolution possible. We applied this methodology to a field study location, collecting 1.4 Tb of images over the course of 30 flights, and location data for 239 ground control points (GCPs). We independently verified changes with orthoimagery, and found that structure-from-motion, software-reported root mean square errors (RMSEs) for both control and check points underestimated the actual error. We found that 3 cm changes are detectable with this methodology, thereby improving our knowledge of a rock’s response to underground explosions.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

The Urban Deployment Model: A Toolset for the Simulation and Performance Characterization of Radiation Detector Deployments in Urban Environments

Static and mobile radiation detectors can be deployed in urban environments for a range of nuclear security applications, including radiological source search-and-tracking scenarios. Modeling detector performance for such applications is challenging, as it does not depend solely on the detector capabilities themselves. Many factors must be taken into consideration, including specific source and background signatures, the topology and constraints of the deployment environment, the presence of nuisance sources, and whether detectors are mobile or static. When considering the simultaneous deployment of multiple, heterogeneous detectors, assessment of the system-wide performance requires the simulation of the individual detectors, and a system-level analysis of the detection performance. In radiological source search-and-tracking scenarios, performance is mostly dominated by the probability of encounter, which depends on the specifics of a given deployment, e.g., static vs. mobile detectors or a combination of both modalities, the number of detectors deployed, the dynamic vs. static setting of false alarm rates, and individual vs. networked operation. The Urban Deployment Model (UDM) toolset was specifically developed to cover the gap in the available generic frameworks for the simulation of radiation detector deployments at city scales. UDM provides a unified and modular framework to support the simulation and performance characterization of heterogeneous detector deployments in urban environments. This paper presents the key components along the UDM workflow.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

NASA NTRS2012

The Invasive Species Forecasting System (ISFS): An iRODS-Based, Cloud-Enabled Decision Support System for Invasive Species Habitat Suitability Modeling

The Invasive Species Forecasting System (ISFS) is an online decision support system that allows users to load point occurrence field sample data for a plant species of interest and quickly generate habitat suitability maps for geographic regions of interest, such as a national park, monument, forest, or refuge. Target customers for ISFS are natural resource managers and decision makers who have a need for scientifically valid, model- based predictions of the habitat suitability of plant species of management concern. In a joint project involving NASA and the Maryland Department of Natural Resources, ISFS has been used to model the potential distribution of Wavyleaf Basketgrass in Maryland's Chesapeake Bay Watershed. Maximum entropy techniques are used to generate predictive maps using predictor datasets derived from remotely sensed data and climate simulation outputs. The workflow to run a model is implemented in an iRODS microservice using a custom ISFS file driver that clips and re-projects data to geographic regions of interest, then shells out to perform MaxEnt processing on the input data. When the model completes, all output files and maps from the model run are registered in iRODS and made accessible to the user. The ISFS user interface is a web browser that uses the iRODS PHP client to interact with the ISFS/iRODS- server. ISFS is designed to reside in a VMware virtual machine running SLES 11 and iRODS 3.0. The ISFS virtual machine is hosted in a VMware vSphere private cloud infrastructure to deliver the online service.

Gill, Roger↗

NASA NTRS2022

Natural Language Understanding and Extraction of Flight Constraints Recorded in Letters of Agreement

This paper presents an automated information extraction and inference technique using natural language processing for extracting flight operational procedures and constraints embedded in heritage air traffic management documents. The extracted flight constraints can be digitized and fit into existing airspace information exchange models such as the Aeronautical Information Exchange Model (AIXM). This approach offers a digitized solution to disseminate airspace operating conditions to diverse air users and stakeholders in the National Airspace System (NAS). Furthermore, the digitized flight procedures can provide operational flexibility for emerging advanced air mobility providers and reduce traffic controller workload while maintaining current safety standards. To demonstrate this process, 1,972 Letters of Agreement (LOAs) have been selected for processing, named entity extraction, constraint identification and extraction. This dataset is derived from a subset of documents related to Air Route Traffic Control Centers (ARTCC) operations. We experimented with various traditional information extraction techniques, state-of-the-art machine learning and deep learning models to perform named entity recognition and pattern recognition on our dataset. We present the results from our experiments and demonstrate 99.0% F-1 score for named entity recognition, and a 96.6% accuracy for our entire workflow up to named entity recognition. We also discuss constraint definitions using generic patterned templates and extensions to this work in applying entity linking to digitally extracting relevant constraints.

Natural Language Processing↗

NASA NTRSDate not supplied

Natural Language Understanding and Extraction of Flight Constraints Recorded in Letters of Agreement

Natural Language Processing↗

DOE OSTI2023

CEGANN: Crystal Edge Graph Attention Neural Network for multiscale classification of materials environment

Abstract We introduce Crystal Edge Graph Attention Neural Network (CEGANN) workflow that uses graph attention-based architecture to learn unique feature representations and perform classification of materials across multiple scales (from atomic to mesoscale) and diverse classes ranging from metals, oxides, non-metals to hierarchical materials such as zeolites and semi-ordered mesophases. CEGANN can classify based on a global, structure-level representation such as space group and dimensionality (e.g., bulk, 2D, clusters, etc.). Using representative materials such as polycrystals and zeolites, we demonstrate its transferability in performing local atom-level classification tasks, such as grain boundary identification and other heterointerfaces. CEGANN classifies in (thermal) noisy dynamical environments as demonstrated for representative zeolite nucleation and growth from an amorphous mixture. Finally, we use CEGANN to classify multicomponent systems with thermal noise and compositional diversity. Overall, our approach is material agnostic and allows for multiscale feature classification ranging from atomic-scale crystals to heterointerfaces to microscale grain boundaries.

36 MATERIALS SCIENCE↗

DOE OSTI2025

JGI Plant Transformation Workshop, May 20-21, 2025

Domestic biomass crops such as sorghum, switchgrass, Miscanthus, and poplar can provide United States industries with renewable feedstocks while also supporting low-input farming systems and strengthening supply chains for biofuels, biochemicals and biomaterials. The U.S. leads globally in biomass crop genomics, yet progress in engineering traits is constrained by slow, genotype-dependent transformation methods and lengthy Design-Build-Test-Learn (DBTL) cycles. At a May 2025 workshop, a panel of experts recommended establishing a DOE Plant Transformation Capability (PTC) to overcome these barriers. The PTC would unite two missions: advancing research to achieve genotype-independent, automated methods, and delivering scalable transformation services through a user-facility model. With expected gains of 10–100x in efficiency, including transformation and cost reduction, the PTC would accelerate the path from discovery to engineered plants, expand community access and training, and support downstream applications and workflows including field trials and regulatory navigation. By enabling rapid and predictable crop engineering, the PTC would strengthen U.S. supply chains, enhance industrial competitiveness, and ensure that DOE’s genomic investments deliver national impact.

09 BIOMASS FUELS↗

DOE OSTI2024

Automated Network Services for Exascale Data Movement

The Large Hadron Collider (LHC) experiments distribute data by leveraging a diverse array of National Research and Education Networks (NRENs), where experiment data management systems treat networks as a “blackbox” resource. After the High Luminosity upgrade, the Compact Muon Solenoid (CMS) experiment alone will produce roughly 0.5 exabytes of data per year. NREN Networks are a critical part of the success of CMS and other LHC experiments. However, during data movement, NRENs are unaware of data priorities, importance, or need for quality of service, and this poses a challenge for operators to coordinate the movement of data and have predictable data flows across multi-domain networks. The overarching goal of SENSE (The Software-defined network for End-to-end Networked Science at Exascale) is to enable National Labs and universities to request and provision end-to-end intelligent network services for their application workflows leveraging SDN (Software-Defined Networking) capabilities. This work aims to allow LHC Experiments and Rucio, the data management software used by CMS Experiment, to allocate and prioritize certain data transfers over the wide area network. In this paper, we will present the current progress of the integration of SENSE, Multi-domain end-to-end SDN Orchestration with QoS (Quality of Service) capabilities, with Rucio, the data management software used by CMS Experiment.

Balcas, Justas↗

DOE OSTI2026

SetGo: Metadata Readiness for Scientific AI Datasets

Scientific datasets intended for AI use require both computational readiness for model training and metadata readiness for discovery, sharing, and reuse. The Readiness Engine for Data Integration (REDI) addresses computational readiness, but no corresponding tool evaluates whether a dataset’s metadata are sufficiently complete, governed, and standards-compliant for publication and agent-based consumption. Existing FAIR assessors operate only on published repository records, and no single system covers FAIR compliance, licensing, provenance, governance, reproducibility, and catalog readiness together. We present SetGo, an open-source Python toolkit that assesses and repairs metadata readiness across these six dimensions before a dataset is published or archived. Applied to four scientific corpora, SetGo surfaces deficiencies that general-purpose tools do not detect: ERA5 climate metadata scores 4% on ACDD 1.3 compliance; materials datasets fail OPTIMADE species-definition requirements; and PDB-derived proteomics data carries licensing terms incompatible with standard SPDX identifiers. Guided enrichment raises overall FAIR scores from 52–57% to 81–91%, and a single setgo publish command pushes to Hugging Face Hub, CKAN, or OpenMetadata with ML Commons Croissant 1.0 metadata sidecars. To support interactive and automated workflows, SetGo integrates with coding agents powered by large language models (LLMs) through a /setgo skill that enables natural-language execution of the full assess–enrich–publish loop, with user involvement limited to supplying missing metadata values.

Wilkinson, Sean [ORNL] (ORCID:0000000214437479)↗