Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

GENTANGLE: integrated computational design of gene entanglements

The design of two overlapping genes in a microbial genome is an emerging technique for adding more reliable control mechanisms in engineered organisms for increased stability. The design of functional overlapping gene pairs is a challenging procedure, and computational design tools are used to improve the efficiency to deploy successful designs in genetically engineered systems. GENTANGLE (Gene Tuples ArraNGed in overLapping Elements) is a high-performance containerized pipeline for the computational design of two overlapping genes translated in different reading frames of the genome. This new software package can be used to design and test gene entanglements for microbial engineering projects using arbitrary sets of user-specified gene pairs.

59 BASIC BIOLOGICAL SCIENCES↗

Disturbed, diffuse, or just missing? A global study of the H I content of Hickson compact groups

Hickson compact groups (HCGs) are dense configurations of four to ten galaxies, whose H I morphology appears to follow an evolutionary sequence of three phases, with gas initially confined to galaxies, then significant amounts spread throughout the intra-group medium, and finally with almost no gas remaining in the galaxies themselves. It has also been suggested that several groups may harbour a diffuse H I component that is resolved out by interferometric observations. The H I deficiency of HCGs is expected to increase as the H I morphological phase progresses along the evolutionary sequence. If this is the case, H I deficiency would be a rough proxy for the age and evolutionary state of a HCG. We aim to test this hypothesis for the first time using a large sample of HCGs and to investigate the evidence for diffuse H I in HCGs. We performed a uniform reduction of all publicly available VLA H I observations (38 HCGs) with a purpose-built pipeline that also maximises the reproducibility of this study. The resulting H I data cubes were then analysed with the latest software tools to perform a manual separation of emission features into those belonging to galaxies and those extending into the intra-group medium. We thereby classified the H I morphological phase of each group as well as quantified their H I deficiency compared to galaxies in isolation. We find little evidence that H I deficiency can be used as a proxy for the evolutionary phase of a compact group in either of the first two phases, with the distribution of H I deficiency being consistent in both. However, for the final phase, the distribution clearly shifts to high H I deficiencies, with more than 90% of the expected H I content typically missing. Across all HCGs studied, we identify a few cases where there is strong evidence for a diffuse gas component in the intra-group medium, which might be detectable with improved observations. We also classify a new sub-phase where groups contain a lone H I -bearing galaxy, but are otherwise devoid of gas. The new morphological phase we have identified is likely the result of an evolved, gas-poor group acquiring a new, gas-rich member. The large spread of H I deficiencies in the first two morphological phases suggests that there is a broad range of initial H I content in HCGs, which is perhaps influenced by large-scale environment, and that the timescale for morphological changes is, in general, considerably shorter than the timescale for the destruction or consumption of neutral gas in these systems.

79 ASTRONOMY AND ASTROPHYSICS↗

An Integrated Framework for Memory-Centric Analysis: From Trace Collection to Co-Design

The memory wall phenomenon—where advances in processor performance significantly outpace those in memory subsystems—poses a fundamental challenge for contemporary computing systems. In memory-bound applications, memory subsystem behavior dominates performance, yet existing analysis approaches present significant limitations: detailed microarchitectural simulators require days to weeks to simulate modest workloads; hardware performance counters provide only aggregate statistics that obscure temporal and spatial access patterns; and scaled simulation approaches face challenges in capturing certain behaviors that emerge at larger scales. These limitations reflect a processor-centric design philosophy increasingly misaligned with memory-bound workloads where detailed understanding of memory access patterns, cache hierarchy interactions, and contention is critical for effective optimization. This paper presents an integrated framework for memory-centric analysis that enables effective hardware-software co-design. We describe practical trace collection techniques, including hardware-assisted processor tracing with minimal overhead and portable software-based instrumentation with statistical sampling. We present multi-perspective analysis methods that examine memory behavior from temporal, sequential, spatial, and relational viewpoints, revealing distinct optimization opportunities invisible in aggregate metrics. We detail an architectural modeling framework that uses sampled traces with temporal interpolation and confidence-based filtering to evaluate cache and memory configurations. Evaluation on representative benchmarks demonstrates that this framework achieves practical accuracy (L2 cache errors of 2.64\%, confidence-filtered L3 errors of 9.92\%, bandwidth errors of 7.33\%) while providing substantial speedup (26.8×) over cycle-accurate simulation, enabling rapid design space exploration. We demonstrate how this integrated framework enables systematic identification of both hardware optimizations (memory controller tuning, bank partitioning, NUMA configuration) and software optimizations (data layout restructuring, prefetching strategies, memory-aware scheduling). Through this comprehensive treatment of the memory-centric analysis pipeline—from trace collection through architectural modeling to co-design application—we provide researchers and practitioners with practical techniques for addressing memory bottlenecks in contemporary computing systems.

Gajaria, Dhruv Mayur↗

The LSST DESC DC2 Simulated Sky Survey

Here, we describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep area of approximately 300 deg 2 , as well as a deep drilling field of approximately 1 deg 2 . We simulate 5 yr of the planned 10 yr survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the data set to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic test bed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time domain cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗

Intelligent Experiments through Real-Time AI: Fast Data Processing and Autonomous Detector Control for High-Energy Nuclear Experiments

The aim of this project is to develop software and hardware for fast real-time data processing and autonomous detector control and calibration for the sPHENIX and the future EIC experiments. Below summarizes Georgia Tech team efforts in the past year: 1. We developed a real-time clustering algorithm and FPGA-based pipeline architecture for processing fired pixel data from ALPIDE sensors in sPHENIX experiments. Our Columnar Clustering Co-Design introduces a hardware-aware, stream-friendly approach that segments pixel data by column pairs using a Column Pair Clustering (CPC) strategy, followed by Cluster Stitching to merge adjacent subclusters. Implemented in Vitis HLS, the pipeline comprises five stages—read-in, subclustering, stitching, analysis, and write-out—connected by tagged HLS streams with custom end-of-event signaling for robust synchronization. We designed a pipelined dataflow model optimized for throughput, low latency, and minimal buffering, enabling scalable clustering across events of arbitrary size. Our system maintains spatial precision via center-of-mass and shape key extraction and efficiently handles edge cases such as fragmented or nested clusters. Compared against DBSCAN in both software and hardware, our approach demonstrates competitive performance under FPGA constraints. 2. We also conducted a comprehensive algorithm-to-hardware co-design of connected component analysis tailored for sPHENIX experiments, focusing on real-time, low-latency processing using FPGAs and High-Level Synthesis (HLS). Starting from a Python-based particle tracking pipeline, the team translated the core logic—graph traversal via DFS and Union-Find—into an HLS-compatible C++ model, replacing dynamic memory and recursion with static arrays and pipelined control flow. The final design includes a fully streamed and dataflow-compatible Union-Find kernel optimized across five iterations, incorporating loop pipelining, array partitioning, AXI/FIFO interface tuning, and function flattening. Experimental results show up to 14.8× speedup over the CPU baseline, reducing per-graph latency to 1.58 μs and demonstrating strong resource efficiency with only ~7k LUTs and zero BRAM usage. The design maintains functional correctness against the Python reference using a Python-based C-simulation framework and Mean Squared Error metrics. This work validates the potential of HLS-driven FPGA designs for edge-level HEP data acquisition, laying a scalable foundation for future integration with real-time detector pipelines and multi-graph processing systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Accelerating Control Systems with GitOps: A Path to Automation and Reliability

GitOps is a foundational approach for modernizing infrastructure by leveraging Git as the single source of truth for declarative configurations. The poster explores how GitOps transforms traditional control system infrastructure, services and applications by enabling fully automated, auditable, and version-controlled infrastructure management. Cloud-native and containerized environments are shifting the ecosystem not only in the IT industry but also within the computational science field, as is the case of CERN and Diamond Light Source among other Accelerator/Science facilities which are slowly shifting towards modern software and infrastructure paradigms. The ACORN project, which aims to modernize Fermilab’s control system infrastructure and software is implementing proven best-practices and cutting-edge technology standards including GitOps, containerization, infrastructure as code and modern data pipelines for control system data acquisition and the inclusion of AI/ML in our accelerator complex.

Gonzalez, M. [Fermilab]↗

Automated Strain Construction for Biosynthetic Pathway Screening in Yeast

Automation accelerates the Design-Build-Test-Learn (DBTL) cycle for synthetic biology; however, most strain construction pipelines lack robotic integration. Here, in this study, we present the workflow design and source code for a modular, integrated protocol that automates the Build step in Saccharomyces cerevisiae. We programmed the Hamilton Microlab VANTAGE to integrate off-deck hardware via its central robotic arm, enabling automated steps that increased throughput to 2,000 transformations per week. We developed a user interface with the Hamilton VENUS software to support on-demand parameter customization. As a proof of concept, we screened a gene library in an engineered yeast strain producing verazine, a key intermediate in the biosynthesis of steroidal alkaloids. Our pipeline rapidly identified pathway bottlenecks and genes that enhanced verazine production by 2.0- to 5-fold. This technical note provides resources for synthetic biologists designing yeast workflows for biofoundries to screen libraries for pathway discovery/optimization, combinatorial biosynthesis, and protein engineering.

automation↗

Multiparameter optical fiber sensing for energy infrastructure through nanoscale light–matter interactions: From hardware to software, science to commercial opportunities

Monitoring of energy infrastructure through robust yet economical sensing platforms is becoming an area of increased importance, with ubiquitous applications including the electrical grid, natural gas and oil transportation pipelines, H2 infrastructure (storage and transportation), carbon storage, power generation, and subsurface environments. Plasmonic and functional nanomaterial enabled fiber optic sensors show excellent promise for a wide range of sensing applications due to their versatility to be engineered for specific analytes of interest while retaining inherent advantages of the optical fiber sensor platform. Through the design of novel sensing layers, the optical transduction mechanism and wavelength dependence can also be tailored for ease of integration with low-cost interrogation systems enabling an inexpensive yet highly functional optical fiber sensing platform. In addition, recent advances in artificial intelligence and machine learning theoretical methods have been leveraged to simultaneously extract multiple parameters through multi-wavelength interrogation such that unique wavelengths can also serve as unique sensing elements, analogous to electronic nose sensor technologies. The concept of an optical fiber based “photonic nose” via multiple interrogation wavelengths and/or sensor nodes offers a compelling platform technology to realize multiparameter speciation of chemical analytes within complex gas mixtures. In this Perspective, we further generalize the notion of multiparameter sensing through the novel “photonic nervous system” concept based upon low-cost, functionalized optical fiber sensor probes monitoring a variety of distinct analyte classes (physical, chemical, electromagnetic, etc.) simultaneously to provide broad situational awareness via integrated sensors.

Su, Yang-Duan (ORCID:0000000214820902)↗

Sky subtraction in an era of low surface brightness astronomy

ABSTRACT The Vera C. Rubin Observatory Wide-Fast Deep sky survey will reach unprecedented surface brightness depths over tens of thousands of square degrees. Surface brightness photometry has traditionally been a challenge. Current algorithms which combine object detection with sky estimation systematically oversubtract the sky, biasing surface brightness measurements at the faint end and destroying or severely compromising low surface brightness light. While it has recently been shown that properly accounting for undetected faint galaxies and the wings of brighter objects can in principle recover a more accurate sky estimate, this has not yet been demonstrated in practice. Obtaining a consistent spatially smooth underlying sky estimate is particularly challenging in the presence of representative distributions of bright and faint objects. In this paper, we use simulations of crowded and uncrowded fields designed to mimic Hyper Suprime-Cam data to perform a series of tests on the accuracy of the recovered sky. Dependence on field density, galaxy type, and limiting flux for detection are all considered. Several photometry packages are utilized: source extractor, gnuastro, and the LSST science pipelines. Each is configured in various modes, and their performance at extreme low surface brightness analysed. We find that the combination of the source extractor software package with novel source model masking techniques consistently produce extremely faint output sky estimates, by up to an order of magnitude, as well as returning high fidelity output science catalogues.

Astronomy & Astrophysics↗

Deployment of ADTimePix3 areaDetector Driver at Neutron and X-ray User Facilities

TimePix3 is a 65k hybrid pixel readout chip with simultaneous Time-of-Arrival (ToA) and Time-over-Threshold (ToT) recording in each pixel*. The chip operates without a trigger signal with a sparse readout where only pixels containing events are read out. The flexible architecture allows 40 MHits/s/cm² readout throughput, using simultaneous readout and acquisition by sharing readout logic with transport logic of superpixel matrix formed using 2x4 structure. The chip ToA records 1.5625 ns time resolution. The X-ray and charged particle events are counted directly. However, indirect neutron counts use 6Li fission in a scintillator matrix, such as ZnS(Ag). The fission space-charge region is limited to 5-9 um. A photon from scintillator material excites a photocathode electron, which is further multiplied in dual-stack MCP. The neutron count event is a cluster of electron events at the chip. We report on the EPICS areaDetector** ADTimePix3 driver that controls Serval*** using json commands. The driver directs data to storage and to a real-time processing pipeline and configures the chip. The time-stamped data are stored in raw .tpx3 file format and passed through a socket where the clustering software identifies individual neutron events. The conventional 2D images are available as images for each exposure frame, and a preview is useful for sample alignment. The areaDetector driver allows integration of time-enhanced capabilities of this detector into SNS beamlines controls and unprecedented time resolution.

Gofron, Kaz↗

A secure measurement unit for an inspection system used in nuclear arms-control verification

Verification of arms control treaties may require information barriers to protect sensitive data acquired during the verification process. These information barriers are commonly implemented in software, but must store and operate on data that are vulnerable to attack and tampering. Here, we present a new Secure Measurement Unit based on a modified pipeline SAR ADC architecture. The idea is to move the information barrier back into the digitization process so that prohibited data is never created. A prototype 65nm CMOS Secure Measurement Unit (SMU) was tested using U-235, U-238, Co-60, Cs-137 and Am-241 sources. Here, the prototype accurately digitized allowable signals while being immune to side-channel attacks on signals within preset forbidden regions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Unified Nanotechnology Format: One Way to Store Them All

The domains of DNA and RNA nanotechnology are steadily gaining in popularity while proving their value with various successful results, including biosensing robots and drug delivery cages. Nowadays, the nanotechnology design pipeline usually relies on computer-based design (CAD) approaches to design and simulate the desired structure before the wet lab assembly. To aid with these tasks, various software tools exist and are often used in conjunction. However, their interoperability is hindered by a lack of a common file format that is fully descriptive of the many design paradigms. Therefore, in this paper, we propose a Unified Nanotechnology Format (UNF) designed specifically for the biomimetic nanotechnology field. UNF allows storage of both design and simulation data in a single file, including free-form and lattice-based DNA structures. By defining a logical and versatile format, we hope it will become a widely accepted and used file format for the nucleic acid nanotechnology community, facilitating the future work of researchers and software developers. Together with the format description and publicly available documentation, we provide a set of converters from existing file formats to simplify the transition. Finally, we present several use cases visualizing example structures stored in UNF, showcasing the various types of data UNF can handle.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Predicting metabolic modules in incomplete bacterial genomes with MetaPathPredict

The reconstruction of complete microbial metabolic pathways using ‘omics data from environmental samples remains challenging. Computational pipelines for pathway reconstruction that utilize machine learning methods to predict the presence or absence of KEGG modules in incomplete genomes are lacking. Here, we present MetaPathPredict, a software tool that incorporates machine learning models to predict the presence of complete KEGG modules within bacterial genomic datasets. Using gene annotation data and information from the KEGG module database, MetaPathPredict employs deep learning models to predict the presence of KEGG modules in a genome. MetaPathPredict can be used as a command line tool or as a Python module, and both options are designed to be run locally or on a compute cluster. Benchmarks show that MetaPathPredict makes robust predictions of KEGG module presence within highly incomplete genomes.

59 BASIC BIOLOGICAL SCIENCES↗

Towards operational atmospheric correction of airborne hyperspectral imaging spectroscopy: Algorithm evaluation, key parameter analysis, and machine learning emulators

Atmospheric correction of airborne hyperspectral imaging spectroscopy (AHIS) to obtain high-quality surface reflectance is the prerequisite for remote sensing applications. Over the last decades, different atmospheric correction methods have been developed based on radiative transfer models (RTMs), however, the relative performances of different algorithms are unclear. Automated operational atmospheric correction methods to process large-volume AHIS data in a high-accurate and high-throughput manner are still lacking. Therefore, this study proposed an operational atmospheric correction pipeline for deriving surface reflectance from AHIS data. To ensure the accuracy and efficiency of the pipeline, we focused on three specific aspects: (1) selecting a suitable RTM for the development of atmospheric lookup tables (LUTs) by comparing the commercial MODerate resolution atmospheric TRANsmission (MODTRAN) and open-sourced Library for Radiative TRANsfer (LibRadTRAN) models, where the widely-used software, Atmospheric/Topographic Correction for Airborne Imagery (ATCOR), was used as benchmarks; (2) identifying key atmospheric correction parameters and determining suitable sources for parameter retrievals including AHIS, Moderate Resolution Imaging Spectroradiometer (MODIS), and AErosol RObotic NETwork (AERONET); and (3) testing the performance of using machine learning emulators to speed up the RTM-based atmospheric correction. Results indicate that (1) atmospheric correction based on MODTRAN LUTs can produce surface reflectance accurately with mean absolute errors < 0.05 and cosine similarities > 0.98 compared to field measurements, which is comparable to the software ATCOR and slightly outperforms the LibRadTRAN LUTs; (2) sobol global sensitivity analysis demonstrates that in the atmospheric correction, visibility and water vapor are two key parameters that can be accurately derived from AHIS in contrast to MODIS or AERONET data; and (3) Random Forest emulators can produce accurate estimations of surface reflectance with mean absolute errors < 0.03 and cosine similarities > 0.98 for higher processing efficiency and determine a suitable set of wavelengths for retrieving atmospheric visibility and water vapor. In conclusion, the proposed atmospheric correction pipeline also improved the four-stream radiative transfer theory for airborne applications by considering adjacent effects from airborne surrounding pixels and can also be applied for atmospheric correction of hyperspectral data from spaceborne missions.

47 OTHER INSTRUMENTATION↗

District EnErgy Planner (DEEP) v2

The District EnErgy Planner - DEEP is an Mixed Integer Linear Programming (MILP) optimization software tool to solve district energy system, technology configuration, operation strategies, and energy retail business schemes. DEEP takes user input energy demand on heating, cooling and electricity demand. Energy loss through pipeline is calculated for heating and cooling from central plants. The optimal system configuration to minimize the district energy system life cycle cost or minimize CO2 emission is calculated based on energy balance equations and various constraints. The recent update on DEEP include a feature of waste heat utilization energy balance based optimization to take into accounts of different temperature bands of heat into account to maximize system efficiency. Another feature is to optimize district energy retail by provide optimal pricing scheme for heating, cooling and electricity to end users to incorporate district to grid demand response.

Feng, Wei↗

PFLOTRAN modeling data and scripts associated with “Refining the Hydrogeologic Framework of a Large River Corridor Model Using Waterborne Transient Electromagnetics”

NOTE: The manuscript associated with this data package is currently in review. The data may be revised based on reviewer feedback. Upon manuscript acceptance, this data package will be updated with the final dataset and additional metadata. This data package is associated with the publication “Refining the Hydrogeologic Framework of a Large River Corridor Model Using Waterborne Transient Electromagnetics” submitted to Water Resources Research (Terry et al. 2025). The data package contains the groundwater modeling dataset from PFLOTRAN software. It includes the python script for mesh generation, boundary condition setting, PFLOTRAN input deck formation and postprocessing. It couples groundwater flow and species transport for Hanford Reach river corridor and pipelines the model generation and processing. This model can be used to easily generate the model and analysis for Hanford site. It can also be adjusted to other hydrologic area with ease. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. The data package consists of 6 folders: (1) “data” contains all necessary data as input and intermediate data for processing; (2) “mesh” contains all mesh related files to generate mesh in Hanford Reach river corridor; (3) “model_run” contains the generated script for PFLOTRAN modeling; (4) “notebooks” contains all the Python script to generate the model; (5) “output” contains all the output from the computation; (6) “postprocessing” contains the Python script to generate scientific figure for manuscript. All files are .csv (comma-separated values), .h5 (HDF5 format), .in (input files), .ipynb (Jupyter notebooks), .p (Python pickle), .png (images), .PNG (images), .py (Python scripts), .pyc (Python bytecode), .r (R scripts), .sh (shell scripts), .txt (text files), .vtu (3D mesh/visualization format), .xz (compressed archive), or .zip (compressed archive).

54 ENVIRONMENTAL SCIENCES↗

Operational Intelligence in the ATLAS Continuous Integration System

Describes the role of the ATLAS Continuous Integration (CI) System in the ATLAS offline software development infrastructure • Outlines the CI system components and processes • Details Operational Intelligence techniques to accelerate CI jobs and lower operating costs • Explains the Directed Acyclic Graph (DAG) approach in CI pipelines • Reports achieved improvements

97 MATHEMATICS AND COMPUTING↗

`SkyPy`: A package for modelling the Universe

SkyPy is an open-source Python package for simulating the astrophysical sky. It comprises a library of physical and empirical models across a range of observables and a command-line script to run end-to-end simulations. The library provides functions that sample realisations of sources and their associated properties from probability distributions. Simulation pipelines are constructed from these models using a YAML-based configuration syntax, while task scheduling and data dependencies are handled internally and the modular design allows users to interface with external software. SkyPy is developed and maintained by a diverse community of domain experts with a focus on software sustainability and interoperability. By fostering development, it provides a framework for correlated simulations of a range of cosmological probes including galaxy populations, large scale structure, the cosmic microwave background, supernovae and gravitational waves. Version 0.4 implements functions that model various properties of galaxies including luminosity functions, redshift distributions and optical photometry from spectral energy distribution templates. Future releases will provide additional modules, for example, to simulate populations of dark matter halos and model the galaxy-halo connection, making use of existing software packages from the astrophysics community where appropriate.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗