Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Accelerating the Inference of the Exa.TrkX Pipeline

Recently, graph neural networks (GNNs) have been successfully used for a variety of particle reconstruction problems in high energy physics, including particle tracking. The Exa.TrkX pipeline based on GNNs demonstrated promising performance in reconstructing particle tracks in dense environments. It includes five discrete steps: data encoding, graph building, edge filtering, GNN, and track labeling. All steps were written in Python and run on both GPUs and CPUs. In this work, we accelerate the Python implementation of the pipeline through customized and commercial GPU-enabled software libraries, and develop a C++ implementation for inferencing the pipeline. The implementation features an improved, CUDA-enabled fixed-radius nearest neighbor search for graph building and a weakly connected component graph algorithm for track labeling. GNNs and other trained deep learning models are converted to ONNX and inferenced via the ONNX Runtime C++ API. The complete C++ implementation of the pipeline allows integration with existing tracking software. We report the memory usage and average event latency tracking performance of our implementation applied to the TrackML benchmark dataset.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

ROSAT Science Data Center

This report provides a summary of the Smithsonian Astrophysical Observatory (SAO) ROSAT SCIENCE DATA CENTER (RSDC) activities for the recent years of our contract. Details have already been reported in the monthly reports. The SAO was responsible for the High Resolution Imager (HRI) detector on ROSAT. We also provided and supported the HRI standard analysis software used in the pipeline processing (SASS). Working with our colleagues at the Max Planck in Garching Germany (MPE), we fixed bugs and provided enhancements. The last major effort in this area was the port from VMS/VAX to VMS/ALPHA architecture. In 1998, a timing bug was found in the HRI standard processing system which degraded the positional accuracy because events accessed incorrect aspect solutions. The bug was fixed and we developed off-line correction routines and provided them to the community. The Post Reduction Off-line Software (PROS) package was developed by SAO and runs in the IRAF environment. Although in recent years PROS was not a contractual responsibility of the RSDC, we continued to maintain the system and provided new capabilities such as the ability to deal with simulated AXAF data in preparation for the NASA call for proposals for Chandra. Our most recent activities in this area included the debugging necessary for newer versions of IRAF which broke some of our software. At SAO we have an operating version of PROS and hope to release a patch even though almost all functionality that was lost was subsequently recovered via an IRAF patch (i.e. most of our problems were caused by an IRAF bug).

Murray, Stephen↗

Presearch Data Conditioning in the Kepler Science Operations Center Pipeline

We describe the Presearch Data Conditioning (PDC) software component and its context in the Kepler Science Operations Center (SOC) pipeline. The primary tasks of this component are to correct systematic and other errors, remove excess flux due to aperture crowding, and condition the raw flux light curves for over 160,000 long cadence (~thirty minute) and 512 short cadence (~one minute) targets across the focal plane array. Long cadence corrected flux light curves are subjected to a transiting planet search in a subsequent pipeline module. We discuss the science algorithms for long and short cadence PDC: identification and correction of unexplained (i.e., unrelated to known anomalies) discontinuities; systematic error correction; and excess flux removal. We discuss the propagation of uncertainties from raw to corrected flux. Finally, we present examples of raw and corrected flux time series for flight data to illustrate PDC performance. Corrected flux light curves produced by PDC are exported to the Multi-mission Archive at Space Telescope [Science Institute] (MAST) and will be made available to the general public in accordance with the NASA/Kepler data release policy.

Twicken, Joseph D.↗

VDiSC: An Open Source Framework for Distributed Smart City Vision and Biometric Surveillance Networks

Recent global growth in the interest of smart cities has led to trillions of dollars of investment toward research and development. These connected cities have the potential to create a symbiosis of technology and society and revolutionize the cost of living, safety, ecological sustainability, and quality of life of societies on a world-wide scale. Some key components of the smart city construct are connected smart grids, self-driving cars, federated learning systems, smart utilities, large-scale public transit, and proactive surveillance systems. While exciting in prospect, these technologies and their subsequent integration cannot be attempted without addressing the potential societal impacts of such a high degree of automation and data sharing. Additionally, the feasibility of coordinating so many disparate tasks will require a fast, extensible, unifying framework. To that end, we propose the Distributed Smart City framework for Vision, or VDiSC. VDiSC serves as a unified biometric API harness that allows for seamless evaluation, deployment, and simple pipeline creation for heterogeneous biometric software. VDiSC additionally provides a fully declarative capability for defining and coordinating custom machine learning and sensor pipelines, allowing the distribution of processes across otherwise incompatible hardware and networks. VDiSC ultimately provides a way to quickly configure, hot-swap, and expand large coordinated or federated systems online without interruptions for maintenance. Because much of the data collected in a smart city contains Personally Identifying Information (PII), VDiSC also provides built-in tools and layers to ensure secure and encrypted streaming, storage, and access of PII data across distributed systems.

Brogan, Joel↗

New Software for Ensemble Creation in the Spitzer-Space-Telescope Operations Database

Some of the computer pipelines used to process digital astronomical images from NASA's Spitzer Space Telescope require multiple input images, in order to generate high-level science and calibration products. The images are grouped into ensembles according to well documented ensemble-creation rules by making explicit associations in the operations Informix database at the Spitzer Science Center (SSC). The advantage of this approach is that a simple database query can retrieve the required ensemble of pipeline input images. New and improved software for ensemble creation has been developed. The new software is much faster than the existing software because it uses pre-compiled database stored-procedures written in Informix SPL (SQL programming language). The new software is also more flexible because the ensemble creation rules are now stored in and read from newly defined database tables. This table-driven approach was implemented so that ensemble rules can be inserted, updated, or deleted without modifying software.

Spitzer Science Center (SSC)↗

ADEPT: A Pedagogical Framework for Integrating Agentic AI with Deterministic Scientific Workflows

The integration of Large Language Models (LLMs) into scientific research promises to accelerate discovery, yet a significant gap remains between the dynamic reasoning of Artificial Intelligence (AI) agents and the static, deterministic nature of canonical scientific workflows. This paper introduces ADEPT (Agentic Discovery and Exploration Platform for Tools), a reference architecture and pedagogical framework explicitly designed to bridge this gap. ADEPT's primary mission is to provide a transparent, "glass-box" environment where researchers and engineers can learn to effectively wrap established scientific software (e.g., BLAST, Nextflow pipelines) and compose it into reliable, agent-driven workflows. We describe its modular, multi-server architecture, which leverages the Model Context Protocol (MCP) for tool serving, LangGraph for robust agentic orchestration, and a secure nsjail-based sandbox for safe code execution. By prioritizing architectural clarity, safety, and modularity, ADEPT serves as an extensible blueprint for building trustworthy AI-augmented systems and fosters the collaborative development necessary to responsibly employ agentic AI for science. We provide practical examples of how to adapt and extend this framework, highlighting its utility in workforce development and AI-readiness capabilities across research and development projects.

97 MATHEMATICS AND COMPUTING↗

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗

PERCEPTIVE: an R shiny $\underline{p}$ipelin$\underline{e}$ for the p$\underline{r}$edi$\underline{c}$tion of $\underline{ep}$igenetic modula$\underline{t}$ors $\underline{i}$n no$\underline{v}$el sp$\underline{e}$cies

Epigenetic processes are central to regulating gene expression, genome stability, and metabolic function across the tree of life; yet, their roles remain underexplored in microalgae, especially as new species continue to be identified and characterized. This is likely due to the cumbersome nature and species-dependent attributes of epigenetic wet-lab methodologies, which preclude the rapid identification of epigenetic modifications and modulators. However, there is high conservation of epigenetic processes from budding yeast to humans; in many cases, one may infer how behavior and function are epigenetically regulated in novel species by identifying epigenetic modulators, or the proteins responsible for conferring epigenetic modifications. Here, to this end, we have developed a graphical software package, titled PERCEPTIVE (pipeline for the prediction of epigenetic modulators in novel species). This platform solely uses the genomic sequence of an algal species, and preexisting information from other model organisms, to predict the epigenetic modulators and associated modifications in algae. Predictions are presented to the user in a graphical interface, which provides literature-based interpretation of results, enabling users to quickly understand potential epigenetic processes in their algal species of interest and plan follow-up experiments. To test PERCEPTIVE, we predicted epigenetic modulators in several feedstock candidate algae species. To validate these predictions, wet-lab studies were performed, including mass spectrometry; these results underscore the high accuracy of PERCEPTIVE predictions. Overall, PERCEPTIVE represents a powerful in silico tool for the research and manipulation of algal species, which does not require a priori knowledge of epigenetics and is accessible to a broad set of investigators.

59 BASIC BIOLOGICAL SCIENCES↗

WizEM

The software provides a flexible pipeline for artificial intelligence (AI) assisted analysis of data collected from an electron microscope. This tool distinctly provides near real-time image quantification results during data collection, incorporating feedback from a user to tune performance.

Akers, Sarah [Pacific Northwest National Laborator↗

Processors, Pipelines, and Protocols for Advanced Modeling Networks

Predictive capabilities arise from our understanding of natural processes and our ability to construct models that accurately reproduce these processes. Although our modeling state-of-the-art is primarily limited by existing computational capabilities, other technical areas will soon present obstacles to the development and deployment of future predictive capabilities. Advancement of our modeling capabilities will require not only faster processors, but new processing algorithms, high-speed data pipelines, and a common software engineering framework that allows networking of diverse models that represent the many components of Earth's climate and weather system. Development and integration of these new capabilities will pose serious challenges to the Information Systems (IS) technology community. Designers of future IS infrastructures must deal with issues that include performance, reliability, interoperability, portability of data and software, and ultimately, the full integration of various ES model systems into a unified ES modeling network.

Coughlan, Joseph↗

Astro-H/Hitomi Data Analysis, Processing, and Archive

Astro-H is the x-ray/gamma-ray mission led by Japan with international participation, launched on February 17, 2016. Soon after launch, Astro-H was renamed Hitomi. The payload consists of four different instruments (SXS, SXI, HXI, and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. On March 27, 2016, JAXA lost contact with the satellite and, on April 28, they announced the cessation of the efforts to restore mission operations. Hitomi collected about one months worth of data with its instruments. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data, along with the plan for the archive. These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and United States.

Angelini, Lorella↗

Data Validation in the Kepler Science Operations Center Pipeline

We present an overview of the Data Validation (DV) software component and its context within the Kepler ScienceOperations Center (SOC) pipeline and overall Kepler Science mission. The SOC pipeline performs a transiting planetsearch on the corrected light curves for over 150,000 targets across the focal plane array. We discuss the DV strategy forautomated validation of Threshold Crossing Events (TCEs) generated in the transiting planet search. For each TCE, atransiting planet model is fitted to the target light curve. A multiple planet search is conducted by repeating the transitingplanet search on the residual light curve after the model flux has been removed; if an additional detection occurs, aplanet model is fitted to the new TCE. A suite of automated tests are performed after all planet candidates have beenidentified. We describe a centroid motion test to determine the significance of the motion of the target photocenterduring transit and to estimate the coordinates of the transit source within the photometric aperture; a series of eclipsingbinary discrimination tests on the parameters of the planet model fits to all transits and the sequences of odd and eventransits; and a statistical bootstrap to assess the likelihood that the TCE would have been generated purely by chancegiven the target light curve with all transits removed.

photometry↗

The updated DESGW processing pipeline for the third LIGO/VIRGO observing run

The DESGW group seeks to identify electromagnetic counterpartsof gravitational wave events seen by the LIGO-VIRGO network, such as thoseexpected from binary neutron star mergers or neutron star- black hole mergers.DESGW was active throughout the first two LIGO observing seasons, followingup several binary black hole mergers and the first binary neutron star merger,GW170817. We describe the modifications to the observing strategy generationand image processing pipeline between the second (ending in August 2017)and third (beginning in April 2019) LIGO observing seasons. The modifica-tions include a more robust observing strategy generator, further parallelizationof the image reduction software and dierence imaging processing pipeline,data transfer streamlining, and a web page listing identified counterpart candi-dates that updates in real time. Taken together, the additional parallelizationsteps enable us to identify potential electromagnetic counterparts within fullycalibrated search images in less than one hour, compared to the 3-5 hours itwould typically take during the first two seasons. These performance improve-ments are critical to the entire EM followup community, as rapid identification(or rejection) of candidates enables detailed spectroscopic followup by multipleinstruments as soon as possible, leading to more information about the environ-ment immediately following such gravitational wave events.

Herner, Kenneth R.↗

Data Validation in the Kepler Science Operations Center Pipeline

We present an overview of the Data Validation (DV) software component and its context within the Kepler Science Operations Center (SOC) pipeline and overall Kepler Science mission. The SOC pipeline performs a transiting planet search on the corrected light curves for over 150,000 targets across the focal plane array. We discuss the DV strategy for automated validation of Threshold Crossing Events (TCEs) generated in the transiting planet search. For each TCE, a transiting planet model is fitted to the target light curve. A multiple planet search is conducted by repeating the transiting planet search on the residual light curve after the model flux has been removed; if an additional detection occurs, a planet model is fitted to the new TCE. A suite of automated tests are performed after all planet candidates have been identified. We describe a centroid motion test to determine the significance of the motion of the target photocenter during transit and to estimate the coordinates of the transit source within the photometric aperture; a series of eclipsing binary discrimination tests on the parameters of the planet model fits to all transits and the sequences of odd and even transits; and a statistical bootstrap to assess the likelihood that the TCE would have been generated purely by chance given the target light curve with all transits removed. Keywords: photometry, data validation, Kepler, Earth-size planets

Wu, Hayley↗

TDCOSMO. X. Automated modeling of nine strongly lensed quasars and comparison between lens-modeling software

When strong gravitational lenses are to be used as an astrophysical or cosmological probe, models of their mass distributions are often needed. We present a new, time-efficient automation code for the uniform modeling of strongly lensed quasars with GLEE, a lens-modeling software for multiband data. By using the observed positions of the lensed quasars and the spatially extended surface brightness distribution of the host galaxy of the lensed quasar, we obtain a model of the mass distribution of the lens galaxy. We applied this uniform modeling pipeline to a sample of nine strongly lensed quasars for which images were obtained with the Wide Field Camera 3 of the Hubble Space Telescope. The models show well-reconstructed light components and a good alignment between mass and light centroids in most cases. We find that the automated modeling code significantly reduces the input time during the modeling process for the user. The time for preparing the required input files is reduced by a factor of 3 from ~3 h to about one hour. The active input time during the modeling process for the user is reduced by a factor of 10 from ~ 10 h to about one hour per lens system. This automated uniform modeling pipeline can efficiently produce uniform models of extensive lens-system samples that can be used for further cosmological analysis. A blind test that compared our results with those of an independent automated modeling pipeline based on the modeling software Lenstronomy revealed important lessons. Quantities such as Einstein radius, astrometry, mass flattening, and position angle are generally robustly determined. Other quantities, such as the radial slope of the mass density profile and predicted time delays, depend crucially on the quality of the data and on the accuracy with which the point spread function is reconstructed. Better data and/or a more detailed analysis are necessary to elevate our automated models to cosmography grade. Nevertheless, our pipeline enables the quick selection of lenses for follow-up and further modeling, which significantly speeds up the construction of cosmography-grade models. This important step forward will help us to take advantage of the increase in the number of lenses that is expected in the coming decade, which is an increase of several orders of magnitude.

79 ASTRONOMY AND ASTROPHYSICS↗

TomoPyUI : a user-friendly tool for rapid tomography alignment and reconstruction

The management and processing of synchrotron and neutron computed tomography data can be a complex, labor-intensive and unstructured process. Users devote substantial time to both manually processing their data ( i.e. organizing data/metadata, applying image filters etc. ) and waiting for the computation of iterative alignment and reconstruction algorithms to finish. In this work, we present a solution to these problems: TomoPyUI , a user interface for the well known tomography data processing package TomoPy . This highly visual Python software package guides the user through the tomography processing pipeline from data import, preprocessing, alignment and finally to 3D volume reconstruction. The TomoPyUI systematic intermediate data and metadata storage system improves organization, and the inspection and manipulation tools (built within the application) help to avoid interrupted workflows. Notably, TomoPyUI operates entirely within a Jupyter environment. Herein, we provide a summary of these key features of TomoPyUI , along with an overview of the tomography processing pipeline, a discussion of the landscape of existing tomography processing software and the purpose of TomoPyUI , and a demonstration of its capabilities for real tomography data collected at SSRL beamline 6-2c.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Optimizing a magnitude-limited spectroscopic training sample for photometric classification of supernovae

ABSTRACT In preparation for photometric classification of transients from the Legacy Survey of Space and Time (LSST) we run tests with different training data sets. Using estimates of the depth to which the 4-m Multi-Object Spectroscopic Telescope (4MOST) Time Domain Extragalactic Survey (TiDES) can classify transients, we simulate a magnitude-limited sample reaching rAB ≈ 22.5 mag. We run our simulations with the software snmachine, a photometric classification pipeline using machine learning. The machine-learning algorithms struggle to classify supernovae when the training sample is magnitude limited, in contrast to representative training samples. Classification performance noticeably improves when we combine the magnitude-limited training sample with a simulated realistic sample of faint high-redshift supernovae observed from larger spectroscopic facilities; the algorithms’ range of average area under receiver operator characteristic curve (AUC) scores over 10 runs increases from 0.547–0.628 to 0.946–0.969 and purity of the classified sample reaches 95 per cent in all runs for two of the four algorithms. By creating new, artificial light curves using the augmentation software avocado, we achieve a purity in our classified sample of 95 per cent in all 10 runs performed for all machine-learning algorithms considered. We also reach a highest average AUC score of 0.986 with the artificial neural network algorithm. Having ‘true’ faint supernovae to complement our magnitude-limited sample is a crucial requirement in optimization of a 4MOST spectroscopic sample. However, our results are a proof of concept that augmentation is also necessary to achieve the best classification results.

79 ASTRONOMY AND ASTROPHYSICS↗

Mass Spectrometer Transient Analysis

This software implements a complete preprocessing pipeline for transient mass spectrometry (MS) data collected during TAP (Temporal Analysis of Products) experiments. It is designed to extract chemically meaningful fluxes from overlapping ion signals by applying a calibrated defragmentation matrix and solving the resulting linear system using non-negative least squares (NNLS) regression. The core script, preprocess_mass_spec.py, performs the following operations: Gain correction: Applies amplifier gain scalars derived from inert-packed calibration pulses to normalize signal intensities across AMUs and acquisition settings. Background subtraction: Removes experiment baselines using user-defined time windows, ensuring compatibility with slow-diffusing species and preventing negative values that would interfere with NNLS. Options to subtract before and after defragmentation. Defragmentation: Constructs a fragmentation matrix A from zeroth moments of calibration pulses (equal molar gas:inert mixtures) and solves Ax=b at each time point, where b is the raw MS signal and x is the estimated species flux. The matrix is normalized to inert signals and accounts for instrument-specific fragmentation behavior. Pulse-mode handling: Supports both averaged and individual pulse modes, enabling statistical treatment of fluxes and calculation of standard deviations. Integration and output: Computes zeroth moments (integrated fluxes) and exports time-resolved and integrated data in CSV format, suitable for downstream kinetic modeling. The software is validated using both virtual TAP simulations (VTAP) and experimental data from propane dehydrogenation (PDH) on CrOx/Al2O3 catalysts. It preserves temporal resolution by applying NNLS point-by-point across the pulse duration (typically 6,000+ time slices per pulse), leveraging the linear superposition principle to reconstruct full flux profiles. The defragmented outputs are compatible with kinetic extraction methods such as the G and Y procedures, which are used to derive rate–concentration relationships from TAP data. The details of these validations are discussed in detail in the supporting manuscript and supporting information. Example data and output files are also included. The methodology is robust to experimental noise and drift, with calibration protocols that account for pulse size effects, MS aging, and inert gas normalization. The software is modular, reproducible, and tailored for high-throughput TAP-MS workflows in catalysis research.

Kristy, Stephen [Idaho National Laboratory (INL), ↗