Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Science Pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

Flexible Learning-Free Segmentation and Reconstruction of Neural Volumes

Imaging is a dominant strategy for data collection in neuroscience, yielding stacks of images that often scale to gigabytes of data for a single experiment. Machine learning algorithms from computer vision can serve as a pair of virtual eyes that tirelessly processes these images, automatically detecting and identifying microstructures. Unlike learning methods, our Flexible Learning-free Reconstruction of Imaged Neural volumes (FLoRIN) pipeline exploits structure-specifc contextual clues and requires no training. This approach generalizes across diferent modalities, including serially-sectioned scanning electron microscopy (sSEM) of genetically labeled and contrast enhanced processes, spectral confocal refectance (SCoRe) microscopy, and high-energy synchrotron X-ray microtomography (μCT) of large tissue volumes. We deploy the FLoRIN pipeline on newly published and novel mouse datasets, demonstrating the high biological fdelity of the pipeline’s reconstructions. FLoRIN reconstructions are of sufcient quality for preliminary biological study, for example examining the distribution and morphology of cells or extracting single axons from functional data. Compared to existing supervised learning methods, FLoRIN is one to two orders of magnitude faster and produces high-quality reconstructions that are tolerant to noise and artifacts, as is shown qualitatively and quantitatively.

47 OTHER INSTRUMENTATION↗

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗

LABEL: Fast and Accurate Lineage Assignment with Assessment of H5N1 and H9N2 Influenza A Hemagglutinins

The evolutionary classification of influenza genes into lineages is a first step in understanding their molecular epidemiology and can inform the subsequent implementation of control measures. We introduce a novel approach called Lineage Assignment By Extended Learning (LABEL) to rapidly determine cladistic information for any number of genes without the need for time-consuming sequence alignment, phylogenetic tree construction, or manual annotation. Instead, LABEL relies on hidden Markov model profiles and support vector machine training to hierarchically classify gene sequences by their similarity to pre-defined lineages. We assessed LABEL by analyzing the annotated hemagglutinin genes of highly pathogenic (H5N1) and low pathogenicity (H9N2) avian influenza A viruses. Using the WHO/FAO/OIE H5N1 evolution working group nomenclature, the LABEL pipeline quickly and accurately identified the H5 lineages of uncharacterized sequences. Moreover, we developed an updated clade nomenclature for the H9 hemagglutinin gene and show a similarly fast and reliable phylogenetic assessment with LABEL. While this study was focused on hemagglutinin sequences, LABEL could be applied to the analysis of any gene and shows great potential to guide molecular epidemiology activities, accelerate database annotation, and provide a data sorting tool for other large-scale bioinformatic studies.

59 BASIC BIOLOGICAL SCIENCES↗

Data Fusion to Enhance Quality Control and Analysis with Instruments at the Marine and Coastal Research Laboratory

Deploying environmental monitoring instruments in the marine environment can be challenging, facing challenges around device survivability, biofouling and corrosion, and consistent data collection. This project explores the use of data fusion – the process of integrating multiple data sources to produce more consistent, accurate, and useful information – to build a consistent long-term monitoring system at the Marine and Coastal Research Laboratory (MCRL) in Sequim, Washington. Unused instruments that had been acquired from past projects were inventoried and deployments planned on the MCRL pier and floating dock. A total of 8 instruments were deployed including a tide gauge, hydrophone, acoustic Doppler current profiler (ADCP), photosynthetically active radiation (PAR) sensors, meteorological station, and three water quality sensors. Deployments were planned to be well-protected around the pier structure and a maintenance schedule was created for cleaning and recalibration. An automated data pipeline was created to aggregate data on edge computers that push data to Amazon Web Services (AWS) cloud storage every 15 minutes, performing automated quality control and data transformations using the Time Series Data Analytical Toolkit (TSDAT). Continued efforts are underway to maintain this system into the future, take a data-driven approach to maintenance scheduling, improve the reliability of the system, and share the data with a variety of end-users.

54 ENVIRONMENTAL SCIENCES↗

Multiplexed Knockouts in the Model Diatom Phaeodactylum by Episomal Delivery of a Selectable Cas9

Marine diatoms are eukaryotic microalgae that play significant ecological and biogeochemical roles in oceans. They also have significant potential as organismal platforms for exploitation to address biotechnological and industrial goals. In order to address both modes of research, sophisticated molecular and genetic tools are required. We presented here new and improved methodologies for introducing CRISPRCas9 to the model diatom Phaeodactylum tricornutum cells and a streamlined protocol for genotyping mutant cell lines with previously unknown phenotypes. First, bacterialconjugation was optimized for the delivery of Cas9 by transcriptionally fusing Cas9 to a selectable marker by the 2A peptide. An episome cloning strategy using both negative and positive selection was developed to streamline CRISPR-episome assembly. Next, cell line picking and genotyping strategies, that utilize manual sequencing curation, TIDE sequencing analysis, and a T7 endonuclease assay, were developed to shorten the time required to generate mutants. Following this new experimental pipeline, both singlegene and two-gene knockout cell lines were generated at mutagenesis efficiencies of 48% and 25%, respectively. Lastly, a protocol for precise gene insertions via CRISPRCas9 targeting was developed using particle-bombardment transformation methods. Overall, the novel Cas9 episome design and improved genotyping methods presented here allow for quick and easy genotyping and isolation of Phaeodactylum mutant cell lines (less than 3 weeks) without relying on a known phenotype to screen for mutants.

59 BASIC BIOLOGICAL SCIENCES↗

Coming-of-Age Characterization of Soil Viruses: A User’s Guide to Virus Isolation, Detection within Metagenomes, and Viromics

The study of soil viruses, though not new, has languished relative to the study of marine viruses. This is particularly due to challenges associated with separating virions from harboring soils. Generally, three approaches to analyzing soil viruses have been employed: (1) Isolation, to characterize virus genotypes and phenotypes, the primary method used prior to the start of the 21st century. (2) Metagenomics, which has revealed a vast diversity of viruses while also allowing insights into viral community ecology, although with limitations due to DNA from cellular organisms obscuring viral DNA. (3) Viromics (targeted metagenomics of virus-like-particles), which has provided a more focused development of ‘virus-sequence-to-ecology’ pipelines, a result of separation of presumptive virions from cellular organisms prior to DNA extraction. This separation permits greater sequencing emphasis on virus DNA and thereby more targeted molecular and ecological characterization of viruses. Employing viromics to characterize soil systems presents new challenges, however. Ones that only recently are being addressed. Here we provide a guide to implementing these three approaches to studying environmental viruses, highlighting benefits, difficulties, and potential contamination, all toward fostering greater focus on viruses in the study of soil ecology.

59 BASIC BIOLOGICAL SCIENCES↗

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

The IPAC Image Subtraction and Discovery Pipeline for the Intermediate Palomar Transient Factory

We describe the near real-time transient-source discovery engine for the intermediate Palomar Transient Factory (iPTF), currently in operations at the Infrared Processing and Analysis Center (IPAC), Caltech. We coin this system the IPAC/iPTF Discovery Engine (or IDE). We review the algorithms used for PSF-matching, image subtraction, detection, photometry, and machine-learned (ML) vetting of extracted transient candidates. We also review the performance of our ML classifier. For a limiting signal-to-noise ratio of 4 in relatively unconfused regions, bogus candidates from processing artifacts and imperfect image subtractions outnumber real transients by approximately equal to 10:1. This can be considerably higher for image data with inaccurate astrometric and/or PSF-matching solutions. Despite this occasionally high contamination rate, the ML classifier is able to identify real transients with an efficiency (or completeness) of approximately equal to 97% for a maximum tolerable false-positive rate of 1% when classifying raw candidates. All subtraction-image metrics, source features, ML probability-based real-bogus scores, contextual metadata from other surveys, and possible associations with known Solar System objects are stored in a relational database for retrieval by the various science working groups. We review our efforts in mitigating false-positives and our experience in optimizing the overall system in response to the multitude of science projects underway with iPTF.

methods: analytical – methods: data analysis –↗

THE SPITZER SURVEY OF STELLAR STRUCTURE IN GALAXIES (S{sup 4}G): MULTI-COMPONENT DECOMPOSITION STRATEGIES AND DATA RELEASE

The Spitzer Survey of Stellar Structure in Galaxies (S{sup 4}G) is a deep 3.6 and 4.5 μm imaging survey of 2352 nearby (<40 Mpc) galaxies. We describe the S{sup 4}G data analysis pipeline 4, which is dedicated to two-dimensional structural surface brightness decompositions of 3.6 μm images, using GALFIT3.0. Besides automatic 1-component Sérsic fits, and 2-component Sérsic bulge + exponential disk fits, we present human-supervised multi-component decompositions, which include, when judged appropriate, a central point source, bulge, disk, and bar components. Comparison of the fitted parameters indicates that multi-component models are needed to obtain reliable estimates for the bulge Sérsic index and bulge-to-total light ratio (B/T), confirming earlier results. Here, we describe the preparations of input data done for decompositions, give examples of our decomposition strategy, and describe the data products released via IRSA and via our web page (www.oulu.fi/astronomy/S4G-PIPELINE4/MAIN). These products include all the input data and decomposition files in electronic form, making it easy to extend the decompositions to suit specific science purposes. We also provide our IDL-based visualization tools (GALFIDL) developed for displaying/running GALFIT-decompositions, as well as our mask editing procedure (MASK-EDIT) used in data preparation. A detailed analysis of the bulge, disk, and bar parameters derived from multi-component decompositions will be published separately.

79 ASTRONOMY AND ASTROPHYSICS↗

High-throughput binding affinity calculations at extreme scales

Background Resistance to chemotherapy and molecularly targeted therapies is a major factor in limiting the effectiveness of cancer treatment. In many cases, resistance can be linked to genetic changes in target proteins, either pre-existing or evolutionarily selected during treatment. Key to overcoming this challenge is an understanding of the molecular determinants of drug binding. Using multi-stage pipelines of molecular simulations we can gain insights into the binding free energy and the residence time of a ligand, which can inform both stratified and personal treatment regimes and drug development. To support the scalable, adaptive and automated calculation of the binding free energy on high-performance computing resources, we introduce the High-throughput Binding Affinity Calculator (HTBAC). HTBAC uses a building block approach in order to attain both workflow flexibility and performance. Results We demonstrate close to perfect weak scaling to hundreds of concurrent multi-stage binding affinity calculation pipelines. This permits a rapid time-to-solution that is essentially invariant of the calculation protocol, size of candidate ligands and number of ensemble simulations. Conclusions As such, HTBAC advances the state of the art of binding affinity calculations and protocols. HTBAC provides the platform to enable scientists to study a wide range of cancer drugs and candidate ligands in order to support personalized clinical decision making based on genome sequencing and drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗

Finding New Cell Wall Regulatory Genes in Populus trichocarpa Using Multiple Lines of Evidence

Understanding the regulatory network controlling cell wall biosynthesis is of great interest in Populus trichocarpa, both because of its status as a model woody perennial and its importance for lignocellulosic products. We searched for genes with putatively unknown roles in regulating cell wall biosynthesis using an extended network-based Lines of Evidence (LOE) pipeline to combine multiple omics data sets in P. trichocarpa, including gene coexpression, gene comethylation, population level pairwise SNP correlations, and two distinct SNP-metabolite Genome Wide Association Study (GWAS) layers. By incorporating validation, ranking, and filtering approaches we produced a list of nine high priority gene candidates for involvement in the regulation of cell wall biosynthesis. We subsequently performed a detailed investigation of candidate gene GROWTH-REGULATING FACTOR 9 (PtGRF9). To investigate the role of PtGRF9 in regulating cell wall biosynthesis, we assessed the genome-wide connections of PtGRF9 and a paralog across data layers with functional enrichment analyses, predictive transcription factor binding site analysis, and an independent comparison to eQTN data. Our findings indicate that PtGRF9 likely affects the cell wall by directly repressing genes involved in cell wall biosynthesis, such as PtCCoAOMT and PtMYB.41, and indirectly by regulating homeobox genes. Furthermore, evidence suggests that PtGRF9 paralogs may act as transcriptional co-regulators that direct the global energy usage of the plant. Using our extended pipeline, we show multiple lines of evidence implicating the involvement of these genes in cell wall regulatory functions and demonstrate the value of this method for prioritizing candidate genes for experimental validation.

network analysis↗

Enabling Dark Energy Measurements from DESI and LSST (Final Technical Report)

This grant enabled efforts to lead the Survey Validation of the Luminous Red Galaxy (LRG) sample for DESI, entailing intensive work to prepare target samples, test their performance with DESI data, and validate that the requirements of the survey for this key sample are met. Luminous Red Galaxies represent the gold standard target class for Baryon Acoustic Observation experiments to study Dark Energy. It also funded work to co-leading the Follow-up Task Force within the LSST Dark Energy Science Collaboration (LSST DESC), which is intended to help develop collaborations with external groups and to produce cross-working group proposals for telescope time, policy proposals and white papers as needed in order to help the collaboration obtain and make use of complementary data which will strengthen LSST Dark Energy constraints. The PI has been particularly engaged in efforts to obtain access to spectroscopic training sets for LSST photometric redshifts in the first years of the survey, which requires developing relationships with groups that are obtaining such data for other purposes. This group has evolved into an External Synergies working group within DESC, also co-led by the PI. These groups have developed white papers for Astro2020, letters of intent and white papers for Snowmass2021, and a response to a DOE-NASA RFI. It also has enabled smaller contributions to improving LSST DESC pipeline infrastructure for photometric redshifts.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

An End-to-End Architecture for Science Goal Driven Observing

New observatories will have greater on-board storage capacity and on-board processing capabilities. The new bottleneck will be download capacity. The cost of downlink time and limitations of bandwidth will end the era where all exposure data is downloaded and all data processing is performed on the ground. In addition, observing campaigns involving inherently variable targets will need scheduling flexibility to focus observing time and data download on exposures that are scientifically interesting. The ability to quickly recognize and react to such events by re-prioritizing the observing schedule will be an essential characteristic for maximizing scientific returns. It will also be a step towards increasing spacecraft autonomy, a major goal of NASA's strategic plan. The science goal monitoring (SGM) system is a proof-of-concept effort to address these challenges. We are developing an interactive distributed system that will use on-board processing and storage combined with event-driven interfaces with ground-based processing and operations, to enable fast re-prioritization of observing schedules, and to minimize time spent on non-optimized observations. SGM is initially aimed towards time-tagged observing modes used frequently in spectroscopic studies of varying targets. In particular, the SGM is collaborating with the proposed MIDEX-class mission Kronos team. The variable targets that Kronos seeks to study make an adaptive system such as SGM particularly valuable for achieving mission goals. However, the architecture and interfaces will also be designed for easy adaptability to other observing platforms, including ground-based systems and to work with different scheduling and pipeline processing systems. This talk will focus on our strategy for developing SGM and the technical challenges that we have encountered. We will discuss the SGM architecture as it applies to the Kronos mission and explain how it is scalable to other missions.

Jones, Jeremy↗

pmartR 2.0 : A Quality Control, Visualization, and Statistics Pipeline for Multiple Omics Datatypes

The pmartR (https://github.com/pmartR/pmartR) package was designed for the quality control (QC) and analysis of mass spectrometry (MS) data, tailored to specific characteristics of proteomic (isobaric or labelled), metabolomic, and lipidomic datasets. Since its initial release, the tool has been expanded to address the needs of its growing userbase and now includes QC and statistics for nuclear magnetic resonance (NMR) metabolomic data, and leverages the DESeq2, edgeR, and limma-voom R packages for some transcriptomic data analyses. These improvements have made progress towards a unified omics processing pipeline for ease of reporting and streamlined statistical purposes. The package’s statistics and visualization capabilities have also been expanded by adding support for paired data and by integrating pmartR with the trelliscopejs R package for the quick creation of trellis displays (https://github.com/hafen/trelliscopejs). Here, we present relevant examples of each of these enhancements to pmartR and highlight how each new feature benefits the omics community.

59 BASIC BIOLOGICAL SCIENCES↗

Automated Strain Construction for Biosynthetic Pathway Screening in Yeast

Automation accelerates the Design-Build-Test-Learn (DBTL) cycle for synthetic biology; however, most strain construction pipelines lack robotic integration. Here, in this study, we present the workflow design and source code for a modular, integrated protocol that automates the Build step in Saccharomyces cerevisiae. We programmed the Hamilton Microlab VANTAGE to integrate off-deck hardware via its central robotic arm, enabling automated steps that increased throughput to 2,000 transformations per week. We developed a user interface with the Hamilton VENUS software to support on-demand parameter customization. As a proof of concept, we screened a gene library in an engineered yeast strain producing verazine, a key intermediate in the biosynthesis of steroidal alkaloids. Our pipeline rapidly identified pathway bottlenecks and genes that enhanced verazine production by 2.0- to 5-fold. This technical note provides resources for synthetic biologists designing yeast workflows for biofoundries to screen libraries for pathway discovery/optimization, combinatorial biosynthesis, and protein engineering.

automation↗

RASTtk: A modular and extensible implementation of the RAST algorithm for building custom annotation pipelines and annotating batches of genomes

The RAST (Rapid Annotation using Subsystem Technology) annotation engine was built in 2008 to annotate bacterial and archaeal genomes. It works by offering a standard software pipeline for identifying genomic features (i.e., protein-encoding genes and RNA) and annotating their functions. Recently, in order to make RAST a more useful research tool and to keep pace with advancements in bioinformatics, it has become desirable to build a version of RAST that is both customizable and extensible. In this paper, we describe the RAST tool kit (RASTtk), a modular version of RAST that enables researchers to build custom annotation pipelines. RASTtk offers a choice of software for identifying and annotating genomic features as well as the ability to add custom features to an annotation job. RASTtk also accommodates the batch submission of genomes and the ability to customize annotation protocols for batch submissions. This is the first major software restructuring of RAST since its inception.

97 MATHEMATICS AND COMPUTING↗

Ligand placement based on prior structures: the guided ligand-replacement method

A new module, Guided Ligand Replacement (GLR), has been developed in Phenix to increase the ease and success rate of ligand placement when prior protein-ligand complexes are available. The process of iterative structure-based drug design involves the X-ray crystal structure determination of upwards of 100 ligands with the same general scaffold (i.e. chemotype) complexed with very similar, if not identical, protein targets. In conjunction with insights from computational models and assays, this collection of crystal structures is analyzed to improve potency, to achieve better selectivity and to reduce liabilities such as absorption, distribution, metabolism, excretion and toxicology. Current methods for modeling ligands into electron-density maps typically do not utilize information on how similar ligands bound in related structures. Even if the electron density is of sufficient quality and resolution to allow de novo placement, the process can take considerable time as the size, complexity and torsional degrees of freedom of the ligands increase. A new module, Guided Ligand Replacement (GLR), was developed in Phenix to increase the ease and success rate of ligand placement when prior protein–ligand complexes are available. At the heart of GLR is an algorithm based on graph theory that associates atoms in the target ligand with analogous atoms in the reference ligand. Based on this correspondence, a set of coordinates is generated for the target ligand. GLR is especially useful in two situations: (i) modeling a series of large, flexible, complicated or macrocyclic ligands in successive structures and (ii) modeling ligands as part of a refinement pipeline that can automatically select a reference structure. Even in those cases for which no reference structure is available, if there are multiple copies of the bound ligand per asymmetric unit GLR offers an efficient way to complete the model after the first ligand has been placed. In all of these applications, GLR leverages prior knowledge from earlier structures to facilitate ligand placement in the current structure.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Intersection of Hydrologic Change and Hydropower in the United States: Needs for Future Research and Practice

Hydropower is crucial for electric‐grid stability in the context of variable renewables but faces threats from changing hydrology. Here, we summarize the state of the science at the intersection of hydropower operations and planning, hydrologic science, and climate. We focus on the United States, outlining research, development, and training needs. Key knowledge gaps include the risk that intensification of compound extreme events poses to future generation, as well as uncertainties surrounding greenhouse gas emissions from hydropower reservoirs with relevance to hydropower's role in energy decarbonization. Quantifying such impacts and reducing uncertainty are critical where possible, but remaining irreducible or deep uncertainty will require new approaches. Future monitoring and modeling methods must provide a better understanding of the complexity inherent in large watersheds that is critical to managing both hydropower and watersheds in the context of hydrologic change. Yet, research and development will have little impact if they do not inform practice. Standardization and consolidation of platforms are essential for data, modeling, and tool translation to local scales and small operators. An enhanced industry‐academia dialog is pivotal for fostering a robust pipeline of hydropower professionals. Collaboration among researchers, policymakers, authorities, and industry stakeholders emerges as a recurring theme, highlighting the imperative for collective efforts.

13 HYDRO ENERGY↗