Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Science Pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

An Adaptive Middleware Framework for Scientific Computing at Extreme Scales

Large computing systems including clusters, clouds, and grids, provide high-performance capabilities that can be utilized for many applications. But as the ubiquity of these systems increases and the scope of analysis being done on them grows, there is a growing need for applications that 1) do not require users to learn the details of high performance systems, and 2) are flexible and adaptive in their usage of these systems to accommodate the best time-to-solution for end users. We introduce a new adaptive interface design and a prototype implementation within the framework of an established middleware framework, MeDICi, for high performance computing systems and describe the applicability of this adaptive design to a real-life scientific workflow. This adaptive framework provides an access model for implementing a processing pipeline using high performance systems that are not local to the data source, making it possible for the compute capabilities at one site to be applied to analysis on data being generated at another site in an automated process. This adaptive design improves overall time-to-solution by moving the data analysis task to the most appropriate resource dynamically, reacting to failures and load fluctuations.

97 MATHEMATICS AND COMPUTING↗

Scatter Removal using Black Body Grids and Determining the Minimum Resolvable Hydrogen Concentration at MARS

This dataset was used to develop and validate a scatter-correction pipeline for neutron radiographs acquired at the MARS beamline, based on the methodology of Carminati et al. (2019). Scatter removal was performed using 36 gadolinium black bodies to characterize and remove the spatially varying neutron scatter field that cannot be eliminated by conventional open-beam normalization. The dataset comprises eight samples - four single-crystal nickel and four polycrystalline austenitic 316L stainless steel. Within each material system, samples were pre-charged with hydrogen gas at four charging pressures (1.5, 5, 12.5, and 20 kpsi). Following scatter correction, the pipeline was applied to all radiographs to recover quantitative attenuation maps. After subtracting the known attenuation contribution of the metal matrix, spatial maps of the hydrogen attenuation coefficient were obtained for each sample. This data was then used to construct a calibration curve relating hydrogen attenuation to hydrogen concentration and to determine the minimum resolvable hydrogen concentration at the MARS beamline. The resulting dataset provides a pressure-resolved calibration standard for quantitative hydrogen mapping using neutron radiography.

36 MATERIALS SCIENCE↗

SDAV Viz July Progress Update: LANL

SDAV Viz July Progress Update: (1) VPIC (Vector Particle in Cell) Kinetic Plasma Simulation Code - (a) Implemented first version of an in-situ adapter based on Paraview CoProcessing Library, (b) Three pipelines: vtkDataSetMapper, vtkContourFilter, vtkPistonContour, (c) Next, resolve issue at boundaries of processor domains; add more advanced viz/analysis pipelines; (2) Halo finding/merger trees - (a) Summer student Wathsala W. from University of Utah is working on data-parallel halo finder algorithm using PISTON, (b) Timo Bremer (LLNL), Valerio Pascucci (Utah), George Zagaris (Kitware), and LANL people are interested in using merger trees for tracking the evolution of halos in cosmo simulations; discussed possible overlap with work by Salman Habib and Katrin Heitmann (Argonne) during their visit to LANL 7/11; (3) PISTON integration in ParaView - Now available from ParaView github.

97 MATHEMATICS AND COMPUTING↗

Spinoff 2015

Topics covered include: 3D Endoscope to Boost Safety, Cut Cost of Surgery; Audio App Brings a Better Night's Sleep Liquid Cooling Technology Increases Exercise Efficiency; Algae-Derived Dietary Ingredients Nourish Animals; Space Grant Research Launches Rehabilitation Chair; Vision Trainer Teaches Focusing Techniques at Home; Aircraft Geared Architecture Reduces Fuel Cost and Noise; Ubiquitous Supercritical Wing Design Cuts Billions in Fuel Costs; Flight Controller Software Protects Lightweight Flexible Aircraft; Cabin Pressure Monitors Notify Pilots to Save Lives; Ionospheric Mapping Software Ensures Accuracy of Pilots' GPS; Water Mapping Technology Rebuilds Lives in Arid Regions; Shock Absorbers Save Structures and Lives during Earthquakes; Software Facilitates Sharing of Water Quality Data Worldwide; Underwater Adhesives Retrofit Pipelines with Advanced Sensors; Laser Imaging Video Camera Sees through Fire, Fog, Smoke; 3D Lasers Increase Efficiency, Safety of Moving Machines; Air Revitalization System Enables Excursions to the Stratosphere; Magnetic Fluids Deliver Better Speaker Sound Quality; Bioreactor Yields Extracts for Skin Cream; Private Astronaut Training Prepares Commercial Crews of Tomorrow; Activity Monitors Help Users Get Optimum Sun Exposure; LEDs Illuminate Bulbs for Better Sleep, Wake Cycles; Charged Particles Kill Pathogens and Round Up Dust; Balance Devices Train Golfers for a Consistent Swing; Landsat Imagery Enables Global Studies of Surface Trends; Ruggedized Spectrometers Are Built for Tough Jobs; Gas Conversion Systems Reclaim Fuel for Industry; Remote Sensing Technologies Mitigate Drought; Satellite Data Inform Forecasts of Crop Growth; Probes Measure Gases for Environmental Research; Cloud Computing Technologies Facilitate Earth Research; Software Cuts Homebuilding Costs, Increases Energy Efficiency; Portable Planetariums Teach Science; Schedule Analysis Software Saves Time for Project Planners; Sound Modeling Simplifies Vehicle Noise Management; Custom 3D Printers Revolutionize Space Supply Chain; Improved Calibration Shows Images' True Colors; Micromachined Parts Advance Medicine, Astrophysics, and More; Metalworking Techniques Unlock a Unique Alloy; Low-Cost Sensors Deliver Nanometer-Accurate Measurements; Electrical Monitoring Devices Save on Time and Cost; Dry Lubricant Smooths the Way for Space Travel, Industry; and Compact Vapor Chamber Cools Critical Components.

Source record↗

An improved Greengenes taxonomy with explicit ranks for ecological and evolutionary analyses of bacteria and archaea

Reference phylogenies are crucial for providing a taxonomic framework for interpretation of marker gene and metagenomic surveys, which continue to reveal novel species at a remarkable rate. Greengenes is a dedicated full-length 16S rRNA gene database that provides users with a curated taxonomy based on de novo tree inference. We developed a ‘taxonomy to tree’ approach for transferring group names from an existing taxonomy to a tree topology, and used it to apply the Greengenes, National Center for Biotechnology Information (NCBI) and cyanoDB (Cyanobacteria only) taxonomies to a de novo tree comprising 408 315 sequences. We also incorporated explicit rank information provided by the NCBI taxonomy to group names (by prefixing rank designations) for better user orientation and classification consistency. The resulting merged taxonomy improved the classification of 75% of the sequences by one or more ranks relative to the original NCBI taxonomy with the most pronounced improvements occurring in under-classified environmental sequences. We also assessed candidate phyla (divisions) currently defined by NCBI and present recommendations for consolidation of 34 redundantly named groups. All intermediate results from the pipeline, which includes tree inference, jackknifing and transfer of a donor taxonomy to a recipient tree (tax2tree) are available for download. The improved Greengenes taxonomy should provide important infrastructure for a wide range of megasequencing projects studying ecosystems on scales ranging from our own bodies (the Human Microbiome Project) to the entire planet (the Earth Microbiome Project). The implementation of the software can be obtained from http://sourceforge.net/projects/tax2tree/.

54 ENVIRONMENTAL SCIENCES↗

Industry Partnerships & Their Role In Reducing Natural Gas Supply Chain Greenhouse Gas Emissions – Phase 2

This analysis is the product of collaboration between Our Nation’s Energy Future (ONE Future) and the United States (U.S.) Department of Energy (DOE) National Energy Technology Laboratory (NETL). This analysis is an update to Phase 1 of the collaboration between ONE Future and NETL. Phase 1 had three objectives: 1. Calculate a greenhouse gas (GHG) emission profile representative of ONE Future’s supply chain, including methane (CH 4 ) emission rates. 2. Compare ONE Future’s emission profile to the emission profile for the U.S. natural gas supply chain. 3. Evaluate specific emission reduction opportunities. Phase 2 has two key enhancements over Phase 1: 1. The reporting year of the data is updated from 2016 to 2017. 2. The emission profiles and the specific emission reduction opportunities are regionalized for the ONE Future supply chain. The ONE Future supply chain is based on data provided by ONE Future members for all their U.S. onshore assets. ONE Future’s data are mostly representative of their participation in the Greenhouse Gas Reporting Program (GHGRP) administered by the Environmental Protection Agency (EPA) and is supplemented by ONE Future facilities that are not required to report to GHGRP. The U.S. scenario is based on NETL’s life cycle analysis (LCA) of natural gas extraction and power generation, which also uses data from the GHGRP (but does not include non-GHGRP facilities). In addition to the data from these sources, NETL accounted for uncertainty due to data variability, data limitations, and variability in liquids unloading frequency and event duration. This report is a re-issue of the 2020 study (published on July 28, 2020). It revises the device count in the mitigation strategy “pipeline pump-down before maintenance” in the Marginal Abatement Cost (MAC) Analysis section of the report (Section 7). Please see the addendum for more details on this revision and resultant changes. This revision does not change most of our conclusions and recommendations. It only changes the total methane reduced from all the mitigation opportunities (low cost and high cost) from 6.5 Bcf CH 4 /yr to 4.8 Bcf CH 4 /yr.

03 NATURAL GAS↗

Agora : Multicomponent simulation for cross-survey science

ABSTRACT Next-generation cosmological surveys will observe large portions of the sky, with significant overlap between them. Multi-wavelength observations will enable us to analyse the same large-scale structure from different angles using a variety of tracers and astrophysical effects. The complex interplay between these observables calls for a model that can accurately and coherently describe their collective behaviour, posing a challenge that can only be met through the use of simulations. In this work, a suite of simulated extragalactic skies is presented, including maps and/or catalogues of cosmic microwave background (CMB) lensing, thermal and kinetic Sunyaev–Zel’dovich (tSZ/kSZ) effects, cosmic infrared background (CIB), radio sources, galaxy overdensity and galaxy weak lensing. Each of these probes is implemented in the lightcone using halo catalogues and/or particles from the Multidark-Planck2 ($\small {MDPL2}$) N-body simulation, and the modelling is calibrated using hydrodynamic simulations and publicly available data. The auto and cross-spectra of the individual probes, as well as the cross-spectra between the observables, are shown to be consistent with theoretical models and measurements from data. The simulation is shown to have a wide range of applications, including forecasting, pipeline testing, and evaluating astrophysical biases in cross-correlation measurements. It is further demonstrated that the simulation products produced in this work have sufficient accuracy to recover the input cosmology when subjected to a full cosmological analysis and are ready for application in real-world analyses for ongoing and future surveys. The simulation products presented in this work can be accessed at: https://yomori.github.io/agora/index.html.

Astronomy & Astrophysics↗

Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances

Purpose: Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology. Methods: We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology. Results: Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping. Conclusions: Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.

60 APPLIED LIFE SCIENCES↗

Strangers in a foreign land: ‘Yeastizing’ plant enzymes

Abstract Expressing plant metabolic pathways in microbial platforms is an efficient, cost‐effective solution for producing many desired plant compounds. As eukaryotic organisms, yeasts are often the preferred platform. However, expression of plant enzymes in a yeast frequently leads to failure because the enzymes are poorly adapted to the foreign yeast cellular environment. Here, we first summarize the current engineering approaches for optimizing performance of plant enzymes in yeast. A critical limitation of these approaches is that they are labour‐intensive and must be customized for each individual enzyme, which significantly hinders the establishment of plant pathways in cellular factories. In response to this challenge, we propose the development of a cost‐effective computational pipeline to redesign plant enzymes for better adaptation to the yeast cellular milieu. This proposition is underpinned by compelling evidence that plant and yeast enzymes exhibit distinct sequence features that are generalizable across enzyme families. Consequently, we introduce a data‐driven machine learning framework designed to extract ‘yeastizing’ rules from natural protein sequence variations, which can be broadly applied to all enzymes. Additionally, we discuss the potential to integrate the machine learning model into a full design‐build‐test cycle.

59 BASIC BIOLOGICAL SCIENCES↗

HiFiAdapterFilt, a memory efficient read processing pipeline, prevents occurrence of adapter sequence in PacBio HiFi reads and their negative impacts on genome assembly

Abstract Background Pacific Biosciences HiFi read technology is currently the industry standard for high accuracy long-read sequencing that has been widely adopted by large sequencing and assembly initiatives for generation of de novo assemblies in non-model organisms. Though adapter contamination filtering is routine in traditional short-read analysis pipelines, it has not been widely adopted for HiFi workflows. Results Analysis of 55 publicly available HiFi datasets revealed that a read-sanitation step to remove sequence artifacts derived from PacBio library preparation from read pools is necessary as adapter sequences can be erroneously integrated into assemblies. Conclusions Here we describe the nature of adapter contaminated reads, their consequences in assembly, and present HiFiAdapterFilt, a simple and memory efficient solution for removing adapter contaminated reads prior to assembly.

59 BASIC BIOLOGICAL SCIENCES↗

NoFold: RNA structure clustering without folding or alignment

Structures that recur across multiple different transcripts, called structure motifs, often perform a similar function—for example, recruiting a specific RNA-binding protein that then regulates translation, splicing, or subcellular localization. Identifying common motifs between coregulated transcripts may therefore yield significant insight into their binding partners and mechanism of regulation. However, as most methods for clustering structures are based on folding individual sequences or doing many pairwise alignments, this results in a tradeoff between speed and accuracy that can be problematic for large-scale data sets. Here we describe a novel method for comparing and characterizing RNA secondary structures that does not require folding or pairwise alignment of the input sequences. Our method uses the idea of constructing a distance function between two objects by their respective distances to a collection of empirical examples or models, which in our case consists of 1973 Rfam family covariance models. Using this as a basis for measuring structural similarity, we developed a clustering pipeline called NoFold to automatically identify and annotate structure motifs within large sequence data sets. We demonstrate that NoFold can simultaneously identify multiple structure motifs with an average sensitivity of 0.80 and precision of 0.98 and generally exceeds the performance of existing methods. We also perform a cross-validation analysis of the entire set of Rfam families, achieving an average sensitivity of 0.57. We apply NoFold to identify motifs enriched in dendritically localized transcripts and report 213 enriched motifs, including both known and novel structures.

59 BASIC BIOLOGICAL SCIENCES↗

Unlocking Short Read Sequencing for Metagenomics

We describe an experimental and computational pipeline yielding millions of reads that can exceed 200 bp with quality scores approaching that of traditional Sanger sequencing. The method combines an automatable gel-less library construction step with paired-end sequencing on a short-read instrument. With appropriately sized library inserts, mate-pair sequences can overlap, and we describe the SHERA software package that joins them to form a longer composite read.

59 BASIC BIOLOGICAL SCIENCES↗

Gas Stopper Developments for Improved Purity and Intensity of Low-Energy, Rare Isotope Ion Beams (Final Technical Report)

This final technical report summarizes the work of the Michigan State University (MSU) team supported by grant # DE-SC0021423 awarded by the Office of Nuclear Physics, Department of Energy. Objectives: The successful fulfillment of the FRIB science mission hinges on ensuring the availability of fast, stopped, and reaccelerated beams consisting of rare isotopes. This project's research and development focus was dedicated to supporting the advancement and creation of a cutting-edge linear gas stopper. The primary aim is to efficiently convert the high-intensity fast beams of rare isotopes provided by FRIB into high-quality, low-energy beams. These beams are essential for conducting stopped beam experiments or for subsequent reacceleration. The overarching goal is to advance technology, aiming to increase the beam rate capability of the linear gas stopper for medium-to-heavy-mass rare isotopes by more than tenfold compared to the currently most effective gas stopper in operation, and to improve the purity of the delivered beams. Project Description: The existing technology employed in gas stopping devices designed for low-energy, rare-isotope beams presents limitations in both the purity of extracted beams and the intensities of injected beams. These limitations are incompatible with the requirements of the recently commissioned rare isotope beam facility, FRIB. Our research and development efforts, aligned with the previously outlined objectives, focused on addressing the most critical aspects for enhancing beam-rate capability and purity. Specifically, advanced particle-in-cell simulations were developed and integrated into a simulation pipeline to explore the efficacy of multi-layer RF carpets on increasing ion transport efficiency with high incoming beam rates that generate space charge fields which can limit it. We also explored the possibility of using a collision-induced-dissociation (CID) gas cell to break up molecular contaminant ions that are generated during the stopping process. A prototype CID gas cell was constructed and tested with beams from an offline ion source, validating the concept with the successful demonstration of breaking of molecular ions. The outcome of this research enabled the formulation a conceptual design for a next-generation linear gas stopping device specifically tailored for FRIB. This device is envisioned to deliver rare-isotope-ion beams at a rate of 10 8 particles per second or higher, accompanied by advancements in purity. Methods employed: This project leverages advancements in technologies initially designed for the Advanced Cryogenic Gas Stopper (ACGS), the current state-of-the-art linear gas stopper, through the use of new simulations and beam purification via collision-induced-dissociation. The methods include: 1. Development of a prototype low-energy, low-pressure CID gas stopper. This prototype features a thin, approximately 20 nm, Si 3 N 4 entrance window designed for dissociating stable and rare-isotope molecular ions. The goal is to enhance beam purification and overall efficiency. 2. Creation of Particle-in-cell (PIC) simulations to assess the advantages of multi-layer RF carpets and multi-point extraction for ion transport efficiency. These simulations rely on the 3DCylPIC package, specifically designed for studying devices of this nature. The goal is to quantify and mitigate ion transport losses due to space charge generated in the stopping process of large numbers of ions. 3. Perform ion transport simulations across an RF carpet using an 8-phase travelling wave and evaluate its performance. Compared to the 4-phase RF carpets used in ACGS, the 8-phase carpets will double the wavelength of the generated traveling wave allowing for larger maximum RF amplitudes that could result in improved ion transport efficiency for high-intensity incoming beams when large space charge fields are present. Impact: Tackling the primary challenges associated with transforming high-energy projectile fragment beams into low-energy beams—specifically, addressing efficiency and purity—holds significant promise for advancing FRIB science. This advancement will particularly impact precision mass measurements, laser spectroscopy of short-lived nuclei, and studies in astrophysics and nuclear reactions using reaccelerated beams. These domains play a crucial role in addressing key questions outlined in the 2023 NSAC long-range plan, spanning nuclear structure, nuclear astrophysics, and fundamental symmetries. Additionally, they contribute to addressing 10 out of the 17 benchmarks identified by the NRC RISAC. The development of a next-generation gas stopping device capable of delivering low-energy, rare-isotope beams at a rate of 10 8 particles per second, or more, with high purity holds the potential to unlock experiments that would otherwise be unfeasible. Furthermore, it is expected to reduce the time required for experiments at FRIB, thereby maximizing scientific output. The research and development activities performed as part of this project bolstered essential competencies at FRIB in beam physics and ion source technologies, provided valuable training opportunities for junior scientists.

43 PARTICLE ACCELERATORS↗

Eukaryotic genomes from a global metagenomic data set illuminate trophic modes and biogeography of ocean plankton

ABSTRACT Metagenomics is a powerful method for interpreting the ecological roles and physiological capabilities of mixed microbial communities. Yet, many tools for processing metagenomic data are neither designed to consider eukaryotes nor are they built for an increasing amount of sequence data. EukHeist is an automated pipeline to retrieve eukaryotic and prokaryotic metagenome-assembled genomes (MAGs) from large-scale metagenomic sequence data sets. We developed the EukHeist workflow to specifically process large amounts of both metagenomic and/or metatranscriptomic sequence data in an automated and reproducible fashion. Here, we applied EukHeist to the large-size fraction data (0.8–2,000 µm) from Tara Oceans to recover both eukaryotic and prokaryotic MAGs, which we refer to as TOPAZ (Tara Oceans Particle-Associated MAGs). The TOPAZ MAGs consisted of >900 environmentally relevant eukaryotic MAGs and >4,000 bacterial and archaeal MAGs. The bacterial and archaeal TOPAZ MAGs expand upon the phylogenetic diversity of likely particle- and host-associated taxa. We use these MAGs to demonstrate an approach to infer the putative trophic mode of the recovered eukaryotic MAGs. We also identify ecological cohorts of co-occurring MAGs, which are driven by specific environmental factors and putative host-microbe associations. These data together add to a number of growing resources of environmentally relevant eukaryotic genomic information. Complementary and expanded databases of MAGs, such as those provided through scalable pipelines like EukHeist, stand to advance our understanding of eukaryotic diversity through increased coverage of genomic representatives across the tree of life. IMPORTANCE Single-celled eukaryotes play ecologically significant roles in the marine environment, yet fundamental questions about their biodiversity, ecological function, and interactions remain. Environmental sequencing enables researchers to document naturally occurring protistan communities, without culturing bias, yet metagenomic and metatranscriptomic sequencing approaches cannot separate individual species from communities. To more completely capture the genomic content of mixed protistan populations, we can create bins of sequences that represent the same organism (metagenome-assembled genomes [MAGs]). We developed the EukHeist pipeline, which automates the binning of population-level eukaryotic and prokaryotic genomes from metagenomic reads. We show exciting insight into what protistan communities are present and their trophic roles in the ocean. Scalable computational tools, like EukHeist, may accelerate the identification of meaningful genetic signatures from large data sets and complement researchers’ efforts to leverage MAG databases for addressing ecological questions, resolving evolutionary relationships, and discovering potentially novel biodiversity.

59 BASIC BIOLOGICAL SCIENCES↗

Space Elevators: Building a Permanent Bridge for Space Exploration and Economic Development

A space elevator is a physical connection from the surface of the Earth to a geo-stationary orbit above the Earth approximately 35,786 km in altitude. Its center of mass is at the geo-stationary point such that it has a 24-hour orbit, and stays over the same point above the equator as the Earth rotates on its axis. The structure is utilized as a transportation and utility system for moving payloads, power, and gases between the surface of the Earth and space. It makes the physical connection from Earth to space in the same way a bridge connects two cities across a body of' water. The space elevator may be an important concept for the future development of space in the latter part of the 21th century. It has the potential to provide mass-transportation to space in the same way highways, railroads, power lines, and pipelines provide mass-transportation across the Earth's surface. The low energy requirements for moving payloads up and down the elevator make it one of only a few concepts that has the potential of lowering the cost to orbit to less than $10 per kilogram. This paper will summarize the findings from a 1999 NASA workshop on Space Elevators held at the NASA Marshall Space Flight Center (MSFC). The workshop was sponsored by the Advanced Projects Office in the Flight Projects Directorate at MSFC, and was organized in cooperation with the Advanced Space Transportation Program at MSFC and the Advanced Concepts Office in the Office of Space Flight at NASA Headquarters. New concepts will be examined for space elevator construction and a number of issues will be discussed that has helped to bring the space elevator concept out of the realm of science fiction and into the realm of possibility. In conclusion, it appears that the space elevator concept may well he possible in the latter part of the 21st century if proper planning and technology development is emphasized to resolve key issues in the development of this advanced space infrastructure concept.

Smitherman, David V., Jr.↗

Flexible Learning-Free Segmentation and Reconstruction of Neural Volumes

Imaging is a dominant strategy for data collection in neuroscience, yielding stacks of images that often scale to gigabytes of data for a single experiment. Machine learning algorithms from computer vision can serve as a pair of virtual eyes that tirelessly processes these images, automatically detecting and identifying microstructures. Unlike learning methods, our Flexible Learning-free Reconstruction of Imaged Neural volumes (FLoRIN) pipeline exploits structure-specifc contextual clues and requires no training. This approach generalizes across diferent modalities, including serially-sectioned scanning electron microscopy (sSEM) of genetically labeled and contrast enhanced processes, spectral confocal refectance (SCoRe) microscopy, and high-energy synchrotron X-ray microtomography (μCT) of large tissue volumes. We deploy the FLoRIN pipeline on newly published and novel mouse datasets, demonstrating the high biological fdelity of the pipeline’s reconstructions. FLoRIN reconstructions are of sufcient quality for preliminary biological study, for example examining the distribution and morphology of cells or extracting single axons from functional data. Compared to existing supervised learning methods, FLoRIN is one to two orders of magnitude faster and produces high-quality reconstructions that are tolerant to noise and artifacts, as is shown qualitatively and quantitatively.

47 OTHER INSTRUMENTATION↗

Machine learning-driven predictive resource management in complex science workflows

Here, the collaborative efforts of large communities in science experiments, often comprising thousands of global members, reflect a monumental commitment to exploration and discovery. Recently, advanced and complex data processing has gained increasing importance in science experiments. Data processing workflows typically consist of multiple intricate steps, and the precise specification of resource requirements is crucial for each step to allocate optimal resources for effective processing. Estimating resource requirements in advance is challenging due to a wide range of analysis scenarios, varying skill levels among community members, and the continuously increasing spectrum of computing options. One practical approach to mitigate these challenges involves initially processing a subset of each step to measure precise resource utilization from actual processing profiles before completing the entire step. While this two-staged approach enables processing on optimal resources for most of the workflow, it has drawbacks such as initial inaccuracies leading to potential failures and suboptimal resource usage, along with overhead from waiting for initial processing completion, which is critical for fast-turnaround analyses. In this context, our study introduces a novel pipeline of machine learning models within a comprehensive workflow management system, the Production and Distributed Analysis (PanDA) system. These models employ advanced machine learning techniques to predict key resource requirements, overcoming challenges posed by limited upfront knowledge of characteristics at each step. Accurate forecasts of resource requirements enable informed and proactive decision-making in workflow management, enhancing the efficiency of handling diverse, complex workflows across heterogeneous resources.

97 MATHEMATICS AND COMPUTING↗