Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Memory acquisition”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

ATLAS FTK Challenge: Simulation of a Billion-fold Hardware Parallelism

During the current LHC shutdown period the ATLAS experiment will upgrade the Trigger and Data Acquisition system to include a hardware tracker coprocessor: the Fast TracKer (FTK). The FTK receives data from the 80 million of channels of the ATLAS silicon detector, identifying charged tracks and reconstructing their parameters at a rate of up to 100 KHz and within 100 microseconds. To achieve this performance, the FTK system identifies candidate tracks utilizing the computing power of a custom ASIC chip with associative memory (AM) designed to perform “pattern matching” at very high speed; track parameters are then calculated using modern FPGAs. A detailed simulation of this massive system has been developed with the goal of supporting the hardware design and studying its impact in the ATLAS online event selection at high LHC luminosities. We present the issues related to emulating FTK on a general-purpose CPU platform, using ATLAS computing Grid resources, and the solutions developed in order to mitigate these problems and allow the studies required to support the system design, construction and installation.

Distributed Computing↗

Scalable Proximity-Based Methods for Large-Scale Analysis of Atom Probe Data

Powered by recent advances in data acquisition technologies, today's state-of-the-art atom probe microscopes yield data sets with sizes ranging from a few million atoms to billions of atoms. Analysis of these atomic data sets within rea-sonable turnaround times is a pressing data analysis challenge for material scientists currently equipped with software systems that do not scale to these massive data sets. Here, we present the shared memory component of a larger ongoing effort to develop a multi-feature data analysis framework capable of analyzing atom probe data of all sizes and scales from desktop multicore machines to large-scale high-performance computing platforms with hybrid (shared and distributed memory) architectures. Our focus here is on a broad class of popular atom probe data analysis methods that rely on core time-consuming k-NN queries. We present a scalable, heuristic algorithm for k-NN queries using three-dimensional range trees. To demonstrate its efficacy, the k-NN algorithm is integrated with two use cases of atom probe data analysis methods and the resulting analysis times are shown to speedup by over 20X on a 32-core Cray XC40 node using workloads up to 8 million atoms, which is already beyond the at-scale capabilities of existing atom probe software. Using this k-NN algorithm, we also introduce a novel parameter estimation method for a class of cluster finding methods, called friends-of-friends (FoF) methods, to completely bypass their expensive pre-processing steps. In each case, we validate the results on a variety of control data sets.

Lu, Hao↗

Distributed seams for gigapixel panoramas

Gigapixel panoramas are an increasingly popular digital image application. They are often created as a mosaic of many smaller images. The mosaic acquisition can take many hours causing the individual images to differ in exposure and lighting conditions. A blending operation is often necessary to give the appearance of a seamless image. The blending quality depends on the magnitude of discontinuity along the image boundaries. Often, new boundaries, or seams, are first computed that minimize this transition. Current techniques based on multi-labeling Graph Cuts are too slow and memory intensive for gigapixel sized panoramas. In this paper, we present a parallel, out-of-core seam computing technique that is fast, has small memory footprint, and is capable of running efficiently on different types of parallel systems. Its maximum memory usage is configurable, in the form of a cache, which can improve performance by reducing redundant disk I/O and computations. It shows near-perfect scaling on symmetric multiprocessing systems and good scaling on clusters and distributed shared memory systems. Lastly, our technique improves the time required to compute seams for gigapixel imagery from many hours (or even days) to just a few minutes, while still producing boundaries with energy that is on-par with Graph Cuts.

97 MATHEMATICS AND COMPUTING↗

A Scalable Online Monitoring System Based on Elasticsearch for Distributed Data Acquisition in Cms

The part of the CMS Data Acquisition (DAQ) system responsible for data readout and event building is a complex network of interdependent distributed applications. To ensure successful data taking, these programs have to be constantly monitored in order to facilitate the timeliness of necessary corrections in case of any deviation from specified behaviour. A large number of diverse monitoring data samples are periodically collected from multiple sources across the network. Monitoring data are kept in memory for online operations and optionally stored on disk for post-mortem analysis. We present a generic, reusable solution based on an open source NoSQL database, Elasticsearch, which is fully compatible and non-intrusive with respect to the existing system. The motivation is to benefit from an off the-shelf software to facilitate the development, maintenance and support efforts. Elasticsearch provides failover and data redundancy capabilities as well as a programming language independent JSON-over-HTTP interface. The possibility of horizontal scaling matches the requirements of a DAQ monitoring system. The data load from all sources is balanced by redistribution over an Elasticsearch cluster that can be hosted on a computer cloud. In order to achieve the necessary robustness and to validate the scalability of the approach the above monitoring solution currently runs in parallel with an existing in-house developed DAQ monitoring system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Active Flash: Out-of-core Data Analytics on Flash Storage

Next generation science will increasingly come to rely on the ability to perform efficient, on-the-fly analytics of data generated by high-performance computing (HPC) simulations, modeling complex physical phenomena. Scientific computing workflows are stymied by the traditional chaining of simulation and data analysis, creating multiple rounds of redundant reads and writes to the storage system, which grows in cost with the ever-increasing gap between compute and storage speeds in HPC clusters. Recent HPC acquisitions have introduced compute node-local flash storage as a means to alleviate this I/O bottleneck. We propose a novel approach, Active Flash, to expedite data analysis pipelines by migrating to the location of the data, the flash device itself. We argue that Active Flash has the potential to enable true out-of-core data analytics by freeing up both the compute core and the associated main memory. By performing analysis locally, dependence on limited bandwidth to a central storage system is reduced, while allowing this analysis to proceed in parallel with the main application. In addition, offloading work from the host to the more power-efficient controller reduces peak system power usage, which is already in the megawatt range and poses a major barrier to HPC system scalability. We propose an architecture for Active Flash, explore energy and performance trade-offs in moving computation from host to storage, demonstrate the ability of appropriate embedded controllers to perform data analysis and reduction tasks at speeds sufficient for this application, and present a simulation study of Active Flash scheduling policies. These results show the viability of the Active Flash model, and its capability to potentially have a transformative impact on scientific data analysis.

99 GENERAL AND MISCELLANEOUS↗

Electron energy loss spectroscopy database synthesis and automation of core-loss edge recognition by deep-learning neural networks

Abstract The ionization edges encoded in the electron energy loss spectroscopy (EELS) spectra enable advanced material analysis including composition analyses and elemental quantifications. The development of the parallel EELS instrument and fast, sensitive detectors have greatly improved the acquisition speed of EELS spectra. However, the traditional way of core-loss edge recognition is experience based and human labor dependent, which limits the processing speed. So far, the low signal–noise ratio and the low jump ratio of the core-loss edges on the raw EELS spectra have been challenging for the automation of edge recognition. In this work, a convolutional-bidirectional long short-term memory neural network (CNN-BiLSTM) is proposed to automate the detection and elemental identification of core-loss edges from raw spectra. An EELS spectral database is synthesized by using our forward model to assist in the training and validation of the neural network. To make the synthesized spectra resemble the real spectra, we collected a large library of experimentally acquired EELS core edges. In synthesize the training library, the edges are modeled by fitting the multi-Gaussian model to the real edges from experiments, and the noise and instrumental imperfectness are simulated and added. The well-trained CNN-BiLSTM network is tested against both the simulated spectra and real spectra collected from experiments. The high accuracy of the network, 94.9%, proves that, without complicated preprocessing of the raw spectra, the proposed CNN-BiLSTM network achieves the automation of core-loss edge recognition for EELS spectra with high accuracy.

36 MATERIALS SCIENCE↗

Drought legacies mediated by trait trade‐offs in soil microbiomes

Abstract Soil microbiomes play a key role in driving biogeochemical cycles of the Earth system. As drought frequency and intensity increase due to climate change, soil microbes and the processes they control will be impacted. Even after a drought ends, microbiomes and other systems take time to recover and may display a memory of previous climate conditions. Still, the mechanisms involved in these legacy effects remain unclear, making it difficult to predict climate and biogeochemical rates in the future. Here, we used a trait‐based microbiome model (DEMENTpy) to implement trade‐off‐mediated mechanisms that may lead to drought legacy effects on litter decomposition. Trade‐offs were assumed to follow the Y‐A‐S framework that defines three primary life‐history strategies of microorganisms: high growth Yield, resource Acquisition, and Stress tolerance. We represented cellular trade‐offs between osmolytes required for drought tolerance and investment in enzymes involved in litter decomposition. Simulations were run under varying levels of drought severity and dispersal. With high levels of dispersal, no legacy effects were predicted by DEMENTpy following drought. With limited dispersal, severe drought resulted in a persistent legacy of altered community‐level traits and reduced litter decomposition. Moderate drought resulted in a transient legacy that disappeared after two years, consistent with recent empirical observations in Southern California ecosystems. These results imply that greater movement along the trade‐off between enzyme investment and osmolyte production resulted in stronger legacy effects. More generally, factors that shift the position of a microbiome in YAS space may alter the legacy outcome following drought. Our trait‐based modeling study motivates additional empirical measurements to quantify YAS traits and trade‐offs that are needed to make accurate predictions of soil microbiome resilience and functioning. Also, our study illustrates an emerging approach for representing trait trade‐offs in microbiomes and vegetation that dictate ecosystem responses to drought and other environmental perturbations.

54 ENVIRONMENTAL SCIENCES↗

Monitoring radiation use in cardiac fluoroscopy imaging procedures

Purpose: Timely identification of systematic changes in radiation delivery of an imaging system can lead to a reduction in risk for the patients involved. However, existing quality assurance programs involving the routine testing of equipment performance using phantoms are limited in their ability to effectively carry out this task. To address this issue, the authors propose the implementation of an ongoing monitoring process that utilizes procedural data to identify unexpected large or small radiation exposures for individual patients, as well as to detect persistent changes in the radiation output of imaging platforms. Methods: Data used in this study were obtained from records routinely collected during procedures performed in the cardiac catheterization imaging facility at St. Andrew's War Memorial Hospital, Brisbane, Australia, over the period January 2008-March 2010. A two stage monitoring process employing individual and exponentially weighted moving average (EWMA) control charts was developed and used to identify unexpectedly high or low radiation exposure levels for individual patients, as well as detect persistent changes in the radiation output delivered by the imaging systems. To increase sensitivity of the charts, we account for variation in dose area product (DAP) values due to other measured factors (patient weight, fluoroscopy time, and digital acquisition frame count) using multiple linear regression. Control charts are then constructed using the residual values from this linear regression. The proposed monitoring process was evaluated using simulation to model the performance of the process under known conditions. Results: Retrospective application of this technique to actual clinical data identified a number of cases in which the DAP result could be considered unexpected. Most of these, upon review, were attributed to data entry errors. The charts monitoring the overall system radiation output trends demonstrated changes in equipment performance associated with relocation of the equipment to a new department. When tested under simulated conditions, the EWMA chart was capable of detecting a sustained 15% increase in average radiation output within 60 cases (<1 month of operation), while a 33% increase would be signaled within 20 cases. Conclusions: This technique offers a valuable enhancement to existing quality assurance programs in radiology that rely upon the testing of equipment radiation output at discrete time frames to ensure performance security.

60 APPLIED LIFE SCIENCES↗

Detection of Isotopes in Urban Source Search Gamma Spectra Using Hopfield Neural Network

Source search campaigns involve the measurement of consecutive background gamma-ray spectra obtained with a mobile detector-spectrometer, such as tellurium-doped sodium iodide NaI(Tl) scintillator detector-spectrometer. Counts are measured with NaI(Tl) detector in short acquisition intervals, such as one second. Detection of isotopes in search data is challenging because of the low signal-to-noise ratio (S/N) of the spectral lines and highly varying background due to naturally occurring radioactive material (NORM). In this work, we investigate the performance of the detection of orphan Cs-137, Am-241, and I-131 isotopes in a gamma ray data set obtained in a search survey. Isotopic detection is performed with a Hopfield Neural Network (HNN), which implements an auto-associative memory by matching incoming measurements to the closest one of its memory-stored signature patterns. HNN is trained on examples of spectra of calibrated isotopic sources with well-resolved spectral lines. During testing, HNN processes spectra with partially complete or distorted isotopic lines. The performance of the HNN detection algorithm is benchmarked with Precision, Recall, and F1 scores. Test results demonstrate the potential of HNN to detect the isotope of interest with high accuracy (F1 > 90%) for all isotopes in this study.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Morris County Improvement Authority, Morris County, New Jersey Renewable Energy Initiative

The Morris County Improvement Authority (Authority), a public body corporate and politic of the State of New Jersey and created and controlled by the County, at the direction of the County and through the Program guaranteed by the County, financed 3.2 MW of solar projects (Solar Projects) at fifteen (15) sites for seven (7) local government units (Local Units) in and including the County. The Program uses a Power Purchase Agreement (PPA) structure, where the Solar Developer constructs, operates and maintains all of the Solar Projects, for the benefit of the Local Units and the Authority, for the maximum State law allowable PPA period of fifteen (15) years. Although all fifteen (15) sites were funded by the Authority, only the Mennen Arena site was considered for the purposes of the required local match funding for this grant. Specifically at the Mennen Arena site, the Authority financed 1.6 MW of solar panels. On October 18, 2013, the DOE Grant was drawn down following completion of the necessary application documents and final execution of an agreement memorializing the contemplated transaction by the Local Units, the County, The Authority and the solar developer. The proceeds of the DOE Grant were then applied to reduce the PPA price to all Local Units across the program and increase the savings from approximately 1/3 to almost half off the existing and forecasted utility pricing over the fifteen (15) year term, without adversely affecting all of the other benefits. With the application of the rate buy down, the price of electricity purchased under the PPA dropped from 10.9 to 7.7 cents/kWh. This made acquisition of renewable energy much more affordable for the Local Units, and it enhanced the success of the program, which will encourage other counties and local units to develop similar programs.

14 SOLAR ENERGY↗

TomocuPy – efficient GPU-based tomographic reconstruction with asynchronous data processing

Fast 3D data analysis and steering of a tomographic experiment by changing environmental conditions or acquisition parameters require fast, close to real-time, 3D reconstruction of large data volumes. Here a performance-optimized TomocuPy package is presented as a GPU alternative to the commonly used central processing unit (CPU) based TomoPy package for tomographic reconstruction. TomocuPy utilizes modern hardware capabilities to organize a 3D asynchronous reconstruction involving parallel read/write operations with storage drives, CPU–GPU data transfers, and GPU computations. In the asynchronous reconstruction, all the operations are timely overlapped to almost fully hide all data management time. Since most cameras work with less than 16-bit digital output, the memory usage and processing speed are furthermore optimized by using 16-bit floating-point arithmetic. As a result, 3D reconstruction with TomocuPy became 20–30 times faster than its multi-threaded CPU equivalent. Full reconstruction (including read/write operations and methods initialization) of a 2048 3 tomographic volume takes less than 7 s on a single Nvidia Tesla A100 and PCIe 4.0 NVMe SSD, and scales almost linearly increasing the data size. To simplify operation at synchrotron beamlines, TomocuPy provides an easy-to-use command-line interface. Efficacy of the package was demonstrated during a tomographic experiment on gas-hydrate formation in porous samples, where a steering option was implemented as a lens-changing mechanism for zooming to regions of interest.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Staff Radiation Doses in a Real-Time Display Inside the Angiography Room

MethodsThe evaluation of a new occupational Dose Aware System (DAS) showing staff radiation doses in real time has been carried out in several angiography rooms in our hospital. The system uses electronic solid-state detectors with high-capacity memory storage. Every second, it archives the dose and dose rate measured and is wirelessly linked to a base-station screen mounted close to the diagnostic monitors. An easy transfer of the values to a data sheet permits further analysis of the scatter dose profile measured during the procedure, compares it with patient doses, and seeks to find the most effective actions to reduce operator exposure to radiation.ResultsThe cumulative occupational doses measured per procedure (shoulder-over lead apron) ranged from 0.6 to 350 {mu}Sv when the ceiling-suspended screen was used, and DSA (Digital Subtraction Acquisition) runs were acquired while the personnel left the angiography room. When the suspended screen was not used and radiologists remained inside the angiography room during DSA acquisitions, the dose rates registered at the operator's position reached up to 1-5 mSv/h during fluoroscopy and 12-235 mSv/h during DSA acquisitions. In such case, the cumulative scatter dose could be more than 3 mSv per procedure.ConclusionReal-time display of doses to staff members warns interventionists whenever the scatter dose rates are too high or the radiation protection tools are not being properly used, providing an opportunity to improve personal protection accordingly.

61 RADIATION PROTECTION AND DOSIMETRY↗

Gaia: segmented germanium detector for high-energy X-ray fluorescence and spectroscopic imaging

We present Gaia, a monolithic array of 96 high-purity germanium pixel detectors integrated with a custom low-noise application-specific integrated circuit (ASIC) and a field-programmable gate array (FPGA)-based data acquisition system. The sensor operates at ∼100 K using a commercial closed-cycle cryocooler, with the in-vacuum electronics thermally isolated from the cold finger to ensure thermal stability. The system demonstrates an average energy resolution of 711 eV at 122 keV, measured using a 57 Co source, and 253 eV at 5.89 keV, measured with 55 Fe across all channels. The readout architecture incorporates a high-performance FPGA paired with a dual-core ARM processor, forming a complete embedded Linux-based computing platform. Communication between the processor and FPGA is handled via memory-mapped I/O, and data are streamed over high-speed gigabit Ethernet. A full-scale 384-pixel Gaia detector, based on this 96-element module, is currently under fabrication.

36 MATERIALS SCIENCE↗

High speed, multi-channel, user programmable digital data acquisition system.

As applications for radiation detection become more demanding, and in turn improvements are made in the technology of radiation detection, there is a need for high speed digital detector readout electronics matching these improvements. Specifically, full control over the on-line processing resources of modern digital electronics is desirable so that researchers can develop custom algorithms for special applications.In the proposed effort, the 500 MHz digital readout electronics previously developed by our company will be redesigned to allow user access to the on-line processing resources. In Phase I, the division of online processing into vendor and user firmware sections has been studied on existing hardware. In Phase II, the hardware will be upgraded to better facilitate the division, and the firmware will be restructured into a robust vendor logic block (providing standard functions such as host I/O, on-board memory I/O, energy computation, MCA spectra, timestamps, waveform capture, run statistics, and triggering and timing) and a user logic block for custom algorithms (with templates and examples for frequently used functions). Investigating several options to divide online processing, it was determined that the most promising approach is to “partition” a single FPGA integrated circuit into a vendor and user section, which is supported in newer devices. The analog front end of the existing electronics proved suitable for most applications, in particular high rate measurements with germanium detectors. The design architecture for new electronics was developed, combining one of the new FPGA device with the analog front end.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

WE-AB-303-09: Rapid Projection Computations for On-Board Digital Tomosynthesis in Radiation Therapy

Purpose: To facilitate fast and accurate iterative volumetric image reconstruction from limited-angle on-board projections. Methods: Intrafraction motion hinders the clinical applicability of modern radiotherapy techniques, such as lung stereotactic body radiation therapy (SBRT). The LIVE system may impact clinical practice by recovering volumetric information via Digital Tomosynthesis (DTS), thus entailing low time and radiation dose for image acquisition during treatment. The DTS is estimated as a deformation of prior CT via iterative registration with on-board images; this shifts the challenge to the computational domain, owing largely to repeated projection computations across iterations. We address this issue by composing efficient digital projection operators from their constituent parts. This allows us to separate the static (projection geometry) and dynamic (volume/image data) parts of projection operations by means of pre-computations, enabling fast on-board processing, while also relaxing constraints on underlying numerical models (e.g. regridding interpolation kernels). Further decoupling the projectors into simpler ones ensures the incurred memory overhead remains low, within the capacity of a single GPU. These operators depend only on the treatment plan and may be reused across iterations and patients. The dynamic processing load is kept to a minimum and maps well to the GPU computational model. Results: We have integrated efficient, pre-computable modules for volumetric ray-casting and FDK-based back-projection with the LIVE processing pipeline. Our results show a 60x acceleration of the DTS computations, compared to the previous version, using a single GPU; presently, reconstruction is attained within a couple of minutes. The present implementation allows for significant flexibility in terms of the numerical and operational projection model; we are investigating the benefit of further optimizations and accurate digital projection sub-kernels. Conclusion: Composable projection operators constitute a versatile research tool which can greatly accelerate iterative registration algorithms and may be conducive to the clinical applicability of LIVE. National Institutes of Health Grant No. R01-CA184173; GPU donation by NVIDIA Corporation.

60 APPLIED LIFE SCIENCES↗

Connect the Dots: In Situ 4-D Seismic Monitoring of CO 2 Storage With Spatio-Temporal CNNs

4-D seismic imaging has been widely used in CO 2 sequestration projects to monitor the fluid flow in the volumetric subsurface region that is not sampled by wells. Ideally, real-time monitoring and near-future forecasting would provide site operators with great insights to understand the dynamics of the subsurface reservoir and assess any potential risks. However, due to obstacles such as high deployment cost, availability of acquisition equipment, exclusion zones around surface structures, only very sparse seismic imaging data can be obtained during monitoring. That leads to an unavoidable and growing knowledge gap over time. The operator needs to understand the fluid flow throughout the project lifetime and the seismic data are only available at a limited number of times. This is insufficient for understanding reservoir behavior. To overcome those challenges, we have developed spatio-temporal neural-network-based models that can produce high-fidelity interpolated or extrapolated images effectively and efficiently. Specifically, our models are built on an autoencoder, and incorporate the long short-term memory (LSTM) structure with a new loss function regularized by optical flow. We validate the performance of our models using real 4-D post-stack seismic imaging data acquired at the Sleipner CO 2 sequestration field. We employ two different strategies in evaluating our models. Numerically, we compare our models with different baseline approaches using classic pixel-based metrics. We also conduct a blind survey and collect a total of 20 responses from domain experts to evaluate the quality of data generated by our models. Finally, via both numerical and expert evaluation, we conclude that our models can produce high-quality 2-D/3-D seismic imaging data at a reasonable cost, offering the possibility of real-time monitoring or even near-future forecasting of the CO 2 storage reservoir.

4-D seismic imaging↗

On the Origin of Reverse Transcriptase-Using CRISPR-Cas Systems and Their Hyperdiverse, Enigmatic Spacer Repertoires

Cas1 integrase is the key enzyme of the clustered regularly interspaced short palindromic repeat (CRISPR)-Cas adaptation module that mediates acquisition of spacers derived from foreign DNA by CRISPR arrays. In diverse bacteria, the cas1 gene is fused (or adjacent) to a gene encoding a reverse transcriptase (RT) related to group II intron RTs. An RT-Cas1 fusion protein has been recently shown to enable acquisition of CRISPR spacers from RNA. Phylogenetic analysis of the CRISPR-associated RTs demonstrates monophyly of the RT-Cas1 fusion, and coevolution of the RT and Cas1 domains. Nearly all such RTs are present within type III CRISPR-Cas loci, but their phylogeny does not parallel the CRISPR-Cas type classification, indicating that RT-Cas1 is an autonomous functional module that is disseminated by horizontal gene transfer and can function with diverse type III systems. To compare the sequence pools sampled by RT-Cas1-associated and RT-lacking CRISPR-Cas systems, we obtained samples of a commercially grown cyanobacterium— Arthrospira platensis . Sequencing of the CRISPR arrays uncovered a highly diverse population of spacers. Spacer diversity was particularly striking for the RT-Cas1-containing type III-B system, where no saturation was evident even with millions of sequences analyzed. In contrast, analysis of the RT-lacking type III-D system yielded a highly diverse pool but reached a point where fewer novel spacers were recovered as sequencing depth was increased. Matches could be identified for a small fraction of the non-RT-Cas1-associated spacers, and for only a single RT-Cas1-associated spacer. Thus, the principal source(s) of the spacers, particularly the hypervariable spacer repertoire of the RT-associated arrays, remains unknown. IMPORTANCE While the majority of CRISPR-Cas immune systems adapt to foreign genetic elements by capturing segments of invasive DNA, some systems carry reverse transcriptases (RTs) that enable adaptation to RNA molecules. From analysis of available bacterial sequence data, we find evidence that RT-based RNA adaptation machinery has been able to join with CRISPR-Cas immune systems in many, diverse bacterial species. To investigate whether the abilities to adapt to DNA and RNA molecules are utilized for defense against distinct classes of invaders in nature, we sequenced CRISPR arrays from samples of commercial-scale open-air cultures of Arthrospira platensis , a cyanobacterium that contains both RT-lacking and RT-containing CRISPR-Cas systems. We uncovered a diverse pool of naturally occurring immune memories, with the RT-lacking locus acquiring a number of segments matching known viral or bacterial genes, while the RT-containing locus has acquired spacers from a distinct sequence pool for which the source remains enigmatic. While the majority of CRISPR-Cas immune systems adapt to foreign genetic elements by capturing segments of invasive DNA, some systems carry reverse transcriptases (RTs) that enable adaptation to RNA molecules. From analysis of available bacterial sequence data, we find evidence that RT-based RNA adaptation machinery has been able to join with CRISPR-Cas immune systems in many, diverse bacterial species. To investigate whether the abilities to adapt to DNA and RNA molecules are utilized for defense against distinct classes of invaders in nature, we sequenced CRISPR arrays from samples of commercial-scale open-air cultures of Arthrospira platensis , a cyanobacterium that contains both RT-lacking and RT-containing CRISPR-Cas systems. We uncovered a diverse pool of naturally occurring immune memories, with the RT-lacking locus acquiring a number of segments matching known viral or bacterial genes, while the RT-containing locus has acquired spacers from a distinct sequence pool for which the source remains enigmatic.

horizontal gene transfer↗

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗