Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel search”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Improving ICARUS Track Reconstruction Algorithms

The ICARUS experiment is part of the Short-Baseline Neutrino (SBN) program at Fermilab. The main goal of the experiment is to investigate the possibility of sterile neutrinos in the O(1 eV) mass region and provide clarification of the anomaly detected from the Liquid Scintillator Neutrino Detector (LSND) and MiniBooNE experiments. The ICARUS-T600 detector is a Liquid Argon Time Projection Chamber (LAr-TPC), that can provide excellent 3D imaging and calorimetric reconstruction of any ionizing particles. This detection technique allows a detailed study of neutrino interactions, spanning a wide energy spectrum (from a few keV to several hundreds of GeV). The detector consists of two identical adjacent modules, filled with a total of 760 tons of ultra-pure liquid argon. Each module houses two LAr-TPCs separated by a common cathode with a maximum drift distance of 1.5 m, equivalent to about 1 ms drift time for the nominal $500$ V/m electric drift field. The anode is made of three parallel wire planes positioned 3 mm apart, where the stainless-steel wires are oriented on each plane at a different angle with respect to the horizontal direction ($+60^\degree$,$-60^\degree$,$0^\degree$). The first two planes (Induction 1 and Induction 2) provide a non-destructive charge measurement, whereas the ionization charge is fully collected by the last collection plane. In total, 53248 wires with a 3 mm pitch and length up to 9 m are installed in the detector. In the first stage of the reconstruction, segments of waveforms corresponding to physical signals (hits) are searched for in the deconvolved wire waveform with a threshold-based hit-finding algorithm. Each hit is then fitted with a Gaussian, whose area is proportional to the number of drift electrons generating the signal. In the second stage of the reconstruction, hits are passed as input to Pandora, a framework software composed of different pattern recognition algorithms, that performs a 3D reconstruction of the full image recorded in the collected event, including the identification of interaction vertices and tracks and showers inside the TPC. These are organized into a hierarchical structure (called slice) of particles generated starting from a primary interaction vertex. In some cases, related to the inefficiencies in the hit detection or excessive deflection of the particle trajectory, Pandora breaks the particle's track into two or more smaller pieces and considers each piece as an independent track. We studied this phenomenon focusing on primary muons from ν_μ CC interactions contained in a single module with a track at least 20 cm long, to exclude delta rays. The study determined that about $7-8\%$ of the muon tracks are broken. Approximately $80\%$ of the times, Pandora assigns all segments of the track to the same slice (intra-slice track split), while in the remaining $20\%$ of the cases, one of the segments is associated with another slice (extra-slice track split). To mitigate this phenomenon, we designed an algorithm that detects and stitches the tracks broken by Pandora for the intra-slice split. In Monte Carlo simulations, the algorithm showed an efficiency exceeding $80\%$ and a purity exceeding $93\%$.

Ricci, Alessandro Maria [Pisa U.; INFN, Pisa] (ORC↗
DOE OSTI2020

STAR Highlights

In this overview talk, I have represented STAR to highlight the results of 19 parallels talks and 38 posters at this Quark Matter Conference. Overall, the contents have 8 major categories: 1. experimental measurements related to the BreitWheeler process and vacuum birefringence; 2. probes of initial states of heavy-ion collisions (parton distribution function and azimuthal correlations in small systems); 3. constraints on temperature-dependent viscosity (η/s(T)) with multiple flow harmonics and rapidity correlations; 4. hard probes using jet structures and heavy flavors; 5. exploration of the origin of global polarization and vorticity; 6. chiral and thermal properties of the QGP; 7. Beam Energy Scan (BES) and the critical point search; 8. upgrades and summary.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Status of DUNE Offline Computing

We summarize the status of Deep Underground Neutrino Experiment (DUNE) Offline Software and Computing program. We describe plans for the computing infrastructure needed to acquire, catalog, reconstruct, simulate and analyze the data from the DUNE experiment and its prototypes in pursuit of the experiment's physics goals of precision measurements of neutrino oscillation parameters, detection of astrophysical neutrinos, measurement of neutrino interaction properties and searches for physics beyond the Standard Model. In contrast to traditional HEP computational problems, DUNE's Liquid Argon Time Projection Chamber data consist of simple but very large (many GB) data objects which share many characteristics with astrophysical images. We have successfully reconstructed and simulated data from 4% prototype detector runs at CERN. The data volume from the full DUNE detector, when it starts commissioning late in this decade will present memory management challenges in conventional processing but significant opportunities to use advances in machine learning and pattern recognition as a frontier user of High Performance Computing facilities capable of massively parallel processing. Our goal is to develop infrastructure resources that are flexible and accessible enough to support creative software solutions as HEP computing evolves.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Enabling rapid COVID-19 small molecule drug design through scalable deep learning of generative models

We improved the quality and reduced the time to produce machine learned models for use in small molecule antiviral design. Our globally asynchronous multi-level parallel training approach strong scales to all of Sierra with up to 97.7% efficiency. We trained a novel, character-based Wasserstein autoencoder that produces a higher quality model trained on 1.613 billion compounds in 23 minutes while the previous state of the art takes a day on 1 million compounds. Reducing training time from a day to minutes shifts the model creation bottleneck from computer job turnaround time to human innovation time. Our implementation achieves 318 PFLOPs for 17.1% of half-precision peak. We will incorporate this model into our molecular design loop enabling the generation of more diverse compounds; searching for novel, candidate antiviral drugs improves and reduces the time to synthesize compounds to be tested in the lab.

Jacobs, Sam Ade↗
DOE OSTI2025

DeepHyper: A Python Package for Massively Parallel Hyperparameter Optimization in Machine Learning

Machine learning models are increasingly applied across scientific disciplines, yet their effectiveness often hinges on heuristic decisions—such as data transformations, training strategies, and model architectures—that are not learned by the models themselves. Automating the selection of these heuristics and analyzing their sensitivity is crucial for building robust and efficient learning workflows. DeepHyper addresses this challenge by democratizing hyperparameter optimization, providing accessible tools to streamline and enhance machine learning workflows from a laptop to the largest supercomputer in the world. Building on top of hyperparameter optimization, it unlocks new capabilities around ensembles of models for improved accuracy and uncertainty quantification. All of these organized around efficient parallel computing.

ensemble↗
DOE OSTI2020

A non-cooperative meta-modeling game for automated third-party calibrating, validating and falsifying constitutive laws with parallelized adversarial attacks

The evaluation of constitutive models, especially for high-risk and high-regret engineering applications, requires efficient and rigorous third-party calibration, validation and falsification. While there are numerous efforts to develop paradigms and standard procedures to validate models, difficulties may arise due to the sequential, manual, and often biased nature of the commonly adopted calibration and validation processes, thus slowing down data collections, hampering the progress towards discovering new physics, increasing expenses and possibly leading to misinterpretations of the credibility and application ranges of proposed models. This work attempts to introduce concepts from game theory and machine learning techniques to overcome many of these existing difficulties. Here, we introduce an automated meta-modeling game where two competing AI agents systematically generate experimental data to calibrate a given constitutive model and to explore its weakness such that the experiment design and model robustness can be improved through competitions. The two agents automatically search for the Nash equilibrium of the meta-modeling game in an adversarial reinforcement learning framework without human intervention. In particular, a protagonist agent seeks to find the more effective ways to generate data for model calibrations, while an adversary agent tries to find the most devastating test scenarios that expose the weaknesses of the constitutive model calibrated by the protagonist. By capturing all possible design options of the laboratory experiments into a single decision tree, we recast the design of experiments as a game of combinatorial moves that can be resolved through deep reinforcement learning by the two competing players. Our adversarial framework emulates idealized scientific collaborations and competitions among researchers to achieve a better understanding of the application range of the learned material laws and prevent misinterpretations caused by conventional AI-based third-party validation. Numerical examples are given to demonstrate the wide applicability of the proposed meta-modeling game with adversarial attacks on both human-crafted constitutive models and machine learning models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

On the Operando Structure of Ruthenium Oxides during the Oxygen Evolution Reaction in Acidic Media

In the search for rational design strategies for oxygen evolution reaction (OER) catalysts, linking the catalyst structure to activity and stability is key. However, highly active catalysts such as IrO x and RuO x undergo structural changes under OER conditions, and hence, structure–activity–stability relationships need to take into account the operando structure of the catalyst. Under the highly anodic conditions of the oxygen evolution reaction (OER), electrocatalysts are often converted into an active form. Here, we studied this activation for amorphous and crystalline ruthenium oxide using X-ray absorption spectroscopy (XAS) and electrochemical scanning electron microscopy (EC-SEM). We tracked the evolution of surface oxygen species in ruthenium oxides while in parallel mapping the oxidation state of the Ru atoms to draw a complete picture of the oxidation events that lead to the OER active structure. Our data show that a large fraction of the OH groups in the oxide are deprotonated under OER conditions, leading to a highly oxidized active material. The oxidation is centered not only on the Ru atoms but also on the oxygen lattice. This oxygen lattice activation is particularly strong for amorphous RuO x . We propose that this property is key for the high activity and low stability observed for amorphous ruthenium oxide.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Grid-based minimization at scale: Feldman-Cousins corrections for light sterile neutrino search

High Energy Physics (HEP) experiments generally employ sophisticated statistical methods to present results in searches of new physics. In the problem of searching for sterile neutrinos, likelihood ratio tests are applied to short-baseline neutrino oscillation experiments to construct confidence intervals for the parameters of interest. The test statistics of the form Δχ2 is often used to form the confidence intervals, however, this approach can lead to statistical inaccuracies due to the small signal rate in the region-of-interest. In this paper, we present a computational model for the computationally expensive Feldman-Cousins corrections to construct a statistically accurate confidence interval for neutrino oscillation analysis. The program performs a grid-based minimization over oscillation parameters and is written in C++. Our algorithms make use of vectorization through Eigen3, yielding a single-core speed-up of 350 compared to the original implementation, and achieve MPI data parallelism by employing DIY. We demonstrate the strong scaling of the application at High-Performance Computing (HPC) sites. We utilize HDF5 along with HighFive to write the results of the calculation to file.

Wospakrik, Marianette↗
DOE OSTI2021

Magnetic structure and multiferroicity of Sc-substituted hexagonal YbFeO 3

The hexagonal rare-earth ferrite RFeO 3 family represents a unique class of multiferroics exhibiting weak ferromagnetism, and a strong coupling between magnetism and structural trimerization is predicted. However, the hexagonal structure for RFeO 3 remains metastable in conventional conditions. We have succeeded in stabilizing the hexagonal structure of polycrystalline YbFeO 3 by partial Sc substitution of Yb. Using bulk magnetometry and neutron diffraction, we find that Yb 0.42 Sc 0.58 FeO 3 orders into a canted antiferromagnetic state with the Néel temperature T N ~ 165K, below which the Fe 3+ moments form the triangular configuration in the ab plane and their in-plane projections are parallel to the [100] axis, consistent with magnetic space group P6 3 c'm'. It is determined that the spin canting is aligned along the c axis, giving rise to the weak ferromagnetism. Furthermore, the Fe 3+ moments reorient toward a new direction below reorientation temperature T R ~ 40K, satisfying magnetic subgroup P6 3 , while the Yb 3+ moments order independently and ferrimagnetically along the c axis at the characteristic temperature T Yb ~ 15K. Interestingly, reproducible modulation of electric polarization induced by magnetic field at low temperature is achieved, suggesting that the delicate structural distortion associated with two-up/one-down buckling of the Yb/Sc planes and tilting of the FeO 5 bipyramids may mediate the coupling between ferroelectric and magnetic orders under magnetic field. Furthermore, the present work represents substantial progress to search for high-temperature multiferroics in hexagonal ferrites and related materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Development of ultra-pure gadolinium sulfate for the Super-Kamiokande gadolinium project

This paper reports the development and detailed properties of about 13 metric tons of gadolinium sulfate octahydrate, Gd 2 (SO 4 ) 3 • 8H 2 O⁠, which has been dissolved into Super-Kamiokande (SK) in the summer of 2020. We evaluate the impact of radioactive impurities in Gd 2 (SO 4 ) 3 • 8H 2 O on diffuse supernova neutrino background searches and solar neutrino observation and confirm the need to reduce radioactive and fluorescent impurities by about three orders of magnitude from commercially available high-purity Gd 2 (SO 4 ) 3 • 8H 2 O⁠. In order to produce ultra-high-purity Gd 2 (SO 4 ) 3 • 8H 2 O, we have developed a method to remove impurities from gadolinium oxide, Gd 2 O 3 , consisting of acid dissolution, solvent extraction, and pH control processes, followed by a high-purity sulfation process. All of the produced ultra-high-purity Gd 2 (SO 4 ) 3 • 8H 2 O is assayed by inductively coupled plasma mass spectrometry and high-purity germanium detectors to evaluate its quality. Because of the long measurement time of high-purity germanium detectors, we have employed several underground laboratories for making parallel measurements including the Laboratorio Subterráneo de Canfranc in Spain, Boulby in the UK, and Kamioka in Japan. In the first half of production, the measured batch purities were found to be consistent with the specifications. However, in the latter half, the Gd 2 (SO 4 ) 3 • 8H 2 O contained one order of magnitude more 228Ra than the budgeted mean contamination. This was correlated with the corresponding characteristics of the raw material Gd 2 O 3 , in which an intrinsically large contamination was present. Based on their modest impact on SK physics, they were nevertheless introduced into the detector. To reduce 228 Ra for the next stage of gadolinium loading to SK, a new process has been successfully established.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Cryogenic light detectors with thermal signal amplification for 0 νββ search experiments

As a step towards the realization of cryogenic-detector experiments to search for neutrinoless double-beta decay (such as CROSS, BINGO, and CUPID), we investigated a batch of 10 Ge light detectors (LDs) assisted by Neganov-Trofimov-Luke (NTL) signal amplification. Each LD was assembled with a large cubic light-emitting crystal (45 mm side) using the recently developed CROSS mechanical structure. The detector array was operated at milli-Kelvin temperatures in a pulse-tube cryostat at the Canfranc underground laboratory in Spain. We achieved good performance with scintillating bolometers from CROSS, made of Li 2 100 MoO 4 crystals and used as reference detectors of the setup, and with all LDs tested (except for a single device that encountered an electronics issue). No leakage current was observed for 8 LDs with an electrode bias up to 100 V. Operating the LDs at an 80 V electrode bias applied in parallel, we obtained a gain of around 9 in the signal-to-noise ratio of these devices, allowing us to achieve a baseline noise RMS of O(10 eV). Thanks to the strong current polarization of the temperature sensors, the time response of the devices was reduced to around half a millisecond in rise time. The achieved performance of the LDs was extrapolated via simulations of pile-up rejection capability for several configurations of the CUPID detector structure. Despite the sub-optimal noise conditions of the LDs (particularly at high frequencies), we demonstrated that the NTL technology provides a viable solution for background reduction in CUPID.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Semantic embedding for quantum algorithms

The study of classical algorithms is supported by an immense understructure, founded in logic, type, and category theory, that allows an algorithmist to reason about the sequential manipulation of data irrespective of a computation’s realizing dynamics. As quantum computing matures, a similar need has developed for an assurance of the correctness of high-level quantum algorithmic reasoning. Parallel to this need, many quantum algorithms have been unified and improved using quantum signal processing (QSP) and quantum singular value transformation (QSVT), which characterize the ability, by alternating circuit ansätze, to transform the singular values of sub-blocks of unitary matrices by polynomial functions. However, while the algebraic manipulation of polynomials is simple (e.g., compositions and products), the QSP/QSVT circuits realizing analogous manipulations of their embedded polynomials are non-obvious. This work constructs and characterizes the runtime and expressivity of QSP/QSVT protocols where circuit manipulation maps naturally to the algebraic manipulation of functional transforms (termed semantic embedding). In this way, QSP/QSVT can be treated and combined modularly, purely in terms of the functional transforms they embed, with key guarantees on the computability and modularity of the realizing circuits. We also identify existing quantum algorithms whose use of semantic embedding is implicit, spanning from distributed search to proofs of soundness in quantum cryptography. The methods used, based in category theory, establish a theory of semantically embeddable quantum algorithms, and provide a new role for QSP/QSVT in reducing sophisticated algorithmic problems to simpler algebraic ones.

Physics↗
DOE OSTI2023

Black-box statistical prediction of lossy compression ratios for scientific data

Lossy compressors are increasingly adopted in scientific research, tackling volumes of data from experiments or parallel numerical simulations and facilitating data storage and movement. In contrast with the notion of entropy in lossless compression, no theoretical or data-based quantification of lossy compressibility exists for scientific data. Users rely on trial and error to assess lossy compression performance. As a strong data-driven effort toward quantifying lossy compressibility of scientific datasets, we provide a statistical framework to predict compression ratios of lossy compressors. Our method is a two-step framework where (i) compressor-agnostic predictors are computed and (ii) statistical prediction models relying on these predictors are trained on observed compression ratios. Proposed predictors exploit spatial correlations and notions of entropy and lossyness via the quantized entropy. We study 8+ compressors on 6 scientific datasets and achieve a median percentage prediction error less than 12%, which is substantially smaller than that of other methods while achieving at least a 8.8× speedup for searching for a specific compression ratio and 7.8× speedup for determining the best compressor out of a collection.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Exploring the Use of Novel Spatial Accelerators in Scientific Applications

Driven by the need to find alternative accelerators which can viably replace GPUs in next-generation Supercomputing systems, this paper proposes a methodology to enable agile application/hardware co-design. The application-first methodology provides the ability to come up with design of accelerators while working with real-world workloads, available accelerators, and system software. The iterative design process targets a set of kernels in a workload for performance estimates that can prune the design space for later phases of detailed architectural evaluations. To this effect, in this paper, a novel data-parallel device model is introduced that simulates the latency of performance-sensitive operations in an accelerator including data transfers and kernel computation using multi-core CPUs. The use of off-the-shelf simulators, such as pre-RTL simulator Aladdin or multiple tools available for exploring the design of deep neural network accelerators (e.g., Timeloop) is demonstrated for evaluation of various accelerator designs using applications with realistic inputs. Examples of multiple device configurations that are instantiable in a system are explored to evaluate the performance benefit of deploying novel accelerators. The proposed device is integrated with a programming model and system software to potentially explore the impacts of high-level programming languages/compilers and low-level effects such as task scheduling on multiple accelerators. We analyze our methodology for a set of applications that represent high-performance computing (HPC) and graph analytics. The applications include a computational chemistry kernel realized using tensor contractions, triangle counting, GraphSAGE and Breadth-first Search. These applications include kernels such as dense matrix-dense matrix multiplication, sparse matrix-spare matrix multiplication, and sparse matrix-dense vector multiplication. Our results indicate potential performance benefits and insights for system design by including accelerators that realize these kernels along-side general purpose accelerators.

AI, codesign, Accelerated Computing, Modeling and ↗
DOE OSTI2023

Comparing Top-Down Proteoform Identification: Deconvolution, PrSM Overlap, and PTM Detection

Generating top-down tandem mass spectra (MS/MS) for complex mixtures of proteoforms has become possible through improvements in fractionation, on-line separation, dissociation, and mass analysis. The algorithms to match tandem mass spectra to sequences have undergone a parallel evolution, with both spectral alignment and peak matching being paired with diverse methods for scoring proteoform-spectral matches (PrSMs). This study assesses state-of-the-art algorithms for top-down identification through three distinct challenges. The first is identifying a large yield of PrSMs while controlling false discovery rate (FDR) in identifying thousands of proteoforms from complex cell lysates via four software workflows: ProSight Proteome Discoverer, TopPIC, Informed Proteomics, and pTop. The second is the deconvolution of data from both Thermo Orbitrap-class and Bruker maXis Q-TOF instruments to produce consistent precursor charge and mass determinations while generating fragment mass lists to optimize identification. The third attempts to detect diverse post-translational modifications (PTMs) in proteoforms from cow milk and human ovarian tissue. The data demonstrate that existing software suites produce admirable sensitivity, in some cases identifying a third of collected tandem mass spectra with FDR controlled below 2%; the overlap in these PrSMs, however, illustrates real value in searching data with multiple search engines. Differences among identification workflows seem to result from each search algorithm incorporating its own deconvolution algorithm. By transmitting deconvolution data from multiple deconvolution routes (Thermo Xtract, Bruker Auto MSn, Mascot Distiller, TopFD, and FLASHDeconv) to the downstream TopPIC search algorithm, we were able to detect common causes of deconvolution disagreement. The detection of PTMs was very inconsistent among search algorithms, with some workflows suggesting as little as 1% of PrSMs from cow’s milk were singly-phosphorylated while other workflows found that 18% of PrSMs were singly-phosphorylated. Taken together, these results make a strong argument for top-down researchers to adopt a standard practice of analyzing each MS/MS experiment with at least two different search engines.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Fast Muon Capture Monitoring in Mu2e with the CAPHRI Detector

The Mu2e experiment at Fermilab will search for the charged lepton flavor violating (CLFV) process of a neutrinoless muon-to-electron conversion in the field of an aluminum nucleus. Reaching the experiment’s target sensitivity requires precise normalization of the physics signal through accurate monitoring of the muon capture rate on the stopping target. For this purpose, the Calorimeter Precise High-Resolution Intensity detector (CAPHRI) has been developed. The detector is composed of four LYSO crystals installed in the upstream disk of the Mu2e calorimeter and read out with the standard calorimeter readout. CAPHRI measures the muon capture rate by detecting the characteristic 1.8~MeV gamma emission line of the $^{27}Al(\mu^−, \nu n \gamma) ^{26}Mg$ nuclear reaction. The fast, precise response enables injection-by-injection monitoring of proton beam intensity fluctuations. We report on the commissioning and performance characterization of CAPHRI. The response of each channel is calibrated at two SiPM overvoltages using both the intrinsic self-emission of the LYSO crystals and cosmic ray signals. In parallel, Monte Carlo simulations are used to evaluate the detector acceptance and the expected signal-to-background ratio under realistic running conditions. Preliminary results show a crystal light yield consistent with expectations and a channel inter-calibration at the 2--4% level. Simulation studies indicate that the detector acceptance and background rejection satisfy the requirements for physics operations, with about 1000 detected events per beam injection at a beam power of 1.5~kW. These results demonstrate that CAPHRI is an effective tool for beam monitoring and signal normalization in Mu2e.

Ciccarella, V. [Frascati; U. Rome La Sapienza (mai↗
DOE OSTI2025

Low-Energy Backgrounds in Solid-State Phonon and Charge Detectors

Solid-state phonon and charge detectors probe the scattering of weakly interacting particles, such as dark matter and neutrinos, through their low recoil thresholds. Recent advancements have pushed sensitivity to eV-scale energy depositions, uncovering previously unseen low-energy excess backgrounds. While some arise from known processes such as thermal radiation, luminescence, and stress, others remain unexplained. This review examines these backgrounds, their possible origins, and parallels to low-energy effects in solids, an understanding of which is essential for interpreting particle interactions at and below the eV scale.

coherent neutrino–nucleus scattering↗
DOE OSTI2026

HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models

As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.

Herron, Emily [ORNL] (ORCID:0000000273008172)↗