Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Large Language Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Transforming the bootstrap: using Transformers to compute scattering amplitudes in planar N = 4 Super Yang-Mills theory

Abstract We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.

Cai, Tianji (ORCID:0000000232359486)↗

Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theory

We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.

Lance, Dixon↗

MOSAIC-CONUS: A Multimodal, Multi-Temporally Paired Dataset for Earth Sciences

Earth embeddings—vector representations of geographic locations indexed in space and time—are emerging as a unifying interface for geospatial AI. However, their quality depends not only on model design, but on how multimodal Earth observation (EO) data are spatially indexed, temporally aligned, and cross-modally associated during pretraining. We introduce MOSAIC-CONUS (Multimodal Observations with Spatially Aligned Imagery, Urban Points of Interest, In-Situ Measurements and Text Captions), a large-scale EO dataset over the contiguous United States, organized around 250,000 stratified point indices that serve as stable spatial keys across seven modalities: active radar, passive optical imagery, lidar-derived elevation, land cover, functional context, hydrometeorological measurements, and textual summaries. Unlike existing EO datasets, MOSAIC-CONUS introduces four contributions not jointly addressed in prior work: 1. an open-source, large-scale multimodal EO corpus structured around point-indexed data designed to support Earth embedding learning; 2. explicit radar-optical pairing tables spanning twelve temporal alignment regimes, formalizing cross-sensor alignment as a controllable variable for analyzing how temporal mismatch across modalities influences learned embeddings quality; 3. a benchmark suite spanning cross-modal retrieval, annual nightlights regression, and basin-held-out streamflow prediction, positioning MOSAIC-CONUS as a benchmark-ready resource for multimodal AI systems; and 4. a language-based embedding layer through co-registered textual summaries, enabling Earth embeddings to function as a queryable interface for agentic AI systems. The dataset and pairing protocols are publicly released.

54 ENVIRONMENTAL SCIENCES↗

Topics in Theoretical Physics

This award supported a broadly based research effort in theoretical particle physics, including research aimed at uncovering the laws of nature at short (subatomic) and long (cosmological) distances. These theoretical developments apply to experiments in laboratories such as CERN, the facility that operates the Large Hadron Collider outside Geneva, as well as to cosmological investigations done using telescopes and satellites. The results reported here apply to physics beyond the so-called Standard Model of particle physics; physics of high energy collisions such as those observed at the Large Hadron Collider; theoretical and mathematical tools and frameworks for describing the laws of nature at short distances; cosmology and astrophysics; and analytic and computational methods to solve theories of short distance physics. Some specific research accomplishments include + Theories of the electroweak interactions, the forces that give rise to many forms of radioactive decay; + Physics of the recently discovered Higgs boson. + Models and phenomenology of dark matter, the mysterious component of the universe, that has so far been detected only by its gravitational effects. + High energy particles in astrophysics and cosmology. + Algorithmic research and Computational methods for physics of and beyond the Standard Model. + Theory and applications of relativity and its possible limitations. + Topological effects in field theory and cosmology. + Conformally invariant systems and AdS/CFT. This award also supported significant training of students and postdoctoral fellows to lead the research effort in particle theory for the coming decades. These students and fellows worked closely with other members of the group as well as theoretical and experimental colleagues throughout the physics community. Many of the research projects funded by this grant arose in response to recently obtained experimental results in the areas of particle physics and cosmology. We describe a few of these below. Relativity is founded on a symmetry property of nature called "Lorentz Invariance". Like all symmetry properties, it is essential to determine precisely how symmetric nature actually is; that is, do the laws of nature fully respect the symmetry or is there room for tiny symmetry violating effects? An important consequence of Lorentz invariance is the existence of a universal limiting velocity for all physical particles. Light travels at this limiting velocity so it is frequently referred to as simply "the speed of light", but relativity requires that ALL particles travel more slowly than this speed. Once the Higgs particle was discovered in 2012 a natural question was whether or not this particle's speed was consistent with relativity. Although the speed of the Higgs particle is not measurable directly, Cohen has shown that, if the maximal speed of the Higgs particle was not precisely the same as the speed of light, then the Higgs would have some unusual properties. In some cases the Higgs would be unstable to some unusual decay modes; in other cases the interactions of the Higgs with other particles would change the properties of these other particles in ways that could be observed in so-called cosmic rays, very energetic particles (such as photons, protons and other atomic nuclei) coming from space. Once these particles hit the upper atmosphere they produce a "shower" of particles that can be seen by ground-based instruments. If the Higgs has a maximal speed that differs even a tiny bit from the speed of light these showers would look quite different from what is observed. In this way Cohen was able to establish that the Higgs travels with a maximal speed that cannot differ from the speed of light by more than one part in a thousand-trillion. This is by far the most precisely determined property of the Higgs particle. Cohen and Schmaltz reviewed evidence from the Large Hadron Collider (LHC), a particle physics experiment operating at the CERN laboratory near Geneva, for a new particle sometimes called a W'. This evidence included certain unexpected by-products in collisions of protons at very high energy. While the evidence was not significant enough to claim a discovery, it was sufficiently intriguing that many particle theorists worked to construct explanations for this signal. Cohen and Schmaltz were able to determine that such explanations are highly constrained by previous experiments involving collisions of very energetic particles. Nevertheless they were able to construct a theory that adequately explains the LHC data and remain consistent with prior experiments. Their explanation predicts the existence of yet another new particle, called a Z', with a mass slightly greater than that of the W'. This additional particle, if it exists, should be seen as more data is collected from the LHC. Amusingly, there is one collision by-product that has already been seen by the CMS experiment at the LHC that supports the existence of this new particle; however, it is not unlikely that this single event is a so-called "background" event, that is a somewhat atypical by-product of a conventional Standard Model process. This theory for the anomalous LHC data will either be confirmed or excluded with further data-taking at the LHC. The ratio of the number of electrons produced in bottom quark decays over the number of muons produced has been measured at the LHC. This ratio is interesting because it can be predicted very precisely from a basic property of the Standard Model: lepton universality. If lepton universality is correct, the ratio of electrons to muons is predicted to be equal to 1. The first measurements of this ratio find a value different from 1 with a statistical significance of about 3 standard deviations. Schmaltz and collaborator proposed a new extension of the Standard Model which can explain the new data. In addition, Schmaltz and collaborators proposed several new measurements of ratios of decay rates which can confirm or rule out the surprising results from the earlier LHC data. The most recent and precise measurements of the cosmic microwave background from the Planck satellite, from a combination of measurements of the dark matter distribution in the universe, and from a measurement of the expansion rate of the universe today show some disagreement when interpreted in terms of the so-called LambdaCDM model. Schmaltz and collaborators proposed an alternative model to LambdaCDM in which the usual cold dark matter is replaced by a new ``dark sector". This sector consists of a cold dark matter particle which interacts with a newly postulated dark radiation component of the universe. The dark radiation can help explain the discrepancy in measurements of the expansion rate, and the dark matter interactions subtly modify the clumping of dark matter at large scales, thus potentially explaining both kinds of tensions in the data. In two publications Schmaltz described the new model and then performed a precision comparison of the predictions of the model with all currently available cosmological data. The results favor the new model at the level of three standard deviations with current data. Quantum Field Theory (QFT) is the language we use to describe quantum systems which are consistent with Einstein’s theory of Special Relativity. In particular, the requirement of Einstein’s theory that signals not travel faster than the speed of light constrains the types of interactions which particles can engage in. One consequence of relativity is that these interactions cannot preserve particle number. The stronger the interactions, the more severe the particle number violation in a given Relativistic QFT. When particle number violation is strong, it becomes very difficult to adequately parameterize the quantum wave function (which characterizes the state of a quantum system). For example, though we can formulate the QFT which describes the strong force as a set of interactions between quark and gluon particles, we have no clear idea how to express the proton state in terms of these quarks and gluons. This is because the proton, though a bound state of quarks and gluons, is not a state of a fixed number of particles due to strong interactions. Yet, understanding the proton state is very important in order to theoretically predict the reaction rates observed at the LHC in Geneva, which is a proton-proton collider. Katz has formulated a new approach to QFT, which among other things offers a way to adequately approximate the quantum wave function of a bound state at strong coupling. The approximation scheme is related to the fact that any sensible QFT (including that of the strong interactions) is at short distances approximately self-similar upon rescaling of space and time. It turns out that keeping track of the response upon this rescaling is important in efficiently parameterizing the state. Katz and collaborators have used this observation to approximate the state of the proton in toy versions of the strong force. In the late 60s Sheldon Glashow, Abdus Salam and Steven Weinberg (1979 Nobel Prize awardees) proposed a theory unifying weak and electromagnetic interaction which assumed the existence of new particles, the W and Z bosons. The W and Z bosons were eventually detected in high-energy collision in a particle accelerator at CERN, and the recent discovery of the Higgs meson at the Large Hadron Collider (LHC), always at CERN, completed the picture. However, deep theoretical considerations indicate that the theory by Glashow, Weinberg and Salam, often referred to as "the standard model" cannot be the whole story: the existence of new particles and new interactions at yet higher energies is widely anticipated. The experiments at the LHC are looking for these, while theorists, like Brower, Rebbi and collaborators, are investigating models for these new interactions. Working in a large national collaboration with access to the most powerful DOE computers Brower, Rebbi and colleagues have been using calculational techniques, similar to those successfully employed for many years to investigate the interactions among quarks in nucleons, to study theories that can describe the expected "beyond the standard model" (BSM) interactions. Their results, which include also a model for dark matter, have been published in several refereed papers in prestigious journals. Various ideas in topologically interesting field theories predict hypothetical objects such as fractional charges and Majorana excitations. However, such fascinating objects have not been seen in particle physics. Nevertheless, these objects demonstrate possible phenomena that quantum field theory can support. Pi used condensed matter physics as a laboratory to study possible realizations and observable effects of these objects predicted by quantum field theory. In recent times there has developed considerable interest among condensed matter field theorists in precisely the same geometrical and topological structures, which were first discovered in particle physics field theories. From particle physicists' point of view, this is an interesting development, since condensed matter provides an arena in which one can concretely realize particle physics ideas. Moreover, particle physicists can learn new ideas from condensed matter physics. Higgs phenomenon is precisely an important particle physics realization of condensed matter ideas. In contrast to the small distance characterizing condensed matter systems, field theory also describes large distance physics characterizing cosmology. Pi worked on various geometrical effects in the standard theory of cosmology, viz general relativity.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Deploying synthetic coevolution and machine learning to engineer protein-protein interactions

Fine-tuning of protein-protein interactions occurs naturally through coevolution, but this process is difficult to recapitulate in the laboratory. We describe a platform for synthetic protein-protein coevolution that can isolate matched pairs of interacting muteins from complex libraries. This large dataset of coevolved complexes drove a systems-level analysis of molecular recognition between Z domain–affibody pairs spanning a wide range of structures, affinities, cross-reactivities, and orthogonalities, and captured a broad spectrum of coevolutionary networks. Furthermore, we harnessed pretrained protein language models to expand, in silico, the amino acid diversity of our coevolution screen, predicting remodeled interfaces beyond the reach of the experimental library. Further, the integration of these approaches provides a means of simulating protein coevolution and generating protein complexes with diverse molecular recognition properties for biotechnology and synthetic biology.

59 BASIC BIOLOGICAL SCIENCES↗

EHR-BERT: A BERT-based model for effective anomaly detection in electronic health records

Objective: Physicians and clinicians rely on data contained in electronic health records (EHRs), as recorded by health information technology (HIT), to make informed decisions about their patients. The reliability of HIT systems in this regard is critical to patient safety. Consequently, better tools are needed to monitor the performance of HIT systems for potential hazards that could compromise the collected EHRs, which in turn could affect patient safety. In this paper, we propose a new framework for detecting anomalies in EHRs using sequence of clinical events. This new framework, EHR-Bidirectional Encoder Representations from Transformers (BERT), is motivated by the gaps in the existing deep-learning related methods, including high false negatives, sub-optimal accuracy, higher computational cost, and the risk of information loss. EHR-BERT is an innovative framework rooted in the BERT architecture, meticulously tailored to navigate the hurdles in the contemporary BERT method; thus, enhancing anomaly detection in EHRs for healthcare applications.Methods: The EHR-BERT framework was designed using the Sequential Masked Token Prediction (SMTP) method. This approach treats EHRs as natural language sentences and iteratively masks input tokens during both training and prediction stages. This method facilitates the learning of EHR sequence patterns in both directions for each event and identifies anomalies based on deviations from the normal execution models trained on EHR sequences.Results: Extensive experiments on large EHR datasets across various medical domains demonstrate that EHR-BERT markedly improves upon existing models. It significantly reduces the number of false positives and enhances the detection rate, thus bolstering the reliability of anomaly detection in electronic health records. This improvement is attributed to the model’s ability to minimize information loss and maximize data utilization effectively.Conclusion: EHR-BERT showcases immense potential in decreasing medical errors related to anomalous clinical events, positioning itself as an indispensable asset for enhancing patient safety and the overall standard of healthcare services. The framework effectively overcomes the drawbacks of earlier models, making it a promising solution for healthcare professionals to ensure the reliability and quality of health data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Improving the Capabilities and Computational Efficiency of the RTE+RRTMGP Radiation Code (Final Report)

This report details progress on the RTE+RRTMGP radiation codes made during the period of performance. RTE+RRTMGP is a set of codes for computing radiative fluxes in planetary atmospheres. RRTMGP uses a k-distribution to provide an optical description (absorption and possibly Rayleigh optical depth) of the gaseous atmosphere, along with the relevant source functions, on a pre-determined spectral grid given temperatures, pressures, and gas concentration. RTE computes fluxes given spectrally-resolved optical descriptions and source functions. Spectrally-resolved fluxes are summarized (“reduced”) via a user extensible class. The initial release of the code and the design choices are described in Pincus et al. 2019; the codes are available on Github. Although RRTMGP was based on current (at the time) empirical spectroscopic data, RTE and RRTMGP were developed in large part to modernize software practices. The design focused on flexibility broadly interpreted: by separating code from data and allowing data to drive computation; in coupling to the host model (e.g. the coupling of clouds to radiative fluxes is user-controlled); with respect to programming languages (computational tasks are accessed via widely-compatible C interfaces); and with respect to hardware (the codes run on a range of CPU and GPU architectures). The code also puts an emphasis on modularity and clarity. RTE+RRTMGP v1.0 was released in September 20219. This award supported the evolution of the RTE+RRTMGP code base to support greater flexibility, accuracy, and efficiency.

54 ENVIRONMENTAL SCIENCES↗

HVAC and Control Templates for the Modelica Buildings Library

This article reports on our experience in creating Modelica models for systems with thousands of configurations and closed-loop controls. The development of such templates required exploration of class parameterization techniques and data structures for handling large sets of equipment parameters. By describing these issues and the approach taken, we show how the Modelica language can support advanced templating logic. The main limitation we encountered relates to parameter assignment and propagation. The interpretation of parameter attributes at user interface runtime, or the handling of non-trivial constructs involving record classes at compile time is not consistently supported by Modelica tools. This leads to choices that are difficult to make when looking for a generic implementation.

Gautier, Antoine↗

Quantifying Groundwater Response and Uncertainty in Beaver‐Influenced Mountainous Floodplains Using Machine Learning‐Based Model Calibration

Abstract Beavers ( Castor canadensis ) alter river corridor hydrology by creating ponds and inundating floodplains, and thereby improving surface water storage. However, the impact of inundation on groundwater, particularly in mountainous alluvial floodplains with permeable gravel/cobble layers overlain by a soil layer, remains uncertain. Numerical modeling across various floodplain structures considers topographic and sediment complexity and multidirectional flow, linking inundation to groundwater response. This study develops a model‐data integration workflow to address uncertainty in groundwater response to beaver‐induced inundations in a mountainous alluvial floodplain in the Upper Colorado River Basin. Uncertain factors include seasonal hydrologic dynamics, hydraulic conductivities, floodplain structures, and meteorological forcings. We employed an ensemble of groundwater models, based on geophysical and hydrologic data, with machine learning‐based calibration using a neural density estimator. This allowed us to quantify the vertical flux from the soil layer to the permeable gravel bed, the down‐valley underflow within the gravel bed, and their ratios. Results show a significant increase in the vertical flux relative to down‐valley underflow, from 2 during dry pond periods to 20 during wet periods, serving as an analogy for conditions without and with beaver ponds. The study highlights the influence of floodplain structure on groundwater storage, water balance, and water quality impacted by beaver ponds. A thick gravel bed layer, with a large down‐valley underflow, minimizes the effect of beaver‐induced inundation on water quality. We emphasize the need for field‐scale measurements of floodplain structure and improved characterization of evapotranspiration changes to reduce uncertainty in groundwater response. Plain Language Summary Beavers change the flow of water in river corridors by creating ponds, expanding wetlands, and flooding floodplains. This increases surface water area, promotes plant growth, and enhances biodiversity. However, the impact of this flooding on groundwater flow is not well understood, especially in mountainous areas with gravel layers where water moves easily beneath soil. In this study, we used numerical modeling to investigate how beaver ponds influence groundwater in a mountainous floodplain of the Upper Colorado River Basin. We adapted a machine learning method to validate our numerical models using multiple field data sets. Our findings show that beaver ponds significantly increase vertical water flow from the soil to the gravel during wet periods, compared to when the ponds are fully drained. The study also highlights the importance of floodplain structure in controlling both water flow in gravel layers along the river direction and vertical flow from the soil to the gravel with the presence of beavers. To reduce uncertainty in groundwater response, we emphasize the need for more field‐scale measurements of floodplain structure, hydraulic properties, and evapotranspiration changes. Key Points Floodplain structures and hydraulic conductivities are important for groundwater response with beaver ponds in mountainous floodplains Large down‐valley underflow in permeability‐stratified floodplains reduces beaver‐induced impacts on groundwater storage and water quality Machine learning‐based model calibration methods are effective for estimating posterior distributions of groundwater model parameters

Wang, Lijing↗

Studying language evolution in the age of big data

The increasing availability of large digital corpora of cross-linguistic data is revolutionizing many branches of linguistics. Overall, it has triggered a shift of attention from detailed questions about individual features to more global patterns amenable to rigorous, but statistical, analyses. This engenders an approach based on successive approximations where models with simplified assumptions result in frameworks that can then be systematically refined, always keeping explicit the methodological commitments and the assumed prior knowledge. Therefore, they can resolve disputes between competing frameworks quantitatively by separating the support provided by the data from the underlying assumptions. These methods, though, often appear as a ‘black box’ to traditional practitioners. In fact, the switch to a statistical view complicates comparison of the results from these newer methods with traditional understanding, sometimes leading to misinterpretation and overly broad claims. We describe here this evolving methodological shift, attributed to the advent of big, but often incomplete and poorly curated, data, emphasizing the underlying similarity of the newer quantitative to the traditional comparative methods and discussing when and to what extent the former have advantages over the latter. In this review, we cover briefly both randomization tests for detecting patterns in a largely model-independent fashion and phylolinguistic methods for a more model-based analysis of these patterns. We foresee a fruitful division of labor between the ability to computationally process large volumes of data and the trained linguistic insight declaring worthy prior commitments and interesting hypotheses in need of comparison.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Scaling Irregular Applications through Data Aggregation and Software Multithreading

Bioinformatics, data analytics, semantic databases, knowledge discovery are emerging high performance application areas that exploit dynamic, linked data structures such as graphs, unbalanced trees or unstructured grids. These data structures usually are very large, requiring significantly more memory than available on single shared memory systems. Additionally, these data structures are difficult to partition on distributed memory systems. They also present poor spatial and temporal locality, thus generating unpredictable memory and network accesses. The Partitioned Global Address Space (PGAS) programming model seems suitable for these applications, because it allows using a shared memory abstraction across distributed-memory clusters. However, current PGAS languages and libraries are built to target regular remote data accesses and block transfers. Furthermore, they usually rely on the Single Program Multiple Data (SPMD) parallel control model, which is not well suited to the fine grained, dynamic and unbalanced parallelism of irregular applications. In this paper we present {\bf GMT} (Global Memory and Threading library), a custom runtime library that enables efficient execution of irregular applications on commodity clusters. GMT integrates a PGAS data substrate with simple fork/join parallelism and provides automatic load balancing on a per node basis. It implements multi-level aggregation and lightweight multithreading to maximize memory and network bandwidth with fine-grained data accesses and tolerate long data access latencies. A key innovation in the GMT runtime is its thread specialization (workers, helpers and communication threads) that realize the overall functionality. We compare our approach with other PGAS models, such as UPC running using GASNet, and hand-optimized MPI code on a set of typical large-scale irregular applications, demonstrating speedups of an order of magnitude.

Aggregation↗

Machine Learning Approaches to Predicting Induced Seismicity and Imaging Geothermal Reservoir Properties

This project developed machine learning (ML) methods, lab data sets, and field data to advance geothermal exploration and geothermal energy production. The work had three focus areas. One involved the development of ML methods to use microearthquakes (MEQs) for imaging geothermal reservoir properties and improving subsurface characterization – most importantly the evolution of permeability within the evolving reservoir. This part of the work included development of ML approaches for automated MEQ location, focal mechanism determination and identification of earthquake precursors. The second area focused on using MEQ signals generated by geothermal exploration and production to predict the relationship between fluid injection and seismicity. Here, we extended to reservoir scale our success in using ML to predict laboratory earthquakes and fault zone stress state. The third focus area was on lab experiments. Here, we developed new ML models for lab earthquake prediction and identification of precursors to failure to improve earthquake forecasting and early warning in geothermal settings. Major outcomes of our work include ML models that learn from MEQ signals during geothermal exploration and production to predict induced seismicity. MEQs occur naturally in connection with drilling and energy production. We developed ML methods to use the seismic waves from these events to characterize the elastic, hydraulic and poromechanical properties of reservoirs. Our work illuminated fracture geometry and the evolution of fracture permeability by incorporating seismic coda wave analysis and ML methods to relate fluid injection and seismicity. We significantly expanded laboratory earthquake prediction to include methods that use both passive measurements of microearthquakes within the lab fault zones and also active source acoustic measurements of fault zone elastic properties. These methods can now predict fault zone stress state, time to failure and the magnitude of lab earthquakes. Our work showed that repetitive stick- slip failure events during frictional sliding (the lab equivalent of earthquakes) are preceded by a cascade of micro-failure events that radiate energy in a manner that foretells unstable failure – manifest as laboratory MEQs. We documented a mapping between fracture properties and statistical attributes of elastic radiation. We extended existing works to geothermal reservoir scale and developed ML methods to determine reservoir permeability, fracture properties, and their evolution during geothermal energy production. An attractive feature of ML algorithms is their ability to handle big datasets and reveal patterns and correlations that may remain invisible to conventional analyses. Our work connected data from field, laboratory and intermediate scales to study permeability, stress, strength, fracture stiffness and geometry. At the field scale we used data from the Newberry Volcano field site, UtahFORGE, EGS Collab, and also the Bedretto underground research lab in Switzerland. These data sets are bridging the gap between the lab scale, theory, and reservoir scale. Our work produced plain language summaries to improve public understanding of DOE research. We also developed openly distributed ML and seismicity datasets for use by all researchers and we published connections between induced seismicity in geothermal areas and reservoir properties including permeability, fracture properties, and stress state. Our models are designed for the large data sets of induced seismicity typically associated with geothermal sites. We produced labeled event catalogs and used them on geothermal data to assess how ML can facilitate geothermal production and exploration. All datasets are available on the GDR Productivity: The project produced 32 publications in peer reviewed journals (two are in review). It supported the work of 6 PhD students, 40 conference presentations, 6 keynote talks at national meetings, and mentoring and professional development for 4 postdoctoral fellows.

15 GEOTHERMAL ENERGY↗

Robotic Planning under Uncertainty in Spatiotemporal Environments in Expeditionary Science

In the expeditionary sciences, spatiotemporally varying environments -- hydrothermal plumes, algal blooms, lava flows, or animal migrations -- are ubiquitous. Mobile robots are uniquely well-suited to study these dynamic, mesoscale natural environments. We formalize expeditionary science as a sequential decision-making problem, modeled using the language of partially-observable Markov decision processes (POMDPs). Solving the expeditionary science POMDP under real-world constraints requires efficient probabilistic modeling and decision-making in problems with complex dynamics and observational models. Previous work in informative path planning, adaptive sampling, and experimental design have shown compelling results, largely in static environments, using data-driven models and information-based rewards. However, these methodologies do not trivially extend to expeditionary science in spatiotemporal environments: they generally do not make use of scientific knowledge such as equations of state dynamics, they focus on information gathering as opposed to scientific task execution, and they make use of decision-making approaches that scale poorly to large, continuous problems with long planning horizons and real-time operational constraints. In this work, we discuss these and other challenges related to probabilistic modeling and decision-making in expeditionary science, and present some of our preliminary work that addresses these gaps. We ground our results in a real expeditionary science deployment of an autonomous underwater vehicle (AUV) in the deep ocean for hydrothermal vent discovery and characterization. Our concluding thoughts highlight remaining work to be done, and the challenges that merit consideration by the reinforcement learning and decision-making community.

Preston, Victoria↗

Improving the Performance of NEML2 with Modern Graph Compilation Backends

NEML2 vectorizes constitutive-model evaluation for large-scale multiphysics simulation, using PyTorch as its tensor backend so that a batch of material-point updates runs on CPU or GPU through a single implementation. In the two prior reports in this series it was a C++-native library, deployed through TorchScript tracing and just-in-time (JIT) compilation; it has since been rewritten from the ground up into a Python-native library deployed through Ahead-of-Time Inductor (AOTInductor), a modern PyTorch graph-compilation backend. The rewrite is driven by a persistent tension, not a language preference: NEML2 composes constitutive models at runtime from a registry of small, independently-authored pieces, and that flexibility is difficult to reconcile with the compile-time knowledge an efficient GPU kernel needs. This report documents the rewrite and the investment that accompanied it: the AOTInductor export pipeline that turns a Python-authored model into a portable, Python-free compiled artifact loadable from pure C++; the eager and compiled runtimes and the new implicit solver layer built on them; a head-to-head benchmark of legacy JIT against AOTInductor; the physics-model catalog and its worked examples; the developer tooling; and the corresponding overhaul of MOOSE’s NEML2 integration that lets MOOSE consume it. A central objective is to examine whether modern PyTorch graph-compilation backends are effective for MOOSE GPU integration. The benchmark answers directly: AOTInductor outperforms legacy JIT on every GPU scenario measured, by 1.0–4.5×. Modern graph-compilation backends are effective for MOOSE GPU integration, and AOTInductor specifically – not compilation in the abstract – is why.

Hu, Gary (Tianchen) [Argonne National Laboratory (↗

Compiler Optimization for Extreme-Scale Scripting

The data-driven task parallelism execution model can support parallel programming models that are well suited for large-scale distributed-memory parallel computing, for example, simulations and analysis pipelines running on clusters and clouds. We describe a novel compiler intermediate representation and optimizations for this execution model, including adaptions of standard techniques alongside novel techniques. These techniques are applied to Swift/T, a high-level scripting language for flexible data flow composition of functions, which may be serial or use lower-level parallel programming models such as MPI and OpenMP. This paper presents preliminary results, indicating that our compiler optimizations reduce communication overhead by 70% to 93% on distributed-memory systems.

Armstrong, Timothy G.↗

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗

MAUD Interface Tool Kit (MILK)

Materials Analysis Using Diffraction (MAUD) is an open source Rietveld refinement program which fits diffraction models to diffraction spectra allowing quantitative diffraction analysis. The scripting interface developed here using python facilitates the choice of parameters to refine during the Rietveld fitting process and provides a simple MPI framework for running MAUD instances in parallel. The scripting language facilitates batch, custom, and reproducible Rietveld refinements of large datasets.

Savage, Daniel↗

Extending the Remote Control Capabilities in the CMS Detector Control System with Remote Procedure Call Services

The CMS Detector Control System (DCS) is implemented as a large distributed and redundant system, with applications interacting and sharing data in multiple ways. The CMS XML-RPC is a software toolkit implementing the standard Remote Procedure Call (RPC) protocol, using the Extensible Mark-up Language (XML) and a custom lightweight variant using the JavaScript Object Notation (JSON) to model, encode and expose resources through the Hypertext Transfer Protocol (HTTP). The CMS XML-RPC toolkit complies with the standard specification of the XML-RPC protocol that allows system developers to build collaborative software architectures with self-contained and reusable logic, and with encapsulation of well-defined processes. The implementation of this protocol introduces not only a powerful communication method to operate and exchange data with web-based applications, but also a new programming paradigm to design service-oriented software architectures within the CMS DCS domain. This paper presents details of the CMS XML-RPC implementation in WinCC Open Architecture (OA) Control Language using an object-oriented approach.

43 PARTICLE ACCELERATORS↗