Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Model Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Wassersplines for Neural Vector Field-Controlled Animation

Much of computer-generated animation is created by manipulating meshes with rigs. While this approach works well for animating articulated objects like animals, it has limited flexibility for animating less structured free-form objects. Here we introduce Wassersplines, a novel trajectory inference method for animating unstructured densities based on recent advances in continuous normalizing flows and optimal transport. The key idea is to train a neurally-parameterized velocity field that represents the motion between keyframes. Trajectories are then computed by advecting keyframes through the velocity field. We solve an additional Wasserstein barycenter interpolation problem to guarantee strict adherence to keyframes. Our tool can stylize trajectories through a variety of PDE-based regularizers to create different visual effects. We demonstrate our tool on various keyframe interpolation problems to produce temporally-coherent animations without meshing or rigging.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Conditional H I Mass Functions and the H I -to-halo Mass Relation in the Local Universe

We present a new H I mass estimator that relates log 10 (M HI /M * ) to a linear combination of four galaxy properties: stellar surface mass density, color index u - r, stellar mass, and concentration index, with the scatter of individual galaxies around the mean H I mass modeled with a Gaussian distribution function. We calibrate the estimator using the xGASS sample, including both H I detection and nondetection, and constrain the model parameters through Bayesian inferences. Tests with mock catalogs demonstrate that our estimator provides unbiased H I masses for optical samples like SDSS. We apply our estimator to the SDSS spectroscopic sample to estimate the H I mass function (HIMF) of local galaxies, as well as the conditional H I mass function in galaxy groups and the H I –halo mass relation. Our HIMF agrees with the ALFALFA measurements at M HI ≳ 5 × 10 9 M ⊙ , but with higher amplitude and a steeper slope at lower masses. We show that this discrepancy is caused primarily by the cosmic variance, which is corrected for the SDSS sample but not for ALFALFA. The total CHIMFs for all halo masses can be described by a single Schechter function, while those of central galaxies show a double-Gaussian profile. The total H I mass in a group increases monotonically with halo mass, but for central galaxies, the H I mass shows weak dependence on halo mass when M h ≳ 10 12 M ⊙ . The observed H I –halo mass relation is not reproduced by current hydrodynamic simulations and semianalytic models of galaxy formation.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1994

The solar cycle variation of ozone in the stratosphere inferred from Nimbus 7 and NOAA 11 satellites

The combined Nimbus 7 solar backscattered ultraviolet (SBUV) and NOAA 11 SBUV/2 ozone data, covering a period of more than a solar cycle (about 15 years), are used to study the UV response of ozone in the stratosphere. The study shows that about 2% change in total column ozone and about 5-7% change in ozone mixing ratio in the upper stratosphere (0.7 to 2 hPa) may be attributed to the change in the solar UV flux over a solar cycle. In the upper stratosphere, where photochemical processes are expected to play a major role, the measured solar cycle variation of ozone is significantly larger than inferred either from the photochemical models or from the ozone response to the 27-day solar UV modulation. For example, the observed solar cycle related change in ozone mixing ratio at 2 hPa is about 1% for 1% change in the solar UV flux near 200 nm. The inferred change in ozone from either the photochemical models or from the 27-day ozone-UV response is about a factor of 2-3 lower than this value.

Chandra, S.↗
NASA NTRS1994

Constraints on hydrothermal heat flux through the oceanic lithosphere from global heat flow

A significant discrepancy exists between the heat flow measured at the seafloor and the higher values predicted by thermal models of the cooling lithosphere. This discrepancy is generally interpreted as indicating that the upper oceanic crust is cooled significantly by hydrothermal circulation. The magnitude of this heat flow discrepancy is the primary datum used to estimate the volume of hydrothermal flow, and the variation in the discrepancy with lithospheric age is the primary constraint on how the hydrothermal flux is divided between near-ridge and off-ridge environments. The resulting estimates are important for investigation of both the thermal structure of the lithosphere and the chemistry of the oceans. We reevaluate the magnitude and age variation of the discrepancy using a global heat flow data set substantially larger than in earlier studies, and the GDHI (Global Depth and Heat Flow) model that better predicts the heat flow. We estimate that of the predicted global oceanic heat flux of 32 x 10(exp 12) W, 34% (11 x 10(exp 12) W) occurs by hydrothermal flow. Approximately 30% of the hydrothermal heat flux occurs in crust younger than 1 Ma, so the majority of this flux is off-ridge. These hydrothermal heat flux estimates are upper bounds, because heat flow measurements require sediment at the site and so are made preferentially at topographic lows, where heat flow may be depressed. Because the water temperature for the near-ridge flow exceeds that for the off-ridge flow, the near-ridge water flow will be even a smaller fraction of the total water flow. As a result, in estimating fluxes from geochemical data, use of the high water temperatures appropriate for the ridge axis may significantly overestimate the heat flux for an assumed water flux or underestimate the water flux for an assumed heat flux. Our data also permit improved estimates of the 'sealing' age, defined as the age where the observed heat flow approximately equals that predicted, suggesting that hydrothermal heat transfer has largely ceased. Although earlier studies suggested major differences in sealing ages for different ocean basins, we find that the sealing ages for the Atlantic, Pacific, and Indian oceans are similar and consistent with the sealing age for the entire data set, 65 +/- 10 Ma. The previous inference of a young (approximately 20 Ma) sealing age for the Pacific appears to have biased downward several previous estimates of the global hydrothermal flux. The heat flow data also provide indirect evidence for the mechanism by which the hydrothermal heat flux becomes small, which has often been ascribed to isolation of the igneous crust from seawater due to the hydraulic conductivity of the intervening sediment. We find, however, that even the least sedimented sites show the systematic increase of the ratio of observed to predicted heat flow with age, although the more sedimented sites have a younger sealing age. Moreover, the heat flow discrepancy persists at heavily sedimented sites until approximately 50 Ma. It thus appears that approximately 100-200 m of sediment is neither necessary nor sufficient to stop hydrothermal heat transfer. We therefore conclude that the age of the crust is the primary control on the fraction of heat transported by hydrothermal flow and that sediment thickness has a lesser effect. This inference is consistent with models in which hydrothermal flow decreases with age due to reduced crustal porosity and hence permeability.

Stein, Carol A.↗
DOE OSTI2026

AI-Based Analytics and Energy Modeling Framework for Characterizing Urban Energy Systems

Developing location-specific district energy models is essential for understanding energy patterns and supporting efficient management and planning decisions. However, accurately characterizing these models remains challenging due to gaps in building characteristics and labor-intensive traditional modeling workflows. To address these challenges, we develop an AI-based framework that integrates top-down and bottom-up building energy data to automate urban energy model characterization. The framework trains multimodal deep learning models using heterogeneous ResStockTM datasets to infer missing building characteristics from varying levels of known information and generate simulation-ready inputs for district-scale energy modeling. It also employs a conditioning-based injection approach to generate ”what-if” scenarios, enabling users to explore retrofit, efficiency, and technology-upgrade pathways. Integrated within URBANoptTM, a bottom-up district energy modeling platform for simulating co-located buildings, the framework infers detailed building-level inputs required for bottom-up simulations. Both localized and generalized AI models are developed to learn relationships across categorical, numerical, and time-series data, enabling reconstruction of missing attributes and generation of targeted upgrade scenarios. We demonstrate this methodology on a residential neighborhood in Baltimore, MD, assessing internal consistency against ResStock reference data and URBANopt simulation, and comparing selected attributes against real-world building characteristics. Results show strong overall predictive accuracy in data completion and scenario generation, with localized and generalized models offering complementary trade-offs between precision and scalability. Overall, our automated framework streamlines energy modeling and provides a reliable framework for urban building energy characterization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

The mass distribution of quasars in optical time-domain surveys

The determination of supermassive black hole (SMBH) masses is the key to understanding the host galaxy build-up and the SMBH mass assembly histories. The SMBH masses of non-local quasars are frequently estimated via the single-epoch virial black-hole mass estimators, which may suffer from significant biases. Here we demonstrate a new approach to infer the mass distribution of SMBHs in quasars by modelling quasar ultraviolet /optical variability. Here, our inferred black hole masses are systematically smaller than the virial ones by 0.3–0.6 dex; the ∼0.3 dex offsets are roughly consistent with the expected biases of the virial black-hole mass estimators. In the upcoming time-domain astronomy era, our methodology can be used to constrain the cosmic evolution of quasar mass distributions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Instantiation of HCML Demonstrating Bayesian Predictive Modeling for Attentional Control

The research team developed models of Attentional Control (AC) that are unique to existing modeling approaches in the literature. The goal was to enable the research team to (1) make predictions about AC and human performance in real-world scenarios and (2) to make predictions about individual characteristics based on human data. First, the team developed a proof-of-concept approach for representing an experimental design and human subjects data in a Bayesian model, then demonstrated an ability to draw inferences about conditions of interest relevant to real-world scenarios. Ultimately, this effort was successful, and we were able to make reasonable (meaning supported by behavioral data) inferences about conditions of interest to develop a risk model for AC (where risk is defined as a mismatch between AC and attentional demand). The team additionally defined a path forward for a human-constrained machine learning (HCML) approach to make predictions about an individual's state based on performance data. The effort represents a successful first step in both modeling efforts and serves as a basis for future work activities. Numerous opportunities for future work have been defined.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

More rain compensation results

To reduce the impact of rain-induced attenuation in the 20/30 GHz band, the attenuation at a specified signal frequency must be estimated and extrapolated forward in time on the basis of a noisy beacon measurement. Several studies have used model based procedures for solving this problem in statistical inference. Perhaps the most widely used model-based paradigm leads to the Kalman filter and its lineal variants. In this formulation, the dynamic features of the attenuation are represented by a state process (x(sub t)). The observation process (y(sub t)) is derived from beacon measurements. Some ideas relating to the signal processing problems related to uplink power control are presented. It is shown that some easily implemented algorithms hold promise for use in estimating rain induced fades. The algorithms were applied to actual data generated at the Virginia Polytechnic Institute and State University (VPI) test facility. Because only one such event was studied, it is not clear that the algorithms will have the same effectiveness when a wide range of events are studied.

Sworder, D. D.↗
DOE OSTI2018

Advanced Analytics Studies Applied to US Department of Veterans Affairs' Corporate Data Warehouse (Initial Draft)

In this report, we describe our experience of applying several advanced analytics algorithms to the US Department of Veterans Affairs’ (VA’s) Corporate Data Warehouse (CDW) electronic health records datasets during FY2017-18. While various algorithms were applied to the CDW data, the main goal of this effort was to provide useful insights into the operational aspects (as opposed to the purely clinical aspect) of this specific implementation. Since there are not many reports in the public literature on advanced analytics applied to this data, this report provides unique insight in this regard. We focused on two machine learning (ML) applications: (1) clinical pathway inference (patterns dominating clinical pathways applied to a male dataset of Stable Ischemic Heart Disease patients as use case) and (2) medical-concept representation learning’s capability to inform and refine cohort membership based on probable patient outcomes. During our study on clinical pathway inference, we used two main modeling techniques: (a) topic probabilistic modeling (latent Dirichlet allocation [LDA]) and (b) feature imbedding. We observed that the applicability of LDA to the pathway inference remains in question. LDA provides results that are intuitive and easy for humans to comprehend; however, when applied to the pathway inference, it also generates pathway patterns that consist of a few dominant pathway components. In addition, LDA provides no information regarding the relationships among components in the same pathway. The outputs of LDA are also found to categorize patients based on their trace data of clinical procedures. However, the results suggest that LDA is biased toward statistically dominant components. This makes it particularly hard to discriminate pathway subbranches. To address the issue, we designed a new pathway inference methodology that integrates temporal ordering of pathway components into LDA results, as existing feature embedding tools such as word2vec are not readily applicable to our task. In addition, we developed our own feature embedding tool customized for patient traces. We conclude this study by suggesting two approaches to use embedding representation of a component into LDA and briefly list our lessons learned and recommendations for future work. For our study on medical-concept representation learning ability to inform and refine cohort membership based on probable patient outcomes, during our empirical evaluation we compared methods from medical-concept learning to standard one of a kind (one-of-K) encoding to evaluate the change in effectiveness as done in Choi, Schuetz, Stewart, and Sun (2016). We performed two primary empirical evaluations. The first evaluation was on a curated collection of 60,000 patients with no more than 1 year of medical history included. The second was a collection of patients with no restriction to the amount of medical history included. The hypothesis across both evaluations is that representation learning is broadly useful independent of downstream processing models; thus, for each experiment, we trained three models—a logistic regression model, a two-layer neural network model, and a nearest-neighbor model—and then averaged the results. Each model was trained using fivefold cross validation. We provided evidence that medical-representation learning improves predictions of the primary diagnosis category of a short patient history through 12 experiments. In addition, we provided evidence that medical-representation learning fails to improve prediction of the primary diagnosis category of an arbitrarily long patient history through 12 experiments. We conclude this study with a brief list of our lessons learned and recommendations for future work. We conclude the report with a discussion of recent technology trends as they relate to our artificial intelligence and, more specifically, to our ML research and approaches as described previously.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Comparison of ANN (MLP), ANFIS, SVM, and RF models for the online classification of heating value of burning municipal solid waste in circulating fluidized bed incinerators

Highlights: • A rapid, cost-effective, and comparative methodology was proposed to evaluate the HVs of burning MSW online. • Models ranked in descending order of performance are ANFIS, RF, SVM, MLP. • A well trained model is capable of characterizing the variation trend of HVs accurately. - Abstract: The heating values, particularly lower heating values of burning municipal solid waste are critically important parameters in operating circulating fluidized bed incineration systems. However, the heating values change widely and frequently, while there is no reliable real-time instrument to measure heating values in the process of incinerating municipal solid waste. A rapid, cost-effective, and comparative methodology was proposed to evaluate the heating values of burning MSW online based on prior knowledge, expert experience, and data-mining techniques. First, selecting the input variables of the model by analyzing the operational mechanism of circulating fluidized bed incinerators, and the corresponding heating value was classified into one of nine fuzzy expressions according to expert advice. Development of prediction models by employing four different nonlinear models was undertaken, including a multilayer perceptron neural network, a support vector machine, an adaptive neuro-fuzzy inference system, and a random forest; a series of optimization schemes were implemented simultaneously in order to improve the performance of each model. Finally, a comprehensive comparison study was carried out to evaluate the performance of the models. Results indicate that the adaptive neuro-fuzzy inference system model outperforms the other three models, with the random forest model performing second-best, and the multilayer perceptron model performing at the worst level. A model with sufficient accuracy would contribute adequately to the control of circulating fluidized bed incinerator operation and provide reliable heating value signals for an automatic combustion control system.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2022

Radiative Losses in the Chromosphere During A C-Class Flare

Context. Solar flares release an enormous amount of energy (~10(exp 32) erg) into the corona. A substantial fraction of this energy is transported to the lower atmosphere, which results in chromospheric heating. The mechanisms that transport energy to the lower solar atmosphere during a flare are still not fully understood. Aims. We aim to estimate the temporal evolution of the radiative losses in the chromosphere at the footpoints of a C-class flare, in order to set observational constraints on the electron beam parameters of a RADYN flare simulation. Methods. We estimated the radiative losses from hydrogen, and singly ionized Ca and Mg using semiempirical model atmospheres, which were inferred from a multiline inversion of observed Stokes profiles obtained with the CRISP and CHROMIS instruments on the Swedish 1-m Solar Telescope. The radiative losses were computed taking into account the effect of partial redistribution and non-local thermodynamic equilibrium. To estimate the integrated radiative losses in the chromosphere, the net cooling rates were integrated between the temperature minimum and the height where the temperature reaches 10 kK. We also compared our time series of radiative losses with those from the RADYN flare simulations. Results. We obtained a high spatial-resolution map of integrated radiative losses around the flare peak time. The stratification of the net cooling rate suggests that the Ca IR triplet lines are responsible for most of the radiative losses in the flaring atmosphere. During the flare peak time, the contribution from Ca II H and K and Mgii h and k lines are strong and comparable to the Ca IR triplet (~32kW m(exp -2)). Since our flare is a relatively weak event, the chromosphere is not heated above 11 kK, which in turn yields a subdued Lyα contribution (~7kW m(exp -2)) in the selected limits of the chromosphere. The temporal evolution of total integrated radiative losses exhibits sharply rising losses (0.4kW m(exp -2) (s(exp -1)) and a relatively slow decay (0.23kW m(exp -2) s(exp -1)). The maximum value of total radiative losses is reached around the flare peak time and can go up to 175kWm􀀀2 for a single pixel located at footpoint. After a small parameter study, we find the best model-data consistency in terms of the amplitude of radiative losses and the overall atmospheric structure with a RADYN flare simulation in the injected energy flux of 5 × 10(exp 10) erg s(exp -1) cm(exp -2).

Chromosphere↗
DOE OSTI2017

The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/Virgo GW170817. VI. Radio Constraints on a Relativistic Jet and Predictions for Late-time Emission from the Kilonova Ejecta

We present Very Large Array (VLA) and Atacama Large Millimeter/submillimeter Array (ALMA) radio observations of GW170817, the first Laser Interferometer Gravitational-wave Observatory (LIGO)/Virgo gravitational wave (GW) event from a binary neutron star merger and the first GW event with an electromagnetic (EM) counterpart. Our data include the first observations following the discovery of the optical transient at both the centimeter (13.7 hr post-merger) and millimeter (2.41 days post-merger) bands. We detect faint emission at 6 GHz at 19.47 and 39.23 days after the merger, but not in an earlier observation at 2.46 days. We do not detect cm/mm emission at the position of the optical counterpart at frequencies of 10–97.5 GHz at times ranging from 0.6 to 30 days post-merger, ruling out an on-axis short gamma-ray burst (SGRB) for energies ≳10{sup 48} erg. For fiducial SGRB parameters, our limits require an observer viewer angle of ≳20°. The radio and X-ray data can be jointly explained as the afterglow emission from an SGRB with a jet energy of ∼10{sup 49}--10{sup 50} erg that exploded in a uniform density environment with n∼10{sup −4}--10{sup −2} cm{sup −3}, viewed at an angle of ∼20°–40° from the jet axis. Using the results of our light curve and spectral modeling, in conjunction with the inference of the circumbinary density, we predict the emergence of late-time radio emission from the deceleration of the kilonova (KN) ejecta on a timescale of ∼5–10 years that will remain detectable for decades with next-generation radio facilities, making GW170817 a compelling target for long-term radio monitoring.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Equation of state at neutron-star densities and beyond from perturbative QCD

We explore the consequences of imposing robust thermodynamic constraints arising from perturbative quantum chromodynamics (QCD) when inferring the dense-matter equation-of-state (EOS). We find that the termination density, up to which the EOS modeling is performed in an inference setup, strongly affects the constraining power of the QCD input. This sensitivity in the constraining power arises from EOSs that have a specific form, with drastic softening immediately above the termination density followed by a strong stiffening. Here we also perform explicit modeling of the EOS down from perturbative-QCD densities to construct a new QCD likelihood function that incorporates additional perturbative-QCD calculations of the sound speed and is insensitive to the termination density, which we make publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Combining Push-Forward Measures and Bayes' Rule to Construct Consistent Solutions to Stochastic Inverse Problems

We formulate, and present a numerical method for solving, an inverse problem for inferring parameters of a deterministic model from stochastic observational data on quantities of interest. The solution, given as a probability measure, is derived using a Bayesian updating approach for measurable maps that finds a posterior probability measure, that when propagated through the deterministic model produces a push-forward measure that exactly matches the observed probability measure on the data. Our approach for finding such posterior measures, which we call consistent Bayesian inference or push-forward based inference, is simple and only requires the computation of the push-forward probability measure induced by the combination of a prior probability measure and the deterministic model. We establish existence and uniqueness of observation-consistent posteriors and present both stability and error analyses. We also discuss the relationships between consistent Bayesian inference, classical/statistical Bayesian inference, and a recently developed measure-theoretic approach for inference. Finally, analytical and numerical results are presented to highlight certain properties of the consistent Bayesian approach and the differences between this approach and the two aforementioned alternatives for inference.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2022

A Detailed Analysis of the Gl 486 Planetary System

Context . The Gl 486 system consists of a very nearby, relatively bright, weakly active M3.5V star at just 8 pc with a warm transiting rocky planet of about 1.3 R ⨁ and 3.0 M ⨁ . It is ideal for both transmission and emission spectroscopy and for testing interior models of telluric planets. Aims . To prepare for future studies, we aim to thoroughly characterise the planetary system with new accurate and precise data collected with state-of-the-art photometers from space and spectrometers and interferometers from the ground. Methods . We collected light curves of seven new transits observed with the CHEOPS space mission and new radial velocities obtained with MAROON-X at the 8.1m Gemini North telescope and CARMENES at the 3.5m Calar Alto telescope, together with previously published spectroscopic and photometric data from the two spectrographs and TESS. We also performed near-infrared interferometric observations with the CHARA Array and new photometric monitoring with a suite of smaller telescopes (AstroLAB, LCOGT, OSN, TJO). This extraordinary and rich data set was the input for our comprehensive analysis. Results . From interferometry, we measure a limb-darkened disc angular size of the star Gl 486 at θ LDD = 0.390 ± 0.018 mas. Together with a corrected Gaia EDR3 parallax, we obtain a stellar radius R * = 0.339 ± 0.015 R ⨀ . We also measure a stellar rotation period at P rot = 49.9 ± 5.5 days, an upper limit to its XUV (5–920 Å) flux informed by new Hubble/STIS data, and, for the first time, a variety of element abundances (Fe, Mg, Si, V, Sr, Zr, Rb) and C/O ratio. Moreover, we imposed restrictive constraints on the presence of additional components, either stellar or sub-stellar, in the system. With the input stellar parameters and the radial-velocity and transit data, we determine the radius and mass of the planet Gl 486 b at R p = 1.343 +0.063 -0.062 R ⨀ and M p = 3.00 +0.13 -0.13 M ⨁ , with relative uncertainties of the planet radius and mass of 4.7% and 4.2%, respectively. From the planet parameters and the stellar element abundances, we infer the most probable models of planet internal structure and composition, which are consistent with a relatively small metallic core with respect to the Earth, a deep silicate mantle, and a thin volatile upper layer. With all these ingredients, we outline prospects for Gl 486 b atmospheric studies, especially with forthcoming James Webb Space Telescope (Webb) observations.

Planetary systems↗
NASA NTRS1977

Use of clear lake as standard reflectors for atmospheric measurement

A method is proposed for using clear lakes as dark backgrounds against which the atmospheric path radiance can be determined from satellite observations. If the path radiance can be determined to sufficient accuracy, the atmospheric extinction can be inferred with suitable radiative transfer models. An extensive program of observation was made to determine the magnitude and variability of the various contributors to the total radiance observed by a satellite. It is shown that the volume and surface reflectance contributions (in the absence of sunglint) are small, constant, and can be modeled accurately enough to make these an insignificant source of error. The sunglint radiance observed in this investigation may be a significant source of error. It is shown that atmospheric extinction can be inferred from the path radiance observation after systematic differences between the model and observations are removed.

Ahern, F. J.↗
NASA NTRS1981

Models of the earth's core

Combined inferences from seismology, high-pressure experiment and theory, geomagnetism, fluid dynamics, and current views of terrestrial planetary evolution lead to models of the earth's core with five basic properties. These are that core formation was contemporaneous with earth accretion; the core is not in chemical equilibrium with the mantle; the outer core is a fluid iron alloy containing significant quantities of lighter elements and is probably almost adiabatic and compositionally uniform; the more iron-rich inner solid core is a consequence of partial freezing of the outer core, and the energy release from this process sustains the earth's magnetic field; and the thermodynamic properties of the core are well constrained by the application of liquid-state theory to seismic and labroatory data.

Stevenson, D. J.↗
DOE OSTI2022

MetaText: Compositional Generalization in Deep Language Models

Compositional generalization, the ability to infer and reason using novel combinations of previously encountered entities and structures, is a trait of great utility across a variety of deep learning tasks; it implies a certain ability to reason using consistent and therefore interpretable rules, to understand the constituent parts of its input at multiple levels of granularity, to be robust to spurious differences between semantically equivalent inputs, and more. We seek to understand the degree to which existing natural language models achieve or fall short of compositional generalization across a variety of tasks, whether there are distinct types of compositional generalization in practice, and to identify avenues of intervention through which we can improve models’ compositional generalization ability. After exploring these questions using an array of models, tasks, and potential interventions we see that large, pretrained language models have encountered sufficient training data to account for a variety of fine-grained compositional behavior but still struggle to reason at the level of phrases or larger language structures. Non-intrusive, data-based interventions in the form of augmenting individual sequences with compressed versions of themselves or deriving new examples from induced grammars prove insufficient to encourage greater levels of compositional reasoning, indicating that future work might benefit most from focusing on changes to a model’s inductive bias at the architectural or loss level, or by integrating compositionality-boosting data interventions into the large-scale pretraining process itself.

97 MATHEMATICS AND COMPUTING↗