Engineering Papers⌕ Search

Engineering topics

Friedland, Gerald

Publications and source records attributed to Friedland, Gerald.

Methods for Explainable Artificial Intelligence

We explored ways of quantifying information in a neural network. This can be used to determine the right size of a network or to infer the way in which a network is processing information. The first year and a half was somewhat exploratory while the last half of the project focused on approaches that seemed to show the most promise. The introduce a new way of computing explainable artificial intelligence (XAI) saliency maps that is several orders of magnitude faster than methods with similar fidelity We call it FastCAM. The method works be combining a Class Activation Map (CAM) method such as GradCAM with a forward activation map computed with a statistic we call SMOE Scale. The addition of the forward activation maps to CAM methods seems to always improve their fidelity. At the same time, computational overhead is not increased by very much. While Gradients with SmoothGrad scores better on some fidelity measures, it is overall not as good and requires more than 1500 times to compute. We demonstrate two completed applications of FastCAM on tasks outside of the LDRD at LLNL. The source code for FastCAM is currently being implanted into Captum, the official XAI toolkit for the popular deep learning toolkit PyTorch. The LDRD currently has 13 publications released to the public. 10 of them are journal length.

97 MATHEMATICS AND COMPUTING↗

A Practical Approach to Sizing Neural Networks

Most approaches to machine learning experiments currently involve tedious hyperparameter tuning. This article first shows how to practically estimate the maximum size of a neural network given a training data set. We present four easily applicable rules to analytically determine the capacity of neural network architectures. The rules are derived based on MacKay’s information theoretic model of supervised machine learning [22]. This allows the comparison of the efficiency of different architectures independently of a task. Second, we introduce and experimentally validate a heuristic method to estimate the neural network capacity requirement for a given dataset and labeling. This allows an estimate of the required size of a neural network for a given problem. We conclude the article with a discussion on the consequences of sizing the network wrongly, which includes both increased computation effort for training as well as reduced generalization capability.

97 MATHEMATICS AND COMPUTING↗

On the Goldbach Conjecture

This note proposes a confirmation proof of Goldbach’s Conjecture from 1742 [gol]: Can every even integer n > 2 be represented as the sum of 2 prime numbers? First, I present an analysis of the nature of multiplication as description length reduction for addition. This allows to explain prime numbers from first principles rather than by definition. Then, I use this analysis to derive that any natural number n must be representable by n – 1 unique two-summand additions. Given that an explicit formula for the number of primes smaller n is unknown, I formulate the contraposition to the conjecture. I then count the maximum number of possible unique additions when the summands are both non-prime or 1 and when one summand is non-prime, prime or 1 and the other is non-prime or 1. This is done using a case distinction for even and odd n. The counting is facilitated by finding the maximum number of positions a plus symbol can be placed into the unary representation of n, given the constraints. The main result is that no even n ≥ 8 exists where all n – 1 additions can be formulated without including the case of the addition of two primes. A side result is that any odd integer n can be represented by 1 addition of 2 primes if and only if n – 2 is prime.

97 MATHEMATICS AND COMPUTING↗

An Isomorphism between Lyapunov Exponents and Shannon's Channel Capacity

We demonstrate that discrete Lyapunov exponents are isomorphic to numeric overflows of the capacity of an arbitrary noiseless and memoryless channel in a Shannon communication model with feedback. The isomorphism allows the understanding of Lyapunov exponents in terms of Information Theory, rather than the traditional definitions in chaos theory. The result also implies alternative approaches to the calculation of related quantities, such as the Kolmogorov Sinai entropy which has been linked to thermodynamic entropy. This work provides a bridge between fundamental physics and information theory. It suggests, among other things, that machine learning and other information theory methods can be employed at the core of physics simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗