Engineering topics
Nebgen, Benjamin
Publications and source records attributed to Nebgen, Benjamin.
Synergy of semiempirical models and machine learning in computational chemistry
Catalyzed by enormous success in the industrial sector, many research programs have been exploring data-driven, machine learning approaches. Performance can be poor when the model is extrapolated to new regions of chemical space, e.g., new bonding types, new many-body interactions. Another important limitation is the spatial locality assumption in model architecture, and this limitation cannot be overcome with larger or more diverse datasets. The outlined challenges are primarily associated with the lack of electronic structure information in surrogate models such as interatomic potentials. Given the fast development of machine learning and computational chemistry methods, we expect some limitations of surrogate models to be addressed in the near future; nevertheless spatial locality assumption will likely remain a limiting factor for their transferability. Here, we suggest focusing on an equally important effort—design of physics-informed models that leverage the domain knowledge and employ machine learning only as a corrective tool. In the context of material science, we will focus on semi-empirical quantum mechanics, using machine learning to predict corrections to the reduced-order Hamiltonian model parameters. The resulting models are broadly applicable, retain the speed of semiempirical chemistry, and frequently achieve accuracy on par with much more expensive ab initio calculations. These early results indicate that future work, in which machine learning and quantum chemistry methods are developed jointly, may provide the best of all worlds for chemistry applications that demand both high accuracy and high numerical efficiency.
AdversarialTensors
This library builds a framework for defending ML models against adversarial attacks. The library will be developed at various stages leading to publication and software release at each stage. We employ tensor decomposition strategies as preprocessing stages for the first stage to provide robustness against the prominent adversarial noise. In the second stage, we develop a latent noise generator capable of generating novel adversarial noise that threatens the existing state-of-the-art defense strategy. In the third stage, we develop a UNSUP-GAN model, where the generator is trained to denoise against latent noise and most adversarial noises. This generator can provide a robust adversarial attack against any unseen attack.
Neural network atomistic potentials for global energy minima search in carbon clusters
This article explores the application of NN atomistic potentials, specifically ANI-1ccx and ANI-nr, for efficient global minima optimization in carbon clusters. Combined with the cluster distance restriction, they are robust PES pre-samplers.
Machine Learning Models Capture Plasmon Dynamics in Ag Nanoparticles
Not Available
Uncertainty-driven dynamics for active learning of interatomic potentials
Machine learning (ML) models, if trained to data sets of high-fidelity quantum simulations, produce accurate and efficient interatomic potentials. Active learning (AL) is a powerful tool to iteratively generate diverse data sets. In this approach, the ML model provides an uncertainty estimate along with its prediction for each new atomic configuration. If the uncertainty estimate passes a certain threshold, then the configuration is included in the data set. Here we develop a strategy to more rapidly discover configurations that meaningfully augment the training data set. The approach, uncertainty-driven dynamics for active learning (UDD-AL), modifies the potential energy surface used in molecular dynamics simulations to favor regions of configuration space for which there is large model uncertainty. The performance of UDD-AL is demonstrated for two AL tasks: sampling the conformational space of glycine and sampling the promotion of proton transfer in acetylacetone. The method is shown to efficiently explore the chemically relevant configuration space, which may be inaccessible using regular dynamical sampling at target temperature conditions.
Active Learning Framework
Machine learning (ML) of interatomic potentials show great promise to accelerate scientific simulation, e.g., by emulating expensive computations at a high accuracy but much reduced computational cost. Training datasets are calculated from computationally expensive ab initio quantum mechanics methods, density functional theory (DFT). Trained on this data, an ML model can be very successful in predicting energy and forces for new atomic configurations. A critical factor is the quality and diversity of the training dataset. Thus, a highly automated approach to dataset construction based on active learning framework is designed suitable for material physics. The active learning scheme begins with fully randomized atomic configurations. Then, many Molecular Dynamics (MD) trajectories are simulated using current ML potentials, where each MD trajectory is initialized to a random disordered configuration. The temperature is varied in order to diversify the sampled configuration during these simulations. The variance of predictions for eight neural networks within an ensemble is analyzed to determine whether the model is operating as expected. This helps in determining whether collecting more data would be helpful to the model by checking the ensemble variance is greater than the threshold. In this case, the MD trajectory is terminated and the final atomic configuration is placed on a queue (SQL database) for DFT calculations and added to training dataset. Periodically, ML model is retrained to the updated training model. This Active Learning loop is iterated until the cost of MD simulations becomes prohibitively expensive. The MD simulations will hopefully be sufficiently robust to support nucleation after many active learning iterations. In this sense, active learning scheme must automatically discover the important low energy and nonequilibrium physics.
Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics
Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that are simultaneously accurate, transferable, and interpretable.
hippynn Python Package
hippynn is a python package for defining, training, and applying neural networks to atomistic systems. In particular, it focuses on Hierarchical Interacting Particle Neural Networks (HIP-NNs), a deep learning architecture for atomistic systems. HIP-NNs take input data describing the properties of atomistic systems (often generated using ab-initio quantum mechanics) to learn fast and accurate models. A trained HIP-NN can predict potential energy surfaces, atomic charges, and more. hippynn uses PyTorch for portable high-performance code, including both CPU and GPU support. hippynn allows for extensive customization, including user-defined models and loss functions, to facilitate future research into extensions of HIP-NN as well as other atomistic deep learning models.
Predicting phosphorescence energies and inferring wavefunction localization with machine learning
We address phosphorescence, a localized phenomenon, by building localization layers into a DNN model of singlet–triplet energy gaps. These layers improve model performance and simultaneously infer the location of spin excitations within molecules.
Bond order predictions using deep neural networks
Machine learning is an extremely powerful tool for the modern theoretical chemist since it provides a method for bypassing costly algorithms for solving the Schrödinger equation. Already, it has proven able to infer molecular and atomic properties such as charges, enthalpies, dipoles, excited state energies, and others. Most of these machine learning algorithms proceed by inferring properties of individual atoms, even breaking down total molecular energy into individual atomic contributions. In this paper, we introduce a modified version of the Hierarchically Interacting Particle Neural Network (HIP-NN) capable of making predictions on the bonds between atoms rather than on the atoms themselves. We train the modified HIP-NN to infer bond orders for a large number of small organic molecules as computed via the Natural Bond Orbital package. We demonstrate that the trained model is extensible to molecules much larger than those in the training set by studying its performance on the COMP6 dataset. This method has applications in cheminformatics and force field parameterization and opens a promising future for machine learning models to predict other quantities that are defined between atoms such as density matrix elements, Hamiltonian parameters, and molecular reactivities.
PYSEQM
PYSEQM is a package for performing semi-empirical quantum mechanical (SEQM) simulations on molecular systems utilizing PyTorch. SEQM simulations determine molecular properties (energy, electron density, dipole moment, ect.) by solving an approximate Schrödinger equation for the motions of electrons in a molecule. The use of PyTorch provides three specific advantages. First, it allows the calculations to be offloaded to GPU accelerators, giving an order of magnitude increase in speed. Second, back propagation is used to get atomic forces (derivative of the total energy with respect to atomic position) at the same computational cost as the energy calculation itself. Finally, the use of PyTorch makes for a natural interface to modern machine learning methods, which can be used to adjust the semi-empirical parameters build into SEQM methods. Additionally, PYSEQM implements various other optimizations for performing quantum mechanics based molecular dynamics, including SP2 for rapid GPU based solution of the self-consistent field algorithm, and the extended Lagrangian method for rapid QM-MD.
The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules
Abstract Maximum diversification of data is a central theme in building generalized and accurate machine learning (ML) models. In chemistry, ML has been used to develop models for predicting molecular properties, for example quantum mechanics (QM) calculated potential energy surfaces and atomic charge models. The ANI-1x and ANI-1ccx ML-based general-purpose potentials for organic molecules were developed through active learning; an automated data diversification process. Here, we describe the ANI-1x and ANI-1ccx data sets. To demonstrate data diversity, we visualize it with a dimensionality reduction scheme, and contrast against existing data sets. The ANI-1x data set contains multiple QM properties from 5 M density functional theory calculations, while the ANI-1ccx data set contains 500 k data points obtained with an accurate CCSD(T)/CBS extrapolation. Approximately 14 million CPU core-hours were expended to generate this data. Multiple QM calculated properties for the chemical elements C, H, N, and O are provided: energies, atomic forces, multipole moments, atomic charges, etc. We provide this data to the community to aid research and development of ML models for chemistry.
Vibronic coupling in asymmetric bichromophores: Theory and application to diphenylmethane-d{sub 5}
A theoretical model based on Fulton and Gouterman dimer Hamiltonian [J. Chem. Phys. 35, 1059 (1961)] is used to understand the jet cooled spectra of partly deuterated diphenylmethane (DPM-d{sub 5}), reported in adjoining paper by Zwier and co-workers [“Vibronic coupling in asymmetric bichromophores: Experimental investigation of diphenylmethane-d5,” J. Chem. Phys. 141, 064316 (2014)]. The model is extended to include multiple vibrational modes, vibrational asymmetries between the monomers, and inter-monomer modes. Time dependent density functional theory and equation of motion coupled cluster electronic structure calculations are used to obtain parameters for the model Hamiltonian. The monomer units for DPM-d{sub 5} are toluene and partially deuterated toluene-d{sub 5}. The comparisons of modeled and experimental spectra show that the model captures the vibronic effects on a multitude of observables, including anomalously intense vibrational states, transition dipole moment directions, and localization/delocalization phenomena of both the vibronic and electronic wavefunctions. Following the success of the model, a qualitative picture of vibronic coupling in DPM-d{sub 5} is built using only three active vibrations. This illustrates the ability of the model to not just qualitatively predict but also phenomenologically explain the vibronic effects observed in spectra of complex bichromophores.
Vibronic coupling in asymmetric bichromophores: Experimental investigation of diphenylmethane-d{sub 5}
Vibrationally and rotationally resolved electronic spectra of diphenylmethane-d{sub 5} (DPM-d{sub 5}) are reported in the isolated-molecule environment of a supersonic expansion. While small, the asymmetry induced by deuteration of one of the aromatic rings is sufficient to cause several important effects that change the principle mechanism of vibronic coupling between the close-lying S{sub 1} and S{sub 2} states, and spectroscopic signatures such coupling produces. The splitting between S{sub 1} and S{sub 2} origins is 186 cm{sup −1}, about 50% greater than its value in DPM-d{sub 0} (123 cm{sup −1}), and an amount sufficient to bring the S{sub 2} zero-point level into near-resonance with the v = 1 level in the S{sub 1} state of a low-frequency phenyl flapping mode, ν{sub R} = 191 cm{sup −1}. Dispersed fluorescence spectra bear clear evidence that Δv(R) = 1 Herzberg-Teller coupling dominates the near-resonant internal mixing between the S{sub 1} and S{sub 2} manifolds. The fluorescence into each pair of Franck-Condon active ring modes shows an asymmetry that suggests incorrectly that the S{sub 1} and S{sub 2} states may be electronically localized. From rotationally resolved studies, the S{sub 0} and S{sub 1} states have been well-fit to asymmetric rotor Hamiltonians while the S{sub 2} state is perturbed and not fit. The transition dipole moment (TDM) orientation of the S{sub 1} state is nearly perpendicular to the C{sub 2} symmetry axes with 66(2)%:3(1)%:34(2)% a:b:c hybrid-type character while that of the S{sub 2} origin contains 50(10)% a:c-type (S{sub 1}) and 50(10)% b-type (S{sub 2}) character. A model is put forward that explains qualitatively the TDM compositions and dispersed emission patterns without the need to invoke electronic localization. The experimental data discussed here serve as a foundation for a multi-mode vibronic coupling model capable of being applied to asymmetric bichromophores, as presented in the work of B. Nebgen and L. V. Slipchenko [“Vibronic coupling in asymmetric bichromophores: Theory and application to diphenylmethane-d{sub 5},” J. Chem. Phys. (submitted)].