Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “activation function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Precise Modeling of a Complex Solenoidal Magnetic Field Using a Combination of Analytic Functions and a PINN

We demonstrate an iterative approach to modeling a sparsely measured magnetic field in a large-bore solenoid. This approach uses a hybrid of traditional and machine learning techniques. The traditional technique is a linear least-squares fit using a series solution to Laplace's equation, while the machine learning technique involves the training of a physics-informed neural network (PINN) on the least-squares fit residuals. We use a newly defined activation function "DELTAsnake," a modification to the snake activation function proposed by Ziyin et al. that allows for stronger curvature and non-monotonicity. The combined model approximately obeys Maxwell's equations to a level sufficient for producing high quality physics simulations and analysis. Our approach is applied to a highly realistic calculation of the expected magnetic field in the Mu2e experiment's Detector Solenoid which includes a simple model for the expected statistical measurement uncertainties. Using ten toy measurement simulations, we demonstrate the capabilities of our model in comparison to the least-squares method alone; the least-squares method alone results in a reduced chi-squared statistic of ${2.15 \pm 0.01}$, while our approach improves the reduced chi-square to ${1.034 \pm 0.005}$. Furthermore, for an average toy simulation, we show that the range of the RMS of the three field component residuals reduces from ${0.07-0.37}$ Gauss to ${0.05-0.07}$ Gauss. We find that this novel method is robust against a realistic systematic uncertainty deriving from Hall probe calibration bias and can be used to significantly reduce the number of measurements required to achieve an accurate model.

Kampa, Cole [Caltech] (ORCID:0000000192972920)↗

Sound-localization-related activation and functional connectivity of dorsal auditory pathway in relation to demographic, cognitive, and behavioral characteristics in age-related hearing loss

Background Patients with age-related hearing loss (ARHL) often struggle with tracking and locating sound sources, but the neural signature associated with these impairments remains unclear. Materials and methods Using a passive listening task with stimuli from five different horizontal directions in functional magnetic resonance imaging, we defined functional regions of interest (ROIs) of the auditory “where” pathway based on the data of previous literatures and young normal hearing listeners ( n = 20). Then, we investigated associations of the demographic, cognitive, and behavioral features of sound localization with task-based activation and connectivity of the ROIs in ARHL patients ( n = 22). Results We found that the increased high-level region activation, such as the premotor cortex and inferior parietal lobule, was associated with increased localization accuracy and cognitive function. Moreover, increased connectivity between the left planum temporale and left superior frontal gyrus was associated with increased localization accuracy in ARHL. Increased connectivity between right primary auditory cortex and right middle temporal gyrus, right premotor cortex and left anterior cingulate cortex, and right planum temporale and left lingual gyrus in ARHL was associated with decreased localization accuracy. Among the ARHL patients, the task-dependent brain activation and connectivity of certain ROIs were associated with education, hearing loss duration, and cognitive function. Conclusion Consistent with the sensory deprivation hypothesis, in ARHL, sound source identification, which requires advanced processing in the high-level cortex, is impaired, whereas the right–left discrimination, which relies on the primary sensory cortex, is compensated with a tendency to recruit more resources concerning cognition and attention to the auditory sensory cortex. Overall, this study expanded our understanding of the neural mechanisms contributing to sound localization deficits associated with ARHL and may serve as a potential imaging biomarker for investigating and predicting anomalous sound localization.

Wu, Junzhi↗

Uncertainty propagation in feed-forward neural network models

We develop new uncertainty propagation methods for feed-forward neural network architectures with leaky ReLU activation functions subject to random perturbations in the input vectors. In particular, we derive analytical expressions for the probability density function (PDF) of the neural network output and its statistical moments as a function of the input uncertainty and the parameters of the network, i.e., weights and biases. A key finding is that an appropriate linearization of the leaky ReLU activation function yields accurate statistical results even for large perturbations in the input vectors. This can be attributed to the way information propagates through the network. We also propose new analytically tractable Gaussian copula surrogate models to approximate the full joint PDF of the neural network output. To validate our theoretical results, we conduct Monte Carlo simulations and a thorough error analysis on a multi-layer neural network representing a nonlinear integro-differential operator between two polynomial function spaces. Our findings demonstrate excellent agreement between the theoretical predictions and Monte Carlo simulations.

MLP networks↗

Extending Parsimonious Bayesian Inference

Parsimonious Bayesian inference is a theoretical framework for efficient data assimilation that seeks to balance increased consistency between predictions and training data against corresponding increases in model complexity. Within this framework, over-training is understood as optimization that encodes excessive information within model parameters while only achieving small improvements between predictions and training data. This project aims to develop practical methods of limiting excess model information during optimization. One key observation is that practical heuristics for parsimonious learning in high-dimensions must balance expressivity, i.e. the ability of the model to capture diverse predictions with only a few non-zero parameters, against discoverability, i.e. the ability to train the model with gradient-based optimization and drive parameters to low information states. As such, we developed logical activation functions that are able to adaptively approximate arbitrary truth tables that define Boolean logic operations within a probabilistic framework. These functions have demonstrated the ability to learn exclusive disjunction (XOR) and conditioned disjunction (if [condition] then [result_if_true] else [result_if_false]) within a single layer of a neural network. To efficiently exploit these activation functions to drive parsimonious learning required several other advances within the domain of variational inference. The most efficient form of complexity suppression is structured sparsification, driving most model parameters to zero while achieving the structural coherence among nonzeros needed for bandwidth reduction. Such models are not only far more efficient at suppressing information-theoretic complexity, they also reduce the other forms of complexity (computations, communication, storage, and the number of dependencies needed to evaluate predictions). Aiming to support enhanced sparsification, this project examined new approaches to high-dimensional variational inference that allow us to calibrate and control parameter uncertainty during optimization. By identifying which parameters can sustain sparsifying perturbations with little impact on prediction quality, we can develop better pruning strategies by framing them as approximate Bayesian inference. These advances also open paths to mitigate concerns with deploying advanced learning methods in resource-constrained environments, such as running models on power-limited or communication-limited devices.

97 MATHEMATICS AND COMPUTING↗

Head on Comparison of Self- and Nano-Assemblies of Gamma Peptide Nucleic Acid Amphiphiles

Peptide nucleic acids (PNAs) are nucleic acid analogs with hybridization properties and enzymatic stability superior to that of DNA. In addition to gene targeting applications, PNAs have garnered significant attention as bio-polymers due to the Watson–Crick-based molecular recognition and flexibility of synthesis. Here, in this work, PNA amphiphiles are engineered using chemically modified gamma PNA (8 mer in length) containing hydrophilic diethylene glycol units at the gamma position and covalently conjugated lauric acid (C12) as a hydrophobic moiety. Gamma PNA ($\gamma$PNA) amphiphiles self-assemble into spherical vesicles. Further, nano-assemblies (NA) are formulated using the amphiphilic $\gamma$PNA as a polymer via ethanol injection-based protocols. Comprehensive head-on comparison of the physicochemical and cellular uptake properties of PNA derived self- and NA is performed. Small-angle neutron and X-ray scattering analysis reveal ellipsoidal morphology of $\gamma$PNA NA that results in superior cellular delivery compate to the spherical self-assembly. Next, the functional activities of $\gamma$PNA self-and NA in lymphoma cells via multiple endpoints, including gene expression, cell viability, and apoptosis-based assays are compared. Overall, it is established that $\gamma$PNA amphiphile is a functionally active bio-polymer to formulate NA for a wide range of biomedical applications.

59 BASIC BIOLOGICAL SCIENCES↗

Deep Learning without Global Optimization by Random Fourier Neural Networks

Here we introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions.

97 MATHEMATICS AND COMPUTING↗

Functional group activation and coupling agent migration induced by plasma treatment in adhesive for enhanced toughness of metal-composite joints

Commercial adhesive paste was additionally activated using low-power oxygen plasma with different plasma exposure durations to investigate its interfacial bonding contribution in adhesively-bonded Aural-5/CFRP-PA6 double cantilever beam (DCB) joints with plasma-treated adherends. The plasma-treated adhesive showed enhanced functional peak intensities in Fourier transfer infrared spectroscopy (FTIR) as plasma exposure time was increased, and some degree of oxidation was also detected via X-ray photoelectron spectroscopy (XPS). A small molecule, dicyandiamde (coupling agent) in adhesive was migrated to adhesive layer, where double concentration of coupling agent was detected after 10 minutes of plasma treatment. As a result, characterization of the adhesive/CFRP-PA6 interface after DCB fracture showed additional chemical bonds formed through amine-initiated epoxy ring opening polymerization, as well as amide bonds and ester bonds. In conclusion, these additional bonds at the adhesive/CFRP-PA6 interfaces led to significantly increased failure extensions and fracture energies of the joints as the plasma exposure time on the adhesive increased, compared to the joints with plasma-treated adherends but non-plasma-treated adhesive as commonly seen in the literature.

36 MATERIALS SCIENCE↗

Physics-informed neural networks for heterogeneous poroelastic media

This study presents a novel physics-informed neural network (PINN) framework for modeling poroelasticity in heterogeneous media with material interfaces. The approach introduces a composite neural network (CoNN) where separate neural networks predict displacement and pressure variables for each material. While sharing identical activation functions, these networks are independently trained for all other parameters. To address challenges posed by heterogeneous material interfaces, the CoNN is integrated with the Interface-PINNs (I-PINNs) framework (Sarma et al., Comput. Methods Appl. Mech. Eng. 429: 117135, 2024), allowing different activation functions across material interfaces. Further, this ensures accurate approximation of discontinuous solution fields and gradients. Performance and accuracy of this combined architecture were evaluated against the conventional PINNs approach, a single neural network (SNN) architecture, and the eXtended PINNs (XPINNs) framework through two one-dimensional benchmark examples with discontinuous material properties. The results show that the proposed CoNN with I-PINNs architecture achieves an RMSE that is two orders of magnitude better than the conventional PINNs approach and is at least 40 times faster than the SNN framework. Compared to XPINNs, the proposed method achieves an RMSE at least one order of magnitude better and is 40% faster.

42 ENGINEERING↗

Restricting electron flow at cytochrome b6f when downstream electron acceptors are severely limited

Abstract Photosynthetic organisms frequently experience abiotic stress that restricts their growth and development. Under such circumstances, most absorbed solar energy cannot be used for CO2 fixation and can cause the photoproduction of reactive oxygen species (ROS) that can damage the photosynthetic reaction centers of PSI and PSII, resulting in a decline in primary productivity. This work describes a biological “switch” in the green alga Chlamydomonas reinhardtii that reversibly restricts photosynthetic electron transport (PET) at the cytochrome b6f (Cyt b6f) complex when the capacity for accepting electrons downstream of PSI is severely limited. We specifically show this restriction in STARCHLESS6 (sta6) mutant cells, which cannot synthesize starch when they are limited for nitrogen (growth inhibition) and subjected to a dark-to-light transition. This restriction represents a form of photosynthetic control that causes diminished electron flow to PSI and thereby prevents PSI photodamage but does not appear to rely on a ΔpH. Furthermore, when electron flow is restricted, the plastid alternative oxidase (PTOX) becomes active, functioning as an electron valve that dissipates some excitation energy absorbed by PSII and allows the formation of a proton motive force (PMF) that would drive some ATP production (potentially sustaining PSII repair and nonphotochemical quenching [NPQ]). The restriction at the Cyt b6f complex can be gradually relieved with continued illumination. This study provides insights into how PET responds to a marked reduction in availability of downstream electron acceptors and the protective mechanisms involved.

Plant Sciences↗

Simplifying activations with linear approximations in neural networks

A key step in Neural Networks is activation. Among the different types of activation functions, sigmoid, tanh, and others involve the usage of exponents for calculation. From a hardware perspective, exponential implementation implies the usage of Taylor series or repeated methods involving many addition, multiplication, and division steps, and as a result are power-hungry and consume many clock cycles. We implement a piecewise linear approximation of the sigmoid function as a replacement for standard sigmoid activation libraries. This approach provides a practical alternative by leveraging piecewise segmentation, which simplifies hardware implementation and improves computational efficiency. In this paper, we detail piecewise functions that can be implemented using linear approximations and their implications for overall model accuracy and performance gain. Our results show that for the DenseNet, ResNet, and GoogLeNet architectures, the piecewise linear approximation of the sigmoid function provides faster execution times compared to the standard TensorFlow sigmoid implementation while maintaining comparable accuracy. Specifically, for MNIST with DenseNet, accuracy reaches 99.91% (Piecewise) vs. 99.97% (Base) with up to 1.31x speedup in execution time. For CIFAR-10 with DenseNet, accuracy improves to 98.97% (Piecewise) vs. 99.40% (Base) while achieving 1.24x faster execution. Similarly, for CIFAR-100 with DenseNet, the accuracy is 97.93% (Piecewise) vs. 98.39% (Base), with a 1.18x execution time reduction. These results confirm the proposed method’s capability to efficiently process large-scale datasets and computationally demanding tasks, offering a practical means to accelerate deep learning models, including LSTMs, without compromising accuracy.

Activation function↗

A human antibody epitope map of Pfs230D1 derived from analysis of individuals vaccinated with a malaria transmission-blocking vaccine

Pfs230 domain 1 (Pfs230D1) is an advanced malaria transmission-blocking vaccine antigen demonstrating high functional activity in clinical trials. However, the structural and functional correlates of transmission-blocking activity are not defined. Here, we characterized a panel of human monoclonal antibodies (hmAbs) elicited in vaccinees immunized with Pfs230D1. These hmAbs exhibited diverse transmission-reducing activity, yet all bound to Pfs230D1 with nanomolar affinity. We compiled epitope-binning data for seventeen hmAbs and structures of nine hmAbs complexes to construct a high-resolution epitope map and revealed that potent transmission-reducing hmAbs bound to one face of Pfs230D1, while non-potent hmAbs bound to the opposing side. The structure of Pfs230D1D2 revealed that non-potent transmission-reducing epitopes were occluded by the second domain. The hmAb epitope map delineated binary hmAb combinations that synergized for extremely high-potency, transmission-reducing activity. Finally, this work provides a high-resolution guide for structure-based design of enhanced immunogens and informs diagnostics that measure the transmission-reducing response.

60 APPLIED LIFE SCIENCES↗

The crystal structure of Grindelia robusta 7,13-copalyl diphosphate synthase reveals active site features controlling catalytic specificity

Diterpenoid natural products serve critical functions in plant development and ecological adaptation and many diterpenoids have economic value as bioproducts. The family of class II diterpene synthases catalyzes the committed reactions in diterpenoid biosynthesis, converting a common geranylgeranyl diphosphate precursor into different bicyclic prenyl diphosphate scaffolds. Enzymatic rearrangement and modification of these precursors generate the diversity of bioactive diterpenoids. We report the crystal structure of Grindelia robusta 7,13-copalyl diphosphate synthase, GrTPS2, at 2.1 Å of resolution. GrTPS2 catalyzes the committed reaction in the biosynthesis of grindelic acid, which represents the signature metabolite in species of gumweed (Grindelia spp., Asteraceae). Grindelic acid has been explored as a potential source for drug leads and biofuel production. The GrTPS2 crystal structure adopts the conserved three-domain fold of class II diterpene synthases featuring a functional active site in the γβ-domain and a vestigial α-domain. Substrate docking into the active site of the GrTPS2 apo protein structure predicted catalytic amino acids. Biochemical characterization of protein variants identified residues with impact on enzyme activity and catalytic specificity. Specifically, mutagenesis of Y457 provided mechanistic insight into the position-specific deprotonation of the intermediary carbocation to form the characteristic 7,13 double bond of 7,13-copalyl diphosphate.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Structural characterization of a soil viral auxiliary metabolic gene product – a functional chitosanase

Metagenomics is unearthing the previously hidden world of soil viruses. Many soil viral sequences in metagenomes contain putative auxiliary metabolic genes (AMGs) that are not associated with viral replication. Here, we establish that AMGs on soil viruses actually produce functional, active proteins. We focus on AMGs that potentially encode chitosanase enzymes that metabolize chitin – a common carbon polymer. We express and functionally screen several chitosanase genes identified from environmental metagenomes. One expressed protein showing endo-chitosanase activity (V-Csn) is crystalized and structurally characterized at ultra-high resolution, thus representing the structure of a soil viral AMG product. This structure provides details about the active site, and together with structure models determined using AlphaFold, facilitates understanding of substrate specificity and enzyme mechanism. Our findings support the hypothesis that soil viruses contribute auxiliary functions to their hosts.

59 BASIC BIOLOGICAL SCIENCES↗

Probe for selectively characterizing enzymes involved in xenobiotic metabolism and method of making and using the same

Activity-based probes that can be used to selectively identify and characterize enzymes that are involved in different phases of xenobiotic metabolism in a host and its microbiota population(s) are described. The activity-based probes described specifically label only their target active enzymes involved in xenobiotic metabolism and therefore provide a measurement of true protein functional activity rather than transcript or protein abundance. The activity-based probes also provide multimodal profiling of these active enzymes. Methods for preparing the activity based probes and exemplary methods for their use also are disclosed.

Wright, Aaron T.↗

Probe for selectively characterizing enzymes involved in xenobiotic metabolism and method of making and using the same

Activity-based probes that can be used to selectively identify and characterize enzymes that are involved in different phases of xenobiotic metabolism in a host and its microbiota population(s) are described. The activity-based probes described specifically label only their target active enzymes involved in xenobiotic metabolism and therefore provide a measurement of true protein functional activity rather than transcript or protein abundance. The activity-based probes also provide multimodal profiling of these active enzymes. Methods for preparing the activity based probes and exemplary methods for their use also are disclosed.

Wright, Aaron T.↗

Combinatorial optimization and spatial remodeling of CYPs to control product profile

Activating inert substrates is a challenge in nature and synthetic chemistry, but essential for creating functionally active molecules. In this work, we used a combinatorial optimization approach to assemble cytochrome P450 monooxygenases (CYPs) and reductases (CPRs) to achieve a target product profile. By creating 110 CYP-CPR pairs and iteratively screening different pairing libraries, we demonstrated a framework for establishing a CYP network that catalyzes six oxidation reactions at three different positions of a chemical scaffold. Target product titer was improved by remodeling endoplasmic reticulum (ER) size and spatially controlling the CYPs’ configuration on the ER. Out of 47 potential products that could be synthesized, 86% of the products synthesized by the optimized network was our target compound quillaic acid (QA), the aglycone backbone of many pharmaceutically important saponins, and fermentation achieved QA titer 2.23 g/L.

59 BASIC BIOLOGICAL SCIENCES↗

Massively parallel and universal approximation of nonlinear functions using diffractive processors

Nonlinear computation is essential for a wide range of information processing tasks, yet implementing nonlinear functions using optical systems remains a challenge due to the weak and power-intensive nature of optical nonlinearities. Overcoming this limitation without relying on nonlinear optical materials could unlock unprecedented opportunities for ultrafast and parallel optical computing systems. Here, we demonstrate that large-scale nonlinear computation can be performed using linear optics through optimized diffractive processors composed of passive phase-only surfaces. In this framework, the input variables of nonlinear functions are encoded into the phase of an optical wavefront—e.g., via a spatial light modulator (SLM)—and transformed by an optimized diffractive structure with spatially varying point-spread functions to yield output intensities that approximate a large set of unique nonlinear functions–all in parallel. We provide proof establishing that this architecture serves as a universal function approximator for an arbitrary set of bandlimited nonlinear functions, also covering wavelength-multiplexed nonlinear functions as well as multi-variate and complex-valued functions that are all-optically cascadable. Our analysis also indicates the successful approximation of typical nonlinear activation functions commonly used in neural networks, including the sigmoid, tanh, ReLU (rectified linear unit), and softplus. We numerically demonstrate the parallel computation of one million distinct nonlinear functions, accurately executed at wavelength-scale spatial density at the output of a diffractive optical processor. Furthermore, we experimentally validated this framework using in situ optical learning and approximated 35 unique nonlinear functions in a single shot using a compact setup consisting of an SLM and an image sensor. These results establish diffractive optical processors as a scalable platform for massively parallel universal nonlinear function approximation, paving the way for new capabilities in analog optical computing based on linear materials.

Rahman, Md Sadman Sakib [University of California,↗