Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “activation function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Adaptive Interface-PINNs (AdaI-PINNs): An Efficient Physics-Informed Neural Networks Framework for Interface Problems

Here, we present an efficient physics-informed neural networks (PINNs) framework, termed Adaptive Interface-PINNs (AdaI-PINNs), to improve the modeling of interface problems with discontinuous coefficients and/or interfacial jumps. This framework is an enhanced version of its predecessor, Interface PINNs or I-PINNs (Sarma et al.; https://doi.org/10.1016/j.cma.2024.117135), which involves domain decomposition and assignment of different predefined activation functions to the neural networks in each subdomain across a sharp interface, while keeping all other parameters of the neural networks identical. In AdaI-PINNs, the activation functions vary solely in their slopes, which are trained along with the other parameters of the neural networks. This makes the AdaI-PINNs framework fully automated without requiring preset activation functions. Comparative studies on one-dimensional, two-dimensional, and three-dimensional benchmark elliptic interface problems reveal that AdaI-PINNs outperform I-PINNs, reducing computational costs by 2-6 times while producing similar or better accuracy.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1991

Total recall in distributive associative memories

Iterative error correction of asymptotically large associative memories is equivalent to a one-step learning rule. This rule is the inverse of the activation function of the memory. Spectral representations of nonlinear activation functions are used to obtain the inverse in closed form for Sparse Distributed Memory, Selected-Coordinate Design, and Radial Basis Functions.

Danforth, Douglas G.↗
NASA NTRS2002

A quantitative comparison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation

Near-infrared spectroscopy (NIRS) has been used to noninvasively monitor adult human brain function in a wide variety of tasks. While rough spatial correspondences with maps generated from functional magnetic resonance imaging (fMRI) have been found in such experiments, the amplitude correspondences between the two recording modalities have not been fully characterized. To do so, we simultaneously acquired NIRS and blood-oxygenation level-dependent (BOLD) fMRI data and compared Delta(1/BOLD) (approximately R(2)(*)) to changes in oxyhemoglobin, deoxyhemoglobin, and total hemoglobin concentrations derived from the NIRS data from subjects performing a simple motor task. We expected the correlation with deoxyhemoglobin to be strongest, due to the causal relation between changes in deoxyhemoglobin concentrations and BOLD signal. Instead we found highly variable correlations, suggesting the need to account for individual subject differences in our NIRS calculations. We argue that the variability resulted from systematic errors associated with each of the signals, including: (1) partial volume errors due to focal concentration changes, (2) wavelength dependence of this partial volume effect, (3) tissue model errors, and (4) possible spatial incongruence between oxy- and deoxyhemoglobin concentration changes. After such effects were accounted for, strong correlations were found between fMRI changes and all optical measures, with oxyhemoglobin providing the strongest correlation. Importantly, this finding held even when including scalp, skull, and inactive brain tissue in the average BOLD signal. This may reflect, at least in part, the superior contrast-to-noise ratio for oxyhemoglobin relative to deoxyhemoglobin (from optical measurements), rather than physiology related to BOLD signal interpretation.

Non-NASA Center↗
DOE OSTI2023

Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware

In-memory computing with emerging non-volatile memory devices (eNVMs) has shown promising results in accelerating matrix-vector multiplications. However, activation function calculations are still being implemented with general processors or large and complex neuron peripheral circuits. Here, we present the integration of Ag-based conductive bridge random access memory (Ag-CBRAM) crossbar arrays with Mott rectified linear unit (ReLU) activation neurons for scalable, energy and area-efficient hardware (HW) implementation of deep neural networks. We develop Ag-CBRAM devices that can achieve a high ON/OFF ratio and multi-level programmability. Compact and energy-efficient Mott ReLU neuron devices implementing ReLU activation function are directly connected to the columns of Ag-CBRAM crossbars to compute the output from the weighted sum current. We implement convolution filters and activations for VGG-16 using our integrated HW and demonstrate the successful generation of feature maps for CIFAR-10 images in HW. Our approach paves a new way toward building a highly compact and energy-efficient eNVMs-based in-memory computing system.

Mott insulators↗
DOE OSTI2023

Activity‐Based Protein Profiling – Enabling Phenotyping of Host‐Associated and Environmental Microbiomes

Abstract Host‐associated and environmental microbiomes play central roles in human health, biogeochemical cycling, response to ecosystem change, and agriculture. Scientific approaches that can reveal the functional activities that contribute to observed phenotypes are needed in microbiome research. Broad characterization of the functional activity of microbes within microbiomes is currently hampered by approaches that rely on inference of function from metagenomes or indirect measurements. Activity‐based protein profiling is uniquely positioned to overcome these challenges and reveal the protein‐level mechanisms associated with microbiome phenotypes. In this review we describe the progress made to date using ABPP in gut microbiome, plant‐microbe interaction, and soil microbiome research, and suggest how ABPP data can be coupled with advanced computational methods to enable phenotype prediction and bioengineering of microbiomes for applied purposes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

ReLU, Sparseness, and the Encoding of Optic Flow in Neural Networks

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

EPR and 31 P ENDOR Characterization of Pseudo-Jahn–Teller Dynamics and N 2 Activation in Functional Nitrogenase Models, P 3 E M(N 2 ) (M = Fe, Co; E = Si, B, C)

Here, the nominally trigonal, pseudo-Jahn-Teller (PJT)-active, S = ½ N 2 -bound transition-metal complexes, P 3 E M(N 2 ), M = Fe, Co, with three in-plane phosphine-ligands and axial donors, E = Si, B, C, include functional nitrogenase models that catalyze reduction of N 2 to NH 3 . We applied EPR, 31 P ENDOR spectroscopy and DFT computations to characterize the PJT-induced distortions of four selected P 3 E M(N 2 ), revealing how the metal-ion and axial ligand E together tune both PJT dynamics and N 2 activation for reduction. Comparisons reveal an unrecognized correlation between PJT distortion, M-E bond elasticity, and N 2 activation, providing guidelines for designing bioinspired N 2 -reduction catalysts.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Waning immunity and IgG4 responses following bivalent mRNA boosting

Messenger RNA (mRNA) vaccines were highly effective against the ancestral SARS-CoV-2 strain, but the efficacy of bivalent mRNA boosters against XBB variants was substantially lower. Here, we show limited durability of neutralizing antibody (NAb) responses against XBB variants and isotype switching to immunoglobulin G4 (IgG4) responses following bivalent mRNA boosting. Bivalent mRNA boosting elicited modest XBB.1-, XBB.1.5-, and XBB.1.16-specific NAbs that waned rapidly within 3 months. In contrast, bivalent mRNA boosting induced more robust and sustained NAbs against the ancestral WA1/2020 strain, suggesting immune imprinting. Following bivalent mRNA boosting, serum antibody responses were primarily IgG2 and IgG4 responses with poor Fc functional activity. In contrast, a third monovalent mRNA immunization boosted all isotypes including IgG1 and IgG3 with robust Fc functional activity. These data show substantial immune imprinting for the ancestral spike and isotype switching to IgG4 responses following bivalent mRNA boosting, with important implications for future booster designs and boosting strategies.

Science & Technology - Other Topics↗
DOE OSTI2026

Precise Modeling of a Complex Solenoidal Magnetic Field Using a Combination of Analytic Functions and a PINN

We demonstrate an iterative approach to modeling a sparsely measured magnetic field in a large-bore solenoid. This approach uses a hybrid of traditional and machine learning techniques. The traditional technique is a linear least-squares fit using a series solution to Laplace's equation, while the machine learning technique involves the training of a physics-informed neural network (PINN) on the least-squares fit residuals. We use a newly defined activation function "DELTAsnake," a modification to the snake activation function proposed by Ziyin et al. that allows for stronger curvature and non-monotonicity. The combined model approximately obeys Maxwell's equations to a level sufficient for producing high quality physics simulations and analysis. Our approach is applied to a highly realistic calculation of the expected magnetic field in the Mu2e experiment's Detector Solenoid which includes a simple model for the expected statistical measurement uncertainties. Using ten toy measurement simulations, we demonstrate the capabilities of our model in comparison to the least-squares method alone; the least-squares method alone results in a reduced chi-squared statistic of ${2.15 \pm 0.01}$, while our approach improves the reduced chi-square to ${1.034 \pm 0.005}$. Furthermore, for an average toy simulation, we show that the range of the RMS of the three field component residuals reduces from ${0.07-0.37}$ Gauss to ${0.05-0.07}$ Gauss. We find that this novel method is robust against a realistic systematic uncertainty deriving from Hall probe calibration bias and can be used to significantly reduce the number of measurements required to achieve an accurate model.

Kampa, Cole [Caltech] (ORCID:0000000192972920)↗
NASA NTRS1980

Effect of sharply lowered muscular activity on the thyroid gland of the white rat

The effect of hypokinesia on the thyroid gland of 200 white rats was studied. The rats were kept in 16x6x6 cm cages for 90 days. The functional activity of the thyroids increased after 24 hrs of partial immobilization and peaked after 15 days. After 30 days of immobilization, the functional activity returned to normal in one third of the test animals and after 60 days in all animals. After 15 days of immobilization, the test animals began to lose weight (in comparison to the controls) and remained underweight for the rest of the test period (up to 90 days). When returned to normal conditions, they caught up with and even overtook in weight the control animals after about 1 month. All changes produced by hypokinesia were reversible after 1 month.

Bekishev, K.↗
DOE OSTI2024

Sound-localization-related activation and functional connectivity of dorsal auditory pathway in relation to demographic, cognitive, and behavioral characteristics in age-related hearing loss

Background Patients with age-related hearing loss (ARHL) often struggle with tracking and locating sound sources, but the neural signature associated with these impairments remains unclear. Materials and methods Using a passive listening task with stimuli from five different horizontal directions in functional magnetic resonance imaging, we defined functional regions of interest (ROIs) of the auditory “where” pathway based on the data of previous literatures and young normal hearing listeners ( n = 20). Then, we investigated associations of the demographic, cognitive, and behavioral features of sound localization with task-based activation and connectivity of the ROIs in ARHL patients ( n = 22). Results We found that the increased high-level region activation, such as the premotor cortex and inferior parietal lobule, was associated with increased localization accuracy and cognitive function. Moreover, increased connectivity between the left planum temporale and left superior frontal gyrus was associated with increased localization accuracy in ARHL. Increased connectivity between right primary auditory cortex and right middle temporal gyrus, right premotor cortex and left anterior cingulate cortex, and right planum temporale and left lingual gyrus in ARHL was associated with decreased localization accuracy. Among the ARHL patients, the task-dependent brain activation and connectivity of certain ROIs were associated with education, hearing loss duration, and cognitive function. Conclusion Consistent with the sensory deprivation hypothesis, in ARHL, sound source identification, which requires advanced processing in the high-level cortex, is impaired, whereas the right–left discrimination, which relies on the primary sensory cortex, is compensated with a tendency to recruit more resources concerning cognition and attention to the auditory sensory cortex. Overall, this study expanded our understanding of the neural mechanisms contributing to sound localization deficits associated with ARHL and may serve as a potential imaging biomarker for investigating and predicting anomalous sound localization.

Wu, Junzhi↗
NASA NTRS1985

The human role in space: Technology, economics and optimization

Man-machine interactions in space are explored in detail. The role and the degree of direct involvement of humans that will be required in future space missions are investigated. An attempt is made to establish valid criteria for allocating functional activities between humans and machines and to provide insight into the technological requirements, economics, and benefits of the human presence in space. Six basic categories of man-machine interactions are considered: manual, supported, augmented, teleoperated, supervised, and independent. Appendices are included which provide human capability data, project analyses, activity timeline profiles and data sheets for 37 generic activities, support equipment and human capabilities required in these activities, and cumulative costs as a function of activity for seven man-machine modes.

Hall, S. B.↗
NASA NTRS1994

Preconditioning electromyographic data for an upper extremity model using neural networks

A back propagation neural network has been employed to precondition the electromyographic signal (EMG) that drives a computational model of the human upper extremity. This model is used to determine the complex relationship between EMG and muscle activation, and generates an optimal muscle activation scheme that simulates the actual activation. While the experimental and model predicted results of the ballistic muscle movement are very similar, the activation function between the start and the finish is not. This neural network preconditions the signal in an attempt to more closely model the actual activation function over the entire course of the muscle movement.

Roberson, D. J.↗
NASA NTRS2001

Expression of extracellular calcium-sensing receptor in human osteoblastic MG-63 cell line

We have previously shown the expression of the extracellular calcium (Ca2+o)-sensing receptor (CaR) in osteoblast-like cell lines, and others have documented its expression in sections of murine, bovine, and rat bone. The existence of the CaR in osteoblasts remains controversial, however, since some studies have failed to document its expression in the same osteoblast-like cell lines. The goals of the present study were twofold. 1) We sought to determine whether the CaR is expressed in the human osteoblast-like cell line, MG-63, which has recently been reported by others not to express this receptor. 2) We investigated whether the CaR, if present in MG-63 cells, is functionally active, since most previous studies have not proven the role of the CaR in mediating known actions of Ca2+o on osteoblast-like cells. We used immunocytochemistry and Western blotting with the specific, affinity-purified anti-CaR antiserum 4637 as well as Northern blot analysis and RT-PCR using a riboprobe and PCR primers specific for the human CaR, respectively, to show readily detectable CaR protein and mRNA expression in MG-63 cells. Finally, we employed the patch-clamp technique to show that an elevation in Ca2+o as well as the specific, allosteric CaR activator NPS R-467 (0.5 microM), but not its less active stereoisomer NPS S-467 (0.5 microM), activate an outward K+ channel in MG-63 cells, strongly suggesting that the CaR in MG-63 cells is not only expressed but is functionally active.

NASA Discipline Regulatory Physiology↗
DOE OSTI2025

Uncertainty propagation in feed-forward neural network models

We develop new uncertainty propagation methods for feed-forward neural network architectures with leaky ReLU activation functions subject to random perturbations in the input vectors. In particular, we derive analytical expressions for the probability density function (PDF) of the neural network output and its statistical moments as a function of the input uncertainty and the parameters of the network, i.e., weights and biases. A key finding is that an appropriate linearization of the leaky ReLU activation function yields accurate statistical results even for large perturbations in the input vectors. This can be attributed to the way information propagates through the network. We also propose new analytically tractable Gaussian copula surrogate models to approximate the full joint PDF of the neural network output. To validate our theoretical results, we conduct Monte Carlo simulations and a thorough error analysis on a multi-layer neural network representing a nonlinear integro-differential operator between two polynomial function spaces. Our findings demonstrate excellent agreement between the theoretical predictions and Monte Carlo simulations.

MLP networks↗