Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “activation function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

In Situ Attenuated Total Reflectance Infrared Spectroelectrochemistry (ATR-IR-SEC) for the Characterization of Molecular Redox Processes on Surface-Proximal Doped Silicon ATR Crystal Working Electrodes

In situ mid-infrared spectroscopy is a powerful technique for understanding the mechanism of CO 2 reduction (CO 2 R) catalysts because it enables the direct detection of catalytic intermediates and products. Moreover, spectroelectrochemistry (SEC), the coupling of spectroscopy with electrochemistry, allows spectroscopic changes to be correlated with applied potentials to reveal potential-dependent intermediates that are often relevant to photoelectrochemical reactions. Hybrid photoelectrodes, comprised of a narrow bandgap semiconductor, like silicon (Si), with a covalently-linked molecular catalyst, are a promising platform for sunlight-driven catalysis, but characterization of the catalytic mechanism(s) is challenging under photoelectrochemical conditions, particularly when the catalyst is present in monolayer or less concentrations. Here, we have developed a new strategy to use multiple-reflection attenuated total reflectance IR spectroscopy (ATR-IR) coupled with electrochemistry to characterize catalysts directly integrated with a semiconductor surface under applied potential. We show that by surface-proximal n-type or p-type doping of the top ~100-200 nm of the crystal surface, Si ATR crystals can be used simultaneously as the internal reflection element and semiconductor working electrode for ATR-IR-SEC measurements. The surface-proximal doping strategy yields a quasi-equipotential surface with excellent infrared transparency that would have been compromised by free carrier absorption if the crystal was uniformly doped. This approach permits the catalytically-active functionalized surface to be directly probed without modification and overcomes signal-to-noise limitations of other strategies that use separately deposited working electrodes on Si ATR crystals. Proof-of-concept ATR-IR-SEC spectra were collected during the reduction and oxidation of monolayers of Re- and Ru-based transition metal carbonyl complexes, respectively, verifying the viability of the technique to probe redox processes associated with CO 2 R catalysts on Si electrode surfaces with high sensitivity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Annihilation-limited long-range exciton transport in high-mobility conjugated copolymer films

A combination of ultrafast, long-range, and low-loss excitation energy transfer from the photoreceptor location to a functionally active site is essential for cost-effective polymeric semiconductors. Delocalized electronic wavefunctions along π-conjugated polymer (CP) backbone can enable efficient intrachain transport, while interchain transport is generally thought slow and lossy due to weak chain–chain interactions. In contrast to the conventional strategy of mitigating structural disorder, amorphous layers of rigid CPs, exemplified by highly planar poly(indacenodithiophene-co-benzothiadiazole) (IDT-BT) donor-accepter copolymer, exhibit trap-free transistor performance and charge-carrier mobilities similar to amorphous silicon. Here, we report long-range exciton transport in HJ -aggregated IDTBT thin-film, in which the competing exciton transport and exciton–exciton annihilation (EEA) dynamics are spectroscopically separated using a phase-cycling-based scheme and shown to depart from the classical diffusion-limited and strong-coupling regime. In the thin film, we find an annihilation-limited mechanism with ≪100% per-encounter annihilation probability, facilitating the minimization of EEA-induced excitation losses. In contrast, excitons on isolated IDTBT chains diffuse over 350 nm with 0.56 cm 2 s −1 diffusivity, before eventually annihilating with unit probability on first contact. We complement the pump–probe studies with temperature-dependent photocurrent and EEA measurements from 295 K to 77 K and find a remarkable correspondence of annihilation rate and photocurrent activation energies in the 140 K to 295 K temperature range.

Shi, Yuping (ORCID:0000000318289415)↗
DOE OSTI2023

Drought Shifts Sorghum Root Metabolite and Microbiome Profiles and Enriches for Pipecolic Acid

Plant-associated microbial communities shift in composition as a result of environmental perturbations, such as drought. It has been shown that Actinobacteria are enriched in plant roots and rhizospheres during drought stress. However, the correlations between microbiome dynamics and plant response to drought are poorly understood. Here we apply a combination of bacterial community composition analysis and plant metabolite profiling in Sorghum bicolor roots, rhizospheres, and soil during drought and drought recovery to investigate potential contributions of host metabolism to shifts in bacterial composition. Our results provide a detailed view of metabolic shifts across the plant root during drought and show that the response to rewatering differs between root and soil; additionally, we identify drought-responsive metabolites that are highly correlated with the observed changes in Actinobacteria abundance. Furthermore, we find that pipecolic acid is a drought-enriched metabolite in sorghum roots, and that exogenous application of pipecolic acid inhibits root growth. Finally, we show that this activity functions independent of the systemic acquired resistance pathway and has the potential to impact Actinobacterial taxa within the root microbiome.

165 RRNA↗
DOE OSTI2023

Raman Wavelength Conversion in Ionic Liquids

Here, we explore the use of room-temperature ionic liquids (ILs) as Raman wavelength converters. ILs provide an engineerable framework to design suitable liquids for wavelength conversion over a broad spectral range, through careful selection of the molecular structures of the IL anions and cations so that specific characteristics can be obtained, such as a desirable Raman shift, low Brillouin scattering, and good optical transmission in the pump and Stokes wavelengths. Applying such criteria, we demonstrate that 1-ethyl-3-methylimidazolium dicyanamide (EMIM DCA) is an effective medium for conversion of 532-nm pulses from a Q-switched Nd :YAG laser to 603 nm. This corresponds to an approximate 2200 cm –1 shift, which can be used to generate mid-infrared radiation through subsequent difference frequency generation for optical pumping of CO 2 lasers. Threefold-higher Raman conversion efficiency is obtained in EMIM DCA compared with water under identical conditions in a proof-of-principle single-pass conversion setup, resulting in an efficient generation of multimillijoule, <6 ns duration, high-quality orange laser pulses in a wavelength region that is difficult to access at high energies. Consequently, we examine ILs representing two other classes of Raman-active functional groups and obtain conversion up to the fifth-order Stokes shift and first anti-Stokes shift. Through the tunable selection of their components and their useful dynamical properties, ILs provide a platform for efficient, simple, and alignment-tolerant high-energy Raman shifting with numerous industrial and technological applications.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

A Bilevel Voltage Regulation Operation for Distribution Systems With Self-Operated Microgrids

The emerging of microgrids in distribution systems has significantly enhanced the resilience of power grids. However, the operators of a distribution system and microgrids therein can be different and have accessibility to different devices. To model the operation of such a grid, this work proposes a bilevel formulation and probes into the voltage regulation operation, considering the interaction between different systems. The proposed bilevel formulation considers the cooperation of active energy resources (AER), transformer tap-changers, and capacitor banks that are controlled by different operators. To facilitate the solution time of the target bilevel optimization, the lower-level problems with different objectives are modeled using deep neural networks (DNNs) which are then converted into a set of constraints. Hence, the bilevel problem can be reformed to a single-level problem. Lastly, the proposed solution procedures are validated using a customized joint system constructed by the IEEE 123-bus system and a real distribution system in Iowa. According to the numerical validation results, the solution time of the proposed nonlinear activation function based DNN model is 69 times faster than other methods in solving voltage regulation with a bilevel structure.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Artificial to Spiking Neural Networks Conversion with Calibration in Scientific Machine Learning

Here, we introduce a method to convert physics-informed neural networks (PINNs), commonly used in scientific machine learning, to spiking neural networks (SNNs), which are expected to have higher energy efficiency compared to traditional artificial neural networks (ANNs). We first extend the calibration technique of SNNs to arbitrary activation functions beyond ReLU, making it more versatile, and we prove a theorem that ensures the effectiveness of the calibration. We successfully convert PINNs to SNNs, enabling computational efficiency for diverse regression tasks in solving multiple differential equations, including the unsteady Navier–Stokes equations. We demonstrate great gains in terms of overall efficiency, including separable PINNs (SPINNs), which accelerate the training process. Overall, this is the first work of this kind and the proposed method achieves relatively good accuracy with low spike rates.

PINN↗
DOE OSTI2023

Architecture-Preserving Provable Repair of Deep Neural Networks

Deep neural networks (DNNs) are becoming increasingly important components of software, and are considered the state-of-the-art solution for a number of problems, such as image recognition. However, DNNs are far from infallible, and incorrect behavior of DNNs can have disastrous real-world consequences. This paper addresses the problem of architecture-preserving V-polytope provable repair of DNNs. A V-polytope defines a convex bounded polytope using its vertex representation. V-polytope provable repair guarantees that the repaired DNN satisfies the given specification on the infinite set of points in the given V-polytope. An architecture-preserving repair only modifies the parameters of the DNN, without modifying its architecture. The repair has the flexibility to modify multiple layers of the DNN, and runs in polynomial time. It supports DNNs with activation functions that have some linear pieces, as well as fully-connected, convolutional, pooling and residual layers. To the best our knowledge, this is the first provable repair approach that has all of these features. We implement our approach in a tool called APRNN. Using MNIST, ImageNet, and ACAS Xu DNNs, we show that it has better efficiency, scalability, and generalization compared to PRDNN and REASSURE, prior provable repair methods that are not architecture preserving.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II

Deep neural network (DNN) potentials are an emerging tool for simulation of dynamical atomistic systems, with the promise of quantum mechanical accuracy at speedups of 10000$\times$. As with other DNN methods, hyperparameters used during training can make a substantial difference in model accuracy, and optimal settings vary with dataset. To enable rapid tuning of hyperparameters for DNN potential training, we developed a scalable multiobjective optimization evolutionary algorithm for supercomputers and tested it on the Summit system at the Oak Ridge Leadership Computing Facility (OLCF). The multiobjective approach is required due to the coupling of two learned values defining the potential: the energy and force. Using a large-scale implementation of the NSGA-II algorithm adapted for training DNN potentials, we discovered several optimal multiobjective combinations, including best choices of activation functions, learning rate scaling scheme, and pairing of the two radial cutoffs used in the three dimensional descriptor function.

Coletti, Mark↗
DOE OSTI2024

Emulator-Based Bayesian Calibration of the CISNET Colorectal Cancer Models

Purpose To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET)'s SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.Methods We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANNs) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.Results The optimal ANN for SimCRC had 4 hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had 1 hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 h for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.Conclusions Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, such as the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating 3 realistic CRC individual-level models using a Bayesian approach.

artificial neural networks↗
DOE OSTI2021

Biophysical characterization and a roadmap towards the NMR solution structure of G0S2, a key enzyme in non-alcoholic fatty liver disease

In the United States non-alcoholic fatty liver disease (NAFLD) is the most common form of chronic liver disease, affecting an estimated 80 to 100 million people. It occurs in every age group, but predominantly in people with risk factors such as obesity and type 2 diabetes. NAFLD is marked by fat accumulation in the liver leading to liver inflammation, which may lead to scarring and irreversible damage progressing to cirrhosis and liver failure. In animal models, genetic ablation of the protein G0S2 leads to alleviation of liver damage and insulin resistance in high fat diets. The research presented in this paper aims to aid in rational based drug design for the treatment of NAFLD by providing a pathway for a solution state NMR structure of G0S2. Here we describe the expression of G0S2 in an E. coli system from two different constructs, both of which are confirmed to be functionally active based on the ability to inhibit the activity of Adipose Triglyceride Lipase. In one of the constructs, preliminary NMR spectroscopy measurements show dominant alpha-helical characteristics as well as resonance assignments on the N-terminus of G0S2, allowing for further NMR work with this protein. Additionally, the characterization of G0S2 oligomers are outlined for both constructs, suggesting that G0S2 may defensively exist in a multimeric state to protect and potentially stabilize the small 104 amino acid protein within the cell. This information presented on the structure of G0S2 will further guide future development in the therapy for NAFLD.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Understanding the Inner-Workings of Language Models Through Representation Dissimilarity

We use model stitching to understand the internal representations of language models. Similar to vision models, we find that "more is better," and representations learned with more data and larger width can improve the performance of weaker models via stitching. We likewise find that certain architecture choices, using GeLU vs SoLU activation functions, influence the quality of learned representations. Finally, model stitching (as opposed to other model diagnostic methods, like mode connectivity) can localize the different generalization strategies of text classifiers under domain shift to certain hidden layers.

Brown, Davis R.↗
DOE OSTI2023

Toward a Photomagnetic Mechanism for f-element separations (Final Technical Report)

The intent behind this proposal was to evaluate magnetic field effects of rare-earth ions on radical pairs. Rare-earth ions are notoriously challenging to separate based on similar thermodynamic characteristics (e.g. solubility) or ion radius, whereas magnetism varies starkly from rare-earth to rare-earth, even when ions are immediately adjacent on the periodic table. Magnetism could be a useful property then for separations, but a mechanism that is effective and selective for magnetic characteristics must be produced. A reactive mechanism for separations could likewise be interesting for selective separations, if a magnetic handle for such a separation could be defined. This proposal sought to understand a specific reactive, magnetic handle for a reactive separations scheme by creating proof of concept model complexes that place rare earths in proximity to photochemically active functional groups.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

BiomeProbe [Abstract]

We propose the development and application of a microtiter plate-based chemical probe assay that can facilitate the rapid quantification of fecal microbiome-associated enzyme functional activities (“gut pharmacomicrobiomics”). Our initial proof-of-concept project with FedImpact is to show that one or more glucuronidase probes can be used for sensitive and reproducible characterization of fecal microbe glucuronidase activities. Glucuronidase enzymes are important for contributing to enterohepatic recycling of therapeutic drug compounds, resulting in impaired or prolonged drug activity. Importantly, the assay to be developed will permit facile ‘swapping’ of chemical probes to characterize different and multiple enzyme activities. Similarly, the assay is naïve to the biological sample to be analyzed.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Leveraging Artificial Intelligence to Predict Novel Eutectic Alloys

The goal of this project was to train an artificial neural network (ANN) to predict the fractional composition and melting point of eutectic alloys using fundamental atomic properties as inputs. The fundamental properties considered include atomic number, atomic weight, atomic radius, valence electron concentration, electronegativity, and electron affinity. The project involved several phases, starting with data preparation, where phase diagram data was harvested from the ASM International database. Approximately 1300 binary eutectics were collected and cleaned to ensure relevance and accuracy. A regression model was selected for training, utilizing a rectified linear unit as the activation function. Various model configurations were evaluated for predictive accuracy, with validation techniques employed to ensure robustness. The model demonstrated predictive capabilities above random guessing and was able to achieve up to 11% accuracy under certain conditions. An ablative test identified atomic radius and valence electron concentration as critical inputs for model performance. Incorporating the melting point of atomic constituents improved accuracy significantly, although ultimately the model’s predictive capability still fell short of the 80% target. This report details the methodology, results, and implications of the research, contributing to the understanding of employing artificial intelligence to predict the phase transition behavior of eutectic alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Early Drug Discovery and Development of Novel Cancer Therapeutics Targeting DNA Polymerase Eta (POLH)

Polymerase eta (or Pol η or POLH) is a specialized DNA polymerase that is able to bypass certain blocking lesions, such as those generated by ultraviolet radiation (UVR) or cisplatin, and is deployed to replication foci for translesion synthesis as part of the DNA damage response (DDR). Inherited defects in the gene encoding POLH (a.k.a., XPV) are associated with the rare, sun-sensitive, cancer-prone disorder, xeroderma pigmentosum, owing to the enzyme’s ability to accurately bypass UVR-induced thymine dimers. In standard-of-care cancer therapies involving platinum-based clinical agents, e.g., cisplatin or oxaliplatin, POLH can bypass platinum-DNA adducts, negating benefits of the treatment and enabling drug resistance. POLH inhibition can sensitize cells to platinum-based chemotherapies, and the polymerase has also been implicated in resistance to nucleoside analogs, such as gemcitabine. POLH overexpression has been linked to the development of chemoresistance in several cancers, including lung, ovarian, and bladder. Co-inhibition of POLH and the ATR serine/threonine kinase, another DDR protein, causes synthetic lethality in a range of cancers, reinforcing that POLH is an emerging target for the development of novel oncology therapeutics. Using a fragment-based drug discovery approach in combination with an optimized crystallization screen, we have solved the first X-ray crystal structures of small novel drug-like compounds, i.e., fragments, bound to POLH, as starting points for the design of POLH inhibitors. The intrinsic molecular resolution afforded by the method can be quickly exploited in fragment growth and elaboration as well as analog scoping and scaffold hopping using medicinal and computational chemistry to advance hits to lead. An initial small round of medicinal chemistry has resulted in inhibitors with a range of functional activity in an in vitro biochemical assay, leading to the rapid identification of an inhibitor to advance to subsequent rounds of chemistry to generate a lead compound. Importantly, our chemical matter is different from the traditional nucleoside analog-based approaches for targeting DNA polymerases.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

An Accuracy-Maximization Approach for Claims Classifiers in Document Content Analytics for Cybersecurity

This paper presents our research approach and findings towards maximizing the accuracy of our classifier of feature claims for cybersecurity literature analytics, and introduces the resulting model ClaimsBERT. Its architecture, after extensive evaluations of different approaches, introduces a feature map concatenated with a Bidirectional Encoder Representation from Transformers (BERT) model. We discuss deployment of this new concept and the research insights that resulted in the selection of Convolution Neural Networks for its feature mapping aspects. We also present our results showing ClaimsBERT to outperform all other evaluated approaches. This new claims classifier represents an essential processing stage within our vetting framework aiming to improve the cybersecurity of industrial control systems (ICS). Furthermore, in order to maximize the accuracy of our new ClaimsBERT classifier, we propose an approach for optimal architecture selection and determination of optimized hyperparameters, in particular the best learning rate, number of convolutions, filter sizes, activation function, the number of dense layers, as well as the number of neurons and the drop-out rate for each layer. Fine-tuning these hyperparameters within our model led to an increase in classification accuracy from 76% obtained with BertForSequenceClassification’s original model to a 97% accuracy obtained with ClaimsBERT.

Ameri, Kimia (ORCID:0000000328791871)↗
DOE OSTI2026

In Vitro Selection of Antibodies Targeting Yersinia pestis Membrane Lipids Using Nanodisc-Based Antigen Presentation

Proteins are the most common targets for antibody discovery and vaccine development, but their sequence variability can limit the breadth of resulting antigens. Lipids represent an alternative class of antigens due to their structural conservation and roles in host–pathogen interactions. Here, we describe the development and optimization of an in vitro antibody selection workflow using lipid-containing nanodiscs as antigen presentation platforms to enable phage and yeast display selections under conditions adapted for these non-protein targets. Lipopolysaccharide (LPS) nanodiscs were first used as a model system to evaluate selection strategies, including competitive and subtractive approaches to reduce non-specific binders, yielding peptide and single-chain variable fragment (scFv) binders that were affinity matured to improve binding signals. The same approach was subsequently used to select scFv antibodies that recognize lipid nanodiscs prepared from Yersinia pestis membrane lipid extracts. These antibodies show binding to lipid nanodiscs derived from Y. pestis, with evidence of selectivity relative to control nanodiscs. Overall, this work establishes a workflow for antibody selection against lipid-containing nanodisc antigens and highlights practical considerations associated with these targets. The approach may be useful for generating affinity reagents to membrane-associated lipids, although further characterization is required to define antigen specificity and functional activity.

59 BASIC BIOLOGICAL SCIENCES↗