Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “label transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗
NASA NTRS1997

Elucidation of the Cross-Link Structure of Nadic-End-Capped Polyimides Using NMR of C-13-Labeled Polymers

Solid NMR of C-13 isotope-labeled samples of PMR-15 was used to follow the cross-linking reaction of the nadic end cap. Some samples were labeled on one of the carbon atoms of the nadic end cap, and others on the methylene carbon atom of the methylenedianiline portion of the polymer. NMR spectra were run on these samples both before and after cross-linking. In this way, direct evidence of the major products of cross-linking under normal cure conditions is provided. The majority (approximately 85%) of the cross-linking derives from olefin polymerization through the double bond of the end cap. Approximately 15% of the products could come from a pathway involving a retro-Diels-Alder reaction. However, all of the products could be explained by a biradical intermediate without a retro-Diels-Alder reaction. Evidence is also presented that the methylene moiety in the methylenedianiline part of the polymer chain also participates in the cross-linking, albeit to a small extent, by a radical transfer reaction. Different cure conditions (higher temperatures, longer times) could change the relative distribution of the products.

Meador, Mary Ann B.↗
DOE OSTI2026

Design and Characterization of the 162.5 MHz RF Component Layout for Fermilab's PIP-II Reference Line

The Proton Improvement Plan II (PIP-II) Reference Line at Fermi National Accelerator Laboratory distributes phase-stable radio-frequency (RF) signals throughout the accelerator. PIP-II requires stable timing and phase reference signals so its accelerating cavities transfer energy to the particle beam at the correct point in each RF cycle. The full system includes 162.5, 325, and 650 MHz sections corresponding to the frequency sections of the PIP-II Linac, with this project focusing on the 162.5 MHz section. The Reference Line must provide a phase stable source signal while responding to phase changes caused by environmental conditions or system drift. Its RF components will be mounted on aluminum heat plates inside a temperature controlled enclosure to further limit temperature-driven phase changes. To prepare the system for manufacture, the project reviewed component functions and dimensions, developed a computer-aided design (CAD) model, and arranged the hardware to support short cable paths, grounding, fastener access, and maintenance. Several full scale, three dimensional printed prototypes allowed the available components to be mounted and inspected. These fit checks revealed mechanical conflicts and guided revisions to component placement, countersink geometry, labeling, and plate thickness. Electrical characterization was also performed on selected RF hardware to compare its measured behavior with the performance metrics that were set for our design. Overall, the project produced a manufacturable 162.5 MHz layout, physical fit-check prototypes, and documented electrical measurements that support review before metal fabrication. The 325 and 650 MHz layouts remain future work because they require additional minor mechanical changes.

Subedi, Harsheet [Unlisted, US, CA] (ORCID:0009000↗
NASA NTRS2017

Recent Radiation Test Results for Trench Power MOSFETs

Single-event effect (SEE) radiation test results are presented for various trench-gate power MOSFETs. The heavy-ion response of the first (and only) radiation-hardened trench-gate power MOSFET is evaluated: the manufacturer SEE response curve is verified and importantly, no localized dosing effects are measured, distinguishing it from other, non-hardened trench-gate power MOSFETs. Evaluations are made of n-type commercial and both n- and p-type automotive grade trench-gate device using ions comparable to of those on the low linear energy transfer (LET) side of the iron knee of the galactic cosmic ray spectrum, to explore suitability of these parts for missions with higher risk tolerance and shorter duration, such as CubeSats. Part-to-part variability of SEE threshold suggests testing with larger sample sizes and applying more aggressive derating to avoid on-orbit failures. The n-type devices yielded expected localized dosing effects including when irradiated in an unbiased (0-V) configuration, adding to the challenge of inserting these parts into space flight missions.

Schottky Diodes↗
DOE OSTI2022

Automated defect identification in electroluminescence images of solar modules

Solar photovoltaic (PV) modules are susceptible to manufacturing defects, mishandling problems or extreme weather events that can limit energy production or cause early device failure. Trained professionals use electroluminescence (EL) images to identify defects in modules, however, field surveys or inline image acquisition can generate millions of EL images, which are infeasible to analyze by rote inspection. Here, we develop a rapid automatic computer vision pipeline (~0.5 seconds/module) to analyze EL images and identify defects including cracks, intra-cell defects, oxygen-induced defects, and solder disconnections. Defect identification is achieved with a machine learning model (Random Forest, ResNet models and YOLO) trained on 762 manually-labeled EL images of PV modules. We compare model performance on an imbalanced real-world validation set containing 134 EL images and determine that ResNet18 and YOLO are the optimal models; we next evaluated these models on a dedicated testing set (129 module images) with resulting macro F1 scores of 0.83 (ResNet18) and 0.78 (YOLO). Using a field EL survey of a PV power plant damaged in a vegetation fire, we analyze 18,954 EL images (2.4 million cells) and inspect the spatial distribution of defects on the solar modules. The results find increased frequency of ‘crack’, ‘solder’ and ‘intra-cell’ defects on the edges of the solar module closest to the ground after fire. We also find an abnormal increase of striation rings on cells which were assumed to be caused mainly in fabrication process. Our methods are published as open-source software. It can also be used to identify other kinds of defects or process different types of solar cells with minor modification on models by transfer learning.

14 SOLAR ENERGY↗
DOE OSTI2023

MnEdgeNet for accurate decomposition of mixed oxidation states for Mn XAS and EELS L2,3 edges without reference and calibration

Accurate decomposition of the mixed Mn oxidation states is highly important for characterizing the electronic structures, charge transfer and redox centers for electronic, and electrocatalytic and energy storage materials that contain Mn. Electron energy loss spectroscopy (EELS) and soft X-ray absorption spectroscopy (XAS) measurements of the Mn L2,3 edges are widely used for this purpose. To date, although the measurements of the Mn L2,3 edges are straightforward given the sample is prepared properly, an accurate decomposition of the mix valence states of Mn remains non-trivial. For both EELS and XAS, 2+, 3+, and 4+ reference spectra need to be taken on the same instrument/beamline and preferably in the same experimental session because the instrumental resolution and the energy axis offset could vary from one session to another. To circumvent this hurdle, in this study, we adopted a deep learning approach and developed a calibration-free and reference-free method to decompose the oxidation state of Mn L2,3 edges for both EELS and XAS. A deep learning regression model is trained to accurately predict the composition of the mix valence state of Mn. To synthesize physics-informed and ground-truth labeled training datasets, we created a forward model that takes into account plural scattering, instrumentation broadening, noise, and energy axis offset. With that, we created a 1.2 million-spectrum database with 1-by-3 oxidation state composition ground truth vectors. The library includes a sufficient variety of data including both EELS and XAS spectra. By training on this large database, our convolutional neural network achieves 85% accuracy on the validation dataset. We tested the model and found it is robust against noise (down to PSNR of 10) and plural scattering (up to t/λ = 1). We further validated the model against spectral data that were not used in training. In particular, the model shows high accuracy and high sensitivity for the decomposition of Mn 3 O 4 , MnO, Mn 2 O 3 , and MnO 2 . The accurate decomposition of Mn 3 O 4 experimental data shows the model is quantitatively correct and can be deployed for real experimental data. Our model will not only be a valuable tool to researchers and material scientists but also can assist experienced electron microscopists and synchrotron scientists in the automated analysis of Mn L edge data.

25 ENERGY STORAGE↗
NASA NTRS2011

Feature Acquisition with Imbalanced Training Data

This work considers cost-sensitive feature acquisition that attempts to classify a candidate datapoint from incomplete information. In this task, an agent acquires features of the datapoint using one or more costly diagnostic tests, and eventually ascribes a classification label. A cost function describes both the penalties for feature acquisition, as well as misclassification errors. A common solution is a Cost Sensitive Decision Tree (CSDT), a branching sequence of tests with features acquired at interior decision points and class assignment at the leaves. CSDT's can incorporate a wide range of diagnostic tests and can reflect arbitrary cost structures. They are particularly useful for online applications due to their low computational overhead. In this innovation, CSDT's are applied to cost-sensitive feature acquisition where the goal is to recognize very rare or unique phenomena in real time. Example applications from this domain include four areas. In stream processing, one seeks unique events in a real time data stream that is too large to store. In fault protection, a system must adapt quickly to react to anticipated errors by triggering repair activities or follow- up diagnostics. With real-time sensor networks, one seeks to classify unique, new events as they occur. With observational sciences, a new generation of instrumentation seeks unique events through online analysis of large observational datasets. This work presents a solution based on transfer learning principles that permits principled CSDT learning while exploiting any prior knowledge of the designer to correct both between-class and withinclass imbalance. Training examples are adaptively reweighted based on a decomposition of the data attributes. The result is a new, nonparametric representation that matches the anticipated attribute distribution for the target events.

Thompson, David R.↗
NASA NTRS1999

Fluorescence Studies of Protein Crystal Nucleation

Fluorescence can be used to study protein crystal nucleation through methods such as anisotropy, quenching, and resonance energy transfer (FRET), to follow pH and ionic strength changes, and follow events occurring at the growth interface. We have postulated, based upon a range of experimental evidence that the growth unit of tetragonal hen egg white lysozyme is an octamer. Several fluorescent derivatives of chicken egg white lysozyme have been prepared. The fluorescent probes lucifer yellow (LY), cascade blue, and 5-((2-aminoethyl)aminonapthalene-1-sulfonic acid (EDANS), have been covalently attached to ASP 101. All crystallize in the characteristic tetragonal form, indicating that the bound probes are likely laying within the active site cleft. Crystals of the LY and EDANS derivatives have been found to diffract to at least 1.7 A. A second group of derivatives is to the N-terminal amine group, and these do not crystallize as this site is part of the contact region between the adjacent 43 helix chains. However derivatives at these sites would not interfere with formation of the 43 helices in solution. Preliminary FRET studies have been carried out using N-terminal bound pyrene acetic acid (Ex 340 nm, Em 376 nm) lysozyme as a donor and LY (Ex -425 nm, Em 525 nm) labeled lysozyme as an acceptor. FRET data have been obtained at pH 4.6, 0.1 M NaAc buffer, at 5 and 7% NaCl, 4 C. The corresponding Csat values are 0.471 and 0.362 mg/ml (approximately 3.3 and approximately 2.5 x 10(exp -5) M respectively). The data at both salt concentrations show a consistent trend of decreasing fluorescence intensity of the donor species (PAA) with increasing total protein concentration. This decrease is more pronounced at 7% NaCl, consistent with the expected increased intermolecular interactions at higher salt concentrations reflected in the lower solubility. The calculated average distance between any two protein molecules at 5 x 10(exp -6) M is approximately 70nm, well beyond the range where any FRET can be expected. Results from these and ongoing studies will be presented.

Pusey, Marc L.↗
DOE OSTI2025

AMF diversity promotes plant community phosphorus acquisition and reduces carbon costs per unit of phosphorus

Plants may benefit from more diverse communities of arbuscular mycorrhizal fungi (AMF), as functional complementarity of AMF may allow for increased resource acquisition, and because a high AMF diversity increases the probability of plants matching with an optimal AMF symbiont. Here, we repeatedly radiolabeled plants and AMF in the glasshouse over c. 9 months to test how AMF species richness (SR) influences the exchange of plant C ( 14 C) for AMF P ( 32 P & 33 P) and resulting shoot nutrients and mass from a biodiversity–ecosystem functioning perspective. Plant P acquisition via AMF increased with sown AMF SR, as did shoot biomass, shoot P, and shoot N. The rate of plant C transferred to AMF for this P (C:P) decreased with sown AMF SR. Plants in plant communities benefit from inoculation with a variety of AMF species via more favorable resource exchange. Surprisingly, this effect did not differ among functionally distinct communities comprised entirely of either legumes, nonlegume forbs, or C3 grasses.

arbuscular mycorrhizal fungi↗
DOE OSTI2022

Mechanism for Acetone and Crotonaldehyde Production during Steam Reforming of Ethanol over La 0.7 Sr 0.3 MnO 3–x Perovskite: Evidence for a Shared C4 Aldol Addition Intermediate

Here a mechanistic study was conducted on the catalytic conversion of ethanol over La 0.7 Sr 0.3 MnO 3–x perovskite catalysts in the presence and absence of water. The study sought insights into the path of C–C coupling toward acetone and crotonaldehyde and also into clarifying whether the lack of previous reports of C–C coupling over La 0.7 Sr 0.3 MnO 3–x (100) could be due to a “pressure gap”. Several types of experiments were performed at 400–800 K: flow experiments with a torr range reactant gas flown over La 0.7 Sr 0.3 MnO 3–x powders; ultra-high vacuum experiments with continuous gas exposures to a La 0.7 Sr 0.3 MnO 3–x (100) single-crystal sample; and torr range continuous gas exposures to a La 0.7 Sr 0.3 MnO 3–x (100) single-crystal sample. When ethanol and water were flown over La 0.7 Sr 0.3 MnO 3–x powders at 400–800 K, the products detected were ethene, acetaldehyde, acetone, crotonaldehyde, CO, CO 2 , and H 2 . Acetone was catalytically produced over both the La 0.7 Sr 0.3 MnO 3–x powder and the La 0.7 Sr 0.3 MnO 3–x (100) single-crystal sample at temperatures of 700–800 K when reaction conditions were on the order of 1 Torr of reactant gas and with an excess of water relative to ethanol (1 ethanol/9 water). Isotopic labeling with deuterium was used to gain insights into the C–C coupling reaction mechanism and paths in species with three and four carbons (C 3 and C 4 species). Additionally, steady-state isotopic transient kinetic analysis (SSITKA) experiments + simulations using carbon labeling of the ethanol feed were performed. Three mechanistic paths were considered for the C–C coupling step: the first two paths, A and B, involve coupling between two intermediates which are both in oxygen vacancies; and the third path, C, involves coupling between one intermediate in an oxygen vacancy and one intermediate outside of an oxygen vacancy. The results suggest that the dominant path to the C 3 product, acetone, depends on the conditions. The less active path (attributed to path A or B) occurs at 600–700 K and involves coupling between two irreversibly bound species. The more active path (attributed to path C) requires an excess of water, becomes dominant at 600–800 K, and involves coupling between one irreversibly bound species and one reversibly bound species. Based on these various observations from experiments and simulations, an elementary step is proposed for acetone formation involving a previously unreported C 4 transition state that is formed after aldol addition. Density functional theory calculations were performed based on this hypothesis, and it confirmed that this specific and previously unreported aldol addition path to acetone does exist and that this path consistent with the experimental data. In this path, C–C formation occurs to create a C 4 intermediate that is bound to an oxygen vacancy, then a hydrogen transfer with C–C bond breaking occurs that results in the production of the acetone molecule. The proposed mechanism is also consistent with the experimental observation that acetone formation has a greater than first-order dependence on the water vapor pressure.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Adaptive Cybersecurity for Distributed Energy Resources (AdCyDER): Online Reinforcement Learning with Stackelberg-Optimized Defenses — Pipeline Architecture, Evaluation Methodology, and Findings from a Synthetic-Data Evaluation

This report documents the design and evaluation of an integrated online-learning pipeline developed within the AdCyDER project for Distributed Energy Resource (DER) cybersecurity. The pipeline couples a Reinforcement Learning (RL) attack classifier — which produces an attack-type probability distribution — with a Stackelberg game-theoretic (GT) defense selector that consumes those distributions alongside SME-encoded priors over (defense, attack) effectiveness pairings and perdefense costs to choose grid-health-preserving defenses. The objective is not attack classification per se but production of distributions that drive effective defense selection through the Stackelberg layer, learned from delayed grid-health feedback rather than labeled attack data. AdCyDER as a whole is broader than the work presented here; this report covers the specific RL/GT loop integration and its evaluation. We present the integrated pipeline (SCADA telemetry with Fronius inverter physics, Suricata IDS, time-windowed aggregation, per-facility LSTM classifier, Stackelberg optimizer, OpenC2 actuators), an experimental campaign of 28 eight-hour iterations across three baseline modes, and a pipeline-ordered diagnostic protocol. The protocol identifies two distinct failure modes within the loop: paired supervised ceilings on the same features establish that the deployed online RL classifier (macro F1 ≈ 0.07) sits at least 4.7× below a same-architecture supervised LSTM (≈ 0.34) and 10–11× below a linear feature-signal ceiling (≈ 0.70–0.79 depending on per-facility isolation), localizing the dominant failure to the training procedure; and the reward signal driving online updates carries weak directional coupling with classifier correctness in the methodology-expected direction (multi-lens convergent: top-decile P(true) records produce more frequent state changes and slightly larger improvements, top-vs-bot Cohen’s 𝑑 ≈ −0.19), but at effect magnitudes too small to drive gradient-based learning at the campaign sample size. The original learning hypothesis is not supported by the data. The primary contributions are the diagnostic methodology — proposed as a transferable falsification protocol for online RL/GT defense pipelines learning from delayed environmental reward — and the open, reproducible experimental infrastructure. We outline reward reformulation as the highest-priority aspirational next step given the underpowered-but-aligned Q6 reading, with hardware-in-the-loop evaluation as the broadest scope-expansion option.

Blakely, Benjamin [Argonne National Laboratory (AN↗
DOE OSTI2021

Bioinspired Activation of N 2 on Asymmetrical Coordinated Fe Grafted 1T MoS 2 at Room Temperature

Inspired by the nitrogen fixation process on MoFe nitrogenase, asymmetrical coordinated Fe grafted onto 1T MoS 2 were successfully synthesized. The unique electron -rich structure with asymmetrical coordination made the 1T Fe 0.1 Mo 0.9 S 2 layered material actively react with water and dinitrogen at room temperature and atmosphere pressure. Subsequently, ammonia can be produced with a yield of 800 pmol (NH 4 + ) g -1 (12.5% yield in mole). The activation, fixation and reduction of dinitrogen were confirmed by isotopically labeled experiments. The location and the specific coordination environment of grafted Fe in Fe-Mo-S were further determined by X-ray absorption spectroscopy analysis. Our work demonstrates that the nitrogen fixation and reduction for ammonia at room temperature without any chemical and electrochemical assistance is distinctly different from traditional bionic -inspired nitrogen fixation process. The mechanism of the activation and reduction of N 2 was further investigated by density functional theory calculation and Raman spectra. Compared with 1T MoS 2 , the enriched electron nature and asymmetrical coordination of Fe in Fe-Mo-S materials play a critical role in the bioinspired activation of N 2 at ambient condition.

1T MoS2↗
DOE OSTI2021

Synergistic Integration of Chemo-Resistive and SERS Sensing for Label-Free Multiplex Gas Detection

Practical sensing applications such as real-time safety alerts and clinical diagnoses require sensor devices to differentiate between various target molecules with high sensitivity and selectivity, yet conventional devices such as oxide-based chemo-resistive sensors and metal-based surface-enhanced Raman spectroscopy (SERS) sensors usually do not satisfy such requirements. In this report a label-free, chemo-resistive/SERS multimodal sensor based on a systematically assembled 3D cross-point multifunctional nanoarchitecture (3D-CMA), which has unusually strong enhancements in both “chemo-resistive” and “SERS” sensing characteristics is introduced. 3D-CMA combines several sensing mechanisms and sensing elements via 3D integration of semiconducting SnO 2 nanowire frameworks and dual-functioning Au metallic nanoparticles. It is shown that the multimodal sensor can successfully estimate mixed-gas compositions selectively and quantitatively at the sub-100 ppm level, even for mixtures of gaseous aromatic compounds (nitrobenzene and toluene) with very similar molecular structures. This is enabled by combined chemo-resistive and SERS multimodal sensing providing complementary information.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

NASA GPM Status and Future Activities

The joint U.S.-Japan Global Precipitation Measurement (GPM) mission is approaching a decade of operations, and continues to pursue research, dataset production, and outreach related to precipitation. Key activities over the last year were the release of an improved “Version 07” of all GPM precipitation and latent heating products, boosting the orbit of the GPM Core Observatory (GPM CO) to 435 km, and improving quality control on precipitation retrievals from the GPM constellation of passive microwave satellites. This presentation summarizes key improvements to the GPM products and provides some examples of the changes between Versions 06 and 07 in algorithm performance. One important operational change that affected Version 07 is that the scanning strategy for the Ka-band radar channel changed in May 2018; all products that depend on Ka were revised to accommodate this change. For example, in Version 07 the Goddard Profiling (GPROF) algorithm has implemented improvements in regions where orographic enhancement and suppression take place and where the surface is snowy/icy, and again covers radiometers reaching back to 1987. The Combined Radar Radiometer Algorithm (CORRA) now incorporates modified drop-size distribution constraints that substantially reduce bias. Revisions to the Convective-Stratiform Heating (CSH) algorithm employ new radiative transfer retrievals as well as accounting for terrain in the vertical coordinates. Each algorithm was adjusted to ensure continuity for each product across the boundary in 2014 between the predecessor Tropical Rainfall Measuring Mission (TRMM) and the GPM CO. The U.S. Science Team’s Integrated Multi-satellitE Retrievals for GPM (IMERG) was upgraded to account for distortions in the probability density function of regional precipitation rates due to weighted averaging in the Kalman filter used for “morphing” the passive microwave data. Maintaining the GPM CO orbital altitude in the the current very active solar cycle has been forcing the use of more fuel than planned and consequently shortening the forecasted life of the mission from the early 2030's to the late 2020's. It was considered vital to regain some of this lifetime to ensure overlap with the upcoming Atmosphere Observing System mission to provide crosscalibration of instruments. To accomplish this, the orbital altitude was raised from 400 to 435 km on 7-8 November 2023. Thereafter, the primary GPM CO algorithms had to be revised to account for the change in observing parameters. By meeting time this action should be complete. Recently, a screening algorithm based on auto-encoding was developed that uncovered 162 orbits (out of the many thousands of orbits across all years and all satellites) of passive microwave retrievals that had highly anomalous values. Removing these defective retrievals has improved the integrity of both the GPROF and IMERG records. However, the nature of the IMERG processing interacted sufficiently badly with the now-discovered anomalous orbits that it was necessary to completely reprocess the IMERG Final Run record, now labeled Version 07B. The presentation also considers major issues that require continued attention, including the use of machine learning algorithms and the operational challenge of swarms of “small”, perhaps short-lived satellites.

GPM↗
DOE OSTI2020

NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations

In the last 50 years there has been a tremendous progress in solving numerically the Navier-Stokes equations using finite differences, finite elements, spectral, and even meshless methods. Yet, in many real cases, we still cannot incorporate seamlessly (multi-fidelity) data into existing algorithms, and for industrial-complexity applications the mesh generation is time consuming and still an art. Moreover, solving ill-posed problems (e.g., lacking boundary conditions) or inverse problems is often prohibitively expensive and requires different formulations and new computer codes. Here, we employ physics-informed neural networks (PINNs), encoding the governing equations directly into the deep neural network via automatic differentiation, to overcome some of the aforementioned limitations for simulating incompressible laminar and turbulent flows. We develop the Navier-Stokes flow nets (NSFnets) by considering two different mathematical formulations of the Navier-Stokes equations: the velocity-pressure (VP) formulation and the vorticity-velocity (VV) formulation. Since this is a new approach, we first select some standard benchmark problems to assess the accuracy, convergence rate, computational cost and flexibility of NSFnets; analytical solutions and direct numerical simulation (DNS) databases provide proper initial and boundary conditions for the NSFnet simulations. The spatial and temporal coordinates are the inputs of the NSFnets, while the instantaneous velocity and pressure fields are the outputs for the VP-NSFnet, and the instantaneous velocity and vorticity fields are the outputs for the VV-NSFnet. This is unsupervised learning and, hence, no labeled data are required beyond boundary and initial conditions and the fluid properties. The residuals of the VP or VV governing equations, together with the initial and boundary conditions, are embedded into the loss function of the NSFnets. No data is provided for the pressure to the VP-NSFnet, which is a hidden state and is obtained via the incompressibility constraint without extra computational cost. Unlike the traditional numerical methods, NSFnets inherit the properties of neural networks (NNs), hence the total error is composed of the approximation, the optimization, and the generalization errors. Here, we empirically attempt to quantify these errors by varying the sampling (“residual”) points, the iterative solvers, and the size of the NN architecture. For the laminar flow solutions, we show that both the VP and the VV formulations are comparable in accuracy but their best performance corresponds to different NN architectures. The initial convergence rate is fast but the error eventually saturates to a plateau due to the dominance of the optimization error. For the turbulent channel flow, we show that NSFnets can sustain turbulence at , but due to expensive training we only consider part of the channel domain and enforce velocity boundary conditions on the subdomain boundaries provided by the DNS data base. We also perform a systematic study on the weights used in the loss function for balancing the data and physics components, and investigate a new way of computing the weights dynamically to accelerate training and enhance accuracy. In the last part, we demonstrate how NSFnets should be used in practice, namely for ill-posed problems with incomplete or noisy boundary conditions as well as for inverse problems. We obtain reasonably accurate solutions for such cases as well without the need to change the NSFnets and at the same computational cost as in the forward well-posed problems. As a result, we also present a simple example of transfer learning that will aid in accelerating the training of NSFnets for different parameter settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

CyBERT: Cybersecurity Claim Classification by Fine-Tuning the BERT Language Model

We introduce CyBERT, a cybersecurity feature claims classifier based on bidirectional encoder representations from transformers and a key component in our semi-automated cybersecurity vetting for industrial control systems (ICS). To train CyBERT, we created a corpus of labeled sequences from ICS device documentation collected across a wide range of vendors and devices. This corpus provides the foundation for fine-tuning BERT’s language model, including a prediction-guided relabeling process. We propose an approach to obtain optimal hyperparameters, including the learning rate, the number of dense layers, and their configuration, to increase the accuracy of our classifier. Fine-tuning all hyperparameters of the resulting model led to an increase in classification accuracy from 76% obtained with BertForSequenceClassification’s original architecture to 94.4% obtained with CyBERT. Furthermore, we evaluated CyBERT for the impact of randomness in the initialization, training, and data-sampling phases. CyBERT demonstrated a standard deviation of ±0.6% during validation across 100 random seed values. Finally, we also compared the performance of CyBERT to other well-established language models including GPT2, ULMFiT, and ELMo, as well as neural network models such as CNN, LSTM, and BiLSTM. The results showed that CyBERT outperforms these models on the validation accuracy and the F1 score, validating CyBERT’s robustness and accuracy as a cybersecurity feature claims classifier.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Microscale Thermophoresis (MST) as a Tool to Study Binding Interactions of Oxygen-Sensitive Biohybrids

Microscale thermophoresis (MST) is a technique used to measure the strength of molecular interactions. MST is a thermophoretic-based technique that monitors the change in fluorescence associated with the movement of fluorescent-labeled molecules in response to a temperature gradient triggered by an IR LASER. MST has advantages over other approaches for examining molecular interactions, such as isothermal titration calorimetry, nuclear magnetic resonance, biolayer interferometry, and surface plasmon resonance, requiring a small sample size that does not need to be immobilized and a high-sensitivity fluorescence detection. In addition, since the approach involves the loading of samples into capillaries that can be easily sealed, it can be adapted to analyze oxygen-sensitive samples. In this Bio-protocol, we describe the troubleshooting and optimization we have done to enable the use of MST to examine protein–protein interactions, protein–ligand interactions, and protein–nanocrystal interactions. The salient elements in the developed procedures include 1) loading and sealing capabilities in an anaerobic chamber for analysis using a NanoTemper MST located on the benchtop in air, 2) identification of the optimal reducing agents compatible with data acquisition with effective protection against trace oxygen, and 3) the optimization of data acquisition and analysis procedures. The procedures lay the groundwork to define the determinants of molecular interactions in these technically demanding systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Computational Imaging for Intelligence in Highly Scattering Aerosols (Final Report)

Natural and man-made degraded visual environments pose major threats to national security. The random scattering and absorption of light by tiny particles suspended in the air reduces situational awareness and causes unacceptable down-time for critical systems and operations. To improve the situation, we have developed several approaches to interpret the information contained within scattered light to enhance sensing and imaging in scattering media. These approaches were tested at the Sandia National Laboratory Fog Chamber facility and with tabletop fog chambers. Computationally efficient light transport models were developed and leveraged for computational sensing. The models are based on a weak angular dependence approximation to the Boltzmann or radiative transfer equation that appears to be applicable in both the moderate and highly scattering regimes. After the new model was experimentally validated, statistical approaches for detection, localization, and imaging of objects hidden in fog were developed and demonstrated. A binary hypothesis test and the Neyman-Pearson lemma provided the highest theoretically possible probability of detection for a specified false alarm rate and signal-to-noise ratio. Maximum likelihood estimation allowed estimation of the fog optical properties as well as the position, size, and reflection coefficient of an object in fog. A computational dehazing approach was implemented to reduce the effects of scatter on images, making object features more readily discernible. We have developed, characterized, and deployed a new Tabletop Fog Chamber capable of repeatably generating multiple unique fog-analogues for optical testing in degraded visual environments. We characterized this chamber using both optical and microphysical techniques. In doing so we have explored the ability of droplet nucleation theory to describe the aerosols generated within the chamber, as well as Mie scattering theory to describe the attenuation of light by said aerosols, and correlated the aerosol microphysics to optical properties such as transmission and meteorological optical range (MOR). This chamber has proved highly valuable and has supported multiple efforts inclusive to and exclusive of this LDRD project to test optics in degraded visual environments. Circularly polarized light has been found to maintain its polarization state better than linearly polarized light when propagating through fog. This was demonstrated experimentally in both the visible and short-wave infrared (SWIR) by imaging targets made of different commercially available retroreflective films. It was found that active circularly polarized imaging can increase contrast and range compared to linearly polarized imaging. We have completed an initial investigation of the capability for machine learning methods to reduce the effects of light scattering when imaging through fog. Previously acquired experimental long-wave images were used to train an autoencoder denoising architecture. Overfitting was found to be a problem because of lack of variability in the object type in this data set. The lessons learned were used to collect a well labeled dataset with much more variability using the Tabletop Fog Chamber that will be available for future studies. We have developed several new sensing methods using speckle intensity correlations. First, the ability to image moving objects in fog was shown, establishing that our unique speckle imaging method can be implemented in dynamic scattering media. Second, the speckle decorrelation over time was found to be sensitive to fog composition, implying extensions to fog characterization. Third, the ability to distinguish macroscopically identical objects on a far-subwavelength scale was demonstrated, suggesting numerous applications ranging from nanoscale defect detection to security. Fourth, we have shown the capability to simultaneously image and localize hidden objects, allowing the speckle imaging method to be effective without prior object positional information. Finally, an interferometric effect was presented that illustrates a new approach for analyzing speckle intensity correlations that may lead to more effective ways to localize and image moving objects. All of these results represent significant developments that challenge the limits of the application of speckle imaging and open important application spaces. A theory was developed and simulations were performed to assess the potential transverse resolution benefit of relative motion in structured illumination for radar systems. Results for a simplified radar system model indicate that significant resolution benefits are possible using data from scanning a structured beam over the target, with the use of appropriate signal processing.

58 GEOSCIENCES↗