Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

ResNet and CycleGAN for pulse shape discrimination of He-4 detector pulses: Recovering pulses conventional algorithms fail to label unanimously

Pulse shape discrimination (PSD) capable detectors, such as He-4, that respond to neutron and gamma-ray 7 interactions have a threshold deposited energy value below which n/γ discrimination vanishes when using 8 conventional PSD algorithms. Recent attempts in applying supervised learning based artificial neural 9 networks for PSD use the pulses in the separated regions to train the networks so they can be used to classify 10 another set of separated pulses. In doing so, pulses previously indistinguishable are not recovered for 11 classification, which would have increased the number of neutron and gamma-ray pulses that could be used 12 for further analysis. Assuming the reason why conventional PSD algorithms have unseparated regions is 13 because the parameter space of the algorithms fail to capture the intrinsic (but subtle) distinguishing 14 behavior of some of the neutron and gamma-ray pulses, a cycle-consistent generative adversarial network 15 (CycleGAN) was trained to amplify those differences and extract well separated neutron and gamma-ray 16 clusters. Results show that, once the network is trained with pulses from separated and unseparated regions, 17 it was able to transform the pulses in the unseparated region to improve the PSD. Subsequent n/γ 18 classification was performed using deep residual network (ResNet) that takes pulses with 512 data points 19 as an input. Two different ResNets were explored – simple ResNet and modified ResNet which takes 20 segmented pulse inputs in the first layer and the corresponding time axis values in the last hidden layer. 21 The later approach enables the network to extract time correlated pulse features to enhance its ability to 22 capture the pulse behaviors relevant for PSD. Although it achieves slightly lower accuracy, 99.41% versus 23 99.89%, based on simply counting the number of correct n/γ labels assigned, compared to the simple 24 ResNet, the modified ResNets architecture was able to decreases the cross-entropy loss function by half, 25 which implies that the correct n/γ labels assigned are less likely to be accidental. PSD parameter 26 distributions based on n/γ classification by ResNet before and after transforming unseparated pulses using 27 CycleGAN show that by enhancing the separation between neutrons and gamma-rays, the transformation 28 helps improve the performance of classifier networks that are trained using labeled dataset. The 29 enhancement of neutron and gamma-ray separation by the CycleGAN increased the PSD figure of merit 30 (FOM) by up to 70% in some regions. Here, the results show that, if a given detector achieves clear separation 31 between neutron and gamma-ray pulses in any energy region, such neural network approaches can help 32 lower the energy threshold for the separation and increasing the number of neutron and gamma-ray pulses 33 that can be used for further analysis.

4He↗

An end-to-end trainable hybrid classical-quantum classifier

Abstract We introduce a hybrid model combining a quantum-inspired tensor network and a variational quantum circuit to perform supervised learning tasks. This architecture allows for the classical and quantum parts of the model to be trained simultaneously, providing an end-to-end training framework. We show that compared to the principal component analysis, a tensor network based on the matrix product state with low bond dimensions performs better as a feature extractor for the input data of the variational quantum circuit in the binary and ternary classification of MNIST and Fashion-MNIST datasets. The architecture is highly adaptable and the classical-quantum boundary can be adjusted according to the availability of the quantum resource by exploiting the correspondence between tensor networks and quantum circuits.

97 MATHEMATICS AND COMPUTING↗

Machine learning based design space exploration for hybrid main-memory design

We develop a machine learning (ML) based design space exploration (DSE) method that builds predictive models for various responses of a hybrid main-memory system. To overcome the challenges associated with latency, capacity, and power of memory systems in future extreme-scale machines, the hybrid memory architectures are being considered in which novel non-volatile memory (NVM) systems augment the traditional DRAM. However, way before their actual design and implementation, these emerging hybrid memory systems need to be simulated and analyzed to fully understand their capabilities and limitations. As the conventional architectural-level memory simulators require significant amounts of computational costs and time, we propose to utilize ML techniques for developing various memory-response models that can instantly provide a predicted response corresponding to any new memory configuration. Specifically, in this work, we apply four supervised ML techniques to build regression models for memory latency, bandwidth, power, and total read/write responses. The training and validation data for the ML methods are generated using NVMain memory simulator for DRAM, NVM, and their hybrid combinations. We demonstrate the results of the ML based memory-DSE method in terms of the learning curve characteristics for hyperparameter tuning and the statistical error analyses of the designed predictive models.

Sen, Satyabrata↗

Any Two Learning Algorithms Are (Almost) Exactly Identical

This paper shows that if one is provided with a loss function, it can be used in a natural way to specify a distance measure quantifying the similarity of any two supervised learning algorithms, even non-parametric algorithms. Intuitively, this measure gives the fraction of targets and training sets for which the expected performance of the two algorithms differs significantly. Bounds on the value of this distance are calculated for the case of binary outputs and 0-1 loss, indicating that any two learning algorithms are almost exactly identical for such scenarios. As an example, for any two algorithms A and B, even for small input spaces and training sets, for less than 2e(-50) of all targets will the difference between A's and B's generalization performance of exceed 1%. In particular, this is true if B is bagging applied to A, or boosting applied to A. These bounds can be viewed alternatively as telling us, for example, that the simple English phrase 'I expect that algorithm A will generalize from the training set with an accuracy of at least 75% on the rest of the target' conveys 20,000 bytes of information concerning the target. The paper ends by discussing some of the subtleties of extending the distance measure to give a full (non-parametric) differential geometry of the manifold of learning algorithms.

Wolpert, David H.↗

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3–20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

42 ENGINEERING↗

The DOE fellows program-a workforce development initiative for the US department of energy

The US Department of Energy Office of Environmental Management (DOE-EM) oversees one of the largest and most technically challenging cleanup programs in the world. The mission of DOE-EM is to complete the safe cleanup of the environmental legacy from five decades of nuclear weapons development and government-sponsored nuclear energy research. Since 1995, Florida International University's Applied Research Center (FIU-ARC) has supported the DOE-EM mission and provided unique research capabilities to address some of these highly technical and difficult challenges. This partnership has allowed FIU-ARC to create a unique infrastructure that is critical for the training and mentoring of science, technology, engineering, and math (STEM) students and has exposed many STEM students to 'hands-on' DOE-EM applied research, supervised by the scientists and engineers at ARC. As a result of this successful partnership between DOE and FIU, DOE requested FIU-ARC to create the DOE-FIU Science and Technology Workforce Development Initiative in 2007. This innovative program was established to create a 'pipeline' of minority STEM students trained and mentored to enter DOE's environmental cleanup workforce. The program was designed to help address DOE's future workforce needs by partnering with academic, government and private companies (DOE contractors) to mentor future minority scientists and engineers in the research, development, and deployment of new technologies and processes addressing DOE's environmental cleanup challenges. Since its inception in 2007, the program has trained and mentored 78 FIU STEM minority students. Although, the program has been in existence for only six years, a total of 75 internships have been conducted at DOE National Laboratories, DOE sites, DOE Headquarters and field offices, and DOE contractors. Over 100 DOE Fellows have participated in the Waste Management (WM) Symposia since 2008 with a total of 84 student posters and 7 oral presentations given at WM. The DOE Fellows participation at WM has resulted in three Best Student Poster Awards (WM09, WM10, and WM11) and one Best Professional Poster Award (WM09). DOE Fellows have also presented their research at ANS DD and R and ANS Robotics Topical meetings and this year two Fellows will present at the International Conference on Environmental Remediation and Radioactive Waste Management (ICEM13) in Brussels, Belgium. Moreover, several of our DOE Fellows have already obtained employment with DOE-EM, other federal agencies, DOE contractors, commercial nuclear power companies, and other STEM industry (GE, Boeing, Lockheed Martin, Johnson and Johnson, Beckman-Coulter, and other top companies). This paper will discuss how DOE Fellows program is training and mentoring FIU STEM students in Department of Energy's Office of Environmental Management technical challenges and research. This training and mentoring has resulted in the development of well-trained and polished young scientists and engineers that will become the future workforce in charge of carrying on DOE-EM's environmental cleanup mission. The paper will showcase FIU's DOE Fellows model and highlight some of the applied research the DOE Fellows have conducted at FIU's Applied Research Center and across the DOE Complex by participating in summer internship assignments. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Training and Mentoring the Next Generation of Scientists and Engineers to Secure Continuity and Successes of the US DOE's Environmental Remediation Efforts - 13387

The DOE Office of Environmental Management (DOE-EM) oversees one of the largest and most technically challenging cleanup programs in the world. The mission of DOE-EM is to complete the safe cleanup of the environmental legacy from five decades of nuclear weapons development and government-sponsored nuclear energy research. Since 1995, Florida International University's Applied Research Center (FIU-ARC) has supported the DOE-EM mission and provided unique research capabilities to address some of these highly technical and difficult challenges. This partnership has allowed FIU-ARC to create a unique infrastructure that is critical for the training and mentoring of science, technology, engineering, and math (STEM) students and has exposed many STEM students to 'hands-on' DOE-EM applied research, supervised by the scientists and engineers at ARC. As a result of this successful partnership between DOE and FIU, DOE requested FIU-ARC to create the DOE-FIU Science and Technology Workforce Development Initiative in 2007. This innovative program was established to create a 'pipeline' of minority STEM students trained and mentored to enter DOE's environmental cleanup workforce. The program was designed to help address DOE's future workforce needs by partnering with academic, government and private companies (DOE contractors) to mentor future minority scientists and engineers in the research, development, and deployment of new technologies and processes addressing DOE's environmental cleanup challenges. Since its inception in 2007, the program has trained and mentored 78 FIU STEM minority students. Although, the program has been in existence for only five years, a total of 75 internships have been conducted at DOE National Laboratories, DOE sites, DOE Headquarters and field offices, and DOE contractors. Over 85 DOE Fellows have participated in the Waste Management Symposia since 2008 with a total of 68 student posters and 7 oral presentations given at WM. The DOE Fellows participation at WM has resulted in three Best Student Poster Awards (WM09, WM10, and WM11) and one Best Professional Poster Award (WM09). DOE Fellows have also presented their research at ANS DD and R and ANS Robotics Topical meetings. Moreover, several of our DOE Fellows have already obtained employment with DOE-EM, other federal agencies, DOE contractors. This paper will discuss how DOE Fellows program is training and mentoring FIU STEM students in Department of Energy's Office of Environmental Management technical challenges and research. This training and mentoring has resulted in the development of well trained and polished young scientists and engineers that will become the future workforce in charge of carrying on DOE-EM's environmental cleanup mission. The paper will showcase FIU's DOE Fellows model and highlight some of the applied research the DOE Fellows have conducted at FIU's Applied Research Center and across the Complex by participating in summer internship assignments. This paper will also present and highlight other Fellowships and internships programs sponsored by National Nuclear Security Agency (NNSA), DOE-EM, NRC, Energy (NE), and other federal agencies targeting workforce development. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

The Center of Excellence for Hypersonics Training and Research at the University of Texas at Austin

Over the period of this grant (1986-92), 23 graduate students were supported by the Center and received education and training in hypersonics through MS and Ph.D. programs. An additional 8 Ph.D. candidates and 2 MS candidates, with their own fellowship support, were attracted to The University of Texas and were recruited into the hypersonics program because of the Center. Their research, supervised by the 10 faculty involved in the Center, resulted in approximately 50 publications and presentations in journals and at national and international technical conferences. To provide broad-based training, a new hypersonics curriculum was created, enabling students to take 8 core classes in theoretical, computational, and experimental hypersonics, and other option classes over a two to four semester period. The Center also developed an active continuing education program. The Hypersonics Short Course was taught 3 times, twice in the USA and once in Europe. Approximately 300 persons were attracted to hear lectures by more than 25 of the leading experts in the field. In addition, a hypersonic aerodynamics short course was offered through AIAA, as well as short courses on computational fluid dynamics (CFD) and advanced CFD. The existence of the Center also enabled faculty to leverage a substantial volume of additional funds from other agencies, for research and graduate student training. Overall, this was a highly successful and highly visible program.

Dolling, David S.↗

Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy

Training machine learning models for radioisotope identification using gamma spectroscopy remains an elusive challenge for many practical applications, largely stemming from the difficulty of acquiring and labeling large, diverse experimental datasets. Simulations can mitigate this challenge, but the accuracy of models trained on simulated data can deteriorate substantially when deployed to an out-of-distribution operational environment. In this study, we demonstrate that unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain, provided unlabeled data from the target domain are available. Conventional supervised techniques are unable to utilize this data because the absence of isotope labels precludes defining a supervised classification loss. Instead, we first pretrain a spectral classifier using labeled synthetic data and subsequently leverage unlabeled target data to align the learned feature representations between the source and target domains. We compare a range of different UDA techniques, finding that minimizing the maximum mean discrepancy (MMD) between source and target feature vectors yields the most consistent improvement to testing scores. For instance, using a custom transformer-based neural network, we achieved a testing accuracy of $0.904 \pm 0.022$ on an experimental LaBr test set after performing unsupervised feature alignment via MMD minimization, compared to $0.754 \pm 0.014$ before alignment. Overall, our results highlight the potential of using UDA to adapt a radioisotope classifier trained on synthetic data for real-world deployment.

Lalor, Peter W.↗

Non-resonant anomaly detection with background extrapolation

Complete anomaly detection strategies that are both signal sensitive and compatible with background estimation have largely focused on resonant signals. Non-resonant new physics scenarios are relatively under-explored and may arise from off-shell effects or final states with significant missing energy. In this paper, we extend a class of weakly supervised anomaly detection strategies developed for resonant physics to the non-resonant case. Machine learning models are trained to reweight, generate, or morph the background, extrapolated from a control region. A classifier is then trained in a signal region to distinguish the estimated background from the data. The new methods are demonstrated using a semi-visible jet signature as a benchmark signal model, and are shown to automatically identify the anomalous events without specifying the signal ahead of time.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Universal linear intensity transformations using spatially incoherent diffractive processors

Abstract Under spatially coherent light, a diffractive optical network composed of structured surfaces can be designed to perform any arbitrary complex-valued linear transformation between its input and output fields-of-view (FOVs) if the total number ( N ) of optimizable phase-only diffractive features is ≥~2 N i N o , where N i and N o refer to the number of useful pixels at the input and the output FOVs, respectively. Here we report the design of a spatially incoherent diffractive optical processor that can approximate any arbitrary linear transformation in time-averaged intensity between its input and output FOVs. Under spatially incoherent monochromatic light, the spatially varying intensity point spread function ( H ) of a diffractive network, corresponding to a given, arbitrarily-selected linear intensity transformation, can be written as H ( m , n ; m ′, n ′) = | h ( m , n ; m ′, n ′)| 2 , where h is the spatially coherent point spread function of the same diffractive network, and ( m , n ) and ( m ′, n ′) define the coordinates of the output and input FOVs, respectively. Using numerical simulations and deep learning, supervised through examples of input-output profiles, we demonstrate that a spatially incoherent diffractive network can be trained to all-optically perform any arbitrary linear intensity transformation between its input and output if N ≥ ~2 N i N o . We also report the design of spatially incoherent diffractive networks for linear processing of intensity information at multiple illumination wavelengths, operating simultaneously. Finally, we numerically demonstrate a diffractive network design that performs all-optical classification of handwritten digits under spatially incoherent illumination, achieving a test accuracy of >95%. Spatially incoherent diffractive networks will be broadly useful for designing all-optical visual processors that can work under natural light.

36 MATERIALS SCIENCE↗

A self-supervised workflow for particle picking in cryo-EM

High-resolution single-particle cryo-EM data analysis relies on accurate particle picking. To facilitate the particle picking process, a self-supervised workflow has been developed. This includes an iterative strategy, which uses a 2D class average to improve training particles, and a progressively improved convolutional neural network for particle picking. To automate the selection of particles, a threshold is defined (%/Res) using the ratio of percentage class distribution and resolution as a cutoff. This workflow has been tested using six publicly available data sets with different particle sizes and shapes, and can automatically pick particles with minimal user input. The picked particles support high-resolution reconstructions at 3.0 Å or better. This workflow is a step towards automated single-particle cryo-EM data analysis at the stage of particle picking. It may be used in conjunction with commonly used single-particle analysis packages such as Relion , cryoSPARC , cisTEM , SPHIRE and EMAN2 .

2D class averages↗

Skiing Simulation

Arvin/Calspan Advanced Technology Center's Dr. Michael Holden is an aerodynamicist who normally works on the reentry characteristics of space vehicles. Calspan donated the use of its wind tunnel for a program aimed at improving the performance of members of the U.S. Ski Team. Dr. Holden, a certified ski instructor, offered to apply his aerodynamics expertise to the design and supervision of the tests. Wind tunnel simulations have become an annual part of the U.S. Ski Team's training for U.S. and international events. Downhill ski racers learn from these simulations which body positions are most effective in which situations.

Source record↗

DCS-Neural-Network Program for Aircraft Control and Testing

A computer program implements a dynamic-cell-structure (DCS) artificial neural network that can perform such tasks as learning selected aerodynamic characteristics of an airplane from wind-tunnel test data and computing real-time stability and control derivatives of the airplane for use in feedback linearized control. A DCS neural network is one of several types of neural networks that can incorporate additional nodes in order to rapidly learn increasingly complex relationships between inputs and outputs. In the DCS neural network implemented by the present program, the insertion of nodes is based on accumulated error. A competitive Hebbian learning rule (a supervised-learning rule in which connection weights are adjusted to minimize differences between actual and desired outputs for training examples) is used. A Kohonen-style learning rule (derived from a relatively simple training algorithm, implements a Delaunay triangulation layout of neurons) is used to adjust node positions during training. Neighborhood topology determines which nodes are used to estimate new values. The network learns, starting with two nodes, and adds new nodes sequentially in locations chosen to maximize reductions in global error. At any given time during learning, the error becomes homogeneously distributed over all nodes.

Jorgensen, Charles C.↗

Dimuon production in the MicroBooNE detector

This analysis studies the production of the rare signal with two muons (dimuon) in the finalstate at µB using the Booster Neutrino Beam (BNB) as source of neutrino interactions. Themain background source for the dimuon signal are pions. Differentiating muons and pions hasbeen challenging at µB because the tracks left by these particles are almost identical. Thisanalysis faces this problem using a supervised classifier algorithm called boosted decisiontree (BDT). A Monte Carlo simulation (MC) enhanced dimuon signal sample was producedto train a BDT capable of recognizing this signal. A set of variables that show somediscrimination characteristics was used to perform the training. A MC neutrino interactionsimulation sample was used to estimate the background. Estimations of dimuon productionin real data samples were made for different cases. Additionally, an analysis of the systematicuncertainties, fake data studies, and BDT performance checks were obtained.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SmileyLlama: modifying large language models for directed chemical space exploration

Here we show that large language models (LLMs) can be transformed via supervised fine-tuning of engineered prompts into SmileyLlama for exploring the chemical space of drug molecules. We benchmark SmileyLlama against pretrained LLMs and chemical language models trained from scratch for generating valid and novel drug-like molecules, and use direct preference optimization to both improve SmileyLlama’s adherence to a prompt and as part of the iMiner reinforcement learning framework to predict molecules with optimized three-dimensional conformations and high binding affinity to drug targets. By training an LLM to speak directly as a chemical language model, while retaining most of its natural language capabilities, we show that SmileyLlama can reliably generate molecules with user-specified properties rather than acting only as a chatbot with knowledge of chemistry or as a virtual assistant. While SmileyLlama is geared toward drug discovery, the supervised fine-tuning/direct preference optimization/LLM framework can be extended to other chemical, biological and materials applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Tree-based algorithms for weakly supervised anomaly detection

Weakly supervised methods have emerged as a powerful tool for model-agnostic anomaly detection at the Large Hadron Collider (LHC). While these methods have shown remarkable performance on specific signatures such as dijet resonances, their application in a more model-agnostic manner requires dealing with a larger number of potentially noisy input features. In this paper, we show that using boosted decision trees as classifiers in weakly supervised anomaly detection gives superior performance compared to deep neural networks. Boosted decision trees are well known for their effectiveness in tabular data analysis. Our results show that they not only offer significantly faster training and evaluation times, but they are also robust to a large number of noisy input features. By using advanced gradient boosted decision trees in combination with ensembling techniques and an extended set of features, we significantly improve the performance of weakly supervised methods for anomaly detection at the LHC. This advance is a crucial step toward a more model-agnostic search for new physics. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

Deep learning for morphological identification of extended radio galaxies using weak labels

Abstract The present work discusses the use of a weakly-supervised deep learning algorithm that reduces the cost of labelling pixel-level masks for complex radio galaxies with multiple components. The algorithm is trained on weak class-level labels of radio galaxies to get class activation maps (CAMs). The CAMs are further refined using an inter-pixel relations network (IRNet) to get instance segmentation masks over radio galaxies and the positions of their infrared hosts. We use data from the Australian Square Kilometre Array Pathfinder (ASKAP) telescope, specifically the Evolutionary Map of the Universe (EMU) Pilot Survey, which covered a sky area of 270 square degrees with an RMS sensitivity of 25–35 $\mu$ Jy beam $^{-1}$ . We demonstrate that weakly-supervised deep learning algorithms can achieve high accuracy in predicting pixel-level information, including masks for the extended radio emission encapsulating all galaxy components and the positions of the infrared host galaxies. We evaluate the performance of our method using mean Average Precision (mAP) across multiple classes at a standard intersection over union (IoU) threshold of 0.5. We show that the model achieves a mAP $_{50}$ of 67.5% and 76.8% for radio masks and infrared host positions, respectively. The network architecture can be found at the following link: https://github.com/Nikhel1/Gal-CAM

Astronomy & Astrophysics↗