MPNet-GRUs: Sentiment Analysis With Masked and Permuted Pre-Training for Language Understanding and Gated Recurrent Units
Not Available
Kai Ning Loh, Nicole↗
SEARCH · Engineering Papers
Results for “Training Analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
FRMAC Gamma Spectroscopist Training : Environmental in-situ batch spectral analysis (Revision 01)
Explore the source record for details and available documents.
Enghauser, Michael (ORCID:0009000470534902)↗
Comparative Study of Large Language Model Architectures on Frontier
Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world’s first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.
Yin, Junqi↗
Classification of Photovoltaic Failures with Hidden Markov Modeling, an Unsupervised Statistical Approach
Failure detection methods are of significant interest for photovoltaic (PV) site operators to help reduce gaps between expected and observed energy generation. Current approaches for field-based fault detection, however, rely on multiple data inputs and can suffer from interpretability issues. In contrast, this work offers an unsupervised statistical approach that leverages hidden Markov models (HMM) to identify failures occurring at PV sites. Using performance index data from 104 sites across the United States, individual PV-HMM models are trained and evaluated for failure detection and transition probabilities. This analysis indicates that the trained PV-HMM models have the highest probability of remaining in their current state (87.1% to 93.5%), whereas the transition probability from normal to failure (6.5%) is lower than the transition from failure to normal (12.9%) states. A comparison of these patterns using both threshold levels and operations and maintenance (O&M) tickets indicate high precision rates of PV-HMMs (median = 82.4%) across all of the sites. Although additional work is needed to assess sensitivities, the PV-HMM methodology demonstrates significant potential for real-time failure detection as well as extensions into predictive maintenance capabilities for PV.
classification↗
Microbial diversity analysis of two full-scale seawater desalination treatment trains provides insights into detrimental biofilm formation
Detrimental biofilms on RO membranes remain a crucial challenge for seawater desalination. Comparative analysis of 16S rRNA gene amplicon sequencing data revealed differences and commonalities of biofilm communities associated with unit operations in the two largest seawater desalination facilities in the U.S., the Claude "Bud" Lewis Carlsbad Desalination Plant and the Tampa Bay Seater Desalination facility. At both plants, feedwater collected at a single time point was a poor indicator of the RO membrane communities, which showed far greater taxa diversity. The analysis of prefilter cartridges from the Carlsbad plant revealed similarly high taxon diversity as the RO module biofilms, with relevant differences. Algal sequences were enriched on the prefilter cartridges as were sequences representing Bdellovibrionota, which are predatory bacteria. Sequences representing opportunistic Gammaproteobacteria (i.e., Shewanella, Woesia) were present in significantly higher relative abundance on the RO membranes than in the prefilter cartridges, suggesting growth of certain taxa in the RO modules. Untargeted metabolomics distinguished intra- and inter-desalination plant biofilm samples, highlighting the potential value of this tool for biofilm monitoring. These findings underscore the value of omics tools for effective microbial monitoring, to understand biofouling dynamics within RO desalination plants, and to provide insight for the development of ecologically-informed biofilm control measures.
59 BASIC BIOLOGICAL SCIENCES↗
Cosmic shear cosmology beyond two-point statistics: a combined peak count and correlation function analysis of DES-Y1
ABSTRACT We constrain cosmological parameters from a joint cosmic shear analysis of peak-counts and the two-point shear correlation functions, as measured from the Dark Energy Survey (DES-Y1). We find the structure growth parameter $S_8\equiv \sigma _8\sqrt{\Omega _{\rm m}/0.3} = 0.766^{+0.033}_{-0.038}$ which, at 4.8 per cent precision, provides one of the tightest constraints on S8 from the DES-Y1 weak lensing data. In our simulation-based method we determine the expected DES-Y1 peak-count signal for a range of cosmologies sampled in four w cold dark matter parameters (Ωm, σ8, h, w0). We also determine the joint covariance matrix with over 1000 realizations at our fiducial cosmology. With mock DES-Y1 data we calibrate the impact of photometric redshift and shear calibration uncertainty on the peak-count, marginalizing over these uncertainties in our cosmological analysis. Using dedicated training samples we show that our measurements are unaffected by mass resolution limits in the simulation, and that our constraints are robust against uncertainty in the effect of baryon feedback. Accurate modelling for the impact of intrinsic alignments on the tomographic peak-count remains a challenge, currently limiting our exploitation of cross-correlated peak counts between high and low redshift bins. We demonstrate that once calibrated, a fully tomographic joint peak-count and correlation functions analysis has the potential to reach a 3 per cent precision on S8 for DES-Y1. Our methodology can be adopted to model any statistic that is sensitive to the non-Gaussian information encoded in the shear field. In order to accelerate the development of these beyond-two-point cosmic shear studies, our simulations are made available to the community upon request.
Harnois-Déraps, Joachim↗
High-speed impact of micron-sized diesel drop trains—Splashing dynamics, secondary droplet formation, and effects of pre-existing film thickness
This work investigates surface impingement of monodispersed diesel drop trains using computational simulations. The three-dimensional simulations include high-speed impact of micrometer-sized fuel drops onto initially dry and wetted stainless steel substrates. The drop size and impact velocity are representative of fuel injection conditions in internal combustion engines (ICEs). The drop trains serve as a simplified representation of fuel spray. To accurately capture the film that naturally develops on initially dry substrates, a dynamic diesel-stainless steel contact angle model was developed using experiments of single drop impact under ICE representative temperature. Simulations are presented for a highly splashing case and the effects of a pre-existing film on splashing dynamics were investigated, including the temporal evolution of splashed mass and film thickness. It was concluded that for thin films, the effect of pre-existing film thickness is less significant than in single drop impingement. Secondary droplet characterization was performed on simulation results, leading to instantaneous and time-averaged distributions of secondary droplet size, velocity magnitude, and trajectory angle. It was found that for each drop impingement, approximately 58% of the splashed mass is from the impinging drop itself, while the remainder of the splashed mass is composed of film liquid. From a high resolution case, details of secondary droplet formation are observed and three distinct phases of secondary droplet formation are identified. The detailed analysis of drop train impingement under engine-relevant conditions serves as a first step toward a robust understanding of fuel impingement and the development of cleaner and more efficient ICEs.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Outage Analysis for Cooperative mmWave UAV Communications With Beam Training Overhead
In this letter, we investigate the tradeoff between reliability and beam training overhead in cooperative millimeter wave (mmWave) unmanned aerial vehicle (UAV) communications. In particular, we analyze how outage probability varies with the number of neighboring UAVs being probed as potential relays for relay selection. Unlike existing work, the limited number of UAV beams as well as UAV orientations are explicitly considered in our analysis. Here we derive the outage probability in closed form and verify our analysis via simulations. The results facilitate us to understand the fundamental limits of cooperative mmWave UAV communications.
99 GENERAL AND MISCELLANEOUS↗
Viscosity in water from first-principles and deep-neural-network simulations
Abstract We report on an extensive study of the viscosity of liquid water at near-ambient conditions, performed within the Green-Kubo theory of linear response and equilibrium ab initio molecular dynamics (AIMD), based on density-functional theory (DFT). In order to cope with the long simulation times necessary to achieve an acceptable statistical accuracy, our ab initio approach is enhanced with deep-neural-network potentials (NNP). This approach is first validated against AIMD results, obtained by using the Perdew–Burke–Ernzerhof (PBE) exchange-correlation functional and paying careful attention to crucial, yet often overlooked, aspects of the statistical data analysis. Then, we train a second NNP to a dataset generated from the Strongly Constrained and Appropriately Normed (SCAN) functional. Once the error resulting from the imperfect prediction of the melting line is offset by referring the simulated temperature to the theoretical melting one, our SCAN predictions of the shear viscosity of water are in very good agreement with experiments.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets
Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.
Chemistry↗
Evaluation of Portable Acceleration Solutions for LArTPC Simulation Using Wire-Cell Toolkit
The Liquid Argon Time Projection Chamber (LArTPC) technology plays an essential role in many current and future neutrino experiments. Accurate and fast simulation is critical to developing efficient analysis algorithms and precise physics model projections. The speed of simulation becomes more important as Deep Learning algorithms are getting more widely used in LArTPC analysis and their training requires a large simulated dataset. Heterogeneous computing is an efficient way to delegate computationally intensive tasks to specialized hardware. However, as the landscape of compute accelerators quickly evolves, it becomes increasingly difficult to manually adapt the code to the latest hardware or software environments. A solution which is portable to multiple hardware architectures without substantially compromising performance would thus be very beneficial, especially for long-term projects such as the LArTPC simulations. In search of a portable, scalable and maintainable software solution for LArTPC simulations, we have started to explore high-level portable programming frameworks that support several hardware backends. In this paper, we present our experience porting the LArTPC simulation code in the Wire-Cell Toolkit to NVIDIA GPUs, first with the CUDA programming model and then with a portable library called Kokkos. Preliminary performance results on NVIDIA V100 GPUs and multi-core CPUs are presented, followed by a discussion of the factors affiecting the performance and plans for future improvements.
Yu, Haiwang↗
LCA-PyTorch
LCA-PyTorch is a code repository which contains PyTorch implementations of the Locally Competitive Algorithm (LCA), which is a biologically-plausible sparse coding model. LCA-PyTorch allows for the training, testing, and analysis of single layer LCA networks, multi-layer LCA networks, and hybrid LCA-based deep neural network models on a wide variety of applications and data types. LCA-PyTorch was developed in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning. LCA-PyTorch interfaces heavily with the open-source PyTorch Python package.
Teti, Michael↗
Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets
Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.
Arasteh, Kourosh [Lawrence Livermore National Labo↗
Harnessing Uncertainty through Functional Data Analysis in Gas Breakthrough Data
Detecting subsurface explosions from radionuclide gas migration through rock fractures is an effective way to identify nuclear activity. Los Alamos National Laboratory (LANL) has developed simulation methods, based on data from the 1962 Hardhat underground nuclear test, to predict gas breakthrough times at the surface. However, these methods rely on an imperfect understanding of the relationship between rock damage and fracture permeability. Our clinic project studies methods for predicting breakthrough curves that characterize total mass produced as a function of time, as well as quantifying the uncertainty associated with these predictions. The model that is currently employed to relate damage to permeability uses an empirically motivated power-law expression, with a range of parameter values that are compatible with the experimental Hardhat data. We develop emulators, built from functional data analysis techniques and trained on simulation data, that rapidly predict the gas breakthrough curve given a damage field and given the parameter values of the power-law equation. Using Bayesian regression, we address the problem of uncertainty quantification in our emulators. Finally, in order to test the robustness of the model, we further validate it on a damage field representing different physical conditions.
58 GEOSCIENCES↗
High-Density Implosions on OMEGA and the National Ignition Facility (NIF) (Final FY23 report for the subcontract B656484 (formerly B640112))
This final FY23 report for the subcontract B656484 (formerly B640112) “High-Density Implosions on Omega and the National Ignition Facility (NIF)” summarizes MIT’s support of LLNL experiments at OMEGA and the NIF with nuclear diagnostics, platforms and analysis, and of training PhD students. Overall, the tasks identified as Statement-Of-Work (SOW) items for FY23 were accomplished.
42 ENGINEERING↗
Graph Neural Networks for Surrogate Modeling of Offshore Floating Platforms
Floating offshore wind turbines (FOWTs) present an significant opportunity to increase renewable energy generation. However, significant challenges remain before FOWTs can be widely commercialized and deployed. In particular, hydrodynamic loading on the platforms can stress the overall structure, damage the mooring systems, and impact power generation. Studying these loads is difficult and often relies on computationally expensive models or experiments. In this work, we explore the use of graph neural networks (GNNs) to construct flexible, data-driven surrogates for hydrodynamic loads on platforms. We leverage the natural graph-like structure of offshore wind platform designs to enable the GNN model to learn to approximate the loads for different wave conditions and structural designs. We demonstrate potential uses for the surrogate by performing parameter sweeps and ridge analysis on the trained model to identify the impacts of different wave and structural features on the loads.
floating offshore wind turbines↗
A User-Friendly GUI Tool for Automated Microstructural Analysis of Fiber-Reinforced Composites and Porous Structures
Understanding and quantifying microstructural features such as fiber orientation and porosity is critical for predicting the mechanical behavior and performance of fiber-reinforced polymer composites. Traditional manual analysis is time-consuming, subjective, and unsuitable for high-throughput datasets. We present a graphical user interface (GUI) application that automates the analysis of microscopy images to extract key microstructural metrics, including fiber orientation tensors, fiber orientation distribution, porosity and pore size distribution. The app integrates multiple image segmentation techniques including global and local thresholding, clustering, and region-based approaches, offering flexibility for different types of image qualities and features. Users can load microstructural images, select regions of interest and segmentation techniques tailored to their image dataset. It also addresses a critical challenge in fiber orientation analysis: the ambiguities caused by touching, overlapping, or partially cut fibers. It supports autorun examples for standardized workflows, enabling reproducible analysis and facilitating training and benchmarking. This tool significantly reduces manual intervention, enhances consistency, and accelerates data generation for structure–property modeling, process optimization, and digital materials research. The tool is intended for use by materials scientists, engineers, and researchers engaged in composite characterization, quality control, and machine learning-based microstructural studies.
Chawla, Komal [ORNL] (ORCID:0000000190327565)↗
A Hydro-particle-mesh Code for Efficient and Rapid Simulations of the Intracluster Medium
We introduce the cosmological HYPER code based on an innovative hydro-particle-mesh (HPM) algorithm for efficient and rapid simulations of gas and dark matter. For the HPM algorithm, we update the approach of Gnedin & Hui to expand the scope of its application from the lower-density intergalactic medium (IGM) to the higher-density intracluster medium (ICM). While the original algorithm tracks only one effective particle species, the updated version separately tracks the gas and dark matter particles, as they do not exactly trace each other on small scales. For the approximate hydrodynamics solver, the pressure term in the gas equations of motion is calculated using robust physical models. In particular, we use a dark matter halo model, ICM pressure profile, and IGM temperature–density relation, all of which can be systematically varied for parameter-space studies. We show that the HYPER simulation results are in good agreement with the halo model expectations for the density, temperature, and pressure radial profiles. Simulated galaxy cluster scaling relations for Sunyaev–Zel’dovich (SZ) and X-ray observables are also in good agreement with mean predictions, with scatter comparable to that found in hydrodynamic simulations. HYPER also produces lightcone catalogs of dark matter halos and full-sky tomographic maps of the lensing convergence, SZ effect, and X-ray emission. These simulation products are useful for testing data analysis pipelines, generating training data for machine learning, understanding selection and systematic effects, and for interpreting astrophysical and cosmological constraints.
79 ASTRONOMY AND ASTROPHYSICS↗