Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transfer Learning Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Foundation models for atomistic simulation of chemistry and materials

Conventional computational methods for modeling chemical and materials systems are limited by system size and timescale, forcing a trade-off between quantum-mechanical accuracy and the sampling needed for realistic observables. Large language and vision foundation models — pre-trained on massive datasets using transformer architectures — have revolutionized many fields. It is thus interesting to ask whether a foundation model — subject to suitable data, parameter scaling and training — could enable learned simulations of chemistry and materials. Here, in this study, we review the field of machine-learned interatomic potentials (MLIPs) and posit that scaling up large and diverse chemical and materials datasets and highly expressive architectures using advanced training strategies should result in models that are: more efficient, transferable, robust to out-of-distribution scenarios, and easier to fine-tune to a variety of downstream physical observables than models trained from scratch on small datasets corresponding to specific, targeted atomistic simulation tasks. We provide specific criteria for creating such large-scale MLIP foundation models, coordinated strategies for their development, evaluation and deployment, and highlight potential emergent capabilities that could transform predictive simulations in chemistry and materials science and accelerate discovery across multiple technological domains.

Yuan, Eric C.-Y. [University of California, Berkel↗

Data for The utility of transfer learning to improve the performance of deep learning in axon segmentation

The utility of transfer learning to improve the performance of deep learning in axon segmentation Data Data: All the input and labeled volumes tf-logs: Tensorflow logs, view with command "tensorboard --logdir [name of folder]" Model Weights: model_weights: the argument list under variable combo indicate 1) no oversampling, 2) no rotation, 3) no learn scheduler, and 4) flipping on all three dimensions, and the additional values indicate 5) elastic deformation percentage, 6) rotate deformation percentage, 7) layer setting , 8) learning rate, and 9) training/validation/test data division suffix (leave '' if not using suffix). Results: Output from inference segment_total_results_validation_final: All validation results and calculations segment_total_results: All test results and calculations Authors The modified code was created for a paper by: Marjolein Oostrom, Michael A. Muniak, Rogene Eichler West, Sarah Akers, Paritosh Pande, Moses Obiri, Wei Wang, Kasey Bowyer, Zhuhao Wu, Lisa Bramer, Tianyi Mao, Bobbie Jo Webb-Robertson The work is adapted from Github TrailMap, which was created by Albert Pun and Drew Friedmann Acknowledgments MO, RMEW, SA, MO, LB, BJWR were supported by the Laboratory Directed Research and Development at Pacific Northwest National Laboratory (PNNL), a Department of Energy facility operated by Battelle under contract DE-AC05-76RLO01830. WW, KB, and ZW were supported in part by a NIH/BRAIN Initiative Grant RF1MH128969. MAM and TM were supported by two NIH/BRAIN Initiative Grants R01NS104944, RF1MH120119 and NIH R01NS081071. This research is affiliated with the Pacific northwest bioMedical Innovation Co-laboratory (PMedIC) collaboration between OHSU and PNNL.

Oostrom, Marjolein T↗

Document Classification Techniques for Aviation Letters of Agreement

Often when working with technical documents, it is helpful to classify them into specific categories. In this paper, we conduct a thorough review of natural language processing techniques to perform this classification task on Letters of Agreement (LOAs), technical aviation documents outlining rules for utilizing US airspace. We evaluate multiple techniques, including Transfer Learning, for representing the text in the documents as embeddings: unigram and bigram Term Frequency Inverse Document Frequency (TFIDF), Word2Vec, Doc2Vec, GloVe and RoBERTa. We investigate a wide range of classification models: K-Nearest Neighbors, Random Forest, Support Vector Machines (SVM), Logistic Regression, Naive Bayes, Feed-Forward Neural Network, Convolutional Neural Networks (CNNs) and Long-Short Term Memory (LSTM). By comparing the different methods, we found the best overall approach for our task was to use unigram TFIDF representations with SVM while also gaining insight into how the other methodologies performed on a small technical datasets.

Aayushi Batra↗

Neural Network‐Based Methods for Ocean Surface Wave Measurement Using Submarine Distributed Acoustic Sensing (DAS)

Two new data-driven models for estimating ocean surface waves from distributed acoustic sensing (DAS) submarine cable strain rate are developed using supervised machine learning on a 10-day data set collected offshore of Oliktok Point, Alaska. The new models were trained on target data from seafloor pressure moorings at three sites spaced evenly along 27.1 km of cable and were benchmarked against an empirical transfer function method previously used to estimate waves from DAS. A model which uses convolutional neural networks to transform 2-km frequency-wavenumber strain spectra to seafloor pressure spectra outperforms the benchmark in wave height prediction (RMSE of 0.15 vs. 0.41 m) and period prediction (0.29 vs. 0.37 s) when evaluated on a held-out test data set. When applied to a DAS data set collected on the same cable 2 years prior, the CNN-based model maintained similar significant wave height performance (RMSE = 0.23 m) relative to available satellite altimetry data. A two-hidden-layer, fully connected neural network which transforms 1-D strain spectra to seafloor pressure spectra also outperforms the benchmark in wave height prediction (RMSE of 0.19 vs. 0.41 m), but does not generalize as well to the prior data. Regression-based machine learning is useful for estimating waves from DAS data when the pressure-strain relationship varies temporally and spatially across different wave conditions. Models can be applied to DAS data to measure waves with higher spatial resolution and longer temporal coverage than traditional methods, which often measure waves only at a single point.

Davis, Jacob R. [Univ. of Washington, Seattle, WA ↗

Numerical modeling based machine learning approach for the optimization of falling - film evaporator in thermal desalination application

Scale formation that drastically increases thermal resistance and reduces freshwater production remains a critical challenge in thermal desalination. Novel designs of falling film evaporator and optimal operating condition hold great promise to mitigate scale formation, and increase heat transfer performance and fresh water production. In this work, CFD simulation based machine learning and multi-objective optimization are performed to identify optimal conditions and tube arrangement for evaporator. Non-dominated sorting genetic algorithm is adopted to determine and analyze the optimal pareto front for multiple objectives in desalination criteria. The errors of training, validation, and testing set are computed to identify an optimal hyperparameter set. For performance ratio, fouling resistance, and water production rate, the average relative error is 2.26%, 3.67%, and 3.24%. At pareto front, both performance ratio and water production rate increase at high temperature with fouling resistance (thermal resistance of the fouling layer) increasing as well. Tradeoffs between mitigating scale formation and enhancing desalination performance are evaluated in optimizations for different objectives. Finally, potential optima are identified and can be applied as guidelines to determine evaporator design and system operating conditions.

42 ENGINEERING↗

A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data Using Unstructured Spatial Discretizations

In this work, we propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov n-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

97 MATHEMATICS AND COMPUTING↗

Microscopy modality transfer of steel microstructures: Inferring scanning electron micrographs from optical microscopy using generative AI

Scanning electron microscopy (SEM) is resource intensive, which limits its throughput in some applications. As an alternative, we propose applying computer vision and machine learning to generate high-quality synthetic SEM micrographs from micrographs obtained using light optical microscopy (LOM). Working with a correlated LOM/SEM dataset of dual-phase steel images, we test generative models of various architectures, including encoder-decoder networks, generative adversarial networks (GANs), and diffusion-based models. We find that the diffusion models significantly outperform other methods on both qualitative and quantitative assessments, while preserving key metallurgical meaning. This work establishes diffusion as the state-of-the-art for microscopy modality transfer and demonstrates the potential of AI-powered microscopy to enhance LOM with micron scale structural recreation.

Computer vision↗

Feasibility of Using Fourier Neural Operators for 3D Elastic Seismic Simulations

Seismic simulations in three-dimensional (3D) Earth models are important in many of the seismological applications related to our lab’s mission, however, accurate high-fidelity simulations are computationally costly. We investigated the capabilities of the newly developed Fourier Neural Operator (FNO) to solve the 3D elastic wave equations for seismic simulations. We generated simulation data for training the FNO model, and analyzed its performance on various test cases, such as the performance with different number of training data, different resolutions, and on more canonical structures. We found the FNO model can reproduce 3D seismic simulations with high accuracy and ~169 times faster on a small spatial grid size of 16 x 16 x16. When applied on higher resolution data, we found that transfer learning with fine-tuning on a small amount of data achieves reasonable results. This feasibility study showed promising results for using FNO’s for 3D seismic simulations and represents the foundation of further research to develop this into a more mature approach for different seismological applications. The potential impact of this project will provide techniques for large-scale or real-time applications of solving PEDs in support of national lab’s programs.

58 GEOSCIENCES↗

Orbital Fluid Transfer System

An automated fluid and power interface system needs to be developed for future space missions which require on orbit consumable replenishment. Current method of fluid transfer require manned vehicles and extravehicular activity. Currently the US does not have an automated capability for consumable transfer on-orbit. This technology would benefit both Space Station and long duration satellites. In order to provide this technology the Automated Fluid Interface System (AFIS) was developed. The AFIS project was an advanced development program aimed at developing a prototype satellite servicer for future space operations. This mechanism could transfer propellants, cryogens, fluids, gasses, electrical power, and communications from a tanker unit to the orbiting satellite. The development of this unit was a cooperative effort between Marshall Space Flight Center in Huntsville, Alabama, and Moog, Inc. in East Aurora, New York. An engineering model was built and underwent substantial development testing at Marshall Space Flight Center (MSFC). While the AFIS is not suitable for spaceflight, testing and evaluation of the AFIS provided significant experience which would be beneficial in building a flight unit. The lessons learned from testing the AFIS provided the foundation for the next generation fluid transfer mechanism, the Orbital Fluid Transfer System (OFTS). The OFTS project was a study contract with MSFC and Moog, Inc. The OFTS was designed for the International Space Station (ISS), but its flexible design could used for long duration satellite missions and other applications. The OFTS was designed to be used after docking. The primary function was to transfer bipropellants and high pressure gases. The other items addressed by this task included propellant storage, hardware integration, safety and control system issues. A new concept for high pressure couplings was also developed. The results of the AFIS testing provided an excellent basis for the OFTS design. The OFTS meet the servicing requirements for ISS and could also provide the automated fluid and power interface system needed for on orbit consumable resupply of spacecraft into the new century.

Johnston, A. S., (Nick)↗

Creation Synthetic Data to Train a Digital Twin to Predict Reactor Operations

Understanding techniques to strengthen the nuclear safeguards regime is crucial in preventing nuclear proliferation due to recent advancements in the nuclear energy industry such as Generation IV reactors and microreactors. Prior to the construction of a nuclear power plant, it is necessary to understand the proliferation potential of the plant's reactor. Digital twins serve as a unique solution to recognizing reactor behavior indicative of nuclear proliferation. A digital twin is defined as a virtual model that works in unison to represent a physical asset, with a transference of data between the virtual and physical assets [1]. This work serves as validation for training a digital twin on synthetic data fabricated via means of Serpent reactor physics and point kinetics equations simulations. In this case this work is based on parameters of Idaho State University's AGN-201 reactor. The synthetic data can then be utilized to train machine learning models in the future to further investigate the utility of these methods. The accuracy of the predicted data is measured against real operational data to verify the reliability of the synthetic data creation methods and decide whether these methods should be used in the future to inform inspectors of a reactor's proliferation potential.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Carbon Capture through Membranes - Leveraging Multiphysics Modeling, Dimensional Analysis and Machine Learning to Scale up and Optimize Devices and Processes for Decarbonization

We study the separation performance using membrane modules through dimensional analysis (DA). We formulate the main process equations to identify relevant dimensionless numbers inherent in the physics. In particular, we identify that the critical step in the separation process is mass transfer through the selective layer. Remarkably, the dimensionless feed flow (DFfeed) emerges as a crucial factor in describing this process. Not only does DFfeed directly appear in the governing equations, but it also holds a physical significance associated with the time scales for the mass transfer across the feed side and through the selective layer. Regarding the output performance variables, we consider the recovery, stage cut, productivity and purity. In this context, we profit from experimental data and CFD simulations to evaluate the separation performance of the modules when varying the input flowrate, the scale of the module, and the CO2 permeance. These datasets enable us to establish correlations between performance metrics and the dimensionless feed flow (DFfeed). Using simple power functions of DFfeed, we obtain R2 coefficients exceeding 0.99, indicating the accuracy of the correlations built in the present work. In the future, we wish to use DA to understand key transport mechanisms, predict and control module performance, and challenge the universality of these findings by testing various gas separations across different membrane modules beyond our case study.

Pedrozo, Hector A.↗

MACHINE LEARNING-ENABLED PREDICTION OF TRANSIENT INJECTION MAP IN AUTOMOTIVE INJECTORS WITH UNCERTAINTY QUANTIFICATION

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.

machine learning↗

A fine pore-preserved deep neural network for porosity analytics of a high burnup U-10Zr metallic fuel

Abstract U-10 wt.% Zr (U-10Zr) metallic fuel is the leading candidate for next-generation sodium-cooled fast reactors. Porosity is one of the most important factors that impacts the performance of U-10Zr metallic fuel. The pores generated by the fission gas accumulation can lead to changes in thermal conductivity, fuel swelling, Fuel-Cladding Chemical Interaction (FCCI) and Fuel-Cladding Mechanical Interaction (FCMI). Therefore, it is crucial to accurately segment and analyze porosity to understand the U-10Zr fuel system to design future fast reactors. To address the above issues, we introduce a workflow to process and analyze multi-source Scanning Electron Microscope (SEM) image data. Moreover, an encoder-decoder-based, deep fully convolutional network is proposed to segment pores accurately by integrating the residual unit and the densely-connected units. Two SEM 250 × field of view image datasets with different formats are utilized to evaluate the new proposed model’s performance. Sufficient comparison results demonstrate that our method quantitatively outperforms two popular deep fully convolutional networks. Furthermore, we conducted experiments on the third SEM 2500 × field of view image dataset, and the transfer learning results show the potential capability to transfer the knowledge from low-magnification images to high-magnification images. Finally, we use a pre-trained network to predict the pores of SEM images in the whole cross-sectional image and obtain quantitative porosity analysis. Our findings will guide the SEM microscopy data collection efficiently, provide a mechanistic understanding of the U-10Zr fuel system and bridge the gap between advanced characterization to fuel system design.

36 MATERIALS SCIENCE↗

Modeling solar magnetic structures

Some ideas in the theoretical study of force-free magnetic fields and magnetostatic fields, which are relevant to the effort of using magnetograph data as inputs to model the quasi-static, large-scale magnetic structures in the solar atmosphere are discussed. Basic physical principles will be emphasized. An attempt will be made to assess what we may learn, physically, from the models based on these ideas. There is prospect for learning useful physics and this ought to be an incentive for intensifying the efforts to improve vector magnetograph technology and to solve the basic radiative-transfer problems encountered in the interpretation of magnetograph raw data.

Low, B. C.↗

Tests characterizing bioprocessor hardware for analytical modeling

The tests outlined in this paper were used to characterize the hardware components of the Salad Machine, a small NASA-developed bioprocessor. The data from these tests are presented, and the methods by which this data can be integrated into system mathematical models are briefly discussed. The subsystems and physical processes discussed include the lighting system, the air loop (condensing heat exchanger and the blower), heat transfer to the surroundings, and leakage. Through this effort it was learned that in the development of a test protocol, care should be taken to order the tests such that environmental parameters, particularly humidity, require as few large adjustments as possible. Sensor calibration and installation take a substantial amount of time, which should be built into the test schedule. Two properties were particularly hard to quantify: the air flow rate and the energy from the lighting system entering into the growth volume. Flow rate can be measured using the appropriate device for the system configuration and airflow. Lighting system radiation level was measured using three methods. The results of these methods varied substantially, putting off conclusive quantification of this value.

Gustavino, S.↗

Generalization Equations for Machine Learners Based on Physical and Abstract Laws

The physical and abstract laws derived from the first principles have been recently exploited to customize and sharpen machine learning (ML) methods and also derive their generalization equations. These laws often encapsulate knowledge that complements datasets and ML models. We present a generic framework that uses these laws to provide ML codes that are transferable across multiple areas, including data transport infrastructures and thermal hydraulics analytics of nuclear reactors. By anchoring on datasets from these areas and the statistical generalization theory, we present a rigorous approach to co-develop ML solutions and the generalization equations that characterize them, by exploiting the structure and constraints from the laws. We present illustrative examples using practical problems from existing literature on the performance characterization of data transport infrastructures, and the sensor error and power level estimation in nuclear reactor systems using sensor measurements of primary and secondary coolant systems, respectively.

Rao, Nageswara↗