Engineering Papers⌕ Search

Engineering topics

Lu, Lu

Publications and source records attributed to Lu, Lu.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Point Adversarial Self-Mining: A Simple Method for Facial Expression Recognition

In this article, we propose a simple yet effective approach, called point adversarial self mining (PASM), to improve the recognition accuracy in facial expression recognition (FER). Unlike previous works focusing on designing specific architectures or loss functions to solve this problem, PASM boosts the network capability by simulating human learning processes: providing updated learning materials and guidance from more capable teachers. Specifically, to generate new learning materials, PASM leverages a point adversarial attack method and a trained teacher network to locate the most informative position related to the target task, generating harder learning samples to refine the network. The searched position is highly adaptive since it considers both the statistical information of each sample and the teacher network capability. Other than being provided new learning materials, the student network also receives guidance from the teacher network. After the student network finishes training, the student network changes its role and acts as a teacher, generating new learning materials and providing stronger guidance to train a better student network. Here, the adaptive learning materials generation and teacher/student update can be conducted more than one time, improving the network capability iteratively. Extensive experimental results validate the efficacy of our method over the existing state of the arts for FER.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Input-response space-filling designs

Traditional space-filling designs are a convenient way to explore throughout an input space of flexible dimension and have design points close to any region where future predictions might be of interest. In some applications, there may be a model connecting the input factors to the response(s), which provides an opportunity to consider the spacing not only in the input space but also in the response space. In this paper, we present an approach for leveraging current understanding of the relationship between inputs and responses to generate designs that allow the experimenter to flexibly balance the spacing in these two regions to find an appropriate design for the experimental goals. Applications where good spacing of the observed response values include calibration problems where the goal is to demonstrate the adequacy of the model across the range of the responses, sensitivity studies where the outputs from a submodel may be used as inputs for subsequent models, and inverse problems where the outputs of a process will be used in the inverse prediction for the unknown inputs. Here, we use the multi-objective optimization method of Pareto fronts to generate multiple non-dominated designs with different emphases on the input and response space-filling criteria from which the experimenter can choose. The methods are illustrated through several examples and a chemical engineering case study.

42 ENGINEERING↗
DOE OSTI2021

Physics-informed machine learning

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Furthermore, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Practical choices for space-filling designs

Space-filling designs are now commonly used as a flexible model-free strategy for providing good coverage throughout an input space of interest for a variety of computer and physical experiment design scenarios. Some of the preliminary choices about how to frame the problem and which type of design to use can have a substantial impact on the success or failure of the experiment, and yet how to make these critical choices is often under-emphasized in the literature. In this paper, we explore several of the practical choices required by the experimenter and describe a sequence of steps to help create an ideal design that matches the goals and constraints of the experiment. These choices include the specification of the input space, the scaling of the variables, the degree of uniformity of the design points across the input space, and the space-filling characteristics. In addition, some new tools for defining the weights to implement non-uniform space-filling designs are provided. Finally, the methods are demonstrated with several illustrative examples and a real-world chemical engineering experiment for carbon capture.

42 ENGINEERING↗
DOE OSTI2021

DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

Electroconvection is a multiphysics problem involving coupling of the flow field with the electric field as well as the cation and anion concentration fields. Here, we use electroconvection as a benchmark problem to put forward a new data assimilation framework, the DeepM&Mnet, for simulating multiphysics and multiscale problems at speeds much faster than standard numerical methods using pre-trained neural networks. We first pre-train DeepONets that can predict independently each field, given general inputs from the rest of the fields of the coupled system. DeepONets can approximate nonlinear operators and are composed of two sub-networks, a branch net for the input fields and a trunk net for the locations of the output field. DeepONets, which are extremely fast, are used as building blocks in the DeepM&Mnet and form constraints for the multiphysics solution along with some sparse available measurements of any of the fields. We demonstrate the new methodology and document the accuracy of each individual DeepONet, and subsequently we present two different DeepM&Mnet architectures that infer accurately and efficiently 2D electroconvection fields for unseen electric potentials. Furthermore, the DeepM&Mnet framework is general and can be applied for building any complex multiphysics and multiscale models based on very few measurements using pre-trained DeepONets in a “plug-and-play” mode.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

It is widely known that neural networks (NNs) are universal approximators of continuous functions. However, a less known but powerful result is that a NN with a single hidden layer can accurately approximate any nonlinear continuous operator. This universal approximation theorem of operators is suggestive of the structure and potential of deep neural networks (DNNs) in learning continuous operators or complex systems from streams of scattered data. Here, in this work, we thus extend this theorem to DNNs. We design a new network with small generalization error, the deep operator network (DeepONet), which consists of a DNN for encoding the discrete input function space (branch net) and another DNN for encoding the domain of the output functions (trunk net). We demonstrate that DeepONet can learn various explicit operators, such as integrals and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations. We study different formulations of the input function space and its effect on the generalization error for 16 different diverse applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Operator learning for predicting multiscale bubble growth dynamics

We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Digital Twin Network (DTN): Concepts, Architecture, and Key Technologies

With the commercial deployment of 5G and the migration of internet from IPv4 to IPv6, the new development of the network technology has highly attracted the attention of the industry. The intelligentization of network is believed as the trend of the new generation of network development. The network provides the foundation for the information transmission in the digital society, and the digitalization of the network itself is the prerequisite for the intelligent development. Facing the goal of the new generation of digital and intelligent network, this paper introduces the new concept of "digital twin network (DTN)", designs the system architecture, and analyzes the key technologies of DTN. By investigating the challenges of DTN, this paper points out the future development direction of digital twin network.

digital twin network↗
DOE OSTI2020

Incorporating uncertainty for enhanced leaderboard scoring and ranking in data competitions

Data competitions have become a popular and cost-effective approach for crowdsourcing versatile solutions from diverse expertise. Current practice relies on the simple leaderboard scoring based on a given set of competition data for ranking competitors and distributing the prize. However, a disadvantage of this practice in many competitions is that a slight difference in the scores due to the natural variability of the observed data could result in a much larger difference in the prize amounts. In this article, we propose a new strategy to quantify the uncertainty in the rankings and scores from using different data sets that share common characteristics with the provided competition data. By using a bootstrap approach to generate many comparable data sets, the new method has four advantages over current practice. Furthermore, during the competition, it provides a mechanism for competitors to get feedback about the uncertainty in their relative ranking. After the competition, it allows the host to gain a deeper understanding of the algorithm performance and their robustness across representative data sets. It also offers a transparent mechanism for prize distribution to reward the competitors more fairly with superior and robust performance. Finally, it has the additional advantage of being able to explore what results might have looked like if competition goals evolved from their original choices. The implementation of the strategy is illustrated with a real data competition hosted by Topcoder on urban radiation search.

42 ENGINEERING↗
DOE OSTI2020

Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness

We report the accuracy of deep learning, i.e., deep neural networks, can be characterized by dividing the total error into three main types: approximation error, optimization error, and generalization error. Whereas there are some satisfactory answers to the problems of approximation and optimization, much less is known about the theory of generalization. Most existing theoretical works for generalization fail to explain the performance of neural networks in practice. To derive a meaningful bound, we study the generalization error of neural networks for classification problems in terms of data distribution and neural network smoothness. We introduce the cover complexity (CC) to measure the difficulty of learning a data set and the inverse of the modulus of continuity to quantify neural network smoothness. A quantitative bound for expected accuracy/error is derived by considering both the CC and neural network smoothness. Although most of the analysis is general and not specific to neural networks, we validate our theoretical assumptions and results numerically for neural networks by several data sets of images. The numerical results confirm that the expected error of trained networks scaled with the square root of the number of classes has a linear relationship with respect to the CC. We also observe a clear consistency between test loss and neural network smoothness during the training process. In addition, we demonstrate empirically that the neural network smoothness decreases when the network size increases whereas the smoothness is insensitive to training dataset size.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Extraction of mechanical properties of materials through deep learning from instrumented indentation

Instrumented indentation has been developed and widely utilized as one of the most versatile and practical means of extracting mechanical properties of materials. This method is particularly desirable for those applications where it is difficult to experimentally determine the mechanical properties using stress–strain data obtained from coupon specimens. Such applications include material processing and manufacturing of small and large engineering components and structures involving the following: three dimensional (3D) printing, thin-film and multilayered structures, and integrated manufacturing of materials for coupled mechanical and functional properties. Here, we utilize the latest developments in neural networks, including a multifidelity approach whereby deep-learning algorithms are trained to extract elastoplastic properties of metals an d alloys from instrumented indentation results using multiple datasets for desired levels of improved accuracy. We have established algorithms for solving inverse problems by recourse to single, dual, and multiple indentation and demonstrate that these algorithms significantly outperform traditional brute force computations and function-fitting methods. Moreover, we present several multifidelity approaches specifically for solving the inverse indentation problem which 1) significantly reduce the number of high-fidelity datasets required to achieve a given level of accuracy, 2) utilize known physical and scaling laws to improve training efficiency and accuracy, and 3) integrate simulation and experimental data for training disparate datasets to learn and minimize systematic errors. The predictive capabilities and advantages of these multifidelity methods have been assessed by direct comparisons with experimental results for indentation for different commercial alloys, including two wrought aluminum alloys and several 3D printed titanium alloys.

3D printed materials, stress-strain behavior, mult↗
DOE OSTI2020

Extraction of mechanical properties of materials through deep learning from instrumented indentation

Instrumented indentation has been developed and widely utilized as one of the most versatile and practical means of extracting mechanical properties of materials. This method is particularly desirable for those applications where it is difficult to experimentally determine the mechanical properties using stress–strain data obtained from coupon specimens. Such applications include material processing and manufacturing of small and large engineering components and structures involving the following: three-dimensional (3D) printing, thin-film and multilayered structures, and integrated manufacturing of materials for coupled mechanical and functional properties. Here, we utilize the latest developments in neural networks, including a multifidelity approach whereby deep-learning algorithms are trained to extract elastoplastic properties of metals and alloys from instrumented indentation results using multiple datasets for desired levels of improved accuracy. We have established algorithms for solving inverse problems by recourse to single, dual, and multiple indentation and demonstrate that these algorithms significantly outperform traditional brute force computations and function-fitting methods. Moreover, we present several multifidelity approaches specifically for solving the inverse indentation problem which 1) significantly reduce the number of high-fidelity datasets required to achieve a given level of accuracy, 2) utilize known physical and scaling laws to improve training efficiency and accuracy, and 3) integrate simulation and experimental data for training disparate datasets to learn and minimize systematic errors. The predictive capabilities and advantages of these multifidelity methods have been assessed by direct comparisons with experimental results for indentation for different commercial alloys, including two wrought aluminum alloys and several 3D printed titanium alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Expansion of Thaumarchaeota habitat range is correlated with horizontal transfer of ATPase operons

Thaumarchaeota are responsible for a significant fraction of ammonia oxidation in the oceans and in soils that range from alkaline to acidic. However, the adaptive mechanisms underpinning their habitat expansion remain poorly understood. Here we show that expansion into acidic soils and the high pressures of the hadopelagic zone of the oceans is tightly linked to the acquisition of a variant of the energy-yielding ATPases via horizontal transfer. Whereas the ATPase genealogy of neutrophilic Thaumarchaeota is congruent with their organismal genealogy inferred from concatenated conserved proteins, a common clade of V-type ATPases unites phylogenetically distinct clades of acidophilic/acid-tolerant and piezophilic/piezotolerant species. A presumptive function of pumping cytoplasmic protons at low pH is consistent with the experimentally observed increased expression of the V-ATPase in an acid-tolerant thaumarchaeote at low pH. Consistently, heterologous expression of the thaumarchaeotal V-ATPase significantly increased the growth rate of E. coli at low pH. Its adaptive significance to growth in ocean trenches may relate to pressure-related changes in membrane structure in which this complex molecular machine must function. Together, our findings reveal that the habitat expansion of Thaumarchaeota is tightly correlated with extensive horizontal transfer of atp operons.

59 BASIC BIOLOGICAL SCIENCES↗