Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Linear Regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66
All sourcesNASA NTRSDOE OSTI
NASA NTRS2000

Model of ASTM Flammability Test in Microgravity: Iron Rods

There is extensive qualitative results from burning metallic materials in a NASA/ASTM flammability test system in normal gravity. However, this data was shown to be inconclusive for applications involving oxygen-enriched atmospheres under microgravity conditions by conducting tests using the 2.2-second Lewis Research Center (LeRC) Drop Tower. Data from neither type of test has been reduced to fundamental kinetic and dynamic systems parameters. This paper reports the initial model analysis for burning iron rods under microgravity conditions using data obtained at the LERC tower and modeling the burning system after ignition. Under the conditions of the test the burning mass regresses up the rod to be detached upon deceleration at the end of the drop. The model describes the burning system as a semi-batch, well-mixed reactor with product accumulation only. This model is consistent with the 2.0-second duration of the test. Transient temperature and pressure measurements are made on the chamber volume. The rod solid-liquid interface melting rate is obtained from film records. The model consists of a set of 17 non-linear, first-order differential equations which are solved using MATLAB. This analysis confirms that a first-order rate, in oxygen concentration, is consistent for the iron-oxygen kinetic reaction. An apparent activation energy of 246.8 kJ/mol is consistent for this model.

Steinberg, Theodore A↗
DOE OSTI2022

Star–Galaxy Image Separation with Computationally Efficient Gaussian Process Classification

Abstract We introduce a novel method for discerning optical telescope images of stars from those of galaxies using Gaussian processes (GPs). Although applications of GPs often struggle in high-dimensional data modalities such as optical image classification, we show that a low-dimensional embedding of images into a metric space defined by the principal components of the data suffices to produce high-quality predictions from real large-scale survey data. We develop a novel method of GP classification hyperparameter training that scales approximately linearly in the number of image observations, which allows for application of GP models to large-size Hyper Suprime-Cam Subaru Strategic Program data. In our experiments, we evaluate the performance of a principal component analysis embedded GP predictive model against other machine-learning algorithms, including a convolutional neural network and an image photometric morphology discriminator. Our analysis shows that our methods compare favorably with current methods in optical image classification while producing posterior distributions from the GP regression that can be used to quantify object classification uncertainty. We further describe how classification uncertainty can be used to efficiently parse large-scale survey imaging data to produce high-confidence object catalogs.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Multi-objective optimization of peel and shear strengths in ultrasonic metal welding using machine learning-based response surface methodology

Ultrasonic metal welding (UMW) is a solid-state joining technique with varied industrial applications. Despite of its numerous advantages, UMW has a relative narrow operating window and is sensitive to variations in process conditions. As such, it is imperative to quantitatively characterize the influence of welding parameters on the resulting joint quality. The quantification model can be subsequently used to optimize the parameters. Conventional response surface methodology (RSM) usually employs linear or polynomial models, which may not be able to capture the intricate, nonlinear input-output relationships in UMW. Furthermore, some UMW applications call for simultaneous optimization of multiple quality indices such as peel strength, shear strength, electrical conductivity, and thermal conductivity. To address these challenges, this paper develops a machine learning (ML)-based RSM to model the input-output relationships in UMW and jointly optimize two quality indices, namely, peel and shear strengths. The performance of various ML methods including spline regression, Gaussian process regression (GPR), support vector regression (SVR), and conventional polynomial regression models with different orders is compared. A case study using experimental data shows that GPR with radial basis function (RBF) kernel and SVR with RBF kernel achieve the best prediction accuracy. The obtained response surface models are then used to optimize a compound joint strength indicator that is defined as the average of normalized shear and peel strengths. In addition, the case study reveals different patterns in the response surfaces of shear and peel strengths, which has not been systematically studied in the literature. While developed for the UMW application, the method can be extended to other manufacturing processes.

42 ENGINEERING↗
NASA NTRS2012

Life Cycle of Midlatitude Deep Convective Systems in a Lagrangian Framework

Deep Convective Systems (DCSs) consist of intense convective cores (CC), large stratiform rain (SR) regions, and extensive non-precipitating anvil clouds (AC). This study focuses on the evolution of these three components and the factors that affect convective AC production. An automated satellite tracking method is used in conjunction with a recently developed multi-sensor hybrid classification to analyze the evolution of DCS structure in a Lagrangian framework over the central United States. Composite analysis from 4221 tracked DCSs during two warm seasons (May-August, 2010-2011) shows that maximum system size correlates with lifetime, and longer-lived DCSs have more extensive SR and AC. Maximum SR and AC area lag behind peak convective intensity and the lag increases linearly from approximately 1-hour for short-lived systems to more than 3-hours for long-lived ones. The increased lag, which depends on the convective environment, suggests that changes in the overall diabatic heating structure associated with the transition from CC to SR and AC could prolong the system lifetime by sustaining stratiform cloud development. Longer-lasting systems are associated with up to 60% higher mid-tropospheric relative humidity and up to 40% stronger middle to upper tropospheric wind shear. Regression analysis shows that the areal coverage of thick AC is strongly correlated with the size of CC, updraft strength, and SR area. Ambient upper tropospheric wind speed and wind shear also play an important role for convective AC production where for systems with large AC (radius greater than 120-km) they are 24% and 20% higher, respectively, than those with small AC (radius=20 km).

Feng, Zhe↗
NASA NTRS2000

Estimation of EOP From VLBI: Direct Approach

The currently adopted strategy of Earth Orientation Parameters (EOP) estimation from Very Long Baseline Interferometry (VLBI) is to estimate six parameters: Universal Time 1 (UT1), UT1 rate, pole positions, and nutation offsets for each 24-hour session independently. Then the resulting time series of raw EOP are filtered and a regression analysis is performed to obtain nutation coefficients, polhode of the pole, and other physical parameters. Thus, the latter parameters are obtained indirectly in two stages. An alternative approach of direct estimation of the final EOP is presented. Pole coordinates and UT1 are considered as a sum of three components: the low-period component that is modeled by a cubic spline, the harmonic component that includes forced nutation, precession and sub-daily variations of EOP, and the stochastic component that is modeled by a linear spline with segment length 1-2 hours. All parameters are obtained in a single LSQ solution using all available data.

POLAR MOTION↗
NASA NTRS2016

Model Development for MODIS Thermal Band Electronic Crosstalk

MODerate-resolution Imaging Spectroradiometer (MODIS) has 36 bands. Among them, 16 thermal emissive bands covering a wavelength range from 3.8 to 14.4 m. After 16 years on-orbit operation, the electronic crosstalk of a few Terra MODIS thermal emissive bands developed substantial issues that cause biases in the EV brightness temperature measurements and surface feature contamination. The crosstalk effects on band 27 with center wavelength at 6.7 m and band 29 at 8.5 m increased significantly in recent years, affecting downstream products such as water vapor and cloud mask. The crosstalk effect is evident in the near-monthly scheduled lunar measurements, from which the crosstalk coefficients can be derived. The development of an alternative approach is very helpful for independent verification.In this work, a physical model was developed to assess the crosstalk impact on calibration as well as in Earth view brightness temperature retrieval. This model was applied to Terra MODIS band 29 empirically to correct the Earth brightness temperature measurements. In the model development, the detectors nonlinear response is considered. The impact of the electronic crosstalk is assessed in two steps. The first step consists of determining the impact on calibration using the on-board blackbody (BB). Due to the detectors nonlinear response and large background signal, both linear and nonlinear coefficients are affected by the crosstalk from sending bands. The second step is to calculate the effects on the Earth view brightness temperature retrieval. The effects include those from affected calibration coefficients and the contamination of Earth view measurements. This model links the measurement bias with crosstalk coefficients, detector non-linearity, and the ratio of Earth measurements between the sending and receiving bands. The correction of the electronic cross talk can be implemented empirically from the processed bias at different brightness temperature. The implementation can be done through two approaches. As routine calibration assessment for thermal infrared bands, the trending over select Earth scenes is processed for all the detectors in a band and the band averaged bias is derived at a certain time. In this case, the correction of an affected band can be made using the regression of the model with band averaged bias and then corrections of detector differences are applied. The second approach requires the trending for individual detectors and the bias for each detector is used for regression with the model. A test using the first approach was made for Terra MODIS band 29 with the biases derived from long-term trending of brightness temperature over ocean and Dome-C.

Chang, Tiejun↗
DOE OSTI2020

MFNets: Multifidelity data-driven networks for Bayesian learning and prediction

This paper presents a multifidelity uncertainty quantification framework called MFNets. We seek to address three existing challenges that arise when experimental and simulation data from different sources are used to enhance statistical estimation and prediction with quantified uncertainty. Specifically, we demonstrate that MFNets can (1) fuse heterogeneous data sources arising from simulations with different parameterizations, e.g simulation models with different uncertain parameters or data sets collected under different environmental conditions; (2) encode known relationships among data sources to reduce data requirements; and (3) improve the robustness of existing multi-fidelity approaches to corrupted data. MFNets construct a network of latent variables (LVs) to facilitate the fusion of data from an ensemble of sources of varying credibility and cost. These LVs are posited as explanatory variables that provide the source of correlation in the observed data. Furthermore, MFNets provide a way to encode prior physical knowledge to enable efficient estimation of statistics and/or construction of surrogates via conditional independence relations on the LVs. We highlight the utility of our framework with a number of theoretical results which assess the quality of the posterior mean as a frequentist estimator and compare it to standard sampling approaches that use single fidelity, multilevel, and control variate Monte Carlo estimators. We also use the proposed framework to derive the Monte Carlo-based control variate estimator entirely from the use of Bayes rule and linear-Gaussian models -- to our knowledge the first such derivation. Finally, we demonstrate the ability to work with different uncertain parameters across different models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Harnessing Machine Learning to Predict MoS 2 Solid Lubricant Performance

Physical vapor deposited (PVD) molybdenum disulfide (MoS 2 ) solid lubricant coatings are an exemplar material system for machine learning methods due to small changes in process variables often causing large variations in microstructure and mechanical/tribological properties. Here, in this work, a gradient boosted regression tree machine learning method is applied to an existing experimental data set containing process, microstructure, and property information to create deeper insights into the process-structure–property relationships for molybdenum disulfide (MoS 2 ) solid lubricant coatings. The optimized and cross-validated models show good predictive capabilities for density, reduced modulus, hardness, wear rate, and initial coefficients of friction. The contribution of individual deposition variables (i.e., argon pressure, deposition power, target conditioning) on coating properties is highlighted through feature importance. The process-property relationships established herein show linear and non-linear relationships and highlight the influence of uncontrolled deposition variables (i.e., target conditioning) on the tribological performance.

MoS2↗
DOE OSTI2024

Model and remote-sensing-guided experimental design and hypothesis generation for monitoring snow-soil–plant interactions

In this study, we develop a machine-learning (ML)-enabled strategy for selecting hillslope-scale ecohydrological monitoring sites within snow-dominated mountainous watersheds, with a particular focus on snow-soil–plant interactions. Data layers rely on spatial data layers from both remote sensing and hydrological model simulations. Specifically, a Landsat-based foresummer drought sensitivity index is used to define the dependency of the annual peak plant productivity on the Palmer drought severity index in the early growing season. Hydrological simulations provide the spatiotemporal dynamics of near-surface soil moisture and snow depth. In this framework, a regression analysis identifies the key hydrological variables relevant to the spatial heterogeneity of drought sensitivity. We then apply unsupervised clustering to these key variables, using the Gaussian mixture model, to group hillslopes into several zones that have divergent relationships regarding soil moisture, snow dynamics, and drought sensitivity. Using the datasets collected in the East River Watershed (Crested Butte, Colorado, United States), results show that drought sensitivity is significantly correlated with model-derived soil moisture and snow-free timing over space and time. The relationship is, however, non-linear, such that the correlation decreases above a threshold elevation and in a heavy snow year due to large snowpacks, lateral flow, and soil storage limitations. Clustering is then able to define the zones that have high or low sensitivity to drought, as well as the mid-elevation regions where sensitivity is associated with the topographic aspect and net potential radiation. In addition, the algorithm identifies the most representative hillslopes with road/trail access within each zone for installing monitoring sites. Our method also aims to significantly increase the use of ML and model-simulation results to guide critical zone and watershed monitoring activities.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2010

Thermal Expansion of Vacuum Plasma Sprayed Coatings

Metallic Cu-8%Cr, Cu-26%Cr, Cu-8%Cr-1%Al, NiAl and NiCrAlY monolithic coatings were fabricated by vacuum plasma spray deposition processes for thermal expansion property measurements between 293 and 1223 K. The corrected thermal expansion, (DL/L(sub 0) varies with the absolute temperature, T, as (DL/L(sub 0) = A(T - 293)(sup 3) + BIT - 293)(sup 2) + C(T - 293) + D, where, A, B, C and D are thermal, regression constants. Excellent reproducibility was observed for all of the coatings except for data obtained on the Cu-8%Cr and Cu-26%Cr coatings in the first heat-up cycle, which deviated from those determined in the subsequent cycles. This deviation is attributed to the presence of residual stresses developed during the spraying of the coatings, which are relieved after the first heat-up cycle. In the cases of Cu-8%Cr and NiAl, the thermal expansion data were observed to be reproducible for three specimens. The linear expansion data for Cu-8% Cr and Cu-26%Cr agree extremely well with rule of mixture (ROM) predictions. Comparison of the data for the Cu-8%Cr coating with literature data for Cr and Cu revealed that the thermal expansion behavior of this alloy is determined by the Cu-rich matrix. The data for NiAl and NiCrAlY are in excellent agreement with published results irrespective of composition and the methods used for processing the materials. The implications of these results on coating GRCop-84 copper alloy combustor liners for reusable launch vehicles are discussed.

Raj, S V.↗
NASA NTRSDate not supplied

Classifying Agnostic Biosignatures using Raman, VNIR, and Elemental Data

How can we use our current wealth of terrestrial data, encompassing biogenic and abiogenic systems, to determine the distinguishing properties of life? SCOBI (Statistical Classification of Biosignature Information) uses machine learning techniques to algorithmically identify combinations of measurements that are “indicative of life”. A set of ~1000 observations, comprising elemental abundance, isotopic fractionation, VNIR reflectance, and (in progress) Raman spectra, have been assembled from existing literature and databases. The observations cover systems classified as “indicative alive” (e.g., cells, vegetation), “indicative non-alive” (e.g., fossils, teeth), “mixed indicative” (e.g., soil, pond water), or “non-indicative” (e.g., rocks, meteorites). VNIR data was preprocessed by linear interpolation from 400-2100 nm and smoothed with a Savitzky-Golay filter. To limit the amount of Earth-biochemistry-specific (non-agnostic) information included, the first five spectral features extracted were number of peaks, number of troughs, mean reflectance, mean peak width, and broadest peak width. To help further emphasize agnostic biosignatures, Earth-specific features such as chlorophylls have been manually flagged so that feature importance with and without them can be compared. Classifiers including k-nearest neighbors (KNN), Gaussian Naïve Bayes (GNB), logistic regression (LR), random forest (RF), and support vector machine (SVM) were implemented, as was a combination voting classifier. Performance metrics included false positive rates, false negative rates, and AUC with 50-50 test/train splits (Monte Carlo simulations). Key takeaways from this stage, prior to the inclusion of Raman spectra, are (1) the overall success rate of 0.933 AUC was most heavily influenced by the elemental abundance data; and (2) VNIR reflectance had the lowest classification performance with 0.52 AUC (58% of objects correctly classified). The next steps are to complete integration of Raman spectral data and to improve the approach to pre-processing and feature extraction for both types of spectral data, such as automated baseline removal, whole spectrum matching, and dimensionality reduction.

Biosignatures↗
DOE OSTI2020

Insights into cosmological structure formation with machine learning.

Our modern understanding of cosmological structure formation posits that small matter density fluctuations present in the early Universe, as traced by the cosmic microwave background, grow via gravitational instability to form extended haloes of dark matter. A theoretical understanding of the structure, evolution and formation of dark matter haloes is an essential step towards unravelling the intricate connection between halo and galaxy formation, needed to test our cosmological model against data from upcoming galaxy surveys. Physical understanding of the process of dark matter halo formation is made difficult by the highly non-linear nature of the haloes' evolution. I describe a new approach to gain physical insight into cosmological structure formation based on machine learning. This approach combines the ability of machine learning algorithms to learn non-linear relationships, with techniques that enable us to physically interpret the learnt mapping. I describe applicatio ns of th e method, with the aim of investigating which aspects of the early universe density field impact the later formation of dark matter haloes. First I present a case where the process of halo formation is turned into a binary classification problem; the algorithm predicts whether or not dark matter `particles' in the initial conditions of a simulation will collapse into haloes of a given mass range. Second, I present its generalization to regression, where the algorithm infers the final mass of the halo to which each particle will later belong. I show that the initial tidal shear does not play a significant role compared to the initial density field in establishing final halo masses. Finally, I demonstrate that extending the framework to deep learning algorithms such as convolutional neural networks allows us to explore connections between the early universe and late time haloes beyond those studied by existing analytic approximations of halo collapse.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1983

Spectral reflectance of surface soils: Relationships with some soil properties

Using a published atlas of reflectance curves and physicochemical properties of soils, a statistical analysis was carried out. Reflectance bands which correspond to five of the wavebands used by NASA's Thematic Mapper were examined for relationships to specific soil properties. The properties considered in this study include: Sand Content, Silt Content, Clay Content, Organic Matter Content, Cation Exchange Capacity, Iron Oxide Content and Moisture Content. Regression of these seven properties on the mean values of five TM bands produced results that indicate that the predictability of the properties can be increased by stratifying the data. The data was stratified by parent material, taxonomic order, temperature zone, moisture zone and climate (combined temperature and moisture). The best results were obtained when the sample was examined by climatic classes. The middle Infra-red bands, 5 and 7, as well as the visible bands, 2 and 3, are significant in the model. The near Infra-red band, band 4, is almost as useful and should be included in any studies. General linear modeling procedures examined relationships of the seven properties with certain wavebands in the stratified samples.

Kiesewetter, C. H.↗
DOE OSTI2023

Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics

Abstract We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of $$C^{\infty }$$ C ∞ bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Regularized Differentiation for Bioburden Density Estimation in Planetary Protection

In this paper, we propose and investigate the performance of two novel shrinkage estimators for bioburden density estimation in planetary protection. The estimators are based on the regularized differentiation of a cumulative count of colony forming units collected throughout the data collecting session or the life cycle of the entire mission. The regularized differentiation recasts the problem of bioburden density estimation as a linear least squares problem. The least squares problem is then solved through regularization techniques, such as truncated singular value decomposition and penalized least squares. The regularization is necessary to avoid noise amplification during the differentiation of noisy data. The two regularization estimators are compared with four other commonly used estimators to simultaneously evaluate the means of multivariable independent Poisson distributions: the maximum likelihood, noninformative Bayes estimator with Jeffreys prior, Empirical Bayes using conjugate gamma-Poisson model with gamma parameters selected by method of moments, and the Clevenson-Zidek estimator. It is shown through computer-simulated data that the regularized differentiation based on ridge regression has the smallest mean-squared error among all estimators. The analysis of shrinkage mechanism implemented by regularized differentiation is performed, and it is shown that the regularized differentiation amounts to performing a weighted averaging of all the samples. The weights are determined by the regularization parameter automatically selected by the L-curve technique. Since the method of least squares makes no distributional assumptions about the data, it presents an attractive technique for bioburden density estimation when there are concerns about the misspecification of the distributional model. The paper concludes with the analysis of the bioburden data collected during InSight mission and directions for future work.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Generalized Brønsted‐Evans‐Polanyi Relationships for Reactions on Metal Surfaces from Machine Learning

Abstract Brønsted‐Evans‐Polanyi (BEP) relationships, i. e., a linear scaling between reaction and activation energies, lie at the core of computational design of heterogeneous catalysts. However, BEPs are not general and often require reparameterization for each class of reactions. Here we construct generalized BEPs (gBEPs), which can predict activation energies for a diverse dataset of reactions of C, O, N and H containing molecules on metal surfaces. In a first step we develop a set of descriptors based on scaling relationships that can capture the change in chemical identity of reactants during the reaction. Subsequently, we use the reaction energy, these descriptors and a single descriptor for the surface structure to parameterize machine learning based regression approaches for the prediction of activation energies. The best approach we developed shows a Mean Absolute Error (MAE) of 0.11 eV for the training set (80 % of the data set) and 0.23 eV for the test set (20 % of the data set). The methodology presented here allows to calculate activation energies within fractions of seconds on a typical personal computer and due to its generality, accuracy and simplicity in application it might prove to be useful in transition metal catalyst design.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2013

Regression Analysis of Top of Descent Location for Idle-thrust Descents

In this paper, multiple regression analysis is used to model the top of descent (TOD) location of user-preferred descent trajectories computed by the flight management system (FMS) on over 1000 commercial flights into Melbourne, Australia. The independent variables cruise altitude, final altitude, cruise Mach, descent speed, wind, and engine type were also recorded or computed post-operations. Both first-order and second-order models are considered, where cross-validation, hypothesis testing, and additional analysis are used to compare models. This identifies the models that should give the smallest errors if used to predict TOD location for new data in the future. A model that is linear in TOD altitude, final altitude, descent speed, and wind gives an estimated standard deviation of 3.9 nmi for TOD location given the trajec- tory parameters, which means about 80% of predictions would have error less than 5 nmi in absolute value. This accuracy is better than demonstrated by other ground automation predictions using kinetic models. Furthermore, this approach would enable online learning of the model. Additional data or further knowl- edge of algorithms is necessary to conclude definitively that no second-order terms are appropriate. Possible applications of the linear model are described, including enabling arriving aircraft to fly optimized descents computed by the FMS even in congested airspace. In particular, a model for TOD location that is linear in the independent variables would enable decision support tool human-machine interfaces for which a kinetic approach would be computationally too slow.

trajectory prediction↗
DOE OSTI2025

Triangle Method for Dense ReLU Layers [SWR-25-72]

This software is an implementation of the methods for initializing and training neural networks to be more efficient per parameter, described more fully below and in the related publication: In theory, depth should make a ReLU network EXPONENTIALLY more efficient by enabling it to produce an exponential number of piecewise linear sections in its output. This reasoning is largely based on the work of mathematicians that have hand-constructed networks that make good use of depth. In practice however, even very deep ReLU networks that have been randomly initialized will behave identically to their shallow counterparts - missing an entire exponential dimension of efficiency. The triangle method is a first attempt at realizing the exponential potential of deep networks. Instead of randomly setting weights, we force pairs of neurons in each layer learn to build triangles (i.e. functions from [0,1] -> [0,1] that look like triangles). This is a very efficient pattern for generating lots of linear pieces because composing two triangular functions doubles the number of pieces with each composition. The triangle method is more than just a different initialization, it is a new paradigm of training. Instead of making direct updates to the matrix weights, we do an extra step of backpropagation to collect the derivatives of the loss function with respect to the shapes of the triangles, training them to tilt left or right. This process essentially holds the networks hand throughout the loss landscape and forces it to always use depth effectively by producing triangular shapes internally. This can produce several orders of magnitude of improvement on convex one-dimensional regression problems. Much more theoretical work is needed to realize its full potential beyond this context, but the implementation in this repository will still work in arbitrary numbers of dimensions. The file Triangle_Method.py is a generalized form of the method that will build each neuron its own custom 1-d convex activation function (with exponential efficiency). Example usage on one dimensional problems can be found in Example_Usage.ipynb and an example of using this in a real neural network can be found in Example_VGG16_CIFAR10.ipynb.

Milkert, Max [National Renewable Energy Laboratory↗