Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “hyperparameter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Revisiting matrix-based inversion of scanning mobility particle sizer (SMPS) and humidified tandem differential mobility analyzer (HTDMA) data

Abstract. Tikhonov regularization is a tool for reducing noise amplification during data inversion. This work introduces RegularizationTools.jl, a general-purpose software package for applying Tikhonov regularization to data. The package implements well-established numerical algorithms and is suitable for systems of up to ∼ 1000 equations. Included is an abstraction to systematically categorize specific inversion configurations and their associated hyperparameters. A generic interface translates arbitrary linear forward models defined by a computer function into the corresponding design matrix. This obviates the need to explicitly write out and discretize the Fredholm integral equation, thus facilitating fast prototyping of new regularization schemes associated with measurement techniques. Example applications include the inversion involving data from scanning mobility particle sizers (SMPSs) and humidified tandem differential mobility analyzers (HTDMAs). Inversion of SMPS size distributions reported in this work builds upon the freely available software DifferentialMobilityAnalyzers.jl. The speed of inversion is improved by a factor of ∼ 200, now requiring between 2 and 5 ms per SMPS scan when using 120 size bins. Previously reported occasional failure to converge to a valid solution is reduced by switching from the L-curve method to generalized cross-validation as the metric to search for the optimal regularization parameter. Higher-order inversions resulting in smooth, denoised reconstructions of size distributions are now included in DifferentialMobilityAnalyzers.jl. This work also demonstrates that an SMPS-style matrix-based inversion can be applied to find the growth factor frequency distribution from raw HTDMA data while also accounting for multiply charged particles. The outcome of the aerosol-related inversion methods is showcased by inverting multi-week SMPS and HTDMA datasets from ground-based observations, including SMPS data obtained at Bodega Marine Laboratory during the CalWater 2/ACAPEX campaign and co-located SMPS and HTDMA data collected at the US Department of Energy observatory located at the Southern Great Plains site in Oklahoma, USA. Results show that the proposed approaches are suitable for unsupervised, nonparametric inversion of large-scale datasets as well as inversion in real time during data acquisition on low-cost reduced-instruction-set architectures used in single-board computers. The included software implementation of Tikhonov regularization is freely available, general, and domain-independent and thus can be applied to many other inverse problems arising in atmospheric measurement techniques and beyond.

54 ENVIRONMENTAL SCIENCES↗

Synthetic Streamflow Datasets Derived from DOE 9505 for Select Texas Basins

This dataset is generated using a Bayesian Hidden Markov Model trained on the DOE 9505 streamflow projection ensemble. A set of 21,000 streamflow realizations are generated for the Colorado, Sabine, and Trinity river basins in Texas. Separate models are trained either using the full 9505 ensemble or a subset based on three hyperparameters: bias correction, downscaling, and hydrological model.

drought↗

Xanthos-Lake Model Source Code

This repository contains the source code for Xanthos-Lake, a lake-modeling extension of the Xanthos framework that introduces a coupled lake component comprising the Xanthos-Lake Snow and Ice Model (xLSIM) and the Xanthos-Lake Water Balance Model (xLWBM). xLSIM is a basin-aware machine-learning model for lake snow, ice, and thermal conditions. It predicts monthly lake ice thickness, snow depth, snow-cover fraction, mixing-layer temperature, and lake ice fraction from meteorological forcing and lake surface-area information. It uses sequence-based deep-learning architectures, including Transformer and hybrid Long Short-Term Memory–Transformer (LSTM–Transformer) models, together with seasonal encoding, multi-lake learning, physical masking, and basin-level cryospheric and non-cryospheric classification. The training workflow uses Ray for scalable execution and includes optional Ray Tune hyperparameter optimization. Model predictions, observations, diagnostics, and feature-importance outputs are written in NetCDF. xLWBM is the water-balance component of the new lake framework. It simulates monthly lake storage, surface area, evaporation, inflow, outflow, and lake–groundwater exchange. It combines physical water-balance equations with calibrated bathymetric relationships, weir-based outlet flow, modified Penman open-water evaporation, groundwater head relaxation, Penman–Monteith snow and ice sublimation, and snow, ice, and thermal conditions supplied by xLSIM. The model calibrates lake parameters against satellite-derived surface-area data, using evaporation-based calibration where surface-area data are unavailable, and supports small, medium, and large lake classes. For large lakes, xLWBM is integrated with the managed-routing workflow so that lake storage and outflow interact directly with downstream river routing and reservoir operations. Together, xLSIM and xLWBM provide Xanthos with a coupled lake-modeling capability. xLSIM supplies the snow, ice, and thermal conditions that affect lake evaporation and snow- and ice-related water exchanges, while xLWBM translates those conditions into dynamic lake storage, surface area, evaporation, and discharge. In return, xLWBM supplies evolving lake surface area to xLSIM. This coupling enables Xanthos to represent lakes as active hydrologic components within basin-scale water-availability and routing simulations.

Machine Learning↗

Hybrid deep learning architecture for general disruption prediction across tokamaks

In this paper, we present a new deep learning disruption prediction algorithm based on important findings from explorative data analysis which effectively allows knowledge transfer from existing devices to new ones, thereby predicting disruptions using very limited disruptive data from the new devices. The explorative data analysis conducted via unsupervised clustering techniques confirms that time-sequence data are much better separators of disruptive and non-disruptive behavior than the instantaneous plasma state data with further advantageous implications for a sequence-based predictor. Based on such important findings, we have designed a new algorithm for multi-machine disruption prediction that achieves high predictive accuracy on the C-Mod (AUC=0.801), DIII-D (AUC=0.947) and EAST (AUC=0.973). tokamaks with limited hyperparameter tuning. Through numerical experiments, we show that boosted accuracy (AUC=0.959) is achieved on EAST predictions by including in the training only 20 disruptive discharges, thousands of non-disruptive discharges from EAST, and combining this with more than a thousand discharges from DIII-D and C-Mod. The improvement of predictive ability obtained by combining disruptive data from other devices is found to be true for all permutations of the three devices. Furthermore, by comparing the predictive performance of each individual numerical experiment, we find that non-disruptive data are machine-specific while disruptive data from multiple devices contain device-independent knowledge that can be used to inform predictions for disruptions occurring on a new device.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

An Algorithm for Detection of Ground and Canopy Cover in Micropulse Photon-Counting Lidar Altimeter Data in Preparation of the ICESat-2 Mission

The Ice, Cloud and Land Elevation Satellite-II (ICESat-2) mission has been selected by NASA as a Decadal Survey mission, to be launched in 2016. Mission objectives are to measure land ice elevation, sea ice freeboard/ thickness and changes in these variables and to collect measurements over vegetation that will facilitate determination of canopy height, with an accuracy that will allow prediction of future environmental changes and estimation of sea-level rise. The importance of the ICESat-2 project in estimation of biomass and carbon levels has increased substantially, following the recent cancellation of all other planned NASA missions with vegetation-surveying lidars. Two innovative components will characterize the ICESat-2 lidar: (1) Collection of elevation data by a multi-beam system and (2) application of micropulse lidar (photon counting) technology. A micropulse photon-counting altimeter yields clouds of discrete points, which result from returns of individual photons, and hence new data analysis techniques are required for elevation determination and association of returned points to reflectors of interest including canopy and ground in forested areas. The objective of this paper is to derive and validate an algorithm that allows detection of ground under dense canopy and identification of ground and canopy levels in simulated ICESat-2-type data. Data are based on airborne observations with a Sigma Space micropulse lidar and vary with respect to signal strength, noise levels, photon sampling options and other properties. A mathematical algorithm is developed, using spatial statistical and discrete mathematical concepts, including radial basis functions, density measures, geometrical anisotropy, eigenvectors and geostatistical classification parameters and hyperparameters. Validation shows that the algorithm works very well and that ground and canopy elevation, and hence canopy height, can be expected to be observable with a high accuracy during the ICESat-2 mission. A result relevant for instrument design is that even the two weaker beam classes considered can be expected to yield useful results for vegetation measurements (93.01-99.57% correctly selected points for a beam with expected return of 0.93 mean signals per shot (msp9) and 72.85% - 98.68% for 0.48 msp (msp4)). Resampling options affect results more than noise levels. The algorithm derived here is generally applicable for analysis of micropulse lidar altimeter data collected over forested areas as well as other surfaces, including land ice, sea ice and land surfaces.

Herzfeld, Ute C.↗

Algorithm for Detection of Ground and Canopy Cover in Micropulse Photon-Counting Lidar Altimeter Data in Preparation for the ICESat-2 Mission

NASA's Ice, Cloud and Land Elevation Satellite-II (ICESat-2) mission is a decadal survey mission (2016 launch). The mission objectives are to measure land ice elevation, sea ice freeboard, and changes in these variables, as well as to collect measurements over vegetation to facilitate canopy height determination. Two innovative components will characterize the ICESat-2 lidar: 1) collection of elevation data by a multibeam system and 2) application of micropulse lidar (photon-counting) technology. A photon-counting altimeter yields clouds of discrete points, resulting from returns of individual photons, and hence new data analysis techniques are required for elevation determination and association of the returned points to reflectors of interest. The objective of this paper is to derive an algorithm that allows detection of ground under dense canopy and identification of ground and canopy levels in simulated ICESat-2 data, based on airborne observations with a Sigma Space micropulse lidar. The mathematical algorithm uses spatial statistical and discrete mathematical concepts, including radial basis functions, density measures, geometrical anisotropy, eigenvectors, and geostatistical classification parameters and hyperparameters. Validation shows that ground and canopy elevation, and hence canopy height, can be expected to be observable with high accuracy by ICESat-2 for all expected beam energies considered for instrument design (93.01%-99.57% correctly selected points for a beam with expected return of 0.93 mean signals per shot (msp), and 72.85%-98.68% for 0.48 msp). The algorithm derived here is generally applicable for elevation determination from photoncounting lidar altimeter data collected over forested areas, land ice, sea ice, and land surfaces, as well as for cloud detection.

Algorithms↗

Automated Pneumothorax Diagnosis using Deep Neural Networks

Thoracic ultrasound can provide information leading to rapid diagnosis of pneumothorax with improved accuracy over the standard physical examination and with higher sensitivity than anteroposterior chest radiography. However, the clinical We have Furthermore, remote environments, such as the battlefield or deep-space exploration, may lack expertise for diagnosing developed an automated image interpretation pipeline for the analysis of thoracic ultrasound data and the classification of pneumothorax events to provide decision support in such situations. Our pipeline consists of image preprocessing, data augmentation, and deep learning architectures for medical diagnosis. In this work, we demonstrate that robust, accurate interpretation of chest images and video can be achieved using deep neural networks. A number of novel image processing techniques were employed to achieve this result. Affine transformations were applied for data augmentation. Hyperparameters were optimized for learning rate, dropout regularization, batch size, and epoch iteration by a sequential model-based Bayesian approach. In addition, we utilized pretrained architecturesinterpretation of a patient medical image is highly operator dependent. certain pathologies., applying transfer learning and fine-tuning techniques to fully connected layers. Our pipeline yielded binary classification validation accuracies of 98.3% for M-mode images and 99.8% with B-mode video frames.

US Army collaboration↗

Reinforcement Learning Applied to Cognitive Space Communications

The future of space exploration depends on robust, reliable communication systems. As the number of such communication systems increase, automation is fast becoming a requirement to achieve this goal. A reinforcement learning solution can be employed as a possible automation method for such systems. The goal of this study is to build a reinforcement learning algorithm which optimizes data throughput of a single actor. A training environment was created to simulate a link within the NASA Space Communication and Navigation (SCaN) infrastructure, using state of the art simulation tools developed by the SCaN Center for Engineering, Networks, Integration, and Communications (SCENIC) laboratory at NASA Glenn Research Center to obtain the closest possible representation of the real operating environment. Reinforcement learning was then used to train an agent inside this environment to maximize data throughput. The simulation environment contained a single actor in low earth orbit capable of communicating with twenty-five ground stations that compose the Near-Earth Network (NEN). Initial experiments showed promising training results, so additional complexity was added by augmenting simulation data with link fading profiles obtained from real communication events with the International Space Station. A grid search was performed to find the optimal hyperparameters and model architecture for the agent. Using the results of the grid search, an agent was trained on the augmented training data. Testing shows that the agent performs well inside the training environment and can be used as a foundation for future studies with added complexity and eventually tested in the real space environment.

Schubert, Carson D.↗

Exploring data-driven modeling of boundary layer transition

Prediction of laminar-turbulent transition in boundary layer flows is an important component of predicting the aerodynamic performance of a number of aerospace configurations. According to the CFD Vision 2030 [1], transition modeling represents acriticalarea in CFD simulation capability that will remain a pacing item for the foreseeable future. The fact thattransition can take placevia either one of a myriad possible paths adds to the challenges inreliable transition predictions, despite a limited knowledge of the relevant input parameters. In the low disturbance environments typical of flight applications, transition is often initiated by small amplitude disturbances in the form of linear instability waves of the laminar boundary layer. These disturbances amplify linearly at first and eventually undergo a sequence of nonlinear interactions that result in transition to turbulence. Because the nonlinear phase is rather rapid, the amplification of boundary layer instabilities is governed by the linearstability theory over a majority of the distance leading up to the onset of transition. Semi-empirical transition correlations based on the linear stability theory have been successful in explaining the observed trends in transition location within a broad class of flows. However, the application of stability theory is highly non-robust and often requires a significant domain expertise. Recent work at the NASA Langley Research Center has beenaimed at bridging the gap between physics based transition analyses such as those based on linear stability theory and practical applications that require transition prediction by users that may not be well versed in transition physics. The applications of deep learning have been at the center of these efforts. This presentation will focus on the progress achieved thus far, highlighting the applications of neural networks to selectedtransition scenarios across a range of Mach numbers and flow configuration, as well as the lessons learnedand remaining challengeswithrespect to the selection of training data and neural networks architectures, hyperparameter tuning, and the physical insights distilled from the otherwise black-box models.

M. R. Malik↗

Use of Design of Experiments in Determining Neural Network Architectures for Loss of Control Detection

We describe empirical methods for selecting a neural network architecture to implement belief state inference on generic commercial transport aircraft. We highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). Our main contribution is the application of a structured method that can be used for machine learning in many aerospace applications. This method optimizes the structure and training parameters of a neural network for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE analysis was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from the NASA T-2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications.

Loss of Control↗

Use of Design of Experiments in Determining Neural Network Architectures for Loss of Control Detection

We describe empirical methods for selecting a neural network architecture to implement belief state inference on generic commercial transport aircraft. We highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). Our main contribution is the application of a structured method that can be used for machine learning in many aerospace applications. This method optimizes the structure and training parameters of a neural network for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE analysis was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from the NASA T 2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications.

Loss of Control↗

Use of Design of Experiments in Determining Neural Network Architectures for Loss of Control Detection

Abstract—We describe empirical methods for selecting a neural network architecture to implement belief state inference on generic commercial transport aircraft. We highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). Our main contribution is the application of a structured method that can be used for machine learning in many aerospace applications. This method optimizes the structure and training parameters of a neural network for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE analysis was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from the NASA T-2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications.

neural networks↗

Predicting Unreinforced Fabric Mechanical Behavior with Recurrent Neural Networks

Unreinforced woven fabrics are widely employed in various high-performance applications, including parachute deployment systems, airbags, and ballistic armor. The analysis of such materials is inherently complex due to the multiscale structure of these materials, and the dependence of macroscale behavior on changes that occur at lower scales. Previously, NASA’s Multiscale Analysis Tool (NASMAT) showed its capability in predicting unreinforced fabric behavior at the macroscale by capturing finite rotations that occur at the mesoscale. Though effective, the tool can face high computational cost for large, complex problems, motivating the need for the development of a surrogate model that can capture the same behavior. A recurrent neural network (RNN) was developed and trained on virtual NASMAT data to mimic the physics-based solutions while improving the computational runtime. The architecture of the RNN to best simulate the fabric behavior was carefully crafted based on heuristic knowledge of predicting physics-based temporal data, manual hyperparameter case studies, and Hyperband optimization.. The resultant model was able to predict a variety of stress-strain curves for fabrics with different mesoscale geometries, and was further validated by comparing to experimental data for the K706 style Kevlar plain-weave fabric, demonstrating the ability of the model to effectively capture the geometric changes in the fabric without explicitly calculating them, as is done in NASMAT. Furthermore, the tool showed its ability to improve on the runtime by a factor of 10 for fabric solutions compared to the multiscale tool, which would further enable the simulation of complex loading scenarios on unreinforced fabrics.

Fabric↗

Application of Support Vector Regression to Derive Crater Depth/Diameter From Satellite Images

Through the study of impact crater shapes, one can draw important conclusions about the nature and evolution of planetary surfaces [e.g., 1-4].In particular, studying the depth (d) to diameter (D)ratio (d/D) of a population of impact craters, in combination with crater count statistics, can yield valuable insights regarding rates of erosion and burial[5]. Motivated by the great abundance of available planetary surface image data, the goal of this project is to develop an efficient way to estimate d/D from satellite images of impact craters for which stereo information is not available [6]. We set out to develop and train a machine learning algorithm to extract d/D from a dataset of synthetic impact crater images for which model d/D is known. The applications of machine learning to planetary science are numerous and diverse [7], including automatic planetary surface mapping [8] and the detection of impact craters [9]. Our algorithm makes use of Support Vector Regression (SVR), which is a type of Support Vector Machine (SVM) [10, 11].SVMs are a branch of supervised machine learning valued for their straightforward implementation and versatility in solving both classification and regression problems. In regression analysis, an SVR algorithm produces a hyperplane function to fit the training data points, as well as an ε-tube that surrounds the hyperplane. Tunable hyperparameters include the width of the ε-tube (ε) and the amount an algorithm is penalized for points which fall outside the ε-tube.

L R Chin↗

Tuning a variational autoencoder for data accountability problem in the Mars Science Laboratory ground data system

The Mars Curiosity rover is frequently sending back engineering and science data that goes through a pipeline of systems before reaching its final destination at the mission operations center making it prone to volume loss and data corruption. A ground data system analysis (GDSA) team is charged with the monitoring of this flow of information and the detection of anomalies in that data in order to request a re-transmission when necessary. This work presents ∆-MADS, a derivative-free optimization method applied for tuning the architecture and hyperparameters of a variational autoencoder trained to detect the data with missing patches in order to assist the GDSA team in their mission.

Lakhmiri, Dounia↗

A Generalized Approach to Aircraft Trajectory Prediction via Supervised Deep Learning

As research advances diverse forms and missions of aircraft, the National Airspace System (NAS) will become increasingly crowded, limiting current communications resources to accommodate aviation operations. Ongoing research proposes a paradigm of airspace communications, such that resources are autonomously and dynamically allocated via intelligent agents; this allocation requires accurate representations of the NAS, including the predicted positions of aircraft. State-of-the-art research emphasizes the importance of a hybrid-recurrent framework for trajectory prediction and compares the impact of commonly considered weather data on prediction accuracy. However, current research has been limited in its scope of efforts, frequently utilizing a unique flight route, architecture, set of weather data, and date range. This article considers the challenges of generalizing hybrid-recurrent predictive models for flight trajectories. Results illustrate an increase in error variance when identical models are trained over a generalized set of flights; this may be mitigated with careful tuning of hyperparameters, both in the network structure and optimization algorithms. Even so, an irreducible vertical error was identified, resulting from the complex takeoff and landing procedures which can not be correlated to functions of weather or additional assumptions of aircraft behavior. Finally, the use of a test route indicates that generalized models still do not possess sufficient knowledge for general aircraft predictions, with mean error increases ranging from 70-500%. These results illustrate the need for continued efforts on improving model versatility, as well as potential limitations for spectrum allocation near airports and other centers.

Nathan Schimpf↗

Low-cost Quantification of Fluid Flow Parameter Sensitivity using Reduced-order Modeling

Uncertainty quantification of computational fluid dynamics (CFD) simulations is a complicated procedure which still relies in many cases on engineering judgment and factors of safety. This is in part because the computational cost of measuring the simulation's sensitivity to all meaningful parameters (e.g., body surface roughness) and hyperparameters (e.g., subiteration convergence criterion) is intractable for even a single simulation. Reduced-order modeling dramatically lowers this computational cost of simulating fluid flows, but usually only where similar data is already available. In this work, fluid reduced-order models are utilized to quantify a flow's sensitivity to certain physical parameters for the purposes of improved uncertainty quantification. Characteristic observability and sensitivity are both explored. The resulting sensitivity quantifications enable more informed CFD frameworks and more rigorous uncertainty bounds on the resulting data.

uncertainty quantification↗

A Robust Machine Learning Schema for Developing, Maintaining, and Disseminating Machine Learning Models

Recent advances in the development of machine learning (ML) algorithms have enabled the creation of predictive models that can improve decision making, decrease computational cost, and improve efficiency in a variety of fields. As an organization begins to develop and implement such models, the data used in the training, validation, and testing of ML models, the model parameters, and the use cases or limitations of the models must be properly stored to ensure models are both fully traceable and used correctly. In the context of predicting material behavior, advances in computationally intense, physics-based modeling of material behavior at various length scales and the emergence of Integrated Computational Materials Engineering (ICME) have driven the need for developing data-driven surrogate models of the physics-based simulation tools using ML techniques. Surrogate model development allows for accurate material behavior prediction at a fraction of the cost of its physics-based counterpart, allowing for multiscale simulations of real-world applications, further enabling the ability to design fit-for-purpose materials for a reasonable computational investment. However, training such models requires extensive data, and thus, effective data management is necessary to reach the full potential that ML can offer to material design and ICME. This paper proposes a generalized, robust schema that allows organizations to store both real (experimental) and virtual (simulation) data used to train ML models and the defining model parameters and architectures within the Granta MI Platform. The developed schema allows for various types of data inputs and outputs, including single point values, time-series data, and images that can be used in the prediction of material behavior, while following outlined best practices for effective data management. An effective schema for ML data and models can help prevent the recreation of virtual/real training data and surrogate models, help reduce the time to create new models similar to existing ones by offering a starting point in the hyperparameter determination stages, minimize resources devoted to verification and validation (V&V) and certification of models, and ensure that data and surrogate models are not misused due to full traceability of both the data and ML model. It also allows organizations access to models that have already been developed, such that they can be used in the design of new materials, enabling the overall goals of ICME.

Brandon L. Hearley↗