Model Card for PickerXL
Explore the source record for details and available documents.
58 GEOSCIENCES↗
Engineering topics
Rose, Derek
Publications and source records attributed to Rose, Derek.
Repeatable Science in Seismic Model Training
Explore the source record for details and available documents.
Torres Polanco, Anibely↗
ICDARTS: Improving the Stability of Cyclic DARTS
Cyclic DARTS (CDARTS) is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently. This training protocol aims to optimize the search process and evaluate the deep evaluation network comprised of discretized candidate operations. However, this approach introduces a loss function for the evaluation network dependent on the search network. The dissimilarity between the evaluation network’s loss function used during the search and retraining phases results in a search network that is a sub-optimal proxy for the final evaluation network accessed during retraining. We present a revised approach that removes the dependency of the evaluation network weights upon those of the search network. In addition, we introduce a modified process for relaxing the search network’s zero operations that allows these operations to be retained in the final evaluation networks.
Herron, Emily↗
Layer Time Control for Large Scale Additive Manufacturing Using High Performance Computing
This work proposes to optimize an additive manufacturing AM process to reduce energy and printing cost. The polymer AM process is inherently dependent on the time-temperature history of each layer to maintain geometric tolerances and mechanical integrity. Our preliminary study shows that regression-based layer time control model using thermal images could result in up to 30% build time reduction for simple geometries. This proposed work would use high-performance computing (HPC) to couple the data-driven model with thermal simulation for better predicting layer temperature profiles, improving throughput of large-scale additive manufacturing, and reducing its energy cost. We have developed a method to optimize a layer deposition time (a.k.a. layer time) for large-scale AM via physics-based simulations. A long layer time leads to an over-cooled surface on which a new layer is deposited, and therefore, it may result in a weak bonding or debonding between layers, cracking, or warping. A short layer time leads to a high temperature of the structure due to insufficient cooling, and therefore, the structure may not be stiff enough and may collapse during manufacturing. Therefore, it is important to estimate the optimal layer time in additive manufacturing for a high-quality product. The temperature of a top layer right before deposition is recommended to be slightly higher than the glass temperature of the material. A temperature cooling was approximated to an exponential function of time, and the optimized layer time was obtained based on a target temperature while maintaining a minimal printing time. The material used is carbon fiber-reinforced polycarbonate (CF/PC), and the large-scale deposition system used is LSAM TM from Thermwood Corporation. Three different layer time cases were used for experiments, and a series of thermal images were obtained via an infra-red (IR) camera during the entire AM processes. AM process simulations were performed using a finite element method and the temperature profiles from the simulation were in good agreements with those from experiments. The layer time optimization was performed based on the temperature profiles from the simulations. A layer temperature with the optimal layer time was confirmed as the target temperature through simulation. In addition to the development of a layer time optimization method, we have developed a numerical framework for AM simulation with element activations in sync with toolpath, based on an open source finite element framework, DEAL.II. A major portion of this work was presented at SAMPE 2022 Conference and Exhibition on May 2022, and published in Proceedings of SAMPE 2022.
42 ENGINEERING↗
Localized Defect Detection from Spatially Mapped, In-Situ Process Data With Machine Learning
In powder bed fusion additive manufacturing, machines are often equipped with in-situ sensors to monitor the build environment as well as machine actuators and subsystems. The data from these sensors offer rich information about the consistency of the fabrication process within a build and across builds. This information may be used for process monitoring and defect detection; however, little has been done to leverage this data from the machines for more than just coarse-grained process monitoring. In this work we demonstrate how these inherently temporal data may be mapped spatially by leveraging scan path information. We then train a XGBoost machine learning model to predict localized defects—specifically soot–using only the mapped process data of builds from a laser powder bed fusion process as input features. The XGBoost model offers a feature importance metric that will help to elucidate possible relationships between the process data and observed defects. Finally, we analyze the model performance spatially and rationalize areas of greater and lesser performance.
3D printing↗
Accurate and Accelerated Neuromorphic Network Design Leveraging A Bayesian Hyperparameter Pareto Optimization Approach
Neuromorphic systems allow for extremely efficient hardware implementations for neural networks (NNs). In recent years, several algorithms have been presented to train spiking NNs (SNNs) for neuromorphic hardware. However, SNNs often provide lower accuracy than their artificial NNs (ANNs) counterparts or require computationally expensive and slow training/inference methods. To close this gap, designers typically rely on reconfiguring SNNs through adjustments in the neuron/synapse model or training algorithm itself. Nevertheless, these steps incur significant design time, while still lacking the desired improvement in terms of training/inference times (latency). Designing SNNs that can mimic the accuracy of ANNs with reasonable training times is an exigent challenge in neuromorphic computing. In this work, we present an alternative approach that looks at such designs as an optimization problem rather than algorithm or architecture redesign. We develop a versatile multiobjective hyperparameter optimization (HPO) for automatically tuning HPs of two state-of-the-art SNN training algorithms, SLAYER and HYBRID. We emphasize that, to the best of our knowledge, this is the first work trying to improve SNNs’ computational efficiency, accuracy, and training time using an efficient HPO. We demonstrate significant performance improvements for SNNs on several datasets without the need to redesign or invent new training algorithms/architectures. Our approach results in more accurate networks with lower latency and, in turn, higher energy efficiency than previous implementations. In particular, we demonstrate improvement in accuracy and more than 5× reduction in the training/inference time for the SLAYER algorithm on the DVS Gesture dataset. In the case of HYBRID, we demonstrate 30% reduction in timesteps while surpassing the accuracy of the state-of-the-art networks on CIFAR10. Further, our analysis suggests that even a seemingly minor change in HPs could change the accuracy by 5 - 6×.
Parsa, Maryam↗
Low Size, Weight, and Power Neuromorphic Computing to Improve Combustion Engine Efficiency
Neuromorphic computing offers one path forward for AI at the edge. However, accessing and effectively utilizing a neuromorphic hardware platform is non-trivial. In this work, we present a complete pipeline for neuromorphic computing at the edge, including a small, inexpensive, low-power, FPGA-based neuromorphic hardware platform, a training algorithm for designing spiking neural networks for neuromorphic hardware, and a software framework for connecting those components. We demonstrate this pipeline on a real-world application, engine control for a spark-ignition internal combustion engine. We illustrate how we connect engine simulations with neuromorphic hardware simulations and training software to produce hardware-compatible spiking neural networks that perform engine control to improve fuel efficiency. We present initial results on the performance of these spiking neural networks and illustrate that they outperform open-loop engine control. We also give size, weight, and power estimates for a deployed solution of this type.
Schuman, Catherine↗
Hyperparameter Optimization in Binary Communication Networks for Neuromorphic Deployment
Training neural networks for neuromorphic deployment is non-trivial. There have been a variety of approaches proposed to adapt back-propagation or back-propagation-like algorithms appropriate for training. Considering that these networks often have very different performance characteristics than traditional neural networks, it is often unclear how to set either the network topology or the hyperparameters to achieve optimal performance. In this work, we introduce a Bayesian approach for optimizing the hyperparameters of an algorithm for training binary communication networks that can be deployed to neuromorphic hardware. We show that by optimizing the hyperparameters on this algorithm for each dataset, we can achieve improvements in accuracy over the previous state-of-the-art for this algorithm on each dataset (by up to 15 percent). This jump in performance continues to emphasize the potential when converting traditional neural networks to binary communication applicable to neuromorphic hardware.
Parsa, Maryam↗
Evolving Energy Efficient Convolutional Neural Networks
As deep neural networks have been deployed in more and more applications over the past half decade and are finding their way into an ever increasing number of operational systems, their energy consumption becomes a concern whether running in the datacenter or on edge devices. Hyperparameter optimization and automated network design for deep learning is a quickly growing field, but much of the focus has remained only on optimizing for the performance of the machine learning task. In this work, we demonstrate that the best performing networks created through this automated network design process have radically different computational characteristics (e.g. energy usage, model size, inference time), presenting the opportunity to utilize this optimization process to make deep learning networks more energy efficient and deployable to smaller devices. Optimizing for these computational characteristics is critical as the number of applications of deep learning continues to expand.
Young, Steven↗
Exascale Deep Learning to Accelerate Cancer Research
Deep learning, through the use of neural networks, has demonstrated remarkable ability to automate many routine tasks when presented with sufficient data for training. The neural network architecture (e.g. number of layers, types of layers, connections between layers, etc.) plays a critical role in determining what, if anything, the neural network is able to learn from the training data. The trend for neural network architectures, especially those trained on ImageNet, has been to grow ever deeper and more complex. The result has been ever increasing accuracy on benchmark datasets with the cost of increased computational demands. In this paper we demonstrate that neural network architectures can be automatically generated, tailored for a specific application, with dual objectives: accuracy of prediction and speed of prediction. Using MENNDL- an HPC-enabled software stack for neural architecture search-we generate a neural network with comparable accuracy to state-of-the-art networks on a cancer pathology dataset that is also 16× faster at inference. The speedup in inference is necessary because of the volume and velocity of cancer pathology data; specifically, the previous state-of-the-art networks are too slow for individual researchers without access to HPC systems to keep pace with the rate of data generation. Our new model enables researchers with modest computational resources to analyze newly generated data faster than it is collected.
Patton, Robert↗
Visualization System for Evolutionary Neural Networks for Deep Learning
Deep learning is actively used in a wide range of fields for scientific discovery. To effectively apply deep learning to a particular problem, it is important to select an appropriate network architecture and other hyper-parameters (at each layer). Evolving architectures and hyper-parameters using a genetic algorithm is one current approach to search the huge space of all possible configurations to find those more optimal for the problem. However, examining an evolutionary process and tuning the genetic algorithm are challenging, pushing most users to treat the process as a black box. To address this challenge, we propose a visualization system for evolutionary neural networks for deep learning. The key feature of our visualization system is to provide a visual analytics environment for evaluating a genetic algorithm in order to improve the underlying operations to reduce time to find good solutions. Our system is able to not only visualize how a genetic algorithm traverses its search space but also allows users to examine evolving networks in-depth to get insights to improve performance through interactive visualization components.
Chae, Junghoon↗
A Two-Tier Convolutional Neural Network for Combined Detection and Segmentation in Biological Imagery
Deep learning techniques have been useful for modern microscopy imaging techniques to further study and analyze biological structures and organs. Convolutional neural networks (CNN) have improved 2D object detection, localization, and segmentation. For imagery containing biological structures with depth, it is especially desirable to perform these tasks in 3D. Traditionally, performing these tasks simultaneously in 3D has proven to be computationally expensive. Currently available methodologies thus largely work to segment 3D objects from 2D images (without context from captured 3D volumes). In this work, we present a novel approach to perform fast and accurate localization, detection, and segmentation of volumes containing cells. Specifically, in our method, we modify and tune two state-of-the-art CNNs, namely 2D YOLOv2 and 3D U-Net, and combine them with a new fusion and image processing algorithms. Annotated volumes in this space are limited, and we have created synthetic data that mimics actual structures for training and testing our proposed approach. Promising results on this test data demonstrate the value of the technique and offers a methodology for 3D cell analysis in real microscopy imagery.
Ziabari, Amir↗
A 2.5d Yolo-Based Fusion Algorithm for 3d Localization Of Cells
Advances in microscopy techniques such as lattice-light-sheet, confocal, two-photon, and electron microscopy have enabled the visualization of 3D image volumes of tightly packed cells, extracellular structures in tissues, organelles, and subcellular components inside cells. These images sampled by 2D projections are often not accurately interpreted even by human experts. As a use case we focus on 3D image volumes of tightly packed nuclei in brain tissue. Due to out-of-plane excitation and low resolution in the z-axis, non-overlapping cells appear as overlapping 3D volumes and make detecting individual cells challenging. On the other hand, running 3D detection algorithms is computationally expensive and infeasible for large datasets. In addition, most existing 3D algorithms are designed to extract 3D objects by identifying the depth in the 2D images. In this work, we propose a YOLO-based 2.5D fusion algorithm for 3D localization of individual cells in densely packed volumes of nuclei. The proposed method fuses 2D detection of nuclei in sagittal, coronal, and axial planes and predicts six coordinates of the 3D bounding cubes around the detected 3D cells. Promising results were obtained on multiple examples of synthetic dense volumes of nuclei imitating confocal microscopy experimental datasets.
Ziabari, Amir↗
Reinforcement Learning-based Traffic Control to Optimize Energy Usage and Throughput (CRADA report)
The US roadways are critical to meeting the mobility and economic needs of the nation. The United States uses 28% of its energy in moving goods and people, with approximately 60% of that utilized by cars, light trucks, and motorcycles. Thus, improved transportation efficiency is vital to America’s economic progress. The increasing congestion and energy resource requirements of transportation systems for metropolitan areas require research in methods to improve and optimize control methods. Coordinating and optimizing traffic in urban areas may introduce hundreds of thousands of vehicles and traffic management systems, which can require high performance computing (HPC) resources to model and manage. In this work, we seek to use machine learning, computer vision, and HPC to improve the energy efficiency aspects of traffic control by leveraging GRIDSMART traffic cameras as sensors for adaptive traffic control, with a sensitivity to the fuel consumption characteristics of the traffic in the camera’s visual field. Traffic control use cases using reinforcement learning have been published and achieved good results. Surveys from DOE national laboratories estimate that the fuel cost of idling is 6 billion gallons wasted annually [Argonne National Laboratory, 2019]. GRIDSMART cameras—an existing, fielded commercial product—sense the presence of vehicles at intersections and replace more conventional sensors (such as inductive loops) to issue calls to traffic control. These cameras, which have horizon-to-horizon view, offer the potential for an improved view of the traffic environment which can be used to generate better control algorithms.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
167-PFlops deep learning for electron microscopy: from learning physics to atomic manipulation
An artificial intelligence system called MENNDL, which used 25,200 NVIDIA Volta GPUs on Oak Ridge National Laboratory's Summit machine, automatically designed an optimal deep learning network in order to extract structural information from raw atomic-resolution microscopy data. In a few hours, MENNDL creates and evaluates millions of networks using a scalable, parallel, asynchronous genetic algorithm augmented with a support vector machine to automatically find a superior deep learning network topology and hyper-parameter set than a human expert can find in months. For the application of electron microscopy, the system furthers the goal of improving our understanding of the electron-beam-matter interactions and real-time image-based feedback, which enables a huge step beyond human capacity towards nanofabricating materials automatically. MENNDL has been scaled to the 4,200 available nodes of Summit achieving a measured 152.5 PFlops, with an estimated sustained performance of 167 PFlops when the entire machine is available.
Patton, Robert↗
Evolving Deep Networks Using HPC
While a large number of deep learning networks have been studiedand published that produce outstanding results on natural imagedatasets, these datasets only make up a fraction of those to whichdeep learning can be applied. These datasets include text data, audiodata, and arrays of sensors that have very different characteristicsthan natural images. As these “best” networks for natural imageshave been largely discovered through experimentation and cannotbe proven optimal on some theoretical basis, there is no reasonto believe that they are the optimal network for these drasticallydifferent datasets. Hyperparameter search is thus often a very im-portant process when applying deep learning to a new problem. Inthis work we present an evolutionary approach to searching thepossible space of network hyperparameters and construction thatcan scale to 18, 000 nodes. This approach is applied to datasets ofvarying types and characteristics where we demonstrate the abilityto rapidly find best hyperparameters in order to enable practitionersto quickly iterate between idea and result.
Young, Steven↗
Fast, Simple Calcium Imaging Segmentation with Fully Convolutional Networks
Calcium imaging is a technique for observing neuron activity as a series of images showing indicator fluorescence over time. Manually segmenting neurons is time-consuming, leading to research on automated calcium imaging segmentation (ACIS). We evaluated several deep learning models for ACIS on the Neurofinder competition datasets and report our best model: U-Net2DS, a fully convolutional network that operates on 2D mean summary images. U-Net2DS requires minimal domain-specific pre/post-processing and parameter adjustment, and predictions are made on full 512×512 images at ≈9K images per minute. It ranks third in the Neurofinder competition (F1=0.57) and is the best model to exclusively use deep learning. We also demonstrate useful segmentations on data from outside the competition. The model’s simplicity, speed, and quality results make it a practical choice for ACIS and a strong baseline for more complex models in the future.
Klibisz, Alex↗