SEARCH · Engineering Papers

Results for “Automatic Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Quantitative multi-image analysis in metals research

Abstract Quantitative multi-image analysis (QMA) is the systematic extraction of new information and insight through the simultaneous analysis of multiple, related images. We present examples illustrating the potential for QMA to advance materials research in multi-image characterization, automatic feature identification, and discovery of novel processing-structure–property relationships. We conclude by discussing opportunities and challenges for continued advancement of QMA, including instrumentation development, uncertainty quantification, and automatic parsing of literature data. Graphical abstract

36 MATERIALS SCIENCE↗

DOE OSTI2016

SE Requirements Development Tool User Guide

The LANL Systems Engineering Requirements Development Tool (SERDT) is a data collection tool created in InfoPath for use with the Los Alamos National Laboratory’s (LANL) SharePoint sites. Projects can fail if a clear definition of the final product requirements is not performed. For projects to be successful requirements must be defined early in the project and those requirements must be tracked during execution of the project to ensure the goals of the project are met. Therefore, the focus of this tool is requirements definition. The content of this form is based on International Council on Systems Engineering (INCOSE) and Department of Defense (DoD) process standards and allows for single or collaborative input. The “Scoping” section is where project information is entered by the project team prior to requirements development, and includes definitions and examples to assist the user in completing the forms. The data entered will be used to define the requirements and once the form is filled out, a “Requirements List” is automatically generated and a Word document is created and saved to a SharePoint document library. SharePoint also includes the ability to download the requirements data defined in the InfoPath from into an Excel spreadsheet. This User Guide will assist you in navigating through the data entry process.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2019

Automatic Analyte-Ion Recognition and Background Removal for Ambient Mass-Spectrometric Data Based on Cross-Correlation

Ambient mass spectrometry is a powerful approach for rapid, high-throughput, and direct sample analysis. Due to the open-air desorption and ionization processes, random fluctuations of ambient conditions can lead to large variances in mass-spectral signals over time. The mass-spectral data also can be further complicated due to multiple analytes present in the sample, background-ion signals stemming from the desorption/ionization source itself, and other laboratory-specific conditions (e.g., ambient laboratory air, nearby hardware). Thus, background removal and analyte-ion recognition can be quite difficult, particularly in non-targeted analyses. Here, we demonstrate the use of a cross-correlation-based approach to exploit chemical information encoded in the time domain to group/categorize mass-spectral peaks from a single analysis dataset. Ions that originate from ambient (or other) background species were readily flagged and removed from spectra; the result was a decrease in mass-spectral complexity by over 70% due to the removal of these background ions. Meanwhile, analyte ions were differentiated and categorized based on their time-domain profiles. With sufficient mass resolving-power and mass-spectral acquisition rate, isolated mass spectra containing ions from the same species in a sample could be extracted, leading to a reduction in mass-spectral complexity by more than 98% in some cases. The cross-correlation approach was tested with different ionization sources as well as reproducible and irreproducible sample introduction. Software built in-house enabled fully automated data processing, which can be performed within a few seconds. Ultimately, this approach provides an additional dimension of analyte separation in ambient mass-spectrometric analyses with information that is already recorded throughout the analysis. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2018

A Virtual Reality Visualization Tool for Neuron Tracing

Tracing neurons in large-scale microscopy data is crucial to establishing a wiring diagram of the brain, which is needed to understand how neural circuits in the brain process information and generate behavior. Automatic techniques often fail for large and complex datasets, and connectomics researchers may spend weeks or months manually tracing neurons using 2D image stacks. We present a design study of a new virtual reality (VR) system, developed in collaboration with trained neuroanatomists, to trace neurons in microscope scans of the visual cortex of primates. We hypothesize that using consumer-grade VR technology to interact with neurons directly in 3D will help neuroscientists better resolve complex cases and enable them to trace neurons faster and with less physical and mental strain. We discuss both the design process and technical challenges in developing an interactive system to navigate and manipulate terabyte-sized image volumes in VR. Using a number of different datasets, we demonstrate that, compared to widely used commercial software, consumer-grade VR presents a promising alternative for scientists.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

TH-A-18C-04: Ultrafast Cone-Beam CT Scatter Correction with GPU-Based Monte Carlo Simulation

Purpose: Scatter artifacts severely degrade image quality of cone-beam CT (CBCT). We present an ultrafast scatter correction framework by using GPU-based Monte Carlo (MC) simulation and prior patient CT image, aiming at automatically finish the whole process including both scatter correction and reconstructions within 30 seconds. Methods: The method consists of six steps: 1) FDK reconstruction using raw projection data; 2) Rigid Registration of planning CT to the FDK results; 3) MC scatter calculation at sparse view angles using the planning CT; 4) Interpolation of the calculated scatter signals to other angles; 5) Removal of scatter from the raw projections; 6) FDK reconstruction using the scatter-corrected projections. In addition to using GPU to accelerate MC photon simulations, we also use a small number of photons and a down-sampled CT image in simulation to further reduce computation time. A novel denoising algorithm is used to eliminate MC scatter noise caused by low photon numbers. The method is validated on head-and-neck cases with simulated and clinical data. Results: We have studied impacts of photo histories, volume down sampling factors on the accuracy of scatter estimation. The Fourier analysis was conducted to show that scatter images calculated at 31 angles are sufficient to restore those at all angles with <0.1% error. For the simulated case with a resolution of 512×512×100, we simulated 10M photons per angle. The total computation time is 23.77 seconds on a Nvidia GTX Titan GPU. The scatter-induced shading/cupping artifacts are substantially reduced, and the average HU error of a region-of-interest is reduced from 75.9 to 19.0 HU. Similar results were found for a real patient case. Conclusion: A practical ultrafast MC-based CBCT scatter correction scheme is developed. The whole process of scatter correction and reconstruction is accomplished within 30 seconds. This study is supported in part by NIH (1R01CA154747-01), The Core Technology Research in Strategic Emerging Industry, Guangdong, China (2011A081402003)

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

Evaluation of a deformable registration algorithm for subsequent lung computed tomography imaging during radiochemotherapy

Purpose: Rating both a lung segmentation algorithm and a deformable image registration (DIR) algorithm for subsequent lung computed tomography (CT) images by different evaluation techniques. Furthermore, investigating the relative performance and the correlation of the different evaluation techniques to address their potential value in a clinical setting. Methods: Two to seven subsequent CT images (69 in total) of 15 lung cancer patients were acquired prior, during, and after radiochemotherapy. Automated lung segmentations were compared to manually adapted contours. DIR between the first and all following CT images was performed with a fast algorithm specialized for lung tissue registration, requiring the lung segmentation as input. DIR results were evaluated based on landmark distances, lung contour metrics, and vector field inconsistencies in different subvolumes defined by eroding the lung contour. Correlations between the results from the three methods were evaluated. Results: Automated lung contour segmentation was satisfactory in 18 cases (26%), failed in 6 cases (9%), and required manual correction in 45 cases (66%). Initial and corrected contours had large overlap but showed strong local deviations. Landmark-based DIR evaluation revealed high accuracy compared to CT resolution with an average error of 2.9 mm. Contour metrics of deformed contours were largely satisfactory. The median vector length of inconsistency vector fields was 0.9 mm in the lung volume and slightly smaller for the eroded volumes. There was no clear correlation between the three evaluation approaches. Conclusions: Automatic lung segmentation remains challenging but can assist the manual delineation process. Proven by three techniques, the inspected DIR algorithm delivers reliable results for the lung CT data sets acquired at different time points. Clinical application of DIR demands a fast DIR evaluation to identify unacceptable results, for instance, by combining different automated DIR evaluation methods.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

Real-time and Autonomous Water Quality Monitoring System Based on Remotely Operated Vehicle

Existing water quality monitoring systems near hydropower facilities are limited by the lack of mobility of the sensors’ carrier platform. Most systems use a buoy, a mounting fixture attached to a solid structure, or a human worker, which significantly limits the selection of the sampling sites and poses safety risks during data collection and equipment maintenance. To improve on this technology, we developed an autonomous water quality monitoring system that can operate in dangerous water environments near hydropower facilities for water sampling at multiple locations. The goal is to enable safe, timely, and comprehensive water-quality data collection; maximize power generation revenue with improved operational control; and reduce Federal Energy Regulatory Commission and state water quality monitoring costs for compliance. The system incorporates a remotely operated vehicle as the mobile monitoring platform, a dissolved oxygen sensor for monitoring water quality, a tether management system for automatically winding the tether, a solar mobile docking platform for suppling power to the ROV, and a web-based graphical user interface for data post-processing and visualization. In addition, preliminary field research are presented to demonstrate the system capabilities.

Salalila, Aljon L.↗

DOE OSTI2018

Resource Aware Intelligent Network Services (RAINS) Final Technical Report

The Resource Aware Intelligent Network Services (RAINS) project conducted research and developed technologies in the area of cyber infrastructure resource modeling and computation. The goal of this work was to provide a foundation to enable intelligent, software defined services which spanned the network AND the resources which connect to the network. A Multi-Resource Service Plane (MRSP) was defined, which allows resource owners/managers to locate and place themselves from a topology and service availability perspective within the dynamic networked cyberinfrastructure ecosystem. The MRSP enables the presentation of integrated topology views and computation results which can include resources across the spectrum of compute, storage, and networks. The RAINS project developed MSRP includes the following key components: i) Multi-Resource Service (MRS) Ontology/Multi-Resource Markup Language (MRML), ii) Resource Computation Engine (RCE), iii) Modular Driver Framework (to allow integration of a variety of external resources). The MRS/MRML is a general and extensible modeling framework that allows for resource owners to model, or describe, a wide variety of resource types. All resources are described using three categories of elements: Resources, Services, and Relationships between the elements. This modeling framework defines a common method for the transformation of cyber infrastructure resources into data in the form of MRML models. In order to realize this infrastructure datification, the RAINS project developed a model based computation system, i.e. “RAINS Computation Engine (RCE)”. The RCE has the ability to ingest, process, integrate, and compute based on automatically generated MRML models. The RCE interacts with the resources thru system drivers which are specific to the type of external network or resource controller. The RAINS project developed a modular and pluggable driver system which facilities a variety of resource controllers to automatically generate, maintain, and distribute MRML based resource descriptions. Once all of the resource topologies are absorbed by the RCE, a connected graph of the full distributed system topology is constructed, which forms the basis for computation and workflow processing. The RCE includes a Modular Computation Element (MCE) framework which allows for tailoring of the computation process to the specific set of resources under control, and the services desired. The input and output of an MCE are both model data based on MRS/MRML ontology and schema. Some of the RAINS project accomplishments include: Development of general and extensible multi-resource modeling framework; Design of a Resource Computation Engine (RCE) system which includes the following key capabilities; Absorb a variety of multi-resource model types and build integrated models; Novel architecture which uses model based communications across the full stack for all Flexible provision of abstract or intent based user facing interfaces; Workflow processing based on model descriptions; Release of the RCE as an open source software; Deployment of RCE in the University of Maryland/Mid-Atlantic Crossroad ScienceDMZ in prototype mode with a plan under way to transition to production; Deployment at the Argonne National Laboratory DTN Facility in prototype mode; Selection of RCE by the DOE SENSE (SDN for End-to-end Networked Science at the Exascale) project as the basis for their orchestration service.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Comparing Automatic CME Detections in Multiple LASCO and SECCHI Catalogs

With the creation of numerous automatic detection algorithms, a number of different catalogs of coronal mass ejections (CMEs) spanning the entirety of the Solar and Heliospheric Observatory ( SOHO ) Large Angle Spectrometric Coronagraph (LASCO) mission have been created. Some of these catalogs have been further expanded for use on data from the Solar Terrestrial Earth Observatory ( STEREO ) Sun Earth Connection Coronal and Heliospheric Investigation (SECCHI) as well. We compare the results from different automatic detection catalogs (Solar Eruption Event Detection System (SEEDS), Computer Aided CME Tracking (CACTus), and Coronal Image Processing (CORIMP)) to ensure the consistency of detections in each. Over the entire span of the LASCO catalogs, the automatic catalogs are well correlated with one another, to a level greater than 0.88. Focusing on just periods of higher activity, these correlations remain above 0.7. We establish the difficulty in comparing detections over the course of LASCO observations due to the change in the instrument image cadence in 2010. Without adjusting catalogs for the cadence, CME detection rates show a large spike in cycle 24, despite a notable drop in other indices of solar activity. The output from SEEDS, using a consistent image cadence, shows that the CME rate has not significantly changed relative to sunspot number in cycle 24. These data, and mass calculations from CORIMP, lead us to conclude that any apparent increase in CME rate is a result of the change in cadence. We study detection characteristics of CMEs, discussing potential physical changes in events between cycles 23 and 24. We establish that, for detected CMEs, physical parameters can also be sensitive to the cadence.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

MPLBM-UT: Multiphase LBM library for permeable media analysis

MPLBM-UT is a specialized lattice-Boltzmann library that makes running single- and two-phase flow simulations in porous media accessible to everyone. We provide a suite of tools to pre-process computational domains for simulation, to set up custom boundary conditions, to run simulations, to post-process simulation outputs, and to visualize simulation results and data. All of these tools are easily accessible to users through the mplbm_utils Python package included in and automatically installed with MPLBM-UT. The high-performance, highly parallel library Palabos is used as the solver backend. MPLBM-UT is easily deployed in a variety of systems, from laptops to supercomputer clusters. MPLBM-UT also features multiple examples and benchmark templates that allow for fast prototyping of different porous media problems. We also provide an interface for reading in different file types and downloading domains from the Digital Rocks Portal to perform simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Change detection and classification of land cover in multispectral satellite imagery using clustering of sparse approximations (CoSA) over learned feature dictionaries

Neuromimetic machine vision and pattern recognition algorithms are of great interest for landscape characterization and change detection in satellite imagery in support of global climate change science and modeling. We present results from an ongoing effort to extend machine vision methods to the environmental sciences, using adaptive sparse signal processing combined with machine learning. A Hebbian learning rule is used to build multispectral, multiresolution dictionaries from regional satellite normalized band difference index data. Land cover labels are automatically generated via our CoSA algorithm: Clustering of Sparse Approximations, using a clustering distance metric that combines spectral and spatial textural characteristics to help separate geologic, vegetative, and hydrologie features. We demonstrate our method on example Worldview-2 satellite images of an Arctic region, and use CoSA labels to detect seasonal surface changes. In conclusion, our results suggest that neuroscience-based models are a promising approach to practical pattern recognition and change detection problems in remote sensing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Automated Classification of Vehicle Movements at Signalized Intersections Using Vehicle Trajectories

Accurate vehicle movement classification through signalized intersections is of paramount importance to the analysis of intersection performance and the optimization of traffic control strategies. Conventional techniques for tracking vehicle turning movements depend on infrastructure-based strategies like human counts, loop detectors, and video analytics, all of which are costly, prone to errors, and spatially constrained. High-frequency trajectory data can be utilized to determine vehicle movement patterns in a scalable and infrastructure-independent method due to the adoption of connected vehicles (CVs). In recent years, several studies have utilized connected vehicle data to generate performance measures. Most of the trajectory-based performance measures approaches, however, require map matching-i.e., extracting geospatial references from maps to identify the movements that individual vehicles make at a signalized intersection. These approaches are often time-consuming and hinder scalability since geographic features need to be provided for an analysis to be conducted. Map matching methods are prone to errors as different map versions change these geographic features. This research presents a novel automatic classification pipeline that uses CV trajectory data to classify vehicle movements at signalized crossings, specifically pass-through left-turn and right-turn maneuvers. The process starts by filtering trips that cross a spatial bounding box that has been defined at the target intersection. Approach and departure headings for each trajectory crossing the boundary are computed and are clustered together to identify dominant movements. The proposed algorithm is used to classify the movement of vehicles at 10 intersections in the state of California, and the results indicate that the algorithm can classify movements at these intersections with varying traffic volumes and road network configurations, all in a map-less framework with no need for conflation of vehicle trajectories to a digital base map.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2015

Towards Exascale Scientific Metadata Management

Advances in technology and computing hardware are enabling scientists from all areas of science to produce massive amounts of data using large-scale simulations or observational facilities. In this era of data deluge, effective coordination between the data production and the analysis phases hinges on the availability of metadata that describe the scientific datasets. Existing workflow engines have been capturing a limited form of metadata to provide provenance information about the identity and lineage of the data. However, much of the data produced by simulations, experiments, and analyses still need to be annotated manually in an ad hoc manner by domain scientists. Systematic and transparent acquisition of rich metadata becomes a crucial prerequisite to sustain and accelerate the pace of scientific innovation. Yet, ubiquitous and domain-agnostic metadata management infrastructure that can meet the demands of extreme-scale science is notable by its absence. To address this gap in scientific data management research and practice, we present our vision for an integrated approach that (1) automatically captures and manipulates information-rich metadata while the data is being produced or analyzed and (2) stores metadata within each dataset to permeate metadata-oblivious processes and to query metadata through established and standardized data access interfaces. We motivate the need for the proposed integrated approach using applications from plasma physics, climate modeling and neuroscience, and then discuss research challenges and possible solutions.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

3-D advanced gas-cooled nuclear reactor fuel channel reconstruction using structure-from-motion - 110

During planned, periodic outages, a selection of fuel channels within the UK fleet of Advanced Gas-cooled Reactor (AGR) cores are inspected using specialist tools which record video footage and other sensory data for each channel which undergoes inspection. Current visualization techniques comprise of manually produced montages by inspection engineers of points of interest (i.e. structural defects) and 2-D panoramic images of the fuel channels automatically produced using bespoke image stitching software. Both techniques however provide limited structural information due to the loss of depth data as a result of the image formation process. By recovering the depth information from the footage, a 3D model could be constructed and subsequently, allow for more accurate profiling of specific defects observed during inspection in addition to obtaining the fuel channels structure using existing footage. This work explores the preliminary application of a 3-D visualization technique known as Structure-from-Motion (SfM) which aims to obtain 3-D information by exploiting image correspondences across multiple viewpoints of the same scene in the RVI footage. This paper investigates the difficulties of applying state-of-the-art SfM to RVI footage and we present new techniques to improve feature correspondence searching in repetitive, nondescript environments. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2023

SolarAPP+ Performance Review (2022 Data)

The Solar Automated Permit Processing Plus (SolarAPP+) platform is an online portal to facilitate and expedite rooftop solar photovoltaic (PV) permitting processes. SolarAPP+ allows PV contractors to upload system specifications, have those specifications automatically reviewed for code compliance, and receive instant approval for code-compliant systems. SolarAPP+ also provides inspection checklists to verify installation practices and adherence to approved designs. SolarAPP+ is available to authorities having jurisdiction (AHJs) at no cost. This report is part of an ongoing series of reviews of SolarAPP+ performance. Consistent with previous performance reviews, we summarize SolarAPP+ adoption trends to date and compare various metrics for PV systems permitted through SolarAPP+ versus systems permitted through conventional AHJ permitting processes. As of the end of 2022, the National Renewable Energy Laboratory (NREL) had contacted over 1,500 AHJs with significant solar permitting volume regarding SolarAPP+. Of those, 607 AHJs had at least expressed interest in the platform. 16 AHJs had begun piloting the platform and 15 of these had publicly launched the platform by the end of 2022. In 2022, 206 installers submitted 11,092 permits through the SolarAPP+ platform, including 708 permits for solar+storage systems. SolarAPP+ permits accounted for around 37% of all permits issued in participating AHJs. We compare permitting timelines through SolarAPP+ to traditional AHJ permitting processes to assess the platform's performance. Consistent with previous SolarAPP+ performance reviews, we find that permitting timelines are significantly shorter for SolarAPP+ projects. Based on median timelines, a typical SolarAPP+ project is permitted and inspected 8 business days sooner than traditional projects. We estimate that automatic SolarAPP+ permitting saved between 3,500 and 13,900 hours of AHJ staff time in 2022. Finally, we find evidence that SolarAPP+ may improve inspection outcomes, with SolarAPP+ projects failing inspections about 28% less frequently than traditional projects.

14 SOLAR ENERGY↗

DOE OSTI2016

Technical Note: SCUDA: A software platform for cumulative dose assessment

Purpose: Accurate tracking of anatomical changes and computation of actually delivered dose to the patient are critical for successful adaptive radiation therapy (ART). Additionally, efficient data management and fast processing are practically important for the adoption in clinic as ART involves a large amount of image and treatment data. The purpose of this study was to develop an accurate and efficient Software platform for CUmulative Dose Assessment (SCUDA) that can be seamlessly integrated into the clinical workflow. Methods: SCUDA consists of deformable image registration (DIR), segmentation, dose computation modules, and a graphical user interface. It is connected to our image PACS and radiotherapy informatics databases from which it automatically queries/retrieves patient images, radiotherapy plan, beam data, and daily treatment information, thus providing an efficient and unified workflow. For accurate registration of the planning CT and daily CBCTs, the authors iteratively correct CBCT intensities by matching local intensity histograms during the DIR process. Contours of the target tumor and critical structures are then propagated from the planning CT to daily CBCTs using the computed deformations. The actual delivered daily dose is computed using the registered CT and patient setup information by a superposition/convolution algorithm, and accumulated using the computed deformation fields. Both DIR and dose computation modules are accelerated by a graphics processing unit. Results: The cumulative dose computation process has been validated on 30 head and neck (HN) cancer cases, showing 3.5 ± 5.0 Gy (mean±STD) absolute mean dose differences between the planned and the actually delivered doses in the parotid glands. On average, DIR, dose computation, and segmentation take 20 s/fraction and 17 min for a 35-fraction treatment including additional computation for dose accumulation. Conclusions: The authors developed a unified software platform that provides accurate and efficient monitoring of anatomical changes and computation of actually delivered dose to the patient, thus realizing an efficient cumulative dose computation workflow. Evaluation on HN cases demonstrated the utility of our platform for monitoring the treatment quality and detecting significant dosimetric variations that are keys to successful ART.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2023

Application of unsupervised deep learning to image segmentation and in-situ contact angle measurements in a CO 2 -water-rock system

Rock surface wettability is a critical property that regulates multiphase flows in porous media, which can be quantified using the surface contact angle (CA). X-ray micro-computed tomography (μCT) provides an effective approach to in-situ measurements of surface CAs. However, the CA measurement accuracy depends significantly on the quality of CT image segmentation, which is the clustering of CT pixels into separate phases. Inspired by this, we developed a deep learning (DL)-based CA measurement workflow. Motivated by the recent tremendous progress in unsupervised learning techniques and aiming to avoid expensive manual data annotations, an unsupervised DL pipeline for CT image segmentation was proposed and implemented, which includes unsupervised model training and post-processing. The unsupervised model training was driven by a novel loss function constrained with feature similarity and spatial continuity and implemented by iterative forward and backward paths; the former clustered the pixel-wise feature vectors extracted by convolution neural networks, whereas the latter updated the parameters using gradient descent. An over-segmentation strategy was adopted for model training. The post-processing steps based on agglomerative hierarchical clustering (AHC) were implemented to further merge the over-segmented model output to the desired cluster number, which is intended to improve the efficiency of image segmentation. The developed unsupervised DL pipeline was compared with other commonly-used image segmentation methods using pixel-wise and physics-based evaluation metrics on a synthetic raw-image dataset, which had a known ground truth. The unsupervised DL pipeline showed the best performance. Next, the segmented images were input to an automatic CA measurement tool, and the results were validated by comparisons with manual measurements. The CA values from the manual and automatic measurements showed similar distributions and statistical properties. The automatic measurement demonstrated a wider spectrum because of the much larger number of measurement data points. The primary novelty of the unsupervised DL pipeline developed in this study lies in the novel loss function and the over-segmentation strategy associated with AHC post-processing. Finally, the workflow has been proven an efficient tool for pore-scale wettability characterization, which has a wide range of applications in fundamental studies of multiphase flows in natural porous media, which have critical implications to geological carbon sequestration, hydrocarbon energy recovery, and contaminant transport in groundwater.

42 ENGINEERING↗

DOE OSTI2017

Considerations on the Use of Custom Accelerators for Big Data Analytics

Accelerators, including Graphic Processing Units (GPUs) for gen- eral purpose computation, many-core designs with wide vector units (e.g., Intel Phi), have become a common component of many high performance clusters. The appearance of more stable and reliable tools tools that can automatically convert code written in high-level specifications with annotations (such as C or C++) to hardware de- scription languages (High-Level Synthesis - HLS), is also setting the stage for a broader use of reconfigurable devices (e.g., Field Pro- grammable Gate Arrays - FPGAs) in high performance system for the implementation of custom accelerators, helped by the fact that new processors include advanced cache-coherent interconnects for these components. In this chapter, we briefly survey the status of the use of accelerators in high performance systems targeted at big data analytics applications. We argue that, although the progress in the use of accelerators for this class of applications has been sig- nificant, differently from scientific simulations there still are gaps to close. This is particularly true for the ”irregular” behaviors exhibited by no-SQL, graph databases. We focus our attention on the limits of HLS tools for data analytics and graph methods, and discuss a new architectural template that better fits the requirement of this class of applications. We validate the new architectural templates by mod- ifying the Graph Engine for Multithreaded System (GEMS) frame- work to support accelerators generated with such a methodology, and testing with queries coming from the Lehigh University Benchmark (LUBM). The architectural template enables better supporting the task and memory level parallelism present in graph methods by sup- porting a new control model and a enhanced memory interface. We show that out solution allows generating parallel accelerators, pro- viding speed ups with respect to conventional HLS flows. We finally draw conclusions and present a perspective on the use of reconfig- urable devices and Design Automation tools for data analytics.

Data Analytics↗