Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Big data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro- Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

analytics-hub↗
DOE OSTI2019

Condition Monitoring of Wind Turbine Planetary Gearboxes Under Different Operating Conditions

Digitally enhanced services for wind power could reduce Operations and Maintenance costs as well as the Levelised Cost Of Energy. Therefore, there is a continuous need for advanced monitoring techniques which can exploit the opportunities of Internet of Things and Big Data Analytics. The heart of wind turbines is an epicyclic gearbox and rolling element bearings are often responsible for machinery stops. The vibration signatures of bearings are rather weak compared to other components. As a result a number of signal processing techniques have been proposed, focusing towards accurate and early fault detection with limited false alarms and missed detections. In Envelope Analysis an envelope of the vibration signal is estimated usually after filtering around a frequency band. Different tools, such as Kurtogram, have been proposed to select the optimum filter parameters. Monitoring techniques have reached a mature level for steady speed and load operating conditions. On the other hand, under chaniging operating conditions, it is still unclear whether the change of the monitoring indicators is due to the change of the machinery's health or due to the change of operating parameters. Recently, the authors have proposed a new tool called IESFOgram, based on Cyclic Spectral Coherence for the automatic selection of the filtering band. In this paper, the performance of the tool is evaluated on the National Renewable Energy Laboratory wind turbine gearbox vibration condition monitoring benchmarking data set which includes various faults with different levels of diagnostic complexity as well as various speed and load operating conditions.

17 WIND ENERGY↗
DOE OSTI2022

Comparing Model Representations of Physiological Limits on Transpiration at a Semi-arid Ponderosa Pine Site

Mechanistic representations of biogeochemical processes in ecosystem models are rapidly advancing, requiring advancements in model evaluation approaches. Here we quantify multiple aspects of model functional performance to evaluate improved process representations in ecosystem models. We compare semi-empirical stomatal models with hydraulic constraints against more mechanistic representations of stomatal and hydraulic functioning at a semi-arid pine site using a suite of metrics and analytical tools. We find that models generally perform similarly under unstressed conditions, but performance diverges under atmospheric and soil drought. The more empirical models better capture synergistic information flows between soil water potential and vapor pressure deficit to transpiration, while the more mechanistic models are overly deterministic. Although models can be parameterized to yield similar functional performance, alternate parameterizations could not overcome structural model constraints that underestimate the unique information contained in soil water potential about transpiration. Additionally, both multilayer canopy and big-leaf models were unable to capture the magnitude of canopy temperature divergence from air temperature, and we demonstrate that errors in leaf temperature can propagate to considerable error in simulated transpiration. This study demonstrates the value of merging underutilized observational data streams with emerging analytical tools to characterize ecosystem function and discriminate among model process representations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Scaling Deep Learning on GPU and Knights Landing clusters

The speed of deep neural networks training has become a big bottleneck of deep learning research and development. For example, training GoogleNet by ImageNet dataset on one Nvidia K20 GPU needs 21 days. To speed up the training process, the current deep learning systems heavily rely on the hardware accelerators. However, these accelerators have limited on-chip memory compared with CPUs. To handle large datasets, they need to fetch data from either CPU memory or remote processors. We use both self-hosted Intel Knights Landing (KNL) clusters and multi-GPU clusters as our target platforms. From an algorithm aspect, current distributed machine learning systems are mainly designed for cloud systems. These methods are asynchronous because of the slow network and high fault-tolerance requirement on cloud systems. We focus on Elastic Averaging SGD (EASGD) to design algorithms for HPC clusters. Original EASGD used round-robin method for communication and updating. The communication is ordered by the machine rank ID, which is inefficient on HPC clusters. First, we redesign four efficient algorithms for HPC systems to improve EASGD's poor scaling on clusters. Async EASGD, Async MEASGD, and Hogwild EASGD are faster \textcolor{black}{than} their existing counterparts (Async SGD, Async MSGD, and Hogwild SGD, resp.) in all the comparisons. Finally, we design Sync EASGD, which ties for the best performance among all the methods while being deterministic. In addition to the algorithmic improvements, we use some system-algorithm codesign techniques to scale up the algorithms. By reducing the percentage of communication from 87% to 14%, our Sync EASGD achieves 5.3x speedup over original EASGD on the same platform. We get 91.5% weak scaling efficiency on 4253 KNL cores, which is higher than the state-of-the-art implementation.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2013

Concept of Operations for Collaboration and Discovery from Big Data Across Enterprise Data Warehouses

The success of data-driven business in government, science, and private industry is driving the need for seamless integration of intra and inter-enterprise data sources to extract knowledge nuggets in the form of correlations, trends, patterns and behaviors previously not discovered due to physical and logical separation of datasets. Today, as volume, velocity, variety and complexity of enterprise data keeps increasing, the next generation analysts are facing several challenges in the knowledge extraction process. Towards addressing these challenges, data-driven organizations that rely on the success of their analysts have to make investment decisions for sustainable data/information systems and knowledge discovery. Options that organizations are considering are newer storage/analysis architectures, better analysis machines, redesigned analysis algorithms, collaborative knowledge management tools, and query builders amongst many others. In this paper, we present a concept of operations for enabling knowledge discovery that data-driven organizations can leverage towards making their investment decisions. We base our recommendations on the experience gained from integrating multi-agency enterprise data warehouses at the Oak Ridge National Laboratory to design the foundation of future knowledge nurturing data-system architectures.

Big Data↗
DOE OSTI2020

Big Data and AI at DoE's Legacy Sites - 20546

More than 30 years have passed since DOE started the decommissioning of nuclear weapon complexes and the clean-up of soil and groundwater. All the sites have been collecting and archiving soil and groundwater monitoring datasets; particularly contaminant concentration time-series. These datasets provide unparalleled opportunities to understand the system behavior (including more fundamental hydrological and geochemical processes, the response to various perturbations, the long-term trend and environmental decay rate towards the regulatory limit). This understanding is critical for providing multiple lines of evidences that can support site closure. In this study, we explore the machine learning (ML) and artificial intelligence (AI) applications to the long-term soil and groundwater management at DoE's legacy sites. ML can improve our understanding of the subsurface systems, which is critical for long-term monitoring and management of the sites, while AI can automate or support some of decision-making processes (e.g., anomaly detection, monitoring well placements). The particular focuses are to develop general algorithms to: (1) to identify distinct spatiotemporal patterns and to identify several groups that have similar temporal behaviors, using unsupervised clustering methods, (2) identify the different temporal scales of hydrological responses to climate perturbations by time-series analysis, and (3) reduce the number of monitoring wells by identifying the minimum sufficient number of wells to capture the heterogeneity of the groundwater contaminant plume and concentration distribution, using the Gaussian Process model. We demonstrate our methodology at the Savannah River Site F-Area. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2018

A View from ORNL: Scientific Data Research Opportunities in the Big Data Age

One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of "Big Data". In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].

Klasky, Scott↗
DOE OSTI2023

A differentiable, physics-informed ecosystem modeling and learning framework for large-scale inverse problems: demonstration with photosynthesis simulations

Photosynthesis plays an important role in carbon, nitrogen, and water cycles. Ecosystem models for photosynthesis are characterized by many parameters that are obtained from limited in situ measurements and applied to the same plant types. Previous site-by-site calibration approaches could not leverage big data and faced issues like overfitting or parameter non-uniqueness. Here we developed an end-to-end programmatically differentiable (meaning gradients of outputs to variables used in the model can be obtained efficiently and accurately) version of the photosynthesis process representation within the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) model. As a genre of physics-informed machine learning (ML), differentiable models couple physics-based formulations to neural networks (NNs) that learn parameterizations (and potentially processes) from observations, here photosynthesis rates. We first demonstrated that the framework was able to correctly recover multiple assumed parameter values concurrently using synthetic training data. Then, using a real-world dataset consisting of many different plant functional types (PFTs), we learned parameters that performed substantially better and greatly reduced biases compared to literature values. Further, the framework allowed us to gain insights at a large scale. Our results showed that the carboxylation rate at 25 °C (V c,max25 ) was more impactful than a factor representing water limitation, although tuning both was helpful in addressing biases with the default values. This framework could potentially enable substantial improvement in our capability to learn parameters and reduce biases for ecosystem modeling at large scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Accelerating Science: A Computing Research Agenda

The emergence of "big data" offers unprecedented opportunities for not only accelerating scientific advances but also enabling new modes of discovery. Scientific progress in many disciplines is increasingly enabled by our ability to examine natural phenomena through the computational lens, i.e., using algorithmic or information processing abstractions of the underlying processes; and our ability to acquire, share, integrate and analyze disparate types of data. However, there is a huge gap between our ability to acquire, store, and process data and our ability to make effective use of the data to advance discovery. Despite successful automation of routine aspects of data management and analytics, most elements of the scientific process currently require considerable human expertise and effort. Accelerating science to keep pace with the rate of data acquisition and data processing calls for the development of algorithmic or information processing abstractions, coupled with formal methods and tools for modeling and simulation of natural processes as well as major innovations in cognitive tools for scientists, i.e., computational tools that leverage and extend the reach of human intellect, and partner with humans on a broad range of tasks in scientific discovery (e.g., identifying, prioritizing formulating questions, designing, prioritizing and executing experiments designed to answer a chosen question, drawing inferences and evaluating the results, and formulating new questions, in a closed-loop fashion). This calls for concerted research agenda aimed at: Development, analysis, integration, sharing, and simulation of algorithmic or information processing abstractions of natural processes, coupled with formal methods and tools for their analyses and simulation; Innovations in cognitive tools that augment and extend human intellect and partner with humans in all aspects of science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Computing environments for reproducibility: Capturing the “Whole Tale”

The act of sharing scientific knowledge is rapidly evolving away from traditional articles and presentations to the delivery of executable objects that integrate the data and computational details (e.g., scripts and workflows) upon which the findings rely. This envisioned coupling of data and process is essential to advancing science but faces technical and institutional barriers. The Whole Tale project aims to address these barriers by connecting computational, data-intensive research efforts with the larger research process transforming the knowledge discovery and dissemination process into one where data products are united with research articles to create "living publications" or tales. The Whole Tale focuses on the full spectrum of science, empowering users in the long tail of science, and power users with demands for access to big data and compute resources. We report here on the design, architecture, and implementation of the Whole Tale environment.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Analytic reconstruction algorithms for triple-source CT with horizontal data truncation

Purpose: This paper explores a triple-source imaging method with horizontal data truncation to enlarge the field of view (FOV) for big objects. Methods: The study is conducted by using theoretical analysis, mathematical deduction, and numerical simulations. The proposed algorithms are implemented in c + + and MATLAB. While the basic platform is constructed in MATLAB, the computationally intensive segments are coded in c + +, which are linked via a MEX interface. Results: A triple-source circular scanning configuration with horizontal data truncation is developed, where three pairs of x-ray sources and detectors are unevenly distributed on the same circle to cover the whole imaging object. For this triple-source configuration, a fan-beam filtered backprojection-type algorithm is derived for truncated full-scan projections without data rebinning. The algorithm is also extended for horizontally truncated half-scan projections and cone-beam projections in a Feldkamp-type framework. Using their method, the FOV is enlarged twofold to threefold to scan bigger objects with high speed and quality. The numerical simulation results confirm the correctness and effectiveness of the developed algorithms. Conclusions: The triple-source scanning configuration with horizontal data truncation cannot only keep most of the advantages of a traditional multisource system but also cover a larger FOV for big imaging objects. In addition, because the filtering is shift-invariant, the proposed algorithms are very fast and easily parallelized on graphic processing units.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2013

Coop Progress Report

My Coop at LLNL has been my first professional experience as an Electrical Engineer. I was tasked to carry out signal processing to analyze data and code in the IDL following standard software development principles. The Coop has met all of my needs to continue my professional career, and I feel more confident as I continue working as a student and professional. It is now a big open question for me as whether to pursue graduate research or industry after I graduate with my B.S. in Electrical Engineering.

42 ENGINEERING↗
DOE OSTI2026

Understanding the Thermal Physics and Metallurgy of Metal Big Area Additive Manufacturing

The research goal of this EPSCoR-DOE partnership is to mitigate defects in parts made using a new type of additive manufacturing (AM) process called metal Big Area Additive Manufacturing (m-BAAM). To realize this goal, the PIs will detect and correct defects in the part as it is being printed by combining fundamental knowledge of the thermal physics and metallurgy of m-BAAM with in-process sensor data. Developed at the DOE-funded Manufacturing Demonstration Facility at Oak Ridge National Laboratory, the m-BAAM process involves one or more robots working together to produce a part by fusing metal wire layer-by-layer using arc welding. The process can print large metal parts such as turbine blades, which is not possible using other AM processes. In addition, m-BAAM production rates are more than ten times faster than other AM processes while requiring one-tenth of the material cost. Despite their potential to become a critical force multiplier in the energy generation industry, m-BAAM parts may fail to print accurately due to retention of heat and uneven cooling. Overheating and anomalous cooling rates in turn can cause inconsistencies in the microstructure, leading to sudden failure when used in safety-critical applications. In other words, flaw formation in m-BAAM parts is governed by the thermal history – intensity and spatial distribution of heat inside the part during printing. The thermal history is a complex function of the part shape and process settings such as welding energy, path taken by the welding torch for deposition (tool path), wire feed rate, among others.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Enabling real-time multi-messenger astrophysics discoveries with deep learning

Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos. In this Expert Recommendation, we review the key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts, and make a number of recommendations to maximize their potential for scientific discovery. These recommendations refer to the design of scalable and computationally efficient machine learning algorithms; the cyber-infrastructure to numerically simulate astrophysical sources, and to process and interpret multi-messenger astrophysics data; the management of gravitational wave detections to trigger real-time alerts for electromagnetic and astroparticle follow-ups; a vision to harness future developments of machine learning and cyber-infrastructure resources to cope with the big-data requirements; and finally, the need to build a community of experts to realize the goals of multi-messenger astrophysics.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

First result of the experimental search for the 9.4 keV solar axion reactions with {sup 83}Kr in the copper proportional counter

The experimental search for solar hadronic axions is started at the Baksan Neutrino Observatory of the Institute for Nuclear Researches of Russian Academy of Science (BNO INR RAS). It is assumed that axions are created in the Sun during M1 transition between the first thermally excited level at 9.4 keV and the ground state in {sup 83}Kr. The experiment is based on axion detection via resonant absorption process by the same nucleus in the detector. The big copper proportional counter filled with krypton is used to detect signals from axions. The experimental setup is situated in the deep underground low background laboratory. No evidence of axion detection were found after the 26.5 days data collection. Resulting new upper limit on axion mass is m{sub A} ≤ 130 eV at 95% C.L.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2012

THE MISSING CAVITIES IN THE SEEDS POLARIZED SCATTERED LIGHT IMAGES OF TRANSITIONAL PROTOPLANETARY DISKS: A GENERIC DISK MODEL

Transitional circumstellar disks around young stellar objects have a distinctive infrared deficit around 10 {mu}m in their spectral energy distributions, recently measured by the Spitzer Infrared Spectrograph (IRS), suggesting dust depletion in the inner regions. These disks have been confirmed to have giant central cavities by imaging of the submillimeter continuum emission using the Submillimeter Array (SMA). However, the polarized near-infrared scattered light images for most objects in a systematic IRS/SMA cross sample, obtained by HiCIAO on the Subaru telescope, show no evidence for the cavity, in clear contrast with SMA and Spitzer observations. Radiative transfer modeling indicates that many of these scattered light images are consistent with a smooth spatial distribution for {mu}m-sized grains, with little discontinuity in the surface density of the {mu}m-sized grains at the cavity edge. Here we present a generic disk model that can simultaneously account for the general features in IRS, SMA, and Subaru observations. Particularly, the scattered light images for this model are computed, which agree with the general trend seen in Subaru data. Decoupling between the spatial distributions of the {mu}m-sized dust and mm-sized dust inside the cavity is suggested by the model, which, if confirmed, necessitates a mechanism, such as dust filtration, for differentiating the small and big dust in the cavity clearing process. Our model also suggests an inwardly increasing gas-to-dust ratio in the inner disk, and different spatial distributions for the small dust inside and outside the cavity, echoing the predictions in grain coagulation and growth models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Automatic Search of Cataclysmic Variables Based on LightGBM in LAMOST-DR7

The search for special and rare celestial objects has always played an important role in astronomy. Cataclysmic Variables (CVs) are special and rare binary systems with accretion disks. Most CVs are in the quiescent period, and their spectra have the emission lines of Balmer series, HeI, and HeII. A few CVs in the outburst period have the absorption lines of Balmer series. Owing to the scarcity of numbers, expanding the spectral data of CVs is of positive significance for studying the formation of accretion disks and the evolution of binary star system models. At present, the research for astronomical spectra has entered the era of Big Data. The Large Sky Area Multi-Object Fiber Spectroscopy Telescope (LAMOST) has produced more than tens of millions of spectral data. the latest released LAMOST-DR7 includes 10.6 million low-resolution spectral data in 4926 sky regions, providing ideal data support for searching CV candidates. To process and analyze the massive amounts of spectral data, this study employed the Light Gradient Boosting Machine (LightGBM) algorithm, which is based on the ensemble tree model to automatically conduct the search in LAMOST-DR7. Finally, 225 CV candidates were found and four new CV candidates were verified by SIMBAD and published catalogs. This study also built the Gradient Boosting Decision Tree (GBDT), Adaptive Boosting (AdaBoost), and eXtreme Gradient Boosting (XGBoost) models and used Accuracy, Precision, Recall, the F1-score, and the ROC curve to compare the four models comprehensively. Experimental results showed that LightGBM is more efficient. The search for CVs based on LightGBM not only enriches the existing CV spectral library, but also provides a reference for the data mining of other rare celestial objects in massive spectral data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Open Source Release of NJOY2016 and NJOY21

NJOY has been the standard for nuclear data processing for many decades. NJOY is a comprehensive computer code used to produce point-wise and multigroup nuclear data application libraries from cross sections and related quantities obtained from evaluated nuclear data in the Evaluated Nuclear Data Format (ENDF). NJOY was first publicly released in 1977 and distributed by what is now the Radiation Safety Information Computational Center (RSICC) at Oak Ridge National Laboratory and the National Energy Software Center (NESC) at Argonne National Laboratory. From that time until the release of NJOY2012, NJOY has been distributed by RSICC and made available to researchers outside the United States by agreements between RSICC and the NEA Databank. NJOY2012 was distributed by the Feynman Center at Los Alamos National Laboratory which did not have agreements with the NEA Databank, thus making it more difficult for non US citizens to obtain copies of NJOY2012. Furthermore, NJOY has had export control restrictions preventing some researchers from obtaining NJOY at all. Two new versions of NJOY have been released, NJOY2016 and NJOY21, in January of 2017. Both NJOY2016 and NJOY21 have been released with no export control restrictions and are distributed with an open-source license. This is a big departure from previous releases and impacts not only who has access to NJOY, but also how updates are made available to users and how collaboration is facilitated among interested developers. In addition to distributing NJOY2016 and NJOY21 with an open-source license, these two new versions provide new capabilities for NJOY which will maintain NJOY as the standard for nuclear data processing into the foreseeable future. We report in this summary about the changes that come with these two new versions of NJOY and how this impacts the users and long-time contributors to NJOY.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗