Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “big data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Emerging Strategies for Modifying Lignin Chemistry to Enhance Biological Lignin Valorization

Biological lignin valorization represents a promising approach contributing to sustainable and economic biorefineries. Here, the low level of valuable lignin–derived products remains a major challenge hindering the implementation of microbial lignin conversion. Lignin's properties play a significant role in determining the efficiency of lignin bioconversion. To date, despite significant progress in the development of biomass pretreatment, lignin fractionation, and fermentation over the last few decades, little efforts have gone into identifying the ideal lignin substrates for an efficient microbial metabolism. In this Minireview, emerging and state–of–the–art strategies for biomass pretreatment and lignin fractionation are summarized to elaborate their roles in modifying lignin structure for bioconversion. Fermentation strategies aimed at enhancing lignin depolymerization for microbial utilization are systematically reviewed as well. With an improved understanding of the ideal lignin structure elucidated by comprehensive metabolic pathways and/or big data analysis, modifying lignin chemistry could be more directional and effective. Ultimately, together with the progress of fermentation process optimization, biological lignin valorization will become more competitive in biorefineries.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

A Parallel Computing Framework for Finding the Supported Solutions to a Biobjective Network Optimization Problem

Abstract Solving multi‐objective combinatorial optimization problems can quickly become computationally challenging when applied to large networks generated from big data. Present‐day first‐rate Mixed Integer Programming (MIP) solvers have parallelism built‐in to take advantage of multicore architectures; but specialized network optimization algorithms that can often solve graph problems more efficiently than a general MIP solver are typically programmed serially. Thus, these network algorithm implementations do not take full advantage of modern multicore computer capabilities. Many parallel computing languages include a fork/join framework into their concurrency packages that allow for simple conversion of recursive serial algorithms to operate in parallel without having to use complicated low‐level message‐passing interfaces. Fork/join is particularly well suited to divide and conquer algorithms such as Non‐Inferior Set Estimation (NISE) method for computing the supported Pareto front of a multi‐objective optimization problem. This work develops a simple general parallel NISE (pNISE) implementation using the Java 7 concurrency tools, and then offers results from a specific implementation of solving a bi‐objective shortest path problem. The results indicate that this method is capable of sizeable speed‐ups on both small‐scale personal computers as well as large‐scale shared memory supercomputers. Finally, other network problems suitable to this method are discussed, including multi‐objective variants of the minimum spanning tree problem, transportation problem, assignment problem, maximum flow problem, and the minimum‐cost flow problem. Copyright © 2015 John Wiley & Sons, Ltd.

Medrano, Fernando Antonio↗
DOE OSTI2023

The United States Department of Energy and National Institutes of Health Collaboration: Medical Care Advances via Discovery in Physical Sciences

Over several months, representatives from the U.S. Department of Energy (DOE) Office of Science and National Institutes of Health (NIH) had a number of meetings that lead to the conclusion that innovations in the Nation's health care could be realized by more directed interactions between NIH and DOE. It became clear that the expertise amassed and instrumentation advances developed at the DOE physical science laboratories to enable cutting-edge research in particle physics could also feed innovation in medical healthcare. To meet their scientific mission, the DOE laboratories created advances in such technologies as particle beam generation, radioisotope production, high-energy particle detection and imaging, superconducting particle accelerators, superconducting magnets, cryogenics, high-speed electronics, artificial intelligence, and big data. To move forward, NIH and DOE initiated the process of convening a joint work- shop which occurred on July 12th and 13th, 2021. Here, this Special Report presents a summary of the findings of the collaborative workshop and introduces the goals of the next one.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

Abstract There is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Dimensionality reduction using elastic measures

With the recent surge in big data analytics for hyperdimensional data, there is a renewed interest in dimensionality reduction techniques. In order for these methods to improve performance gains and understanding of the underlying data, a proper metric needs to be identified. This step is often overlooked, and metrics are typically chosen without consideration of the underlying geometry of the data. Here, in this paper, we present a method for incorporating elastic metrics into the t-distributed stochastic neighbour embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP). We apply our method to functional data, which is uniquely characterized by rotations, parameterization and scale. If these properties are ignored, they can lead to incorrect analysis and poor classification performance. Through our method, we demonstrate improved performance on shape identification tasks for three benchmark data sets (MPEG-7, Car data set and Plane data set of Thankoor), where we achieve 0.77, 0.95 and 1.00 F1 score, respectively.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

GoFFish: A Sub-Graph Centric Framework for Large-Scale Graph Analytics1

Large scale graph processing is a major research area for Big Data exploration. Vertex centric programming models like Pregel are gaining traction due to their simple abstraction that allows for scalable execution on distributed systems naturally. However, there are limitations to this approach which cause vertex centric algorithms to under-perform due to poor compute to communication overhead ratio and slow convergence of iterative superstep. In this paper we introduce GoFFish a scalable sub-graph centric framework co-designed with a distributed persistent graph storage for large scale graph analytics on commodity clusters. We introduce a sub-graph centric programming abstraction that combines the scalability of a vertex centric approach with the flexibility of shared memory sub-graph computation. We map Connected Components, SSSP and PageRank algorithms to this model to illustrate its flexibility. Further, we empirically analyze GoFFish using several real world graphs and demonstrate its significant performance improvement, orders of magnitude in some cases, compared to Apache Giraph, the leading open source vertex centric implementation. We map Connected Components, SSSP and PageRank algorithms to this model to illustrate its flexibility. Further, we empirically analyze GoFFish using several real world graphs and demonstrate its significant performance improvement, orders of magnitude in some cases, compared to Apache Giraph, the leading open source vertex centric implementation.

Simmhan, Yogesh↗
DOE OSTI2015

TECA: Petascale pattern recognition for climate science

Climate Change is one of the most pressing challenges facing humanity in the 21st century. Climate simulations provide us with a unique opportunity to examine effects of anthropogenic emissions. Highresolution climate simulations produce “Big Data”: contemporary climate archives are ≈ 5PB in size and we expect future archives to measure on the order of Exa-Bytes. In this work, we present the successful application of TECA (Toolkit for Extreme Climate Analysis) framework, for extracting extreme weather patterns such as Tropical Cyclones, Atmospheric Rivers and Extra-Tropical Cyclones from TB-sized simulation datasets. TECA has been run at full-scale on Cray XE6 and IBM BG/Q systems, and has reduced the runtime for pattern detection tasks from years to hours. TECA has been utilized to evaluate the performance of various computational models in reproducing the statistics of extreme weather events, and for characterizing the change in frequency of storm systems in the future.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

SHMemCache: Enabling Memcached on the OpenSHMEM Global Address Model, In: OpenSHMEM 2016: OpenSHMEM and Related Technologies. Enhancing OpenSHMEM for Hybrid Environments

Memcached is a popular key-value memory store for big data applications. Its performance is directly related to the underlying run-time systems including the communication protocols. OpenSHMEM is a strong run-time system that supports data access to both local and remote memory through a simple shared-memory addressing model. In view of the communication compatibilities between Memcached and OpenSHMEM, we propose to integrate the programmability and portability of OpenSHMEM for supporting Memcached on a wide variety of HPC systems. In this paper, we present the design and implementation of SHMemCache, an OpenSHMEM-based communication conduit for Memcached, which can expand the deployment scope of Memcached to various leadership facilities with OpenSHMEM run-time.

Computer Science↗
DOE OSTI2018

Portable SHMEMCache: A High-Performance Key-Value Store on OpenSHMEM and MPI

The integration of Big Data frameworks and HPC capabilities has drawn enormous interests in recent years. SHMEMCache is a distributed key-value store built on the OpenSHMEM global address space. It has solved several practical issues in leveraging OpenSHMEM’s one-sided operations for a distributed key-value store and providing efficient key-value operations on both commodity machines and supercomputers. However, being based solely on OpenSHMEM, SHMEMCache cannot leverage one-sided operations from a variety of software packages. This results in several limitations for SHMEMCache. First, we cannot make SHMEMCache available to a wider range of platforms. Second, an opportunity for potential performance improvement is missed. Third, there is a lack of deep understanding about how different one-sided operations can fit in with SHMEMCache and other distributed key-values in general. For example, the one-sided operations in OpenSHMEM and MPI have many differences in their interfaces, memory semantics and synchronization methods, all of which can have distinct implications and also increase the complexity in supporting both OpenSHMEM and MPI for SHMEMCache. Therefore, we have taken on an effort on leveraging different one-sided operations for SHMEMCache and proposed a design of portable SHMEMCache. Based on this new framework, we have supported both OpenSHMEM and MPI for SHMEMCache. We have also conducted an extensive set of experiments to compare the performance of the two versions on both commodity machines and the Titan supercomputer.

Computer Science↗
DOE OSTI2021

Introduction to the special issue on smart transportation

Transportation is getting smarter and smarter, with the prominence of connected automated vehicle technologies in the global auto industry’s near-term growth strategies, of big data analytics and unprecedented access to sensing data of mobility, and of integration of this analytics into the optimization of mobility and transport. Further, these developments are setting off a wave of smart transportation innovations, which are featured by new methods and applications driven by various forms of sensor data such as GPS, CAN bus, LIDA, images, etc. At the same time, complexities surrounding the use, conflation, and processing of disparate data in near real-time is of essence for the design and development of futuristic smart transportation.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Special Issue: Geostatistics and Machine Learning

Abstract Recent years have seen a steady growth in the number of papers that apply machine learning methods to problems in the earth sciences. Although they have different origins, machine learning and geostatistics share concepts and methods. For example, the kriging formalism can be cast in the machine learning framework of Gaussian process regression. Machine learning, with its focus on algorithms and ability to seek, identify, and exploit hidden structures in big data sets, is providing new tools for exploration and prediction in the earth sciences. Geostatistics, on the other hand, offers interpretable models of spatial (and spatiotemporal) dependence. This special issue on Geostatistics and Machine Learning aims to investigate applications of machine learning methods as well as hybrid approaches combining machine learning and geostatistics which advance our understanding and predictive ability of spatial processes.

58 GEOSCIENCES↗
DOE OSTI2018

Additive Manufacturing of Nickel Superalloys: Opportunities for Innovation and Challenges Related to Qualification

Innovative designs for turbines can be achieved by advances in nickel-based superalloys and manufacturing methods, including the adoption of additive manufacturing. In this regard, selective electron beam melting (SEBM) and selective laser melting (SLM) of nickel-based superalloys do provide distinct advantages. Furthermore, the direct energy deposition (DED) processes can be used for repair and reclamation of nickel alloy components. The current paper explores opportunities for innovation and qualification challenges with respect to deployment of AM as a disruptive manufacturing technology. Here in the first part of the paper, fundamental correlations of processing parameters to defect tendency and microstructure evolution will be explored using DED process. In the second part of the paper, opportunities for innovation in terms of site-specific control of microstructure during processing will be discussed. In the third part of the paper, challenges in qualification of AM parts for service will be discussed and potential methods to alleviate these issues through in situ process monitoring, and big data analytics are proposed.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Chapter 16 - Predictive Analytics for Comprehensive Energy Systems State Estimation

Energy sustainability is a subject of concern to many nations in the modern world. It is critical for electric power systems to diversify energy supply to include systems with different physical characteristics, such as wind energy, solar energy, electrochemical energy storage, thermal storage, bio-energy systems, geothermal, and ocean energy. Each system has its own range of control variables and targets. To be able to operate such a complex energy system, big-data analytics become critical to achieve the goal of predicting energy supplies and consumption patterns, assessing system operation conditions, and estimating system states - all providing situational awareness to power system operators. This chapter presents data analytics and machine learning-based approaches to enable predictive situational awareness of the power systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

miRNA editing landscape reveals miR-34c regulated spermatogenesis through structure and target change in pig and mouse

Highlights: • We integrated transcriptome of spermatogenesis and found miRNA regulate spermatogenesis through miRNA editing. • We compared with different species and found that the distribution of miRNA editing during spermatogenesis is conservative. • We identified miR-34c, which is edited frequently at all stages during spermatogenesis, regulates its target genes through the RNA structure changing and shows dysfunction when it is edited. Spermatogenesis has a close relationship with male infertility. MicroRNAs (miRNAs) play crucial roles in their regulation of target genes during spermatogenesis. A huge dataset of high-throughput sequencing all over the world provides the basis to dig the cryptic molecular mechanism. But how to take advantage of the big data and unearth the miRNA regulation is still a challenging problem. Here we integrated transcriptome of spermatogenesis and found miRNA regulate spermatogenesis through miRNA editing. We then compared different species and found that the distributions of miRNA editing site number and editing types among different cell types during spermatogenesis are conservative. Interesting, we further found that nearly half of the editing events occurred in the seed region in both mouse and pig. Finally, we foundmiR-34c, which is edited frequently at all stages during spermatogenesis, regulates its target genes through the RNA structure changing and shows dysfunction when it is edited. Summary, we depicted the overall profile of miRNA editing during spermatogenesis in mouse and pig and reveal miR-34c may play its roles through miRNA editing.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Machine learning-assisted design of metal–organic frameworks for hydrogen storage: A high-throughput screening and experimental approach

Various theoretical approaches, including big data and high-throughput screening techniques, have been explored in developing new materials due to their significant potential time-saving advantages. However, it remains a significant challenge to experimentally realize new materials that are predicted. In this study, we propose a novel materials design strategy that utilizes machine-learning (ML) techniques to predict new porous materials that show promise for hydrogen storage and are likely to be feasible to synthesize. By leveraging ML techniques and metal–organic framework (MOF) databases, we are able to predict the synthesizability of MOF structures. This is evidenced by the successful synthesis of a new vanadium-based MOF that exhibits excellent performance for cryogenic H 2 storage. Notably, the total gravimetric and volumetric H 2 uptakes are as high as 9.0 wt% and 50.0 g/L at 77 K and 150 bar. This ML-assisted materials design offers an efficient and promising approach for developing hydrogen storage materials.

08 HYDROGEN↗
DOE OSTI2018

Machine learning materials physics: Surrogate optimization and multi-fidelity algorithms predict precipitate morphology in an alternative to phase field dynamics

Machine learning has been effective at detecting patterns and predicting the response of systems that behave free of natural laws. Examples include learning crowd dynamics, recommender systems and autonomous mobility. There also have been applications to the search for new materials that draw upon big-data classification problems. Though, when it comes to physical systems governed by conservation laws, the role of machine learning has been more limited. Here, we present our recent work in exploring the role of machine learning methods in discovering, or aiding, the search for physics. Specifically, we focus on using machine learning algorithms to represent high-dimensional free energy surfaces with the goal of identifying precipitate morphologies in alloy systems. Traditionally, this problem has been approached by combining phase field models, which impose first-order dynamics, with elasticity, to traverse a free energy landscape in search of minima. Equilibrium precipitate morphologies occur at these minima. Here, we exploit the machine learning methods to represent high-dimensional data, combined with surrogate optimization, sensitivity analysis and multifidelity modeling as an alternate framework to explore phenomena controlled by energy extremization. This combination of data-driven methods offers an alternative to the imposition of first-order dynamics via phase field methods, and represents one aspect of applying machine learning to materials physics.

42 ENGINEERING↗
DOE OSTI2021

Machine learning in materials science: From explainable predictions to autonomous design

The advent of big data and algorithmic developments in the field of machine learning (and artificial intelligence, in general) have greatly impacted the entire spectrum of physical sciences, including materials science. Materials data, measured or computed, combined with various techniques of machine learning have been employed to address a myriad of challenging problems, such as, development of efficient and predictive surrogate models for a range of materials properties, screening and down-selection of novel candidate materials for targeted applications, new methodologies to improve and further expedite molecular and atomistic simulations, with likely many more important developments to come in the foreseeable future. While the applications thus far have provided a glimpse of the true potential data-enabled routes have to offer, it has also become clear that further progress in this direction hinges on our ability to understand, explain and rationalize findings of a machine learning model in light of the domain-knowledge. This focused review provides an overview of the main areas where machine learning has been widely and successfully used in materials science. Subsequently, a brief discussion of several techniques that have been helpful in extracting physically-meaningful insights, causal relationships and design-centric knowledge from materials data is provided. Finally, we identify some of the imminent opportunities and challenges that materials community faces in this exciting and rapidly growing field.

36 MATERIALS SCIENCE↗
DOE OSTI2020

State-of-the-art on research and applications of machine learning in the building life cycle

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Finally, findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗