Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “big data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

An overview of the operation architectures and energy management system for multiple microgrid clusters

The emerging novel energy infrastructures, such as energy communities, smart building-based microgrids, electric vehicles enabled mobile energy storage units raise the requirements for a more interconnective and interoperable energy system. It leads to a transition from simple and isolated microgrids to relatively large-scale and complex interconnected microgrid systems named multi-microgrid clusters. In order to efficiently, optimally, and flexibly control multi-microgrid clusters, cross-disciplinary technologies such as power electronics, control theory, optimization algorithms, information and communication technologies, cyber-physical, and big-data analysis are needed. This paper introduces an overview of the relevant aspects for multi-microgrids, including the outstanding features, architectures, typical applications, existing control mechanisms, as well as the challenges.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Digital Twin Framework Utilizing Machine Learning for Robust Predictive Maintenance: Enhancing Tire Health Monitoring

We introduce a novel digital twin (DT) framework for the predictive maintenance of long-term physical systems. Using monitoring tire health as an application, we show how the DT framework can be used to enhance automotive safety and efficiency, and how the technical challenges can be overcome using a three-step approach. First, to manage the data complexity over a long operation span, we employ data reduction techniques to concisely represent physical tires using historical performance and usage data. Relying on these data, for fast real-time prediction, we train a transformer-based model offline on our concise dataset to predict future tire health over time, represented as remaining casing potential (RCP). Based on our architecture, our model quantifies both epistemic and aleatoric uncertainties, providing reliable confidence intervals around predicted RCP. Second, to incorporate real-time data, we update the predictive model in the DT framework, ensuring its accuracy throughout its lifespan with the aid of hybrid modeling and the use of the discrepancy function. Third, to assist decision-making in predictive maintenance, we implement a tire state decision algorithm, which strategically determines the optimal timing for tire replacement based on RCP forecasted by our transformer model. This approach ensures that our DT accurately predicts system health, continually refines its digital representation, and supports predictive maintenance decisions. Furthermore, our framework effectively embodies a physical system, leveraging big data and machine learning (ML) for predictive maintenance, model updates, and decision-making.

advanced computing infrastructure↗
NASA NTRSDate not supplied

Communication Network Awareness Machine System Phase I Development: The Intelligent Party-Line Schema

As NextGen continues toward the full implementation of a Net-Centric Architecture (N-CA)it will inherently provide a continuous increase to the Three-Vs components (Volume, Velocity, and Variety) of big data . This will create an insurmountable environment for direct-action aviation personnel (DAAP)as the DAAP’s natural abilities to manage and process data into actionable information will be overmatched by the Three-Vs. Therefore, conducting operations within a N-CA requires that new tools and applications be researched and developed to aid the DAAP’s ability to understand and manage data, mitigate non-normals, create contingency plans and actions. This paper will describe a research area at NASA Langley Research Center known as the Intelligent Party-Line (IPL).

Intelligent Party-Line↗
DOE OSTI2022

Melt pool temperature measurement and monitoring during laser powder bed fusion based additive manufacturing via single-camera two-wavelength imaging pyrometry (STWIP)

Melt pool (MP) temperature is one of the determining factors and a key signature for evaluating the properties of printed components in metal additive manufacturing (AM). The state-of-the-art measurement systems are hindered, primarily by the large-scale data acquisition and processing demands. In this work, we introduce a novel coaxial, high-speed, single-camera two-wavelength imaging pyrometer (STWIP) system as opposed to the typical utilization of multiple cameras for measuring MP temperature profiles in laser powder bed fusion (LPBF) processes. Developed on a commercial LPBF machine (EOS M290), the STWIP system demonstrated its ability to quantitatively monitor the MP temperature and its variation for 50 layers at high framerates (>30,000 fps) for a real-world application (standard fatigue specimens) print. High performance computing is employed to analyze the acquired big data (MP images), for determining each MP's average temperature and 2D temperature profile. The MP temperature evolution in the gage section of a fatigue specimen is also examined at a temporal resolution of 1 ms, by evaluating the MP temperatures in the samples' first, middle, and last layers. This report is the first of its kind on monitoring MP temperature distribution and evolution at such a large, detailed scale for longer durations in practical applications.

42 ENGINEERING↗
DOE OSTI2014

DOE Advanced Scientific Computing Advisory Subcommittee (ASCAC) Report: Top Ten Exascale Research Challenges

Exascale computing systems are essential for the scientific fields that will transform the 21st century global economy, including energy, biotechnology, nanotechnology, and materials science. Progress in these fields is predicated on the ability to perform advanced scientific and engineering simulations, and analyze the deluge of data. On July 29, 2013, ASCAC was charged by Patricia Dehmer, the Acting Director of the Office of Science, to assemble a subcommittee to provide advice on exascale computing. This subcommittee was directed to return a list of no more than ten technical approaches (hardware and software) that will enable the development of a system that achieves the Department's goals for exascale computing. Numerous reports over the past few years have documented the technical challenges and the non¬-viability of simply scaling existing computer designs to reach exascale. The technical challenges revolve around energy consumption, memory performance, resilience, extreme concurrency, and big data. Drawing from these reports and more recent experience, this ASCAC subcommittee has identified the top ten computing technology advancements that are critical to making a capable, economically viable, exascale system.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2017

Autonomous Energy Grids: Preprint

With much higher levels of distributed energy resources - variable generation, energy storage, and controllable loads just to mention a few - being deployed into power systems, the data deluge from pervasive metering of energy grids, and the shaping of multi-level ancillary-service markets, current frameworks to monitoring, controlling, and optimizing large-scale energy systems are becoming increasingly inadequate. This position paper outlines the concept of 'Autonomous Energy Grids' (AEGs) - systems that are supported by a scalable, reconfigurable, and self-organizing information and control infrastructure, can be extremely secure and resilient (self-healing), and self-optimize themselves in real-time for economic and reliable performance while systematically integrating energy in all forms. AEGs rely on scalable, self-configuring cellular building blocks that ensure that each 'cell' can self-optimize when isolated from a larger grid as well as partaking in the optimal operation of a larger grid when interconnected. To realize this vision, this paper describes the concepts and key research directions in the broad domains of optimization theory, control theory, big-data analytics, and complex system modeling that will be necessary to realize the AEG vision.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Modelling urban-scale occupant behaviour, mobility, and energy in buildings: A survey

The proliferation of urban sensing, IoT, and big data in cities provides unprecedented opportunities for a deeper understanding of occupant behaviour and energy usage patterns at the urban scale. This enables data-driven building and energy models to capture the urban dynamics, specifically the intrinsic occupant and energy use behavioural profiles that are not usually considered in traditional models. Although there are related reviews, none have investigated urban data for use in modelling occupant behaviour and energy use at multiple scales, from buildings to neighbourhood to city. This survey paper aims to fill this gap by providing a critical summary and analysis of the works reported in the literature. We present the different sources of occupant-centric urban data that are useful for data-driven modelling and categorise the range of applications and recent data-driven modelling techniques for urban behaviour and energy modelling, along with the traditional stochastic and simulation-based approaches. Finally, we present a set of recommendations for future directions in data-driven modelling of occupant behaviour and energy in buildings at the urban scale.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2016

Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy

Since its inception over two decades ago, Kelvin probe force microscopy (KPFM) has become the standard technique for characterizing electrostatic, electrochemical and electronic properties at the nanoscale. In this work, we present a purely digital, software-based approach to KPFM utilizing big data acquisition and analysis methods. General Mode (G-Mode) KPFM, works by capturing the entire photodetector data stream, typically at the sampling rate limit, followed by subsequent de-noising, analysis and compression of the cantilever response. We demonstrate that the G-Mode approach allows simultaneous multi-harmonic detection, combined with on-the-fly transfer function correction required for quantitative CPD mapping. The KPFM approach outlined in this work significantly simplifies the technique by avoiding cumbersome instrumentation optimization steps (i.e. lock in parameters, feedback gains etc.), while also retaining the flexibility to be implemented on any atomic force microscopy platform. We demonstrate the added advantages of G-Mode KPFM by allowing simultaneous mapping of CPD and capacitance gradient (C') channels as well as increased flexibility in data exploration across frequency, time, space, and noise domains. As a result, G-Mode KPFM is particularly suitable for characterizing voltage sensitive materials or for operation in conductive electrolytes, and will be useful for probing electrodynamics in photovoltaics, liquids and ionic conductors.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Thermochemical Data Fusion Using Graph Representation Learning

Large databases are required for “Big Data” applications in catalysis and materials science. Thermochemical databases can be created by combining data from various sources and by correcting low-fidelity datasets to higher accuracy with minimal computation. To achieve this “data fusion”, thermochemical quantities of interest, calculated at various levels of density functional theory (DFT), need to be mapped to the same, high levels of theory. In this work, a graph theoretical, statistical framework is proposed for such tasks. Subgraph frequencies are shown to provide a natural representation for learning these fusion maps. The maps are linear and are learnt with automated descriptor selection. Using a dataset of as few as ~1% from the QM9 database of 133,885 molecules, these models can predict multiple thermochemical quantities at a higher level of theory with an accuracy of 1 kcal/mol. Here, the method is explainable, generalizable, and provides a diagnostic tool for outlier identification

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRSDate not supplied

NASA’s Prototype Spectral Water Inversion Processor and Emulator (SWIPE): Towards Global Coastal and Inland Water Quality and Algal Biodiversity Monitoring

Degradation of Earth’s inland water resources due to anthropogenic perturbations and climate anomalies at both local and global scales continues to place human health at substantial risk. There is now a growing necessity to develop pragmatic approaches that allow timely and effective extrapolation of local processes, to spatially resolved global products, and to promote operational and sustainable resource policy management. This presentation will provide updates on NASA’s prototype open-source aquatic modeling platform, Spectral Water Inversion Processor and Emulator (SWIPE), which is a comprehensive, multi-faceted modeling platform for both forward and inverse modeling of diverse aquatic ecosystems from the benthos to top-of-atmosphere (TOA). SWIPE provides a cohesive application which leverages recent advancements in particle modeling, Big Data analytics, and machine learning to develop a high-fidelity synthetic training ground for sensitivity studies and algorithm development for multispectral or upcoming hyperspectral missions. Some of the prominent features of SWIPE to be discussed include: 1. Advanced hyperspectral modeling of globally diverse algal and non-algal particles using a novel two-layer coated sphere scattering model and radiative transfer modeling, 2. Massive, highly detailed synthetic spectral libraries of Analysis-Ready-Data (ARD) which include spectral libraries of particle microphysics, water biogeophysical and optical properties, as well as surface and TOA reflectances at 1 nm resolution, 3. An ensemble of pre-built analytic, machine learning, and deep learning inversion algorithms for various water quality and biodiversity related retrieval parameters and uncertainty quantification, 4. Sensor-agnostic water quality inversion at wide ranging spatial and spectral resolutions including a codebase for seamless application in the Google Earth Engine and NASA Earth Exchange (NEX) for planetary scale analysis. SWIPE will be a fully open-source platform based in python with comprehensive documentation, tutorials, and options for distributed computing on high performance computing clusters or on single, local machines. Further, we will discuss how we envision SWIPE contributing towards a global analysis of coastal and inland water quality dynamics.

top-of-atmosphere (TOA)↗
DOE OSTI2025

A 28 nm multiply-accumulate ASIC architecture for on-chip data compression in MHz frame rate X-ray and electron pixel detectors

Modern X-ray detector systems urgently require compact, efficient, and fast data compression schemes to handle the transmission of big data from pixel arrays, enabling frame rates in the MHz regime. Here, in this work, a data compression ASIC that implements a streaming fixed-length lossy compression scheme is introduced and analyzed, proving the feasibility and benefits of on-chip compression. The compression scheme utilizes a vector matrix product logic, which performs a number of floating-point multiplications, additions, and accumulations. The logic is verified, synthesized, and shown to fit in the area resource available for the X-ray detector under study, which comprises 192 × 168 pixels each of 12-bit width, and having a total area of 20 mm× 20 mm, about 2 mm× 20 mm of which are available for the digital logic. Several system architectures, precisions, and compression ratios ranging from 100 to 250 were analyzed to pave the way for on-chip fixed-length compression (e.g., principal component analysis, singular value decomposition) and data reduction (e.g., azimuthal integration) for X-ray and electron detectors.

Data compression↗
DOE OSTI2024

Accelerating Lossy and Lossless Compression on Emerging BlueField DPU Architectures

Data compression has become a crucial technique in addressing performance bottlenecks caused by increasing data volumes in High-Performance Computing (HPC), Big Data, and Deep Learning (DL). Despite its potential to boost system performance, recent studies have identified significant challenges with existing compression methods, mainly due to their high computational demands amidst continuously growing data sizes. Concurrently, the advent of Data Processing Units (DPUs), equipped with programmable System-on-Chip (SoC) and specialized compression accelerators, offers a promising opportunity to alter the landscape of data compression. This paper explores the complexities and potential of leveraging NVIDIA BlueField DPUs to accelerate lossy and lossless compression. Towards this, we introduce PEDAL, an innovative library that leverages the hardware capabilities of DPUs to unify and optimize data compression designs. Moreover, we seamlessly co-design PEDAL with the popular MPICH MPI library, demonstrating up to 101x speedup in compression time and 88x decrease in communication latency. Drawing on these achievements, we share our experience with various research communities about accelerating data compression on DPUs in communication-oriented HPC scenarios.

Li, Yuke↗
DOE OSTI2016

TU-D-201-04: Veracity of Data Elements in Radiation Oncology Incident Learning Systems

Purpose: Incident learning systems encompass volumes, varieties, values, and velocities of underlying data elements consistent with the V’s of big data. Veracity, the 5th V however exists only if there is high inter-rater reliability (IRR) within the data elements. The purpose of this work was to assess IRR in the nationally deployed RO-ILS: Radiation Oncology-Incident Learning System (R) sponsored by the American Society for Radiation Oncology (ASTRO) and the American Association of Physicists in Medicine (AAPM). Methods: Ten incident reports covering a wide range of scenarios were created in standardized narrative and video formats and disseminated to 67 volunteers of multiple disciplines from 26 institutions along with two published narratives from the International Commission of Radiological Protection to assess IRR on a nationally representative level. The volunteers were instructed to independently enter the associated data elements in a test version of RO-ILS over a 3-week period. All responses were aggregated into a spreadsheet to assess IRR using free-marginal kappa metrics. Results: 48 volunteers from 21 institutions completed all reports in the study period. The average kappa score for all raters across all critical data elements was 0.659 [range 0.326–1.000]. Statistically significant differences (p <0.05) were noted between reporters of different disciplines and raters with varying levels of experience. Kappa scores were high for event classification (0.781) and contributory factors (0.777) and low for likelihood-of-harm (0.326). IRR was highest among AAPM-ASTRO members (0.672) and lowest among trainees (0.463). Conclusion: A moderate-to-substantial level of IRR in RO-ILS was noted in this study. Although the number of events reviewed in this study was small, opportunities for improving the taxonomy for the lower scoring data elements as well as specific educational targets for training were identified by assessing data veracity quantitatively. This is expected to improve the quality of the data garnered from RO-ILS.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2014

Power Management Techniques for Data Centers: A Survey

With growing use of internet and exponential growth in amount of data to be stored and processed (known as ``big data''), the size of data centers has greatly increased. This, however, has resulted in significant increase in the power consumption of the data centers. For this reason, managing power consumption of data centers has become essential. In this paper, we highlight the need of achieving energy efficiency in data centers and survey several recent architectural techniques designed for power management of data centers. We also present a classification of these techniques based on their characteristics. This paper aims to provide insights into the techniques for improving energy efficiency of data centers and encourage the designers to invent novel solutions for managing the large power dissipation of data centers.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Early Fault Detection in Particle Accelerator Power Electronics Using Ensemble Learning

Early fault detection and fault prognosis are crucial to ensure efficient and safe operations of complex engineering systems such as the Spallation Neutron Source (SNS) and its power electronics (high voltage converter modulators). Following an advanced experimental facility setup that mimics SNS operating conditions, the authors successfully conducted 21 early fault detection experiments, where fault precursors are introduced in the system to a degree enough to cause degradation in the waveform signals, but not enough to reach a real fault. Nine different machine learning techniques based on ensemble trees, convolutional neural networks, support vector machines, and hierarchical voting ensembles are proposed to detect the fault precursors. Although all 9 models have shown a perfect and identical performance during the training and testing phase, the performance of most models has decreased in the next test phase once they got exposed to realworld data from the 21 experiments. The hierarchical voting ensemble, which features multiple layers of diverse models, maintains a distinguished performance in early detection of the fault precursors with 95% success rate (20/21 tests), followed by adaboost and extremely randomized trees with 52% and 48% success rates, respectively. The support vector machine models were the worst with only 24% success rate (5/21 tests). The study concluded that a successful implementation of machine learning in the SNS or particle accelerator power systems would require a major upgrade in the controller and the data acquisition system to facilitate streaming and handling big data for the machine learning models. In addition, this study shows that the best performing models were diverse and based on the ensemble concept to reduce the bias and hyperparameter sensitivity of individual models.

43 PARTICLE ACCELERATORS↗
NASA NTRS2018

FAIRness and Usability for Open-Access Omics Data Systems

Omics data sharing is especially crucial to the biological research community, and the last decade or two has seen a huge rise in collaborative analysis systems, databases, and knowledge bases for omics and other systems biology data. We assessed the "FAIRness" of NASA's GeneLab Data Systems (GLDS) along with four similar kinds of systems in the research omics data domain, using 14 FAIRness metrics. 14 metrics. The range of Pass ratings was 29-79% of the 14 metrics, Partial Pass 0-21%, and Fail 7-50%. The range of overall FAIRness scores was 5-12 (out of 14). The systems we evaluated performed the best in the areas of data findability and accessibility, and worst in the area of data interoperability. We propose two new principles that Big Data systems, in particular, should consider for increasing data accessibility. We relate our experiences implementing semantic integration of omics data from several systems for the federated querying and retrieval functions of the GLDS, given the shortcomings in data interoperability of these systems.

Berrios, Daniel C.↗
DOE OSTI2016

Measurement-based performance profiles and dynamics of UDT over dedicated connections

Wide-area data transfers in high-performance computing and big data scenarios are increasingly being carried over dedicated network connections that provide high capacities at low loss rates. UDP-based transport protocols are expected to be particularly well-suited for such transfers but their performance is relatively unexplored over a wide range of connection lengths, compared to TCP over shared connections. We present extensive throughput measurements of UDP-based Data Transfer (UDT) over a suite of physical and emulated 10 Gbps connections. In sharp contrast to current UDT analytical models, these measurements indicate much more complex throughput dynamics that are sensitive to the connection modality, protocol parameters, and round-trip times. Lyapunov exponents estimated from the Poincare maps of UDT traces clearly indicate regions of instability and complex dynamics. We propose a simple model based on the ramp-up and sustainment regimes of a generic transport protocol, which qualitatively illustrates the dominant monotonicity and concavity properties of throughput profiles and relates them to Lyapunov exponents. These measurements and analytical results together enable us to comprehensively evaluate UDT performance and select parameters to achieve high throughput, and they also provide guidelines for designing effective transport protocols for dedicated connections.

Foster, Ian↗