Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Big data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Optoelectronic Properties of C60 and C70 Fullerene Derivatives: Designing and Evaluating Novel Candidates for Efficient P3HT Polymer Solar Cells

Ten novel fullerene-derivatives (FDs) of C60 and C70 had been designed as acceptor for polymer solar cell (PSC) by employing the quantitative structure-property relationship (QSPR) model, which was developed strategically with a reasonably big pool of experimental power conversion efficiency (PCE) data. The QSPR model was checked and validated with stringent parameter and reliability of predicted PCE values of all designed FDs. They were assessed by the applicability domain (AD) and process randomization test. The predicted PCE of FDs range from 7.96 to 23.01. The obtained encouraging results led us to the additional theoretical analysis of the energetics and UV-Vis spectra of isolated dyes employing Density functional theory (DFT) and Time-dependent-DFT (TD-DFT) calculations using PBE/6-31G(d,p) and CAM-B3LYP/6-311G(d,p) level calculations, respectively. The FD4 is the best C60-derivatives candidates for PSCs as it has the lowest exciton binding energy, up-shifted lowest unoccupied molecular orbital (LUMO) energy level to increase open-circuit voltage (VOC) and strong absorption in the UV region. In case of C70-derivatives, FD7 is potential candidate for future PSCs due to its strong absorption in UV-Vis region and lower exciton binding energy with higher VOC. Our optoelectronic results strongly support the developed QSPR model equation. Analyzing QSPR model and optoelectronic parameters, we concluded that the FD1, FD2, FD4, and FD10 are the most potential candidates for acceptor fragment of fullerene-based PSC. The outcomes of tactical molecular design followed by the investigation of optoelectronic features are suggested to be employed as a significant resource for the synthesis of FDs as an acceptor of PSCs.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Towards Real-Time Detection and Tracking of Spatio-Temporal Features: Blob-Filaments in Fusion Plasma [Spatio-Temporal Features in Large Irregular Data: Blob-Filaments in Fusion Plasma]

This work was originally motivated the need to identify and track blob filaments, a feature often associated with instability in magnetically confined fusion plasma. Understanding and mitigating such instability would improve fusion reactors and make fusion a truly inexhaustible source of clean energy. Similar spatio-temporal features are important in many other applications, for example, ignition kernels in combustion and tumor cells in a medical image. This work presents an approach for extracting these features by dividing the overall task into three steps: local identification of feature cells, grouping feature cells into extended feature, and tracking movement of feature through overlapping in space. Through our extensive work in parallelization, we demonstrate that this approach can effectively make use of a large number of compute nodes to detect and track blobs in fusion plasma. On a set of 4.3TB data, we observed linear speedup on 1024 processes and completed blob detection in less than three milliseconds using Edison, a Cray XC30 system at NERSC.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2018

Introduction to Analysis Methods for Big Earth Data

Big Earth Data are too big to be tractable to simple data inspection and require models to make sense of all the data. Useful models for Big Earth Data may be physical, statistical, or machine learning based. In many cases, hybrid models combine attributes of two or more of these types.

parallel processing (computers)↗
DOE OSTI2019

Big Hill 2018 InSAR Analysis U.S. Strategic Petroleum Reserve

The historical subsidence surveys shot over the U.S. Strategic Petroleum Reserve Big Hill site, located in southeastern Texas, have indicated surface uplift since 2002. In order to better understand and substantiate the surface behavior inferred from annual elevation measurements, InSAR (interferometric synthetic aperture radar) data was acquired. InSAR involves the processing of multiple satellite synthetic aperture radar scenes acquired across the same location of the Earth's surface at different times to map surface deformation. The analysis of the data can detect millimeters of motion spanning days, months, year and decades, across specific sites. The InSAR analysis indicates the fastest subsidence rates are over the north central region of the site, specifically centered over caverns 104 and 103. Subsidence rates decrease towards both the west and east, with the western side subsiding at greater rate than the eastern edge. There is some uplift noted, off the site and off the dome to the east. Overall, the subsidence pattern is in line with subsidence behavior expected over a cavern field. In investigating the validity of the uplift measured during the ground surveys it was discovered that reference location can impact results. An exercise was conducted that took the current InSAR data and presented two varying results dependent on the reference location, either on or off the dome. The conclusion was that if the reference is located on the dome, as it has been for years for the ground surveys, the reference location is moving too, giving the appearance of uplift.

02 PETROLEUM↗
DOE OSTI2021

FOA 1861 Data Curation Overview

This document describes the process executed to collect, examine, and consolidate Phasor Measurement Unit (PMU) data from multiple transmission operators into a common dataset. The consolidated PMU data set was further anonymized and distributed to the Department of Energy Funding Opportunity Announcement (FOA) 1861 Big Data Analysis of Synchrophasor Data awardees.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS2012

A Systems Modeling Approach for Risk Management of Command File Errors

The main cause of commanding errors is often (but not always) due to procedures. Either lack of maturity in the processes, incompleteness of requirements or lack of compliance to these procedures. Other causes of commanding errors include lack of understanding of system states, inadequate communication, and making hasty changes in standard procedures in response to an unexpected event. In general, it's important to look at the big picture prior to making corrective actions. In the case of errors traced back to procedures, considering the reliability of the process as a metric during its' design may help to reduce risk. This metric is obtained by using data from Nuclear Industry regarding human reliability. A structured method for the collection of anomaly data will help the operator think systematically about the anomaly and facilitate risk management. Formal models can be used for risk based design and risk management. A generic set of models can be customized for a broad range of missions.

probabilistic risk↗
DOE OSTI2013

Feature Tracking and Visualization of Madden-Julian Osciallation in Climate Simulation

Madden-Julian Oscillation (MJO) is one of the less understood aspects of tropical meteorology, which plays a significant role in tropical intra-seasonal variations in rain, temperature and winds over the Indian and Pacific Oceans. In this paper, we present an integrated analysis and visualization framework for MJO episodes simulated by a high resolution regional model. To distinguish MJOs from other weather phenomena, our framework utilizes domain knowledge to track MJOs as finding the globally optimized properties in the data. In addition to enhancing the animation with feature tracking, our visualization system also integrates different visualization components such as Virtual Globe and Hovmoller Diagrams to visualize large scale events both in space and time. By linking all of these visualization components on a web-based interface, scientists can identify cloud and environmental processes associated with the initiation and eastward propagation of MJO more easily.

big-data visualization↗
DOE OSTI2022

Automated defect identification in electroluminescence images of solar modules

Solar photovoltaic (PV) modules are susceptible to manufacturing defects, mishandling problems or extreme weather events that can limit energy production or cause early device failure. Trained professionals use electroluminescence (EL) images to identify defects in modules, however, field surveys or inline image acquisition can generate millions of EL images, which are infeasible to analyze by rote inspection. Here, we develop a rapid automatic computer vision pipeline (~0.5 seconds/module) to analyze EL images and identify defects including cracks, intra-cell defects, oxygen-induced defects, and solder disconnections. Defect identification is achieved with a machine learning model (Random Forest, ResNet models and YOLO) trained on 762 manually-labeled EL images of PV modules. We compare model performance on an imbalanced real-world validation set containing 134 EL images and determine that ResNet18 and YOLO are the optimal models; we next evaluated these models on a dedicated testing set (129 module images) with resulting macro F1 scores of 0.83 (ResNet18) and 0.78 (YOLO). Using a field EL survey of a PV power plant damaged in a vegetation fire, we analyze 18,954 EL images (2.4 million cells) and inspect the spatial distribution of defects on the solar modules. The results find increased frequency of ‘crack’, ‘solder’ and ‘intra-cell’ defects on the edges of the solar module closest to the ground after fire. We also find an abnormal increase of striation rings on cells which were assumed to be caused mainly in fabrication process. Our methods are published as open-source software. It can also be used to identify other kinds of defects or process different types of solar cells with minor modification on models by transfer learning.

14 SOLAR ENERGY↗
DOE OSTI2014

Occupancy schedules learning process through a data mining framework

Building occupancy is a paramount factor in building energy simulations. Specifically, lighting, plug loads, HVAC equipment utilization, fresh air requirements and internal heat gain or loss greatly depends on the level of occupancy within a building. Developing the appropriate methodologies to describe and reproduce the intricate network responsible for human-building interactions are needed. Extrapolation of patterns from big data streams is a powerful analysis technique which will allow for a better understanding of energy usage in buildings. A three-step data mining framework is applied to discover occupancy patterns in office spaces. First, a data set of 16 offices with 10 minute interval occupancy data, over a two year period is mined through a decision tree model which predicts the occupancy presence. Then a rule induction algorithm is used to learn a pruned set of rules on the results from the decision tree model. Finally, a cluster analysis is employed in order to obtain consistent patterns of occupancy schedules. Furthermore, the identified occupancy rules and schedules are representative as four archetypal working profiles that can be used as input to current building energy modeling programs, such as EnergyPlus or IDA-ICE, to investigate impact of occupant presence on design, operation and energy use in office buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2016

What can the CMB tell about the microphysics of cosmic reheating?

In inflationary cosmology, cosmic reheating after inflation sets the initial conditions for the hot big bang. We investigate how CMB data can be used to study the effective potential and couplings of the inflaton during reheating to constrain the underlying microphysics. If there is a phase of preheating that is driven by a parametric resonance or other instability, then the thermal history and expansion history during the reheating era depend on a large number of microphysical parameters in a complicated way. In this case the connection between CMB observables and microphysical parameters can only established with intense numerical studies. Such studies can help to improve CMB constraints on the effective inflaton potential in specific models, but parameter degeneracies usually make it impossible to extract meaningful best-fit values for individual microphysical parameters. If, on the other hand, reheating is driven by perturbative processes, then it can be possible to constrain the inflaton couplings and the reheating temperature from CMB data. This provides an indirect probe of fundamental microphysical parameters that most likely can never be measured directly in the laboratory, but have an immense impact on the evolution of the cosmos by setting the stage for the hot big bang.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Clinical Natural Language Processing for Radiation Oncology: A Review and Practical Primer

Natural language processing (NLP), which aims to convert human language into expressions that can be analyzed by computers, is one of the most rapidly developing and widely used technologies in the field of artificial intelligence. Natural language processing algorithms convert unstructured free text data into structured data that can be extracted and analyzed at scale. In medicine, this unlocking of the rich, expressive data within clinical free text in electronic medical records will help untap the full potential of big data for research and clinical purposes. Recent major NLP algorithmic advances have significantly improved the performance of these algorithms, leading to a surge in academic and industry interest in developing tools to automate information extraction and phenotyping from clinical texts. Thus, these technologies are poised to transform medical research and alter clinical practices in the future. Radiation oncology stands to benefit from NLP algorithms if they are appropriately developed and deployed, as they may enable advances such as automated inclusion of radiation therapy details into cancer registries, discovery of novel insights about cancer care, and improved patient data curation and presentation at the point of care. However, challenges remain before the full value of NLP is realized, such as the plethora of jargon specific to radiation oncology, nonstandard nomenclature, a lack of publicly available labeled data for model development, and interoperability limitations between radiation oncology data silos. Successful development and implementation of high quality and high value NLP models for radiation oncology will require close collaboration between computer scientists and the radiation oncology community. Here, we present a primer on artificial intelligence algorithms in general and NLP algorithms in particular; provide guidance on how to assess the performance of such algorithms; review prior research on NLP algorithms for oncology; and describe future avenues for NLP in radiation oncology research and clinics.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2014

Results and Lessons Learned From the DOE Commercial Building Partnerships: Preprint

Over the course of 5 years, NREL worked with commercial building owners and their design teams in the DOE Commercial Building Partnerships (CBP) to cut energy consumption by 50% in new construction (versus code) and by 30% in existing building pilot projects (versus code or pre-retrofit operational energy use depending on the preference of the Partner) using strategies that could be replicated across their building portfolios. A number of different building types were addressed, including supermarket, retail merchandise, combination big box (general merchandise and food sales), high rise office space, and warehouse. The projects began in pre-design and included a year of measurement data to evaluate performance against design expectations. Focused attention was required throughout the entire process to achieve a design with the potential to hit the energy performance target and to operate the resulting building to reach this potential. This paper will report quantitative results and cover both the technical and the human sides of CBP, including the elements that were required to succeed and where stumbling blocks were encountered. It will also address the impact of energy performance goals and intensive energy modeling on the design process innovations and best practices.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

Next Generation Workload Management System For Big Data on Heterogeneous Distributed Computing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS and ALICE are the largest collaborations ever assembled in the sciences and are at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, both experiments rely on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System (WMS) for managing the workflow for all data processing on hundreds of data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. The scale is demonstrated by the following numbers: PanDA manages O(10 2 ) sites, O(10 5 ) cores, O(10 8 ) jobs per year, O(10 3 ) users, and ATLAS data volume is O(10 17 ) bytes. In 2013 we started an ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF). The project titled 'Next Generation Workload Management and Analysis System for Big Data' (BigPanDA) is funded by DOE ASCR and HEP. Extending PanDA to clouds and LCF presents new challenges in managing heterogeneity and supporting workflow. The BigPanDA project is underway to setup and tailor PanDA at the Oak Ridge Leadership Computing Facility (OLCF) and at the National Research Center "Kurchatov Institute" together with ALICE distributed computing and ORNL computing professionals. Our approach to integration of HPC platforms at the OLCF and elsewhere is to reuse, as much as possible, existing components of the PanDA system. Finally, we will present our current accomplishments with running the PanDA WMS at OLCF and other supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Where Is the Provenance? Ethical Replicability and Reproducibility in GIScience and Its Critical Applications

As replicability and reproducibility (R&R) crises develop within emerging convergent inquiry, ethical use of provenance information is central to the establishment and preservation of trust in critical applications of GIScience and geospatial technologies. Today large volumes of geospatial data are generated at high velocity from satellite sensors and unmanned aircraft systems, citizen sensors, geolocation-based data services, global navigation satellite systems, and so on. The extensive use of these data for applications such as disaster and humanitarian response raises the issue of R&R from competing perspectives of location privacy and geospatial data quality. Although geospatial data can be integrated and linked with contextual information to identify individuals’ movements, steps taken to ensure privacy can complicate the multiuser development of high-quality geospatial workflows. Provenance information as digital records of historical (retrospective) and potential future (prospective) geospatial processes is often overlooked, misunderstood, or inadequately addressed. We explore the relationship between provenance information, location privacy, and geospatial data quality in the context of R&R with a focus on disaster analytics. Here, we argue that in the era of big data and deep learning, GIScientists and associated institutions bear greater responsibility both for geospatial workflow quality and for location privacy. Given vastly heterogenous computational landscapes, we provide practical recommendations for ethically driven provenance and R&R research and development within the GIScience community and beyond.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2018

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro- Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

analytics-hub↗
DOE OSTI2019

Condition Monitoring of Wind Turbine Planetary Gearboxes Under Different Operating Conditions

Digitally enhanced services for wind power could reduce Operations and Maintenance costs as well as the Levelised Cost Of Energy. Therefore, there is a continuous need for advanced monitoring techniques which can exploit the opportunities of Internet of Things and Big Data Analytics. The heart of wind turbines is an epicyclic gearbox and rolling element bearings are often responsible for machinery stops. The vibration signatures of bearings are rather weak compared to other components. As a result a number of signal processing techniques have been proposed, focusing towards accurate and early fault detection with limited false alarms and missed detections. In Envelope Analysis an envelope of the vibration signal is estimated usually after filtering around a frequency band. Different tools, such as Kurtogram, have been proposed to select the optimum filter parameters. Monitoring techniques have reached a mature level for steady speed and load operating conditions. On the other hand, under chaniging operating conditions, it is still unclear whether the change of the monitoring indicators is due to the change of the machinery's health or due to the change of operating parameters. Recently, the authors have proposed a new tool called IESFOgram, based on Cyclic Spectral Coherence for the automatic selection of the filtering band. In this paper, the performance of the tool is evaluated on the National Renewable Energy Laboratory wind turbine gearbox vibration condition monitoring benchmarking data set which includes various faults with different levels of diagnostic complexity as well as various speed and load operating conditions.

17 WIND ENERGY↗
NASA NTRS2017

Engine Icing Data - An Analytics Approach

Engine icing researchers at the NASA Glenn Research Center use the Escort data acquisition system in the Propulsion Systems Laboratory (PSL) to generate and collect a tremendous amount of data every day. Currently these researchers spend countless hours processing and formatting their data, selecting important variables, and plotting relationships between variables, all by hand, generally analyzing data in a spreadsheet-style program (such as Microsoft Excel). Though spreadsheet-style analysis is familiar and intuitive to many, processing data in spreadsheets is often unreproducible and small mistakes are easily overlooked. Spreadsheet-style analysis is also time inefficient. The same formatting, processing, and plotting procedure has to be repeated for every dataset, which leads to researchers performing the same tedious data munging process over and over instead of making discoveries within their data. This paper documents a data analysis tool written in Python hosted in a Jupyter notebook that vastly simplifies the analysis process. From the file path of any folder containing time series datasets, this tool batch loads every dataset in the folder, processes the datasets in parallel, and ingests them into a widget where users can search for and interactively plot subsets of columns in a number of ways with a click of a button, easily and intuitively comparing their data and discovering interesting dynamics. Furthermore, comparing variables across data sets and integrating video data (while extremely difficult with spreadsheet-style programs) is quite simplified in this tool. This tool has also gathered interest outside the engine icing branch, and will be used by researchers across NASA Glenn Research Center. This project exemplifies the enormous benefit of automating data processing, analysis, and visualization, and will help researchers move from raw data to insight in a much smaller time frame.

Engine Icing↗
DOE OSTI2022

Comparing Model Representations of Physiological Limits on Transpiration at a Semi-arid Ponderosa Pine Site

Mechanistic representations of biogeochemical processes in ecosystem models are rapidly advancing, requiring advancements in model evaluation approaches. Here we quantify multiple aspects of model functional performance to evaluate improved process representations in ecosystem models. We compare semi-empirical stomatal models with hydraulic constraints against more mechanistic representations of stomatal and hydraulic functioning at a semi-arid pine site using a suite of metrics and analytical tools. We find that models generally perform similarly under unstressed conditions, but performance diverges under atmospheric and soil drought. The more empirical models better capture synergistic information flows between soil water potential and vapor pressure deficit to transpiration, while the more mechanistic models are overly deterministic. Although models can be parameterized to yield similar functional performance, alternate parameterizations could not overcome structural model constraints that underestimate the unique information contained in soil water potential about transpiration. Additionally, both multilayer canopy and big-leaf models were unable to capture the magnitude of canopy temperature divergence from air temperature, and we demonstrate that errors in leaf temperature can propagate to considerable error in simulated transpiration. This study demonstrates the value of merging underutilized observational data streams with emerging analytical tools to characterize ecosystem function and discriminate among model process representations.

54 ENVIRONMENTAL SCIENCES↗