Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

Drone flight visible images, digital elevation maps, and geotiffs, Teller, Seward Peninsula, Alaska, USA, July 2017

Remote sensing data collected from Lawrence Berkeley National Laboratory’s (LBNL) unmanned aerial system (UAS) quadcopter platform – Inspire-1 – operated by the Ameriflux Management Project (ameriflux.lbl.gov) Technical Team. The Insprire-1 is a visible imagery UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) at regular intervals along the flight path. This package includes data from a flight path over the NGEE Arctic Teller Site, west of mile marker 27 Bob Blodgett Highway in the Seward Peninsula, Alaska in July 2017. Derived image products include ortho-mosaiced RGB, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, and a digital terrain model (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions are also included. Unprocessed and processed data products are included in this package (processing levels 0-2). This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Drone flight visible images, digital elevation maps, and geotiffs, Kougarok, Seward Peninsula, Alaska, USA, July 2017

Remote sensing data collected from Lawrence Berkeley National Laboratory’s (LBNL) unmanned aerial system (UAS) quadcopter platform – Inspire-1 – operated by the Ameriflux Management Project (ameriflux.lbl.gov) Technical Team. The Insprire-1 is a visible imagery UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) at regular intervals along each flight path. This package includes data from four flight paths over the NGEE Arctic Kougarok Site, west of mile marker 64 on Kuzitrin Road in the Seward Peninsula, Alaska in July 2017. Derived image products include ortho-mosaiced RGB, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, and a digital terrain model (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions are also included. Unprocessed and processed data products are included in this package (processing levels 0-2). This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Diamonds in the rough: a strong case for the inclusion of weak-intensity X-ray diffraction data

Here, new evidence is provided to show that the inclusion of weak-intensity, high-resolution X-ray diffraction data helps to improve the quality of experimental phases by imposing proper constraints on electron-density models during noncrystallographic symmetry averaging. Overwhelming evidence exists to show that the inclusion of weak-intensity, high-resolution X-ray diffraction data helps improve the refinement of atomic models by imposing strong constraints on individual and overall temperature B factors and thus the quality of crystal structures. Some researchers consider these data to be of little value and opt to discard them during data processing, particularly at medium and low resolution, at which individual B factors of atomic models cannot be refined. Here, new evidence is provided to show that the inclusion of these data helps to improve the quality of experimental phases by imposing proper constraints on electron-density models during noncrystallographic symmetry (NCS) averaging. Using electron-density correlation coefficients as criteria, the resolution of data has successfully been extended from 3.1 to 2.5 Å resolution with redundancy-independent merging R factors from below 100% to about 310%. It is further demonstrated that phase information can be fully extracted from observed amplitudes through de novo NCS averaging. Averaging starts with uniform density inside double-shelled spherical masks and NCS matrices that are derived from bound heavy-atom clusters at the vertices of cuboctahedrally symmetric protein particles.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A NOVEL TOOL FOR IMPROVING THE DATA COLLECTION PROCESS DURING CONTROL ROOM MODERNIZATION HUMAN-SYSTEM INTERFACE TESTING AND EVALUATION ACTIVITIES

The U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program is developing a scientific basis to extend the existing U.S. nuclear power plant (NPP) operating life beyond the current 60-year licensing period and to ensure NPP long-term reliability, productivity, safety, and security. Under the Advanced Instrumentation, Information, and Control (II&C) Systems Technologies pathway, the LWRS Program conducts targeted research and development to address long-term aging and obsolescence of existing instrumentation and control technologies, as well as to implement and test new digital II&C technology that enables broad innovation and business improvement in the NPP operating model. To this end, the role of human factors engineering (HFE) is important as part of ensuring that the control room modernization (CRM) activities account for the operator’s cognitive and physiological characteristics. Part of the HFE process for CRM entails conducting tests and evaluations (T&Es) that will inform the design of new human-system interface (HSI) technologies. Qualitative data that captures operator feedback and important human-system interactions are an important part of T&E. However, the analyzing this qualitative data in order to provide actionable design recommendations can be time-consuming and error prone. This paper presents an early version of a tool that can be used to analyze this qualitative data to (1) sift through freeform text for improved design issue identification, (2) maintain consistency in theme coding conventions, and (3) readily provide actionable design guidance that can be traceable to state-of-the-art HFE guidelines such as NUREG-0700. A description of this tool is discussed next.

99 GENERAL AND MISCELLANEOUS↗

DeepAdversaries: examining the robustness of deep learning models for galaxy morphology classification

With increased adoption of supervised deep learning methods for work with cosmological survey data, the assessment of data perturbation effects (that can naturally occur in the data processing and analysis pipelines) and the development of methods that increase model robustness are increasingly important. In the context of morphological classification of galaxies, we study the effects of perturbations in imaging data. In particular, we examine the consequences of using neural networks when training on baseline data and testing on perturbed data. We consider perturbations associated with two primary sources: (a) increased observational noise as represented by higher levels of Poisson noise and (b) data processing noise incurred by steps such as image compression or telescope errors as represented by one-pixel adversarial attacks. We also test the efficacy of domain adaptation techniques in mitigating the perturbation-driven errors. We use classification accuracy, latent space visualizations, and latent space distance to assess model robustness in the face of these perturbations. For deep learning models without domain adaptation, we find that processing pixel-level errors easily flip the classification into an incorrect class and that higher observational noise makes the model trained on low-noise data unable to classify galaxy morphologies. On the other hand, we show that training with domain adaptation improves model robustness and mitigates the effects of these perturbations, improving the classification accuracy up to 23% on data with higher observational noise. Domain adaptation also increases up to a factor of ${\approx}2.3$ the latent space distance between the baseline and the incorrectly classified one-pixel perturbed image, making the model more robust to inadvertent perturbations. Successful development and implementation of methods that increase model robustness in astronomical survey pipelines will help pave the way for many more uses of deep learning for astronomy.

79 ASTRONOMY AND ASTROPHYSICS↗

What can we learn from N 2 O isotope data? – Analytics, processes and modelling

The isotopic composition of nitrous oxide (N 2 O) provides useful information for evaluating N 2 O sources and budgets. Due to the co–occurrence of multiple N 2 O transformation pathways, it is, however, challenging to use isotopic information to quantify the contribution of distinct processes across variable spatiotemporal scales. Here, we present an overview of recent progress in N 2 O isotopic studies and provide suggestions for future research, mainly focusing on: analytical techniques; production and consumption processes; and interpretation and modelling approaches. Comparing isotope–ratio mass spectrometry (IRMS) with laser absorption spectroscopy (LAS), we conclude that IRMS is a precise technique for laboratory analysis of N 2 O isotopes, while LAS is more suitable for in situ/inline studies and offers advantages for site–specific analyses. When reviewing the link between the N 2 O isotopic composition and underlying mechanisms/processes, we find that, at the molecular scale, the specific enzymes and mechanisms involved determine isotopic fractionation effects. In contrast, at plot–to–global scales, mixing of N 2 O derived from different processes and their isotopic variability must be considered. We also find that dual isotope plots are effective for semi–quantitative attribution of co–occurring N 2 O production and reduction processes. More recently, process–based N 2 O isotopic models have been developed for natural abundance and 15 N–tracing studies, and have been shown to be effective, particularly for data with adequate temporal resolution. Here, despite the significant progress made over the last decade, there is still great need and potential for future work, including development of analytical techniques, reference materials and inter–laboratory comparisons, further exploration of N 2 O formation and destruction mechanisms, more observations across scales, and design and validation of interpretation and modelling approaches. Synthesizing all these efforts, we are confident that the N 2 O isotope community will continue to advance our understanding of N 2 O transformation processes in all spheres of the Earth, and in turn to gain improved constraints on regional and global budgets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Using SciDB to Support Photon Science Data Analysis

Array data analytic systems like SciDB hold great potential to accelerate processing data from SLAC's Linac Coherent Light Source and other experiments. SciDB is unique in its ability to integrate storage and processing of array data efficiently, providing both space-efficient storage and out-of-memory efficient parallel array processing. We describe a recent effort to leverage SciDB to store and process LCLS data. The work includes development of software to import data into SciDB, subsequent benchmarks, and interactive manipulation of data in SciDB.

43 PARTICLE ACCELERATORS↗

FPGA-based prototype of portable environmental radiation monitor

This new portable radiological environmental monitor consists of 2 main components, Gamma ionization chamber and a FPGA-based electronic enclosure linked to convivial software for treatment and analyzing. The HPIC ion chamber is the heart of this radiation measurement system and is running in range from 0 to 100 mR/h, so that the sensitivity at the output is 20 mV/μR/h, with a nearly flat energy response from 0,07 to 10 MEV. This paper presents a contribution for developing a new nuclear measurement data acquisition system based on Cyclone III FPGA Starter Kit ALTERA, and a user-friendly software to run real-time control and data processing. It was developed to substitute the older radiation monitor RSS-112 PIC installed in CNESTEN's Laboratory in order to improve some of its functionalities related to acquisition time and data memory capacity. As for the associated acquisition software, it was conceived under the virtual LabView platform from National Instrument, and offers a variety of system setup for radiation environmental monitoring. It gives choice to display both the statistical data and the dose rate. Statistical data shows a summary of current data, current time/date and dose integrator values, and the dose rate displays the current dose rate in large numbers for viewing from a distance as well as the date and time. The prototype version of this new instrument and its data processing software has been successfully tested and validated for viewing and monitoring the environmental radiation of Moroccan nuclear center. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Toward artifact-free data in Hadamard transform-based double multiplexing of ion mobility-Orbitrap mass spectrometry

The integration of ion mobility spectrometry (IMS) with trap-based mass spectrometer (MS) such as Orbitrap using the dual gate approach suffers from low duty cycle. Efforts to improve the duty cycle involve the utilization of Hadamard transform based double multiplexing which significantly improve the signal to noise ratio and duty cycle of the ion mobility – Orbitrap mass spectrometry (IM – Orbitrap MS) platform. However, significant fluctuations in ion intensity and the temporal shifts in the encoded data give rise to artifacts and noise in the demultiplexed data which significantly reduce the data quality and negate the benefits of multiplexing. We propose a new approach that identifies the true IM peaks and helps in eliminating the artifacts in the demultiplexed data leading to a decrease in false positives in subsequent data processing. The algorithm takes an analytical approach to first identify the position of the IM peak in the temporal domain, and then demultiplex and identify the true data making it easier for subsequent data processing. After the application of the algorithm, the quality of the IM-Orbitrap MS measurements was greatly improved because of the reduction in artifacts.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Process Knowledge Data Gathering and Reporting in Support of Decommissioning [Slides]

Summary of recent ORAU decommissioning activities at the Oak Ridge National Laboratory (ORNL) and the East Tennessee Technology Park (ETTP). Project objective was to generate approved Waste Lot Profiles for legacy facilities scheduled for demolition and shipment to the Environmental Management Waste Management Facility (EMWMF) or appropriate alternate facility. The form and content of process knowledge (PK) reports were developed with input from the EMWMF Waste Acceptance Criteria (WAC) Attainment Team and regulators. PK may be defined as the knowledge of the design and the history of operations that occurs during the life cycle of a facility (paraphrased from SRNL guidance) - similar to the MARSSIM historical site assessment. Some types of PK data used to decommission ORNL and ETTP facilities include: (1) Design drawings; (2) Historical documents [e.g., History of the Oak Ridge National Laboratory by Thomas (1963) and A Brief History of the Chemical Technical Division (ORNL/M-2733)]; (3) Historical photographs; (4) Radiological survey reports; (5) Facility-specific databases - (a) Spill history, (b) Waste Information Tracking System (WITS), and (c) Hazardous Materials Management Information System (HMMIS); (6) Facility walkdown summary reports; and (7) Living memory data. Facility walkdowns are critical for worker safety planning and to assure on-the-ground-conditions match historical descriptions. For Oak Ridge operations, investigators also document the nature and number of items requiring special handling or disposition planning, such as the following: (1) Items containing polychlorinated biphenyls, asbestos, lead, or refrigerants; (2) Items with physical WAC restriction (e.g., large items, pipes, and concrete); and (3) Too 'hot' for EMWMF. Special emphasis was made to interview facility managers, scientists, technicians, or anyone with direct knowledge of process-related activities. Interviews often led to more contact names and reports but also offered anecdotal accounts of releases, process-related operations, maintenance activities, and other relevant information not addressed in the written record. 'Fun' part of PK data gathering. Often got not-so-useful information such as, 'The operations manager was a jerk and we all hated him.' PK data are used to indicate the presence or absence of contaminants. Multiple lines of investigation are necessary for characterization planning and to help determine which disposal facility is best suited for targeted wastes. The model used by ORAU assisted remediation contractors and EMWMF managers by identifying anomalous waste and items requiring special handling.

54 ENVIRONMENTAL SCIENCES↗

False alarms and the IMS infrasound network: Understanding the factors influencing the creation of false events

The International Monitoring System (IMS) infrasound network has been designed to acquire the necessary data to detect and locate explosions in the atmosphere with a yield equivalent to 1 kiloton of TNT anywhere on Earth. A major associated challenge is the task of automatically processing data from all IMS infrasound stations to identify possible nuclear tests for subsequent review by analysts. This paper is the first attempt to quantify the false alarm rate (FAR) of the IMS network, and in particular to assess how the FAR is affected by the numbers and distributions of detections at each infrasound station. To ensure that the results are sufficiently general, and not dependent entirely on one detection algorithm, the assessment is based on two detection algorithms that can be thought of as end members in their approach to the trade-off between missed detections and false alarms. The results show that the FAR for events formed at only two arrays is extremely high (ranging from 10’s to 100’s of false events per day across the IMS network, depending on the detector tuning). It is further shown that the FAR for events formed at three or more IMS arrays is driven by ocean-generated waves (microbaroms), despite efforts within both detection algorithms for avoiding these signals, indicating that further research into this issue is merited. Overall, the results highlight the challenge of processing data from a globally sparse network of stations to detect and form events. On the basis of these results, it is shown that further work on understanding signal and noise characteristics is needed before attempting to automatically form events detected at only two arrays on the global network. In particular, more work is required to reduce false alarms caused by the detection of microbarom signals.

47 OTHER INSTRUMENTATION↗

The application of quaternions and other spatial representations to the reconstruction of re-entry vehicle motion.

The analysis of spacecraft kinematics and dynamics requires an efficient scheme for spatial representation. While the representation of displacement in three dimensional Euclidean space is straightforward, orientation in three dimensions poses particular challenges. The unit quaternion provides an approach that mitigates many of the problems intrinsic in other representation approaches, including the ill-conditioning that arises from computing many successive rotations. This report focuses on the computational utility of unit quaternions and their application to the reconstruction of re-entry vehicle (RV) motion history from sensor data. To this end they will be used in conjunction with other kinematic and data processing techniques. We will present a numerical implementation for the reconstruction of RV motion solely from gyroscope and accelerometer data. This will make use of unit quaternions due to their numerical efficacy in dealing with the composition of many incremental rotations over a time series. In addition to signal processing and data conditioning procedures, algorithms for numerical quaternion-based integration of gyroscope data will be addressed, as well as accelerometer triangulation and integration to yield RV trajectory. Actual processed flight data will be presented to demonstrate the implementation of these methods.

47 OTHER INSTRUMENTATION↗

Development of ML FPGA Filter for Particle Identification and Tracking in Real Time

Real-time data processing is a frontier field in experimental particle physics. Machine Learning methods are widely used and have proven to be very powerful in particle physics. The growing computational power of modern FPGA boards allows us to add more sophisticated algorithms for real time data processing. Many tasks could be solved using modern Machine Learning (ML) algorithms which are naturally suited for FPGA architectures. The FPGA-based machine learning algorithm provides an extremely low, sub-microsecond, latency decision and makes information-rich data sets for event selection. We report work has started to evaluate an FPGA based Machine Learning (ML) algorithm for a real-time particle identification and tracking with Transition Radiation Detector (TRD) and e/m calorimeter. The first target is the GlueX experiment, with a plan to build a TRD based on GEM technology. GlueX trigger latency is 3.3 μs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Heterogeneity-Aware Task Scheduler for Spark

Big data processing systems such as Spark are employed in an increasing number of diverse applications-such as machine learning, graph computation, and scientific computing-each with dynamic and different resource needs. These applications increasingly run on heterogeneous hardware, e.g., with out-of-core accelerators. However, big data platforms do not factor in the multi-dimensional heterogeneity of applications and hardware. This leads to a fundamental mismatch between the application and hardware characteristics, and the resource scheduling adopted in big data platforms. For example, Hadoop and Spark consider only data locality when assigning tasks to nodes, and typically disregard the hardware capabilities and suitability to specific application requirements. In this paper, we present RUPAM, a heterogeneity-aware task scheduling system for big data platforms, which considers both task-level resource characteristics and underlying hardware characteristics, as well as preserves data locality. RUPAM adopts a simple yet effective heuristic to decide the dominant scheduling factor (e.g., CPU, memory, or I/O), given a task in a particular stage. Our experiments show that RUPAM is able to improve the performance of representative applications by up to 62.3% compared to the standard Spark scheduler.

Computer Science↗

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗

RDataFrame: Easy Parallel ROOT Analysis at 100 Threads

The Physics programmes of LHC Run III and HL-LHC challenge the HEP community. The volume of data to be handled is unprecedented at every step of the data processing chain: analysis is no exception. Physicists must be provided with first-class analysis tools which are easy to use, exploit bleeding edge hardware technologies and allow to seamlessly express parallelism. This document discusses the declarative analysis engine of ROOT, RDataFrame, and gives details about how it allows to profitably exploit commodity hardware as well as high-end servers and manycore accelerators thanks to the synergy with the existing parallelised ROOT components. Real-life analyses of LHC experiments’ data expressed in terms of RDataFrame are presented, highlighting the programming model provided to express them in a concise and powerful way. The recent developments which make RDataFrame a lightweight data processing framework are described, such as callbacks and I/O capabilities. Finally, the flexibility of RDataFrame and its ability to read data formats other than ROOT’s are characterised, as an example it is discussed how RDataFrame can directly read and analyse LHCb’s raw data format MDF.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Darshan for HEP applications

Modern HEP workflows must manage increasingly large and complex data collections. HPC facilities may be employed to help meet these workflows’ growing data processing needs. However, a better understanding of the I/O patterns and underlying bottlenecks of these workflows is necessary to meet the performance expectations of HPC systems.Darshan is a lightweight I/O characterization tool that captures concise views of HPC application I/O behavior. It intercepts application I/O calls at runtime, records file access statistics for each process, and generates log files detailing application I/O access patterns.Typical HEP workflows include event generation, detector simulation, event reconstruction, and subsequent analysis stages. A study of the I/O behavior of the ATLAS simulation and filtering stage, and the CMS simulation workflow using Darshan is presented, including insights into the I/O operations and data access size.

Wang, Rui↗