Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Missing data problem”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Reprocessing Microflare Data

The report concerns work on detecting and cataloging solar microflares using an automated. An accompanying figure represents the solar microflare distribution during the period of April 1991 to November 1992, the height of solar activity after the launch of CGRO. It also shows the distribution extending below the distribution obtained at GSFC by manual means. We have implemented significant refinements in the search algorithm. The algorithm in its simplest form searches for transient events and based upon the distribution of the signal among the different BATSE detectors, we can assign it to be of solar origin if the signal distribution conforms to what one expects from a burst or transient from that direction. One of the major problems in an earlier effort was to search for microflares and large flares simultaneously. The requirement for a dynamic range of almost 10(exp 4) resulted in ambiguous identifications at the low side of the distribution. We have since restricted the search to events with peak count rates under 2000/s. Larger events are easily identified in the manual search, so we have chosen not to duplicate that work. The second problem was that missing counts existed below channel 0 in the BATSE Large Area Detector (LAD) data. These have been recovered and are now included in the search process. This provides data below 20 keV, and as we get closer to the thermal part of the spectrum, it provides greater sensitivity. The third problem was that too many BATSE detectors were used in the search. Detectors with pointing directions far from the Sun, although detecting the event, had poorly known responses. Detectors greater than approximately 60 degrees off the Sun are no longer included in the search process. By reducing the systematic errors with the large off-axis detectors we can conduct more rigorous statistical tests of a candidate event to ascertain whether it originated from the solar direction. We have reprocessed the period in the early mission that covers solar maximum and constructed the microflare distribution shown in the figure. The results of the automated search start to deviate from the manual search results below about 1000/s. Not only do we now have this distribution but we have a database of solar microflares that was used to construct the distribution. This database contains the signal at higher energy channels as well as that in channel zero (and below). From this one can, using software at GSFC, construct a photon spectrum for some of the larger microflares. It can also be used in other solar studies, especially those that correlate the X-ray flux with emission at other wavelengths. With some additional effort we hope to integrate this database into the corresponding one residing at the Solar Data Analysis Center at GSFC. The entire CGRO mission's data can now be reprocessed to obtain the microflare distribution at all phases of the solar cycle. This work is in progress. The results of this work will be presented in forthcoming scientific workshops and conferences.

Ryan, James M.↗

What Is in Umbilicaria pustulata? A Metagenomic Approach to Reconstruct the Holo-Genome of a Lichen

Lichens are valuable models in symbiosis research and promising sources of biosynthetic genes for biotechnological applications. Most lichenized fungi grow slowly, resist aposymbiotic cultivation, and are poor candidates for experimentation. Obtaining contiguous, high-quality genomes for such symbiotic communities is technically challenging. Here, we present the first assembly of a lichen holo-genome from metagenomic whole-genome shotgun data comprising both PacBio long reads and Illumina short reads. The nuclear genomes of the two primary components of the lichen symbiosis—the fungus Umbilicaria pustulata (33 Mb) and the green alga Trebouxia sp. (53 Mb)—were assembled at contiguities comparable to single-species assemblies. The analysis of the read coverage pattern revealed a relative abundance of fungal to algal nuclei of ~20:1. Gap-free, circular sequences for all organellar genomes were obtained. The bacterial community is dominated by Acidobacteriaceae and encompasses strains closely related to bacteria isolated from other lichens. Gene set analyses showed no evidence of horizontal gene transfer from algae or bacteria into the fungal genome. Our data suggest a lineage-specific loss of a putative gibberellin-20-oxidase in the fungus, a gene fusion in the fungal mitochondrion, and a relocation of an algal chloroplast gene to the algal nucleus. Major technical obstacles during reconstruction of the holo-genome were coverage differences among individual genomes surpassing three orders of magnitude. Moreover, we show that GC-rich inverted repeats paired with nonrandom sequencing error in PacBio data can result in missing gene predictions. This likely poses a general problem for genome assemblies based on long reads.

54 ENVIRONMENTAL SCIENCES↗

Learning Memento archive routing with Character-based Artificial Neural Networks

This white paper describes a series of tests that were performed to determine if a neural network could learn patterns from a service that maintains a cache of routing decisions for the discovery of version information for discrete web-at-large URLs. Labeled training data was derived from a log file that records the by-archive availability of Memento availability for a given URL. Training data sets were generated from this log file on a by-archive basis (thus making it a binary classification problem). Each training data set consisted of equal numbers of hit and miss URLs, selected at random. The URLs were converted to a normalized numeric representation where each integer in a URL training vector represents a character in that URL. The corresponding label indicates whether or not Mementos were available for that URL in the selected archive. The training data matrix and the label vector became input to a neural network. A number of neural network architectures and network hyperparameters were explored, however the log entries themselves were used as-is, without any feature engineering, beyond the aforementioned normalization.

97 MATHEMATICS AND COMPUTING↗

Bayesian Analysis of the Cosmic Microwave Background

There is a wealth of cosmological information encoded in the spatial power spectrum of temperature anisotropies of the cosmic microwave background! Experiments designed to map the microwave sky are returning a flood of data (time streams of instrument response as a beam is swept over the sky) at several different frequencies (from 30 to 900 GHz), all with different resolutions and noise properties. The resulting analysis challenge is to estimate, and quantify our uncertainty in, the spatial power spectrum of the cosmic microwave background given the complexities of "missing data", foreground emission, and complicated instrumental noise. Bayesian formulation of this problem allows consistent treatment of many complexities including complicated instrumental noise and foregrounds, and can be numerically implemented with Gibbs sampling. Gibbs sampling has now been validated as an efficient, statistically exact, and practically useful method for low-resolution (as demonstrated on WMAP 1 and 3 year temperature and polarization data). Continuing development for Planck - the goal is to exploit the unique capabilities of Gibbs sampling to directly propagate uncertainties in both foreground and instrument models to total uncertainty in cosmological parameters.

methods - statistical↗

Final Report: Quantification of Uncertainty in Extreme Scale Computations (QUEST)

QUEST (\url{www.quest-scidac.org}) is a SciDAC Institute that is focused on uncertainty quantification (UQ) in large-scale scientific computations. Our goals are to (1) advance the state of the art in UQ mathematics, algorithms, and software; and (2) provide modeling, algorithmic, and general UQ expertise, together with software tools, to other SciDAC projects, thereby enabling and guiding a broad range of UQ activities in their respective contexts. QUEST is a collaboration among six institutions (Sandia National Laboratories, Los Alamos National Laboratory, the University of Southern California, Massachusetts Institute of Technology, the University of Texas at Austin, and Duke University) with a history of joint UQ research. Our vision encompasses all aspects of UQ in leadership-class computing. This includes the well-founded setup of UQ problems; characterization of the input space given available data/information; local and global sensitivity analysis; adaptive dimensionality and order reduction; forward and inverse propagation of uncertainty; handling of application code failures, missing data, and hardware/software fault tolerance; and model inadequacy, comparison, validation, selection, and averaging. The nature of the UQ problem requires the seamless combination of data, models, and information across this landscape in a manner that provides a self-consistent quantification of requisite uncertainties in predictions from computational models. Accordingly, our UQ methods and tools span an interdisciplinary space across applied math, information theory, and statistics. The MIT QUEST effort centers on statistical inference and methods for surrogate or reduced-order modeling. MIT personnel have been responsible for the development of adaptive sampling methods, methods for approximating computationally intensive models, and software for both forward uncertainty propagation and statistical inverse problems. A key software product of the MIT QUEST effort is the MIT Uncertainty Quantification library, called MUQ (\url{muq.mit.edu}).

97 MATHEMATICS AND COMPUTING↗

DShaper: An approach for handling missing low-Q data in pair distribution function analysis of nanostructured systems

In this work we discuss the potential problems and currently available solutions in modeling powder-diffraction based pair-distribution function (PDF) data from systems where morphological feature information content includes distances in the nanometer length scale, such as finite nanoparticles, nanoporous networks, and nanoscale precipitates in bulk materials. The implications of an experimental finite minimum Q-value are addressed by simulation, which also demonstrates the advantages of combining PDF data with small angle scattering data (SAS). In addition, we introduce a simple Fortran90 code, DShaper , which may be incorporated into PDF data fitting routines in order to approximate the so-called shape-function for any atomistic model.

36 MATERIALS SCIENCE↗

SHELXT – Integrated space-group and crystal-structure determination

SHELXT automates routine small-molecule structure determination starting from single-crystal reflection data, the Laue group and a reasonable guess as to which elements might be present. The new computer program SHELXT employs a novel dual-space algorithm to solve the phase problem for single-crystal reflection data expanded to the space group P1. Missing data are taken into account and the resolution extended if necessary. All space groups in the specified Laue group are tested to find which are consistent with the P1 phases. After applying the resulting origin shifts and space-group symmetry, the solutions are subject to further dual-space recycling followed by a peak search and summation of the electron density around each peak. Elements are assigned to give the best fit to the integrated peak densities and if necessary additional elements are considered. An isotropic refinement is followed for non-centrosymmetric space groups by the calculation of a Flack parameter and, if appropriate, inversion of the structure. The structure is assembled to maximize its connectivity and centred optimally in the unit cell. SHELXT has already solved many thousand structures with a high success rate, and is optimized for multiprocessor computers. It is, however, unsuitable for severely disordered and twinned structures because it is based on the assumption that the structure consists of atoms.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Atmospheric Disturbance Environment Definition

Traditionally, the application of atmospheric disturbance data to airplane design problems has been the domain of the structures engineer. The primary concern in this case is the design of structural components sufficient to handle transient loads induced by the most severe atmospheric "gusts" that might be encountered. The concern has resulted in a considerable body of high altitude gust acceleration data obtained with VGH recorders (airplane velocity, V, vertical acceleration, G, altitude, H) on high-flying airplanes like the U-2 (Ehernberger and Love, 1975). However, the propulsion system designer is less concerned with the accelerations of the airplane than he is with the airflow entering the system's inlet. When the airplane encounters atmospheric turbulence it responds with transient fluctuations in pitch, yaw, and roll angles. These transients, together with fluctuations in the free-stream temperature and pressure will disrupt the total pressure, temperature, Mach number and angularity of the inlet flow. For the mixed compression inlet, the result is a disturbed throat Mach number and/or shock position, and in extreme cases an inlet unstart can occur (cf. Section 2.1). Interest in the effects of inlet unstart on the vehicle dynamics of large, supersonic airplanes is not new. Results published by NASA in 1962 of wind tunnel studies of the problem were used in support of the United States Supersonic Transport program (SST) (White, at aI, 1963). Such studies continued into the late 1970's. However, in spite of such interest, there never was developed an atmospheric disturbance database for inlet unstart analysis to compare with that available for the structures load analysis. Missing were data for the free-stream temperature and pressure disturbances that also contribute to the unStart problem.

Tank, William G.↗

Statistical Performance of Forced Oscillation Detectors in the Presence of Missing Measurements

In bulk power systems, measurement-based monitoring for large oscillations can help maintain system reliability. One of the challenges encountered in a recent field demonstration was the unavailability of measurements due to underlying measurement quality or communication problems. During the demonstration, the oscillation detector ignored a measurement location if even 10 seconds of data was missing. To extend the detector's ability to operate in these conditions, this paper evaluates the impact of three methods for addressing missing data. The strengths and weaknesses of each approach are evaluated using theoretical expressions for the probability of detection along with results from simulated data and publicly available field measurements. Based on these results, a suitable approach is identified that can extend the oscillation detector's performance when large segments of data are missing.

Follum, James D.↗

Explaining Missing Data in Graphs: A Constraint-based Approach

Abstract: This paper introduces a constraint-based approach to clarify missing values in graphs. Our method capitalizes on a set S of graph data constraints. An explanation is a sequence of operational enforcement of S towards the recovery of interested yet missing data (e.g., attribute values, edges). We show that constraint-based approach helps us to understand not only why a value is missing, but also how to recover the missing value. We study S-explanation problem, which is to compute the optimal explanations with guarantees on the informativeness and conciseness. We show the problem is in ?P^2 for established graph data constraints such as graph keys and graph association rules. We develop an efficient bidirectional algorithm to compute optimal explanations, without enforcing S on the entire graph. We also show our algorithm can be easily extended to support graph refinement within limited time, and to explain missing answers. Using real-world graphs, we experimentally verify the effectiveness and efficiency of our algorithms.

Data Analytics↗

Inverse problem in the large momentum effective theory framework

One proposal to compute parton distributions from first principles is the large momentum effective theory (LaMET), which requires the Fourier transform of matrix elements computed nonperturbatively. Lattice quantum chromodynamics (QCD) provides calculations of these matrix elements over a finite range of Fourier harmonics that are often noisy or unreliable in the largest computed harmonics. It has been suggested that enforcing an exponential decay of the missing harmonics helps alleviate this issue. Using nonperturbative data, we show that the uncertainty introduced by this inverse problem in a realistic setup remains significant without very restrictive assumptions, and that the importance of the exact asymptotic behavior is minimal for values of 𝑥 where the framework is currently applicable. We show that the crux of the inverse problem lies in harmonics of the order of 𝜆 = 𝑧⁢𝑃 𝑧 ∼ 5–15, where the signal in the lattice data is often barely existent in current studies, and the asymptotic behavior is not firmly established. We stress the need for more sophisticated techniques to account for this inverse problem, whether in the LaMET or related frameworks like the short-distance factorization. We also address a misconception that, with available lattice methods, the LaMET framework allows a “direct” computation of the 𝑥-dependence, whereas the alternative short-distance factorization only gives access to moments or fits of the 𝑥-dependence.

Dutrieux, Hervé [Aix-Marseille Université, Marseil↗

Solar Microflare with BATSE

Our work on detecting and cataloging solar microflares using an automated method is illustrated in the accompanying figure. The figure represents the solar microflare distribution during the period of April 1991 to November 1992, the height of solar activity after the launch of The Compton Gamma Ray Observatory (CGRO). It also shows the distribution extending below the distribution obtained at Goddard Space Flight Center (GSFC) by manual means. We have implemented significant refinements in the search algorithm. The algorithm in its simplest form searches for transient events and based upon the distribution of the signal among the different Burst and Transient Source Experiment (BATSE) detectors, we can assign it to be of solar origin if the signal distribution conforms to what one expects from a burst or transient from that direction. One of the major problems in the earlier effort was to search for microflares and large flares simultaneously. The requirement for a dynamic range of almost 10 (exp 4) resulted in ambiguous identifications at the low side of the distribution. We have since restricted the search to events with peak count rates under 2000 s (exp -1). Larger events are easily identified in the manual search, so we have chosen not to duplicate that work. The second problem was that missing counts existed below channel 0 in the Burst and Transient Source Experiment Large Area Detector data (BATSE LAD). These have been recovered and are now included in the search process. This provides data below 20 keV, and as we get closer to the thermal part of the spectrum, it provides greater sensitivity. The third problem was that too many BATSE detector were used in the search. Detectors with pointing directions far from the Sun, although detecting the event, had poorly known responses. Detectors greater than approximately 60 deg. off the Sun are no longer included in the search process. By reducing the systematic errors with the large off-axis detectors we can conduct more rigorous statistical tests of a candidate event to ascertain whether it originated from the solar direction. We have reprocessed the period in the early mission that covers solar maximum and constructed the microflare distribution shown in the figure. The results of the automated search start to deviate from the manual search results below about 1000 s (exp -1). Not only do we now have this distribution but we have a database of solar microflares that was used to construct the distribution. This database contains the signal at higher energy channels as well as that in channel zero (and below). From this one can, using software at GSFC, construct a photon spectrum for some of the larger microflares. It can also be used in other solar studies, especially those that correlate the X-ray flux with emission at other wavelengths. With some additional effort we hope to integrate this database into the corresponding one residing at the Solar Data Analysis Center at GSFC. The entire CGRO mission's data can now be reprocessed to obtain the microflare distribution at all phases of the solar cycle. This work is in progress. The results of this work will be presented in forthcoming scientific workshops and conferences.

Ryan, James M.↗

Simplified Models for a First Characterization of New Physics at the LHC

Low-energy SUSY and several other theories that address the hierarchy problem predict pair-production at the LHC of particles with Standard Model quantum numbers that decay to jets, missing energy, and possibly leptons. If an excess of such events is seen in LHC data, a theoretical framework in which to describe it will be essential to constraining the structure of the new physics. We propose a basis of four deliberately simplified models, each specified by only 2-3 masses and 4-5 branching ratios, for use in a first characterization of data. Fits of these simplified models to the data furnish a quantitative presentation of the jet structure, electroweak decays, and heavy-flavor content of the data, independent of detector effects. These fits, together with plots comparing their predictions to distributions in data, can be used as targets for describing the data within any full theoretical model.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

SU-E-T-469: Implementation of VAs Web-Based Radiotherapy Incident Reporting and Analysis System (RIRAS)

Purpose: This Web-based Radiotherapy Incident Reporting and Analysis System (RIRAS) is a tool to improve quality of care for radiation therapy patients. This system is an important facet of continuing effort by our community to maintain and improve safety of radiotherapy.Material and Methods: VA’s National Radiation Oncology Program office has embarked on a program to electronically collect adverse events and good-catch data of radiation treatment of over 25,000 veterans treated with radiotherapy annually. This VA-Intranet based software design has made use of dataset taxonomies and data dictionaries defined in AAPM/ASTRO reports on error reporting. We used proven industrial and medical event reporting techniques to avoid several common problems faced in effective data collection such as incomplete data due to data entry fatigue by the reporters, missing data due to data difficult to obtain or not familiar to most reporters, missing reports due to fear of reprisal etc. This system encompasses the entire feedback loop of reporting an incident, analyzing it for salient details, and developing interventions to prevent it from happening again. The analysis reports with corrective, learning actions are shared with the reporter/facility and made public to the community (after deidentification) as part of the learning process. Results: Till date 50 incident/good catches have been reported in RIRAS and we have completed analysis on 100% of these reports. This is done due to the fact that each reported incidents is investigated and a complete analysis/patient-safety-work-product report is generated by radiation oncology domain-experts. Conclusions Because of the completeness of the data, the system has enabled us to analyze process steps and track trends of major errors which in the future will lead to implementing system wide process improvement steps and safe standard operating procedures for each radiotherapy treatment modality/technique and fulfills our goal of “Effecting Quality While Treating Safely”. RIRAS developed and copyrighted by TSG Innovations Inc.

60 APPLIED LIFE SCIENCES↗

Iterative Strain-Gage Balance Calibration Data Analysis for Extended Independent Variable Sets

A new method was developed that makes it possible to use an extended set of independent calibration variables for an iterative analysis of wind tunnel strain gage balance calibration data. The new method permits the application of the iterative analysis method whenever the total number of balance loads and other independent calibration variables is greater than the total number of measured strain gage outputs. Iteration equations used by the iterative analysis method have the limitation that the number of independent and dependent variables must match. The new method circumvents this limitation. It simply adds a missing dependent variable to the original data set by using an additional independent variable also as an additional dependent variable. Then, the desired solution of the regression analysis problem can be obtained that fits each gage output as a function of both the original and additional independent calibration variables. The final regression coefficients can be converted to data reduction matrix coefficients because the missing dependent variables were added to the data set without changing the regression analysis result for each gage output. Therefore, the new method still supports the application of the two load iteration equation choices that the iterative method traditionally uses for the prediction of balance loads during a wind tunnel test. An example is discussed in the paper that illustrates the application of the new method to a realistic simulation of temperature dependent calibration data set of a six component balance.

Ulbrich, Norbert Manfred↗

Underground hydrogen storage leakage detection and characterization based on machine learning of sparse seismic data

Underground hydrogen storage (UHS) is considered as a scalable approach for massive storage and seasonal extraction of hydrogen (H 2 ). Although conventional leakage detection and characterization methods based on time-lapse seismic imaging and inversion generally apply to H 2 leakage detection problem, a high-fidelity yet cost effective geophysics approach is still missing to reliably inform leakage location and properties based on very sparse data. In response, we develop a novel supervised machine learning method to detect and characterize H 2 leakage from UHS. The input to our neural network are sparse time-lapse seismic waveforms, while the output from the neural network includes the spatial location and physical properties of a H 2 leakage. Here, we generate high-quality time-lapse waveforms using the elastic-wave equations to train the neural network. We train and validate our machine learning model and find that it attains high accuracy in using extremely sparse time-lapse seismic data to detect and characterize H 2 leakage. Our investigation is the first systematic study that focuses on applying machine learning to subsurface H 2 leakage detection and characterization and could potentially serve as a cost-effective geophysical tool for underground hydrogen leakage detection and characterization with high fidelity.

08 HYDROGEN↗

Flow field Reconstruction for Inhomogeneous Turbulence using Data and Physics Driven Models

A methodology combining Large Eddy Simulation (LES) trained data and a physics driven wave packet model to obtain a reduced order reconstruction for broadband, three-dimensional, temporally stationary but spatially inhomogeneous, incompressible turbulence. Wake turbulence generated by an axisymmetric dragging disk with a turbulent co-flow serves as the benchmark test case. We begin by studying the proper-orthogonal decomposition of the turbulent fluctuations taken from a high-resolution LES to first identify whether the fields demonstrate a low-rank character. It is argued that the presence of the turbulent co-flow results in a largely broadband character lacking any tonal properties. This is especially true for Strouhal numbers greater than 1 and only a small fraction of energy is contained in the leading order Kelvin-Helmholtz modes. As such reconstructions and reduced order modeling purely relying on data from LES does not appear to be a lucrative solution - contrary to problems with strongly tonal character. To supplement the missing energy from a low order truncated mode expansion, we utilize a physics based super-resolution (enrichment) algorithm that relies on spatio-temporally localized Gabor wave packets whose time evolution is described using a set of ordinary differential equations. The reconstructed flow has single- and two-point correlations that are consistent with the reference high resolution simulation data.

SLS↗

Neutrino masses, neutrino oscillations, and cosmological implications

Theoretical concepts and motivations for considering neutrinos having finite masses are discussed and the experimental situation on searches for neutrino masses and oscillations is summarized. The solar neutrino problem, reactor, deep mine and accelerator data, tri decay experiments and double beta-decay data are considered and cosmological implications and astrophysical data relating to neutrino masses are reviewed. The neutrino oscillation solution to the solar neutrino problem, the missing mass problem in galaxy halos and galaxy cluster galaxy formation and clustering, and radiative neutrino decay and the cosmic ultraviolet background radiation are examined.

Stecker, F. W.↗