Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The Lick Observatory Supernova Search follow-up program: photometry data release of 70 SESNe

We present BVRI and unfiltered (Clear) light curves of 70 stripped-envelope supernovae (SESNe), observed between 2003 and 2020, from the Lick Observatory Supernova Search follow-up program. Our SESN sample consists of 19 spectroscopically normal SNe Ib, 2 peculiar SNe Ib, six SNe Ibn, 14 normal SNe Ic, 1 peculiar SN Ic, 10 SNe Ic-BL, 15 SNe IIb, 1 ambiguous SN IIb/Ib/c, and 2 superluminous SNe. Our follow-up photometry has (on a per-SN basis) a mean coverage of 81 photometric points (median of 58 points) and a mean cadence of 3.6 d (median of 1.2 d). From our full sample, a subset of 38 SNe have pre-maximum coverage in at least one passband, allowing for the peak brightness of each SN in this subset to be quantitatively determined. We describe our data collection and processing techniques, with emphasis toward our automated photometry pipeline, from which we derive publicly available data products to enable and encourage further study by the community. Using these data products, we derive host-galaxy extinction values through the empirical colour evolution relationship and, for the first time, produce accurate rise-time measurements for a large sample of SESNe in both optical and infrared passbands. By modelling multiband light curves, we find that SNe Ic tend to have lower ejecta masses and lower ejecta velocities than SNe Ib and IIb, but higher 56Ni masses.

Lick Observatory↗
DOE OSTI2016

redMaGiC: Selecting luminous red galaxies from the DES Science Verification data

Here, we introduce redMaGiC, an automated algorithm for selecting luminous red galaxies (LRGs). The algorithm was specifically developed to minimize photometric redshift uncertainties in photometric large-scale structure studies. redMaGiC achieves this by self-training the colour cuts necessary to produce a luminosity-thresholded LRG sample of constant comoving density. We demonstrate that redMaGiC photo-zs are very nearly as accurate as the best machine learning-based methods, yet they require minimal spectroscopic training, do not suffer from extrapolation biases, and are very nearly Gaussian. We apply our algorithm to Dark Energy Survey (DES) Science Verification (SV) data to produce a redMaGiC catalogue sampling the redshift range z ϵ [0.2, 0.8]. Our fiducial sample has a comoving space density of 10 –3 (h –1 Mpc) –3 , and a median photo-z bias (zspec – zphoto) and scatter (σz/(1 + z)) of 0.005 and 0.017, respectively. The corresponding 5σ outlier fraction is 1.4 per cent. We also test our algorithm with Sloan Digital Sky Survey Data Release 8 and Stripe 82 data, and discuss how spectroscopic training can be used to control photo-z biases at the 0.1 per cent level.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

Joint Measurement of Lensing-Galaxy Correlations Using SPT and DES SV Data

We measure the correlation of galaxy lensing and cosmic microwave background lensing with a set of galaxies expected to trace the matter density field. The measurements are performed using pre-survey Dark Energy Survey (DES) Science Verification optical imaging data and millimeter-wave data from the 2500 square degree South Pole Telescope Sunyaev-Zel’dovich (SPT-SZ) survey. The two lensing-galaxy correlations are jointly fit to extract constraints on cosmological parameters, constraints on the redshift distribution of the lens galaxies, and constraints on the absolute shear calibration of DES galaxy lensing measurements. We show that an attractive feature of these fits is that they are fairly insensitive to the clustering bias of the galaxies used as matter tracers. The measurement presented in this work confirms that DES and SPT data are consistent with each other and with the currently favored ΛCDM cosmological model. It also demonstrates that joint lensing-galaxy correlation measurement considered here contains a wealth of information that can be extracted using current and future surveys.

cosmic background radiation↗
DOE OSTI2024

Development of a Data Overflow Protection System for Super-Kamiokande to Maximize Data from Nearby Supernovae

Neutrinos from very nearby supernovae, such as Betelgeuse, are expected to generate more than ten million events over 10 s in Super-Kamokande (SK). At such large event rates, the buffers of the SK analog-to-digital conversion board (QBEE) will overflow, causing random loss of data that are critical for understanding the dynamics of the supernova explosion mechanism. In order to solve this problem, two new data-acquisition (DAQ) modules were developed to aid in the observation of very nearby supernovae. The first of these, the SN module, is designed to save only the number of hit photomultiplier tubes during a supernova burst and the second, the Veto module, prescales the high-rate neutrino events to prevent the QBEE from overflowing based on information from the SN module. In the event of a very nearby supernova, these modules allow SK to reconstruct the time evolution of the neutrino event rate from beginning to end using both QBEE and SN module data. This paper presents the development and testing of these modules together with an analysis of supernova-like data generated with a flashing laser diode. We demonstrate that the Veto module successfully prevents DAQ overflows for Betelgeuse-like supernovae as well as the long-term stability of the new modules. During normal running the Veto module is found to issue DAQ vetos a few times per month resulting in a total dead-time less than 1 ms, and does not influence ordinary operations. Additionally, using simulation data we find that supernovae closer than 800 pc will trigger the Veto module, resulting in a prescaling of the observed neutrino data.

F20 Instrumentation and technique↗
DOE OSTI2023

Joint analysis of Dark Energy Survey Year 3 data and CMB lensing from SPT and P l a n c k . II. Cross-correlation measurements and cosmological constraints

Cross-correlations of galaxy positions and galaxy shears with maps of gravitational lensing of the cosmic microwave background (CMB) are sensitive to the distribution of large-scale structure in the Universe. Such cross-correlations are also expected to be immune to some of the systematic effects that complicate correlation measurements internal to galaxy surveys. We present measurements and modeling of the cross-correlations between galaxy positions and galaxy lensing measured in the first three years of data from the Dark Energy Survey with CMB lensing maps derived from a combination of data from the 2500 deg 2 SPT-SZ survey conducted with the South Pole Telescope and full-sky data from the Planck satellite. The CMB lensing maps used in this analysis have been constructed in a way that minimizes biases from the thermal Sunyaev Zel’dovich effect, making them well suited for cross-correlation studies. The total signal-to-noise of the cross-correlation measurements is 23.9 (25.7) when using a choice of angular scales optimized for a linear (nonlinear) galaxy bias model. Here we use the cross-correlation measurements to obtain constraints on cosmological parameters. For our fiducial galaxy sample, which consist of four bins of magnitude-selected galaxies, we find constraints of Ω m =0.272$^{+0.032}_{-0.052}$ and S 8 ≡σ8$\sqrt{Ω_m/0.3}$=0.736$^{+0.032}_{-0.028}$ (Ω m =0.245$^{+0.026}_{-0.044}$ and S 8 =0.734$^{+0.035}_{-0.028}$) when assuming linear (nonlinear) galaxy bias in our modeling. Considering only the cross-correlation of galaxy shear with CMB lensing, we find Ω m =0.270$^{+0.043}_{-0.061}$ and S 8 =0.740$^{+0.034}_{-0.029}$. Our constraints on S8 are consistent with recent cosmic shear measurements, but lower than the values preferred by primary CMB measurements from Planck.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Symmetry-mode analysis for local structure investigations using pair distribution function data

Symmetry-adapted distortion modes provide a natural way of describing distorted structures derived from higher-symmetry parent phases. Structural refinements using symmetry-mode amplitudes as fit variables have been used for at least ten years in Rietveld refinements of the average crystal structure from diffraction data; more recently, this approach has also been used for investigations of the local structure using real-space pair distribution function (PDF) data. Here, the value of performing symmetry-mode fits to PDF data is further demonstrated through the successful application of this method to two topical materials: TiSe2, where a subtle but long-range structural distortion driven by the formation of a charge-density wave is detected, and MnTe, where a large but highly localized structural distortion is characterized in terms of symmetry-lowering displacements of the Te atoms. Here, the analysis is performed using fully open-source code within the DiffPy framework via two packages developed for this work: isopydistort, which provides a scriptable interface to the ISODISTORT web application for group theoretical calculations, and isopytools, which converts the ISODISTORT output into a DiffPy-compatible format for subsequent fitting and analysis. These developments expand the potential impact of symmetry-adapted PDF analysis by enabling high-throughput analysis and removing the need for any commercial software.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Unwarping GISAXS data

Explore the source record for details and available documents.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2018

Social media based NPL system to find and retrieve ARM data: Concept paper

Information connectivity and retrieval has a role in our daily lives. The most pervasive source of online information is databases. The amount of data is growing at rapid rate and database technology is improving and having a profound effect. Almost all online applications are storing and retrieving information from databases. One challenge in supplying the public with wider access to informational databases is the need for knowledge of database languages like Structured Query Language (SQL). Although the SQL language has been published in many forms, not everybody is able to write SQL queries. Another challenge is that it may not be practical to make the public aware of the structure of the database. There is a need for novice users to query relational databases using their natural language. To solve this problem, many natural language interfaces to structured databases have been developed. The goal is to provide more intuitive method for generating database queries and delivering responses. Social media makes it possible to interact with a wide section of the population. Through this medium, and with the help of Natural Language Processing (NLP) we can make the data of the Atmospheric Radiation Measurement Data Center (ADC) more accessible to the public. We propose an architecture for using Apache Lucene/Solr [1], OpenML [2,3], and Kafka [4] to generate an automated query/response system with inputs from Twitter5, our Cassandra DB, and our log database. Using the Twitter API and NLP we can give the public the ability to ask questions of our database and get automated responses.

Devarakonda, Ranjeet↗
DOE OSTI2017

Management, Analysis, and Visualization of Experimental and Observational Data – The Convergence of Data and Computing

Scientific user facilities—particle accelerators, telescopes, colliders, supercomputers, light sources, sequencing facilities, and more—operated by the U.S. Department of Energy (DOE) Office of Science (SC) generate ever increasing volumes of data at unprecedented rates from experiments, observations, and simulations. At the same time there is a growing community of experimentalists that require real-time data analysis feedback, to enable them to steer their complex experimental instruments to optimized scientific outcomes and new discoveries. Recent efforts in DOE-SC have focused on articulating the data-centric challenges and opportunities facing these science communities. Key challenges include difficulties coping with data size, rate, and complexity in the context of both real-time and post-experiment data analysis and interpretation. Solutions will require algorithmic and mathematical advances, as well as hardware and software infrastructures that adequately support data-intensive scientific workloads. This paper presents the summary findings of a workshop held by DOE-SC in September 2015, convened to identify the major challenges and the research that is needed to meet those challenges.

Bethel, E. Wes↗
DOE OSTI2020

Data-driven predictive probability density function control of fiber length stochastic distribution shaping in refining process

Pulp is the most important raw material in paper in paper industries, whose Fiber Length Stochastic Distribution (FLSD) shaping directly determines the energy consumption and paper quality of the subsequent papermaking processes. However, the mean and variance are insufficient to describe the output FLSD shaping, which display non-Gaussian distribution properties. Therefore, the traditional control, method based on the mean and variance of fiber length is difficult to control the FLSD shaping effectively. In this paper, a novel data-driven predictive probability density function (PDF) control method is proposed for output FLSD shaping in refining process. Primarily, in order to improve the approximation accuracy of the PDF employing RBF neural network, the parameters (center value and width) of each RBF basis functions are tuned via utilizing iterative learning control (ILC) law, and the corresponding estimations of weights law can be obtained. Secondly, considering that the conventional linear model of weights vector has drawbacks of low accuracy and weak generalization ability, random vector functional link networks (RVFLNs) based data driven nonlinear modelling method is employed to characterize the prediction model between the input variables and weights vector. Finally, in order to reduce the randomness of the output FLSD, a minimum entropy control method under mean constraint is employed to design the predictive PDF controller for the output FLSD shaping with the help of the established nonlinear stochastic distribution model. Both simulations and experiments are given to illustrate the effectiveness and practicability of the proposed method.

Probability debsity function, Fiber length distrib↗
DOE OSTI2022

A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

A key challenge in scientific simulation is that the simulation outputs often require intensive I/O and storage space to store the results for effective post hoc analysis. This article focuses on a quality-aware adaptive temporal data selection and reconstruction problem where the goal is to adaptively select simulation data samples at certain key timesteps in situ and reconstruct the discarded samples with quality assurance during post hoc analysis. This problem is motivated by the limitation of current solutions that a significant amount of simulation data samples are either discarded or aggregated during the sampling process, leading to inaccurate modeling of the simulated phenomena. Two unique challenges exist: 1) the sampling decisions have to be made in situ and adapted to the dynamics of the complex scientific simulation data; 2) the reconstruction error must be strictly bounded to meet the application requirement. To address the above challenges, we develop DeepSample , an error-controlled convolutional neural network framework, that jointly integrates a set of coherent multi-branch deep decoders to effectively reconstruct the simulation data with rigorous quality assurance. The results on two real-world scientific simulation applications show that DeepSample significantly outperforms other state-of-the-art methods on both sampling efficiency and reconstructed simulation data quality.

Zhang, Yang↗
DOE OSTI2018

Synchrophasor Data Correction under GPS Spoofing Attack: A State Estimation-Based Approach

GPS spoofing attack (GSA) has been shown to be one of the most imminent threats to almost all cyber-physical systems incorporated with the civilian GPS signal. Specifically, for our current agenda of the modernization of the power grid, this may greatly jeopardize the benefits provided by the pervasively installed phasor measurement units (PMU). In this paper, we consider the case where synchrophasor data from PMUs are compromised due to the presence of a single GSA, and show that it can be corrected by signal processing techniques. In particular, we introduce a statistical model for synchrophasorbased power system state estimation (SE), and then derive the spoofing-matched algorithms for synchrophasor data correction against GPS spoofing attack. Different testing scenarios in IEEE 14-, 30-, 57-, 118-bus systems are simulated to show the proposed algorithms’ performance on GSA detection and state estimation. Numerical results demonstrate that our proposed algorithms can consistently locate and correct the spoofed synchrophasor data with good accuracy as long as the system observability is satisfied. The accuracy of state estimation is significantly improved compared with the traditional weighted least square method and approaches the performance under the Genie-aided method.

Phasor measurement units, Global Positioning Syste↗
DOE OSTI2019

Machine learning in a data-limited regime: Augmenting experiments with synthetic data uncovers order in crumpled sheets

Machine learning has gained widespread attention as a powerful tool to identify structure in complex, high-dimensional data. However, these techniques are ostensibly inapplicable for experimental systems where data are scarce or expensive to obtain. Here, we introduce a strategy to resolve this impasse by augmenting the experimental dataset with synthetically generated data of a much simpler sister system. Specifically, we study spontaneously emerging local order in crease networks of crumpled thin sheets, a paradigmatic example of spatial complexity, and show that machine learning techniques can be effective even in a data-limited regime. This is achieved by augmenting the scarce experimental dataset with inexhaustible amounts of simulated data of rigid flat-folded sheets, which are simple to simulate and share common statistical properties. This considerably improves the predictive power in a test problem of pattern completion and demonstrates the usefulness of machine learning in bench-top experiments where data are good but scarce.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Measurement of the photon identification efficiencies with the ATLAS detector using LHC Run-1 data

The algorithms used by the ATLAS Collaboration to reconstruct and identify prompt photons are described. Measurements of the photon identification efficiencies are reported, using 4.9 fb –1 of pp collision data collected at the LHC at √s = 7 TeV and 20.3 fb –1 at √s = 8 TeV. The efficiencies are measured separately for converted and unconverted photons, in four different pseudorapidity regions, for transverse momenta between 10 GeV and 1.5 TeV. The results from the combination of three data-driven techniques are compared to the predictions from a simulation of the detector response, after correcting the electromagnetic shower momenta in the simulation for the average differences observed with respect to data. Data-to-simulation efficiency ratios used as correction factors in physics measurements are determined to account for the small residual efficiency differences. These factors are measured with uncertainties between 0.5% and 10% in 7 TeV data and between 0.5% and 5.6% in 8 TeV data, depending on the photon transverse momentum and pseudorapidity.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

A Data Processing Pipeline To Extract A Knowledge Graph From Heterogeneous Data For Socio-technical Analysis Of Critical Infrastructure Influence

The code is written in Python and consists of the following pipeline that is implemented in Apache Airflow. This pipeline intends to understand the companies that are directly or indirectly involved with a type of critical infrastructure system at some point in that system's lifecycle. The pipeline takes a configuration file that specifies a list of initial companies to consider, a geographic region of interest, and a set of SEC form types as well as other data sources (e.g. CrunchBase) from which to extract entities and relations. There are four main components to this pipeline as currently implemented: Entity Extraction, Network Construction, Analysis, and Visualization. First, Entity Extraction, is implemented as the `topear-extract_organizations` Apache Airflow workflow. Given an initial query that specifies a geographic region of interest and a time interval, the software will extract CI facilities of interest and organizations that have a direct influence relationship to those facilities (e.g. ownership). During the course of the LDRD, we focused on Electric Vehicle charging stations and this information is available via the Department of Energy (DOE) database on fueling stations maintained by NREL. Within the context of the DOE CESER project, we have focused on Battery Energy Storage Systems (BESS). Second, the Network Extraction component will iteratively construct a social network graph given the set of organizations and people extracted in the previous step. Organizations (and eventually People if desired) are then fed as a query to the `topgear-construct_social_network` Apache Airflow workflow which given a set of initial companies and data sets (e.g. SEC EDGAR form types, OpenCorporates, Crunchbase). This Airflow workflow will iteratively query such data sources to discover relationships with new organizations and people. For example, this module can iteratively query SEC EDGAR for metadata that documents the number of each type of form for the given set of companies and their location. This forms metadata represents a catalog of data sources from SEC EDGAR for the extracted social network knowledge graph. The pipeline then downloads these forms from the website and saves them in a build directory for further processing. These documents are then parsed for entities and relations. Again, we note that in additional to SEC data sources, this step can also pull in information on organizations via API services such as CrunchBase and OpenCorporates or bulk data sources. At the end of this step, the resultant social network, the Critical Infrastructure network, and the edges that encode relationships between organizations and CI facilities, form the Adversarial Socio-Technical Network (ASTN) that informs the analysis. Third, the Analysis component processes these generated ASTN. Previously, that has included the ability to compare prevalence of different vendors for a given infrastructure component type across different regions as well as identify common public and private investors across those vendors. This was demonstrated for EV Charging Stations across several different metropolitan areas within an IEEE PES GridEdge publication. More recently, we have looked at ways to identify infrastructure owners and operators of BESS with the most nameplate capacity across different states as well as other indictors of risk resulting from changes in ownership over time. Finally, the Visualization component consists of an HTML/CSS/JS framework by which users can interact geospatial, operational, and organizational relationships across a given portfolio of Critical Infrastructure facilities. The objective is to provide a library of UI/UX modules that can be repurposed for stakeholder-specific dashboards. All of the modules are related via a common event model that enables UI actions in one view to percolate across the other views.

Weaver, Gabriel [Idaho National Laboratory (INL), ↗
DOE OSTI2026

Data for Spatial Analysis of Cell Patterning to Aid Genetic and Phenotypic Understanding of Grass Stomatal Density: A Case Study in Maize

Biological processes involve complex hierarchies where composite traits result from multiple component traits. However, holistically understanding of how sets of component traits interact to underpin genotype-to-phenotype relationships is generally lacking. Stomatal density (SD) is a tractable model system for exploring how high-throughput phenotyping (HTP) data could be exploited by a new spatial analysis approach to better understand a developmentally and functionally important trait. SD is a composite trait, resulting from various components related to cell identity and size, which are themselves governed by a series of spatio-developmental processes. Data from 192 recombinant inbred lines of maize [Zea mays (L.)] were analyzed by a new stomatal patterning phenotype (SPP) to (1) describe the average spatial probability distribution of the nearest neighboring stomata; (2) derive a core set of component traits related to cell size, cell packing, and positional probabilities; (3) build a structural equation model of component traits underlying SD; and (4) identify stomatal patterning quantitative trait loci (QTL). The core set of SPP-derived traits explained 74% of the variation in SD. Analyzing SPP component traits allowed some loci previously identified as generic SD QTL to be recognized as specific to lateral versus longitudinal elements of stomatal patterning. Therefore, this study highlights how novel insights can be gained by decomposing a composite trait (e.g., SD) into a set of component traits that were present in HTP data but not previously exploited.

AI/ML↗
DOE OSTI2021

A Data-Driven Framework for Automated Detection of Aircraft-Generated Signals in Seismic Array Data Using Machine Learning

Abstract Ground motions associated with aircraft overflights can cover a significant portion of the seismic data collected by shallowly emplaced seismometers, such as new nodal and Distributed Acoustic Sensing systems. This article describes the first published framework for automated detection of aircraft on single channel and multichannel seismic data. The seismic data are converted to spectrograms in a sliding time window and classified as aircraft or nonaircraft in each window using a deep convolutional neural network trained with analyst-labeled data. A majority voting scheme is used to convert the output from the sequence of sliding time windows onto a decision time sequence for each channel and to combine the binary classifications on the decision time sequences across multiple channels. Precision, recall, and F-score are used to quantify the detection performance of the algorithm on nodal data using fourfold time-series cross validation. By applying our framework to data from the Sage Brush Flats nodal array in Southern California, we provide a benchmark performance and demonstrate the advantage of using an array of sensors.

Geochemistry & Geophysics↗