Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Using Temporal Information from Human Mobility Data to Detect Anchor Points

Spatiotemporal mobility data are available in massive quantities, but large quantities of data typically include fewer variables or data fields. Often, the only available fields are User ID, Longitude, Latitude, Timestamp (ULLT). This raises an important question: how much can we infer about human mobility patterns using only these four fields? With ULLT data, we do not know individuals' socioeconomic status information or when they are visiting their anchor points (AP) or locations (such as homes, places of employment, or schools), and it is a modern challenge to use this data to infer these characteristics. When detecting anchor locations with limited input information, verification and validation (VV) are significant challenges. This paper addresses the problem of identifying individuals' anchor locations using only temporal information from spatiotemporal datasets with limited attributes. Our approach does not explicitly use latitude and longitude during analysis. Locationbased information is only employed in the preprocessing stage to identify periods of movement (trips) and stops (dwelling). Beyond this step, all analysis is based on temporal patterns. In theory, if stops and dwell times could be detected through alternative means, our method could function entirely without location-based input. We demonstrate this methodology on the 2017 National Household Travel Survey (NHTS) data, because it includes a carefully designed and collected time use survey with representative sampling and labeled ground truth. The high-quality survey data allows us to test the accuracy of our methods because NHTS contains intended place labels and agent/user characteristics. We have also applied our validated AP identification algorithm on very large-scale GPS based trajectory data for Patterns-of-Life (PoL) assessment and other applications, but due to space limit that could not be presented here.

McBride, Liz [ORNL] (ORCID:0000000286925869)↗
DOE OSTI2022

Inference of bipolar neutrino flavor oscillations near a core-collapse supernova based on multiple measurements at Earth

Neutrinos in compact-object environments, such as core-collapse supernovae, can experience various kinds of collective effects in flavor space, engendered by neutrino-neutrino interactions. These include “bipolar” collective oscillations, which are exhibited by neutrino ensembles where different flavors dominate at different energies. Considering the importance of neutrinos in the dynamics and nucleosynthesis in these environments, it is desirable to ascertain whether an Earth-based detection could contain signatures of bipolar oscillations that occurred within a supernova envelope. To that end, we, in this study, continue examining a cost-function formulation of statistical data assimilation (SDA) to infer solutions to a small-scale model of neutrino flavor transformation. SDA is an inference paradigm designed to optimize a model with sparse data. Our model consists of two monoenergetic neutrino beams with different energies emanating from a source and coherently interacting with each other and with a matter background, with radially varying interaction strengths. We attempt to infer flavor transformation histories of these beams using simulated measurements of the flavor content at locations “in vacuum” (that is, far from the source), which could in principle correspond to Earth-based detectors. Within the scope of this small-scale model, we found that: (i) based on such measurements, the SDA procedure is able to infer whether bipolar oscillations had occurred within the protoneutron star envelope, and (ii) if the measurements sample the full amplitude of the neutrino oscillations in vacuum, then the amplitude of the prior bipolar oscillations is well predicted. This result intimates that the inference paradigm can well complement numerical integration codes, via its ability to infer flavor evolution at physically inaccessible locations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Recommended conventions for reporting results from direct dark matter searches

Abstract The field of dark matter detection is a highly visible and highly competitive one. In this paper, we propose recommendations for presenting dark matter direct detection results particularly suited for weak-scale dark matter searches, although we believe the spirit of the recommendations can apply more broadly to searches for other dark matter candidates, such as very light dark matter or axions. To translate experimental data into a final published result, direct detection collaborations must make a series of choices in their analysis, ranging from how to model astrophysical parameters to how to make statistical inferences based on observed data. While many collaborations follow a standard set of recommendations in some areas, for example the expected flux of dark matter particles (to a large degree based on a paper from Lewin and Smith in 1995), in other areas, particularly in statistical inference, they have taken different approaches, often from result to result by the same collaboration. We set out a number of recommendations on how to apply the now commonly used Profile Likelihood Ratio method to direct detection data. In addition, updated recommendations for the Standard Halo Model astrophysical parameters and relevant neutrino fluxes are provided. The authors of this note include members of the DAMIC, DarkSide, DARWIN, DEAP, LZ, NEWS-G, PandaX, PICO, SBC, SENSEI, SuperCDMS, and XENON collaborations, and these collaborations provided input to the recommendations laid out here. Wide-spread adoption of these recommendations will make it easier to compare and combine future dark matter results.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey

Growing interest in Artificial Intelligence (AI) has resulted in a surge in demand for faster methods of Machine Learning (ML) model training and inference. This demand for speed has prompted the use of high performance computing (HPC) systems that excel in managing distributed workloads. Because data is the main fuel for AI applications, the performance of the storage and I/O subsystem of HPC systems is critical. In the past, HPC applications accessed large portions of data written by simulations or experiments or ingested data for visualizations or analysis tasks. ML workloads perform small reads spread across a large number of random files. This shift of I/O access patterns poses several challenges to modern parallel storage systems. In this paper, we survey I/O in ML applications on HPC systems, and target literature within a 6-year time window from 2019 to 2024. We define the scope of the survey, provide an overview of the common phases of ML, review available profilers and benchmarks, examine the I/O patterns encountered during offline data preparation, training, and inference, and explore I/O optimizations utilized in modern ML frameworks and proposed in recent literature. Lastly, we seek to expose research gaps that could spawn further R&D.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Landmark-embedded Gaussian process with applications for functional data modeling

In practice, we often need to infer the value of a target variable from functional observation data. A challenge in this task is that the relationship between the functional data and the target variable is very complex: the target variable not only influences the shape but also the location of the functional data. In addition, due to the uncertainties in the environment, the relationship is probabilistic, that is, for a given fixed target variable value, we still see variations in the shape and location of the functional data. To address this challenge, we present a landmark-embedded Gaussian process model that describes the relationship between the functional data and the target variable. A unique feature of the model is that landmark information is embedded in the Gaussian process model so that both the shape and location information of the functional data are considered simultaneously in a unified manner. Gibbs-Metropolis-Hasting algorithm is used for model parameters estimation and target variable inference. The performance of the proposed framework is evaluated by extensive numerical studies and a case study of nano-sensor calibration.

42 ENGINEERING↗
DOE OSTI2020

Flow-based likelihoods for non-Gaussian inference

We investigate the use of data-driven likelihoods to bypass a key assumption made in many scientific analyses, which is that the true likelihood of the data is Gaussian. In particular, we suggest using the optimization targets of flow-based generative models, a class of models that can capture complex distributions by transforming a simple base distribution through layers of nonlinearities. We call these flow-based likelihoods (FBL). We analyze the accuracy and precision of the reconstructed likelihoods on mock Gaussian data, and show that simply gauging the quality of samples drawn from the trained model is not a sufficient indicator that the true likelihood has been learned. We nevertheless demonstrate that the likelihood can be reconstructed to a precision equal to that of sampling error due to a finite sample size. We then apply FBLs to mock weak lensing convergence power spectra, a cosmological observable that is significantly non-Gaussian (NG). We find that the FBL captures the NG signatures in the data extremely well, while other commonly used data-driven likelihoods, such as Gaussian mixture models and independent component analysis, fail to do so. This suggests that works that have found small posterior shifts in NG data with data-driven likelihoods such as these could be underestimating the impact of non-Gaussianity in parameter constraints. By introducing a suite of tests that can capture different levels of NG in the data, we show that the success or failure of traditional data-driven likelihoods can be tied back to the structure of the NG in the data. Here, unlike other methods, the flexibility of the FBL makes it successful at tackling different types of NG simultaneously. Because of this, and consequently their likely applicability across datasets and domains, we encourage their use for inference when sufficient mock data are available for training.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

A Differentiable Hydrology Approach for Modeling With Time-Varying Parameters

In the analysis of observational data with mathematical models, it is often desirable to let one or more model parameters vary with time to account for changing environmental conditions or to offer more flexibility. A standard approach for dynamic models is to allow for the number of parameters to grow as a function of time, leading to a high-dimensional inference problem for long data records. We propose using Hamiltonian Monte Carlo, a gradient-based Markov chain Monte Carlo (MCMC) method, to perform Bayesian inference for time-varying parameters (TVP) in hydrology models. As derivatives of model error functions with regard to parameters are not available in closed form, we implemented the GR4J rainfall-runoff models in Theano/PyMC3, and Jax/NumPyro, allowing for the application of automatic differentiation techniques. A simulation experiment assessing the viability of these methods for recovering underlying temporal variation in parameters indicates that a discrete Gaussian random walk-based prior appears to be best suited across a range of scenarios while the continuous-time Gaussian process compares relatively poorly. In an analysis of 20 years of daily real-world streamflow records from the Model Parameter Estimation Experiment (MOPEX) hydrology data set with increasing data sparsity and at multiple temporal resolutions, we find that TVP inference with gradient-based MCMC is a flexible and reliable approach for analyzing hydrology models with dynamic parameter sets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data

High-energy large-scale particle colliders produce data at high speed in the order of 1 terabytes per second in nuclear physics and petabytes per second in high energy physics. Developing real-time data compression algorithms to reduce such data at high throughput to fit permanent storage has drawn increasing attention. Specifically, at the newly constructed sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC), a time projection chamber is used as the main tracking detector, which records particle trajectories in a volume of three-dimensional (3D) cylinder. The resulting data are usually very sparse with occupancy around 10.8%. Such sparsity presents a challenge to conventional learning-free lossy compression algorithms, such as SZ, ZFP, and MGARD. The 3D convolutional neural network (CNN)-based approach, Bicephalous Convolutional Autoencoder (BCAE), outperforms traditional methods both in compression rate and reconstruction accuracy. BCAE can also utilize the computation power of graphical processing units suitable for deployment in a modern heterogeneous highperformance computing environment. This work introduces two BCAE variants: BCAE++ and BCAE-2D. BCAE++ achieves a 15% better compression ratio and a 77% better reconstruction accuracy measured in mean absolute error compared with BCAE. BCAE-2D treats the radial direction as the channel dimension of an image, resulting in a 3× speedup in compression throughput. In addition, we demonstrate an unbalanced autoencoder with a larger decoder can improve reconstruction accuracy without significantly sacrificing throughput. Lastly, we observe both the BCAE++ and BCAE-2D can benefit more from using half-precision mode in throughput (76 - 79% increase) without loss in reconstruction accuracy. The source code and links to data and pretrained models can be found at https://github.com/BNL-DAQ-LDRD/NeuralCompression_v2

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1993

Convective and stratiform rain: Multichannel microwave sensing over oceans

Measurements made by the Special Sensor Microwave/Imager (SSM/I) radiometer over the oceans, at 19, 37, and 85 GHz in dual polarization, are used to develop a model to classify rain into light-stratiform, moderately convective, and heavy convective types in the mesoscale convective systems (MCS). It is observed that the bulk of the 19- and 37-GHz data are linearly correlated with respect to one another, and generally increase together in brightness as the mean rain rate in the field of view (FOV) of the radiometer increases. However, a significant fraction of the data from these channels departs from this linear relationship, reflecting the nonuniform rain that is convective vs. the relatively light stratiform rain. It is inferred from the SSM/I data, in a MCS, when the slope dT sub 3/dT sub 19 is greater than unity there are optically thin clouds which produce light uniform rain. On the other hand, when dT sub 3/dT sub 19 is close to unity, the rain cells have an open structure and correspond to the convective type of rain. The openings between the cells are apparently a result of the downdrafts and/or entrainment. Relatively low values of 85-GHz brightness temperatures that are present when dT sub 37/dT sub 19 is close to unity support these views and, in addition, leads us to conclude that when the convection is heavy this brightness temperature decreases due to scattering by hydrometeors. On the basis of this explanation of the SSM/I data, an empirical rain retrieval algorithm is developed. Radar backscatter observations over the Atlantic Ocean next to Florida are used to demonstrate the applicability of this method. Three monthly mean maps of rainfall over the oceans from 50 degrees N to 50 degrees S, are presented to illustrate the ability of this method to sense seasonal and interannual variations of rain.

Prabhakara, C.↗
NASA NTRS2004

Classification of Particle Shapes from Lidar Depolarization Ratios in Convective Ice Clouds Compared to in situ Observations During CRYSTAL-FACE

This manuscript describes a method to class@ cirrus cloud ice particle shape using lidar depolarization measurements as a basis for segregating different particle shape regimes. Measurements from the ER-2 Cloud Physics Lidar (CPL) system during CRYSTAL-FACE provide the basis for this work. While the CPL onboard the ER-2 aircraft was providing remote sensing measurements of cirrus clouds, the Cloud Particle Imager (CPI) onboard the WB-57 aircraft was flying inside those same clouds to sample particle sizes. The results of classifying particle shapes using the CPL data are compared to the in situ measurements made using the CPI , and there is found to be good agreement between the particle shape inferred from the CPL data and that actually measured by the CPI. If proven practical, application of this technique to spaceborne observations could lead to large-scale classification of cirrus cloud particle shapes.

Noel, Vincent↗
NASA NTRS2010

Molybdenum Valence in Basaltic Silicate Melts

The moderately siderophile element molybdenum has been used as an indicator in planetary differentiation processes, and is particularly relevant to core formation [for example, 1-6]. However, models that apply experimental data to an equilibrium differentiation scenario infer the oxidation state of molybdenum from solubility data or from multivariable coefficients from metal-silicate partitioning data [1,3,7]. Partitioning behavior of molybdenum, a multivalent element with a transition near the J02 of interest for core formation (~IW-2) will be sensitive to changes in JO2 of the system and silicate melt structure. In a silicate melt, Mo can occur in either 4+ or 6+ valence state, and Mo6+ can be either octahedrally or tetrahedrally coordinated. Here we present first XANES measurements of Mo valence in basaltic run products at a range of P, T, and JO2 and further quantify the valence transition of Mo.

Danielson, L. R.↗
DOE OSTI2026

Machine Learning on Heterogeneous, Edge, and Quantum Hardware for Particle Physics (ML-HEQUPP)

The next generation of particle physics experiments will face a new era of challenges in data acquisition, due to unprecedented data rates and volumes along with extreme environments and operational constraints. Harnessing this data for scientific discovery demands real-time inference and decision-making, intelligent data reduction, and efficient processing architectures beyond current capabilities. Crucial to the success of this experimental paradigm are several emerging technologies, such as artificial intelligence and machine learning (AI/ML) and silicon microelectronics, and the advent of quantum algorithms and processing. Their intersection includes areas of research such as low-power and low-latency devices for edge computing, heterogeneous accelerator systems, reconfigurable hardware, novel codesign and synthesis strategies, readout for cryogenic or high-radiation environments, and analog computing. This white paper presents a community-driven vision to identify and prioritize research and development opportunities in hardware-based ML systems and corresponding physics applications, contributing towards a successful transition to the new data frontier of fundamental science.

Gonski, Julia [SLAC]↗
DOE OSTI2021

A genome-scale phylogeny of the kingdom Fungi

Phylogenomic studies using genome-scale amounts of data have greatly improved understanding of the tree of life. Despite the diversity, ecological significance, and biomedical and industrial importance of fungi, evolutionary relationships among several major lineages remain poorly resolved, especially those near the base of the fungal phylogeny. To examine poorly resolved relationships and assess progress toward a genome-scale phylogeny of the fungal kingdom, we compiled a phylogenomic data matrix of 290 genes from the genomes of 1,644 species that includes representatives from most major fungal lineages. We also compiled 11 data matrices by subsampling genes or taxa from the full data matrix based on filtering criteria previously shown to improve phylogenomic inference. Analyses of these 12 data matrices using concatenation- and coalescent-based approaches yielded a robust phylogeny of the fungal kingdom, in which ~85% of internal branches were congruent across data matrices and approaches used. We found support for several historically poorly resolved relationships as well as evidence for polytomies likely stemming from episodes of ancient diversification. By examining the relative evolutionary divergence of taxonomic groups of equivalent rank, we found that fungal taxonomy is broadly aligned with both genome sequence divergence and divergence time but also identified lineages where current taxonomic circumscription does not reflect their levels of evolutionary divergence. Finally, our results provide a robust phylogenomic framework to explore the tempo and mode of fungal evolution and offer directions for future fungal phylogenetic and taxonomic studies.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1983

Quantile Data Analysis of Image Data

Quantile data analysis and functional statistical inference methods are introduced and applied to provide representations of spectral data which may lead to simple statistical discriminators effective for the estimation of ground truth from satellite spectral measurements. To estimate the ground truth of a pixel, the probability of each possible ground truth is estimated, given observed (estimated) quantile theoretic statistical characteristics of the multispectral image data corresponding to the pixel and its neighboring pixels. A strategy for determining which statistical characteristics discriminate best is described. Results are reported of quantile data analysis of an extensive collection of training files of image data.

Parzen, E.↗
NASA NTRS1987

A multiprocessing architecture for real-time monitoring

A multiprocessing architecture for performing real time monitoring and analysis using knowledge-based problem solving techniques is discussed. To handle asynchronous inputs and perform in real time, the system consists of three or more separate processes which run concurrently on one or more processors and communicate via a message passing scheme. The Data Management Process gathers, compresses, scales and sends the incoming telemetry data to other tasks. The Inference Process consists of a proprietary high performance inference engine that runs at 1000 rules per second using telemetry data to perform real time analysis on the state and health of the Space Telescope. The multiprocessing architecture has been interfaced to a simulator and is able to process the incoming telemetry in real time.

Laffey, Thomas J.↗
NASA NTRS2011

Isotherm Sensor Calibration Program for Mars Science Laboratory Heat Shield Flight Data Analysis

Seven instrumented sensor plugs were installed on the Mars Science Laboratory heat shield in December 2008 as part of the Mars Science Laboratory Entry, Descent, and Landing Instrumentation (MEDLI) project. These sensor plugs contain four in-depth thermocouples and one Hollow aErothermal Ablation and Temperature (HEAT) sensor. The HEAT sensor follows the time progression of a 700 C isotherm through the thickness of a thermal protection system (TPS) material. The data can be used to infer char depth and, when analyzed in conjunction with the thermocouple data, the thermal gradient through the TPS material can also be determined. However, the uncertainty on the isotherm value is not well defined. To address this uncertainty, a team at NASA Ames Research Center is carrying out a HEAT sensor calibration test program. The scope of this test program is described, and initial results from experiments conducted in the laboratory to study the isotherm temperature of the HEAT sensor are presented. Data from the laboratory tests indicate an isotherm temperature of 720 C 60 C. An overview of near term arc jet testing is also given, including preliminary data from 30.48cm 30.48cm PICA panels instrumented with two MEDLI sensor plugs and tested in the NASA Ames Panel Test Facility. Forward work includes analysis of the arc jet test data, including an evaluation of the isotherm value based on the instant in time when it reaches a thermocouple depth.

Santos, Jose A.↗
NASA NTRS2014

Using Historical Data to Automatically Identify Air-Traffic Control Behavior

This project seeks to develop statistical-based machine learning models to characterize the types of errors present when using current systems to predict future aircraft states. These models will be data-driven - based on large quantities of historical data. Once these models are developed, they will be used to infer situations in the historical data where an air-traffic controller intervened on an aircraft's route, even when there is no direct recording of this action.

trajectory generation↗
DOE OSTI2024

Physics-inspired spatiotemporal-graph AI ensemble for the detection of higher order wave mode signals of spinning binary black hole mergers

We present a new class of AI models for the detection of quasi-circular, spinning, non-precessing binary black hole mergers whose waveforms include the higher order gravitational wave modes ($\ell$, |m|) = {(2,2), (2,1), (3,3), (3,2), (4,4)}, and mode mixing effects in the $\ell$ = 3, |m| = 2 harmonics. These AI models combine hybrid dilated convolution neural networks to accurately model both short- and long-range temporal sequential information of gravitational waves; and graph neural networks to capture spatial correlations among gravitational wave observatories to consistently describe and identify the presence of a signal in a three detector network encompassing the Advanced LIGO and Virgo detectors. We first trained these spatiotemporal-graph AI models using synthetic noise, using 1.2 million modeled waveforms to densely sample this signal manifold, within 1.7 h using 256 NVIDIA A100 GPUs in the Polaris supercomputer at the Argonne Leadership Computing Facility. This distributed training approach exhibited optimal classification performance, and strong scaling up to 512 NVIDIA A100 GPUs. With these AI ensembles we processed data from a three detector network, and found that an ensemble of 4 AI models achieves state-of-the-art performance for signal detection, and reports two misclassifications for every decade of searched data. We distributed AI inference over 128 GPUs in the Polaris supercomputer and 128 nodes in the Theta supercomputer, and completed the processing of a decade of gravitational wave data from a three detector network within 3.5 h. Finally, we fine-tuned these AI ensembles to process the entire month of February 2020, which is part of the O3b LIGO/Virgo observation run, and found 6 gravitational waves, concurrently identified in Advanced LIGO and Advanced Virgo data, and zero false positives. This analysis was completed in one hour using one NVIDIA A100 GPU.

79 ASTRONOMY AND ASTROPHYSICS↗