Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sample Handling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Bayesian approach to the study of white dwarf binaries in LISA data: The application of a reversible jump Markov chain Monte Carlo method

The Laser Interferometer Space Antenna (LISA) defines new demands on data analysis efforts in its all-sky gravitational wave survey, recording simultaneously thousands of galactic compact object binary foreground sources and tens to hundreds of background sources like binary black hole mergers and extreme-mass ratio inspirals. We approach this problem with an adaptive and fully automatic Reversible Jump Markov Chain Monte Carlo sampler, able to sample from the joint posterior density function (as established by Bayes theorem) for a given mixture of signals ''out of the box'', handling the total number of signals as an additional unknown parameter beside the unknown parameters of each individual source and the noise floor. We show in examples from the LISA Mock Data Challenge implementing the full response of LISA in its TDI description that this sampler is able to extract monochromatic Double White Dwarf signals out of colored instrumental noise and additional foreground and background noise successfully in a global fitting approach. We introduce 2 examples with fixed number of signals (MCMC sampling), and 1 example with unknown number of signals (RJ-MCMC), the latter further promoting the idea behind an experimental adaptation of the model indicator proposal densities in the main sampling stage. We note that the experienced runtimes and degeneracies in parameter extraction limit the shown examples to the extraction of a low but realistic number of signals.

79 ASTRONOMY AND ASTROPHYSICS↗

Cosmological inference from an emulator based halo model. II. Joint analysis of galaxy-galaxy weak lensing and galaxy clustering from HSC-Y1 and SDSS

Here, we present high-fidelity cosmology results from a blinded joint analysis of galaxy-galaxy weak lensing (ΔΣ) and projected galaxy clustering (w p ) measured from the Hyper Suprime-Cam Year-1 (HSC-Y1) data and spectroscopic Sloan Digital Sky Survey (SDSS) galaxy catalogs in the redshift range 0.15 < z < 0.7. We define luminosity-limited samples of SDSS galaxies to serve as the tracers of w p in three spectroscopic redshift bins, and as the lens samples for ΔΣ. For the ΔΣ measurements, we select a single sample of 4×10 6 source galaxies over 140 deg 2 from HSC-Y1 with photometric redshifts (photo z) greater than 0.75, enabling a better handle of photo- z errors by comparing the ΔΣ amplitudes for the three lens redshift bins. The deep, high-quality HSC-Y1 data enable significant detections of the ΔΣ signals, with integrated signal-to-noise ratio S/N ~ 15 in the range 3 ≤ R/[h –1 Mpc] ≤ 30 for the three lens samples, despite the small area coverage. For cosmological parameter inference, we use an input galaxy-halo connection model built on the dark emulator package (which uses an ensemble set of high-resolution N-body simulations and enables fast, accurate computation of the clustering observables) with a halo occupation distribution that includes nuisance parameters to marginalize over modeling uncertainties. We model the ΔΣ and wp measurements on scales from R≃3 and 2h –1 Mpc , respectively, up to 30 h –1 Mpc (therefore excluding the baryon acoustic oscillations information) assuming a flat Λ CDM cosmology, marginalizing over about 20 nuisance parameters and demonstrating the robustness of our results to them. With various tests using mock catalogs described in Miyatake et al. [preceding paper, Phys. Rev. D 106, 083519 (2022)], we show that any bias in the clustering amplitude S 8 ≡ σ 8 (Ω m /0.3) 0.5 due to uncertainties in the galaxy-halo connection is less than ~ 50 % of the statistical uncertainty of S 8 , unless the assembly biaseffect is unexpectedly large. Our best-fit models have S 8 = 0.795$_{-0.042}^{+0.049}$ (mode and 68% credible interval) for the flat Λ CDM model; we find tighter constraints on the quantity S 8 (α = 0.17)≡σ 8 (Ω m /0.3) 0.17 = 0.745$_{-0.031}^{+0.039}$.

79 ASTRONOMY AND ASTROPHYSICS↗

VIPIR: A High-Throughput Drop-Weight Impact Instrument for Imaging the Initiation and Propagation of Reactions in Energetic Materials

Characterizing the handling safety and sensitivity of explosives has been a challenging area of study for over 60 years. Historically one of the most accessible and widely utilized experiments has been the drop-weight impact test, which involves dropping a weight on a small sample sandwiched between two anvils. Because this experiment generally only utilizes sound thresholds to determine whether or not a sample reacted, the physical and chemical properties governing sensitivity remain convolved. Better understanding of chemical and material characteristics is needed to give the chemistry and engineering communities a predictive tool to determine the handling sensitivity of explosives prior to pursuing expensive and potentially hazardous synthesis and formulation operations. Here, we are developing a high throughput drop tower instrument capable of imaging the deformation and flow of energetic materials during impact and the resulting thermal ignition and propagation events. This instrument is based on previous designs but has been modified for higher throughput and tailorable modifications in the future. Herein, we present key design features that enable high-speed visible and thermal imaging of explosive initiation by sub-shock impacts, as well as preliminary results in which ignition sites were observed in an erythritol tetranitrate sample.

47 OTHER INSTRUMENTATION↗

DOE/DOT Crude Oil Characterization Research Study, Task 2 Test Report on Evaluating Crude Oil Sampling and Analysis Methods, Revision 1 - Winter Sampling

The Crude Oil Characterization Research Study is designed to evaluate whether crude oils currently transported in North America, including those produced from “tight” formations, exhibit physical or chemical properties that are distinct from conventional crudes, and how these properties associate with combustion hazards that may be realized during transportation and handling. The current report presents results from Task 2, investigating which commercially available methods can accurately and reproducibly collect and analyze crude oils for vapor pressure and composition, including dissolved gases. This issue, Revision 1 – Winter Sampling, incorporates additional seasonal data and compositional analysis results that have become available since publication of a prior report, SAND2017-12482, released in December 2017. Both reports compare performance of commercially available methods to that of a well-established mobile laboratory system that currently serves as the baseline instrument system for the U.S. Strategic Petroleum Reserve Crude Oil Vapor Pressure Program. The experimental matrix evaluates the performance of selected methods for (i) capturing, transporting, and delivering hydrocarbon fluid samples from the field to the analysis laboratory, coupled with (ii) analyzing for properties related to composition and volatility of the oil, including vapor pressure, gas-oil ratio, and dissolved gases and light hydrocarbons. Several combinations of sample capture and analysis were observed to perform well in both summer and winter sampling environments, though conditions apply that need to be considered carefully for given applications. Methods that perform well from Task 2 will then be utilized in subsequent Task 3 (combustion studies) and Task 4 (compositional analyses of multiple crude types), to be addressed in subsequent reports.

02 PETROLEUM↗

Evaluation of Aluminum-clad Spent Nuclear Fuel during Drying and Dry Storage

The DOE SNF Working Group ASNF Subgroup and the U.S. Nuclear Waste Technical Review Board have identified knowledge gaps regarding the management of aluminum-clad spent nuclear fuel (ASNF).[1, 2] The objectives of this research are to observe, sample and analyze the nature of aluminum surface corrosion and corrosion layer chemistry relevant to the ASNF performance over the duration of the dry storage segment of the fuel life cycle. The primary goal is to build an understanding of ASNF behavior, from reactor service and wet storage through drying and dry storage, with parallel testing of surrogate materials. The premise is that residual water affiliated with the aluminum oxide from corrosion during reactor service and wet storage, continues to influence ASNF performance during dry storage, and that even after successful drying the surface conditions of ASNF have yet to be satisfactorily characterized. The initial scope collected and analyzed specimens from irradiated non-fuel components of aluminum-clad Advanced Test Reactor (ATR) fuel elements that are routinely removed at the end of reactor service life. Techniques have been developed for remote material sampling from ATR elements from vented dry storage at the Irradiated Fuel Storage Facility (IFSF). These techniques involve 1) nipping off a small corner of the aluminum near where the end box had been removed and 2) scraping the exterior (non-fueled) plate surface to collect an oxide sample. This second effort targets ATR elements expected to have seen the longest residence in storage, both in the CPP-603 pool (with poor water quality) and in dry storage at IFSF, within the fuel handling scheduled to consolidate ATR at IFSF and receive the remaining inventory at CPP-666. Follow on work has been proposed to nip corners and collect scrapings immediately following ATR fuel drying, before placement in dry storage. Sample analysis will use multiple techniques to understand the chemical composition and morphology of the surface oxides. A simultaneous effort is progressing to develop surrogate materials to enable rigorous controlled experiments. Consistently produced, these surrogate materials can then be used to demonstrate limitations inherent to remote sampling and to validate performance models under relevant constraints. The combined efforts will substantiate ASNF performance and fuel management decisions regarding drying and packaging over a more complete range of life cycle conditions. [1] DOE Spent Nuclear Fuel Working Group, Aluminum-Clad Spent Nuclear Fuel: Technical Considerations and Challenges for Extended (>50 Years) Dry Storage, DOE/ID RPT#: 1575, prepared by Aluminum-Clad SNF Sub Working Group, June 2017. [2] U.S. Nuclear Waste Technical Review Board, Management and Disposal of U.S. Department of Energy Spent Nuclear Fuel, A Report to the United States Congress and the Secretary of Energy, December 2017, p. 143.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A Machine Learning-Based Vulnerability Analysis for Cascading Failures of Integrated Power-Gas Systems

This article proposes a cascading failure simulation (CFS) method and a hybrid machine learning method for vulnerability analysis of integrated power-gas systems (IPGSs). The CFS method is designed to study the propagating process of cascading failures between the two systems, generating data for machine learning with initial states randomly sampled. The proposed method considers generator and gas well ramping, transmission line and gas pipeline tripping, island issue handling and load shedding strategies. Then, a hybrid machine learning model with a combined random forest (RF) classification and regression algorithms is proposed to investigate the impact of random initial states on the vulnerability metrics of IPGSs. Extensive case studies are carried out on three test IPGSs to verify the proposed models and algorithms. Simulation results show that the proposed models and algorithms can achieve high accuracy for the vulnerability analysis of IPGSs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Manifold Learning-Based Polynomial Chaos Expansions for High-Dimensional Surrogate Models

In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension reduction technique which allows us to reduce the dimensionality of the data and identify meaningful geometric descriptions in a parsimonious and inexpensive manner. Polynomial chaos expansion is then used to construct a mapping between the stochastic input parameters and the diffusion coordinates of the reduced space. An adaptive clustering technique is proposed to identify an optimal number of clusters of points in the latent space. The similarity of points allows us to construct a number of geometric harmonic emulators which are finally utilized as a set of inexpensive pretrained models to perform an inverse map of realizations of latent features to the ambient space and thus perform accurate out-of-sample predictions. Thus, the proposed method acts as an encoder-decoder system which is able to automatically handle very high-dimensional data while simultaneously operating successfully in the small-data regime. The method is demonstrated on two benchmark problems and on a system of advection-diffusion-reaction equations which model a first-order chemical reaction between two species. In all test cases, the proposed method is able to achieve highly accurate approximations which ultimately lead to the significant acceleration of UQ tasks.

42 ENGINEERING↗

Adaptive Algebraic Multigrid for Finite Element Elliptic Equations with Random Coefficients

This thesis presents a two-grid algorithm based on Smoothed Aggregation Spectral Element Agglomeration Algebraic Multigrid (SA-ρAMGe) combined with adaptation. The aim is to build an efficient solver for the linear systems arising from discretization of second-order elliptic partial differential equations (PDEs) with stochastic coefficients. Examples include PDEs that model subsurface flow with random permeability field. During a Markov Chain Monte Carlo (MCMC) simulation process, that draws PDE coefficient samples from a certain distribution, the PDE coefficients change, hence the resulting linear systems to be solved change. At every such step the system (discretized PDE) needs to be solved and the computed solution used to evaluate some functional(s) of interest that then determine if the coefficient sample is acceptable or not. The MCMC process is hence computationally intensive and requires the solvers used to be efficient and fast. This fact that at every step of MCMC the resulting linear system changes, makes an already existing solver built for the old problem perhaps not as efficient for the problem corresponding to the new sampled coefficient. This motivates the main goal of our study, namely, to adapt an already existing solver to handle the problem (with changed coefficient) with the objective to achieve this goal to be faster and more efficient than building a completely new solver from scratch. Our approach utilizes the local element matrices (for the problem with changed coefficients) to build local problems associated with constructed by the method agglomerated elements (a set of subdomains that cover the given computational domain). We solve a generalized eigenproblem for each set in a subspace spanned by the previous local coarse space (used for the old solver) and a vector, component of the error, that the old solver cannot handle. A portion of the spectrum of these local eigen-problems (corresponding to eigenvalues close to zero) form the coarse basis used to define the new two-level method of our interest. We illustrate the performance of this adaptive two-level procedure with a large set of numerical experiments that demonstrate its efficiency over building the solvers from scratch.

97 MATHEMATICS AND COMPUTING↗

Final Technical Report DOE/GO/13142-1

This research adds to the understanding of the areas of residual starch and biomass conversion to alcohol, by providing data from pilot plant equipment of larger scale than the minimum required to give commercially scalable data. Instrumentation and control is in place to capture the information produced, for economic and technical evaluation. The impact of rheology, recycle streams, and residence time distributions on the technical and economic performance can be assessed. Various processes can be compared technically and economically because the pilot plants are readily modifiable. Several technologies for residual starch yield improvement have been identified, implemented, and patent applications filed. Various biomass-to-ethanol processes have been compared and one selected for technical optimization and commercialization. The technical and economic feasibility of the current simplified biomass conversion process is being confirmed by intensive pilot plant efforts as of this writing. Optimization of the feedstock handling and pretreatment is occurring to increase the alcohol yield above the minimum commercially viable level already demonstrated. Samples of biomass residue and reactor blowdown condensate are being collected to determine the technical and economic performance of the high-water-recycle waste treatment system being considered for the process. The project is of benefit to the public because it is advancing the efforts to achieve low-cost fermentable substrates for conversion to transportation fuels. This process combines the hydrolysis of agricultural residues with novel enzymes and organisms to convert the sugars released to transportation fuels. The process development is taking place at a scale allowing commercial development to proceed at a rapid pace.

09 BIOMASS FUELS↗

13 TeV pp collisions, Exotics type, MADGRAPH/PY8 generator: tev13pp_mg5_chaHT_tbeta_tb

Production of charged Higgs (H-) (with different masses) associated with t (top) using MG5_aMC_v2_6_1 at LO QCD. PYTHI8 is used for H+ decays, PS and hadronisation. Charged higgs decay to t plus b-quark (done in Pythia8) in 100% cases. The cross sections are for tan(beta)=1-7. NNPDF23_LO used for PDF. It uses 5 flavour number scheme (b-quark is a part of proton PDF). Pythia8 is used for showering and hadronisation and deacys. H+ masses ranged from 500 GeV to 8000 GeV (included to the names of files as *_m[MASS]). Cross section for each file depends on generated masses. 5,000 events per file. LHE events from MG5 are in "tev13_mg5_HT5F.tgz" Samples are created with the help of Xiaoping Wang and Carlos E.M. Wagner. Decays: Decays of H+ are handled by PYTHIA8. Events are created assuming charged Higgs to decay to b-quark and to-quarks with 100% branching ratio. No specific decay of top quarks was used. To create realistic cross sections, you should multiply the obtained cross sections by the branching ratio [H- to b-t~]. Slimming: Particle records are slimmed (all stable with pT>0.3 GeV) and (PID=5 || PID=6) or PID>22 && PID<38) or PID>10 && PID<17).

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Structural and Theoretical Studies Indicate that the Cylindrical Protease ClpP Samples Extended and Compact Conformations

The highly conserved ClpP protease consists of two heptameric rings that interact by the interdigitation of an {alpha}-helix {beta} strand handle domain motif to form a tetradecameric cylinder. We previously proposed that protease dynamics results in the temporary unstructuring of interacting pairs of handle domains, opening transient equatorial side pores that allow for peptide egress. Here, we report the structure of an Escherichia coli ClpP mutant in which each opposing pair of protomers is linked by a disulfide bond. This structure resembles the compact structures of Streptococcus pneumoniae, Mycobacterium tuberculosis, and Plasmodium falciparum ClpPs, rather than the active, extended structures that have previously been determined for E. coli ClpPs. The structural data, along with normal mode analysis, support a model whereby the ClpP cylinder switches dynamically between an active extended state required for substrate degradation and an inactive compact state allowing peptide product release.

59 BASIC BIOLOGICAL SCIENCES↗

VitroJet: new features and case studies

Single-particle cryo-electron microscopy has become a widely adopted method in structural biology due to many recent technological advances in microscopes, detectors and image processing. Before being able to inspect a biological sample in an electron microscope, it needs to be deposited in a thin layer on a grid and rapidly frozen. The VitroJet was designed with this aim, as well as avoiding the delicate manual handling and transfer steps that occur during the conventional grid-preparation process. Since its creation, numerous technical developments have resulted in a device that is now widely utilized in multiple laboratories worldwide. It features plasma treatment, low-volume sample deposition through pin printing, optical ice-thickness measurement and cryofixation of pre-clipped Autogrids through jet vitrification. This paper presents recent technical improvements to the VitroJet and the benefits that it brings to the cryo-EM workflow. A wide variety of applications are shown: membrane proteins, nucleosomes, fatty-acid synthase, Tobacco mosaic virus, lipid nanoparticles, tick-borne encephalitis viruses and bacteriophages. These case studies illustrate the advancement of the VitroJet into an instrument that enables accurate control and reproducibility, demonstrating its suitability for time-efficient cryo-EM structure determination.

59 BASIC BIOLOGICAL SCIENCES↗

A Comprehensive Analysis of Ontogeny of Renal Drug Transporters: mRNA Analyses, Quantitative Proteomics, and Localization

Human renal membrane transporters play key roles in the disposition of renally cleared drugs and endogenous substrates, but their ontogeny is largely unknown. Using 184 human postmortem frozen renal cortical tissues (preterm newborns to adults) and a subset of 62 tissue samples, we measured the mRNA levels of 11 renal transporters and the transcription factor pregnane X receptor (PXR) with quantitative real‐time polymerase chain reaction, and protein abundance of nine transporters using liquid chromatography tandem mass spectrometry selective reaction monitoring, respectively. Expression levels of p‐glycoprotein, urate transporter 1, organic anion transporter 1, organic anion transporter 3, and organic cation transporter 2 increased with age. Protein levels of multidrug and toxin extrusion transporter 2‐K and breast cancer resistance protein showed no difference from newborns to adults, despite age‐related changes in mRNA expression. Multidrug and toxin extrusion transporter 1, glucose transporter 2, multidrug resistance‐associated protein 2, multidrug resistance‐associated protein 4 (MRP4), and PXR expression levels were stable. Using immunohistochemistry, we found that MRP4 localization in pediatric samples was similar to that in adult samples. Collectively, our study revealed that renal drug transporters exhibited different rates and patterns of maturation, suggesting that renal handling of substrates may change with age.

60 APPLIED LIFE SCIENCES↗

Mapping wall-to-wall fractional cover of Arctic tundra plant functional types in Alaska using 20-m spatial resolution satellite imagery and harmonized plot observations

Estimates of fractional cover (fCover) across given land surfaces are used to assess, and often model, vegetation composition and diversity, which are crucial for understanding the health and functioning of terrestrial ecosystems. Remote sensing provides a useful means for scaling local, plot-measured fCover estimates to regional scales. Leveraging a recently synthesized and harmonized plot database, this study generated wall-to-wall maps of fCover for six Alaskan-Arctic plant functional types (PFT), including non-vascular plants, forbs, graminoids, and deciduous and evergreen shrubs, using 20-m satellite data (Sentinel-1, Sentinel-2, ArcticDEM) using a machine learning regression approach, specifically the random forest (RF) algorithm, which is well-suited for handling nonlinear relationships and high-dimensional satellite datasets. This study additionally addressed the spatio-temporal inconsistencies e.g., sampling scale, plot size, and collection year in plot measured fCover by adopting a multivariate outlier detection approach—Cook’s distance—to identify high-quality plots for model training and validation. Our approach achieves high accuracy (R 2 = 0.59–0.93, root mean squared errors = 0.02–0.10 for all PFTs) between plot-observed and satellite-derived fCover when using high-quality plot samples. The mapped fCover characterizes the spatial patterns of different PFTs across the tundra biome at a 20-m resolution, providing key information needed for improved representation of Arctic tundra vegetation in terrestrial biosphere models to better understand climate-vegetation feedback across the Arctic tundra.

Arctic tundra↗

Observation of a topological insulator Dirac cone reshaped by non-magnetic impurity resonance

The massless Dirac electrons found at topological insulator surfaces are thought to be influenced very little by weak, non-magnetic disorder. However, a resonance effect of strongly perturbing non-magnetic impurities has been theoretically predicted to change the dispersion and physical nature of low-energy quasiparticles, resulting in unique particle-like states that lack microscopic translational symmetry. Here we report the direct observation of impurities reshaping the surface Dirac cone of the model three-dimensional topological insulator bismuth selenide. A pronounced kink-like dispersion feature is observed in disorder-enriched samples, and found to be closely associated with the anomaly caused by impurity resonance in the surface state density of states, as observed by dichroic angle-resolved photoemission spectroscopy. The experimental observation of these features, which closely resemble theoretical predictions, has significant implications for the properties of topological Dirac cones in applied scenarios that commonly feature point-defect disorder at surfaces or interfaces. Topological insulators - influence of surface impurities: The electronic properties of topological insulators are robust against perturbations, including the presence of non-magnetic impurities. However, surface impurities can give rise to resonant states near the Dirac point, and if their density becomes sufficiently high it is predicted that they can substantially modify the dispersion of the Dirac cone and develop a collective behaviour that results in the formation of particle-like states that lack microscopic translational symmetry. L. Andrew Wray at Purdue University and at the New York University Shanghai, and colleagues, used angle-resolved photoemission spectroscopy to experimentally observe the reshaping of the surface Dirac cone in a defect-rich sample of the topological insulator Bi 2 Se 3 . These results indicate that surface impurities can provide a useful handle to control the properties of topological insulators.

36 MATERIALS SCIENCE↗

Decayheatml

This code is designed to predict and analyze the decay heat generated in molten salt reactors (MSRs) using a hybrid approach that combines machine learning and segmented polynomial fitting. The accurate prediction of decay heat is essential for reactor safety and the optimization of spent fuel storage. The code operates through several key components: 1) Data Architecture: It incorporates a modular data architecture that handles various MSR-specific operational parameters such as power density, humidity content, and air ingress. These parameters are sampled using Sobol sequences to ensure comprehensive coverage of operational uncertainties. 2) Machine Learning Framework: The code employs a diverse set of machine learning models, including polynomial regression, decision trees, random forests, gradient boosting, support vector regression, k-nearest neighbors, multi-layer perceptrons, and symbolic regression. These models are trained to predict decay heat over a wide temporal range, from immediate shutdown up to 10,000 years. 3) Region-Optimized Training: The temporal domain is divided into multiple regions, each modeled separately to capture distinct decay heat characteristics across different time scales. This approach significantly improves the accuracy and interpretability of predictions. 4) Segmented Polynomial Interpretation (SPI): The SPI method translates machine learning predictions into piecewise polynomial equations. These equations are physically interpretable and can be directly integrated into existing engineering workflows and safety analyses. 5) Front-End Interfaces: The code includes both a Jupyter notebook interface for research development and a Streamlit web application for operational deployment. These interfaces allow users to interactively explore decay heat predictions, adjust operational parameters, and visualize results in real-time. 6) Applications: The framework supports various applications, including safety system validation and spent fuel container optimization. It enables real-time evaluation of worst-case decay heat scenarios, informing the design of passive safety systems and optimizing container designs for long-term storage. Overall, this code provides a robust, accurate, and user-friendly tool for predicting decay heat in MSRs, enhancing reactor safety, and optimizing spent fuel management.

Retamales, Mauricio Eduardo Tano [Idaho National L↗

Communication-avoiding optimization methods for distributed massive-scale sparse inverse covariance estimation

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ≈819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.

97 MATHEMATICS AND COMPUTING↗

Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation.

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ~819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.

Koanantakool, Penporn↗