Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “bottleneck analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Inactivation of 4-Oxalocrotonate Tautomerase by 5-Halo-2-hydroxy-2,4-pentadienoates

5-Halo-2-hydroxy-2,4-pentadienoates (5-halo-HPDs) are reportedly generated in the bacterial catabolism of halogenated aromatic hydrocarbons by the meta-fission pathway. The 5-halo-HPDs, where the halogen can be bromide, chloride, or fluoride, result in the irreversible inactivation of 4-oxalocrotonate tautomerase (4-OT), which precedes the enzyme that generates them. The loss of activity is due to the covalent modification of the nucleophilic amino-terminal proline. Mass spectral and crystallographic analysis of the modified enzymes indicates that inactivation of 4-OT by 5-chloro- and 5-bromo-2-hydroxy-2,4-pentadienoate follows a mechanism different from that for the inactivation of 4-OT by 5-fluoro-2-hydroxy-2,4-pentadienoate. The 5-chloro and 5-bromo derivatives undergo 4-OT-catalyzed tautomerization to their respective α,β-unsaturated ketones followed by attack at C5 (by the prolyl nitrogen) with concomitant loss of the halide. For the 5-fluoro species, the presence of a small amount of the α,β-unsaturated ketone could result in a Michael addition of the prolyl nitrogen to C4 followed by protonation at C3. The fluoride is not eliminated. Here, these observations suggest that the inactivation of 4-OT by a downstream metabolite could hamper the efficacy of the pathway, which is the first time that such a bottleneck has been reported for the meta-fission pathway.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2019

Tensor-decomposition techniques for ab initio nuclear structure calculations: From chiral nuclear potentials to ground-state energies

The computational resources needed to generate the ab initio solution of the nuclear many-body problem for increasing mass number and/or accuracy necessitates innovative developments to improve upon (i) the storage of many-body operators and (ii) the scaling of many-body methods used to evaluate nuclear observables. The storing and efficient handling of many-body operators with high particle ranks is currently one of the major bottlenecks limiting the applicability range of ab initio studies with respect to mass number and accuracy. Recently, the application of tensor decomposition techniques to many-body tensors has proven highly beneficial to reduce the computational cost of ab initio calculations in quantum chemistry and solid-state physics. The impact of applying state-of-the-art tensor factorization techniques to modern nuclear Hamiltonians derived from chiral effective field theory is investigated. Subsequently, the error induced by the tensor decomposition of the input Hamiltonian on ground-state energies of closed-shell nuclei calculated via second-order many-body perturbation theory is benchmarked. The first proof-of-principles application of tensor-decomposition techniques to the nuclear Hamiltonian is performed. Two different tensor formats are investigated by systematically benchmarking the approximation error on matrix elements stored in various bases of interest. The analysis is achieved while including normal-ordered three-nucleon interactions that are nowadays used as input to the most advanced ab initio calculations in medium-mass nuclei. With the aid of the factorized Hamiltonian, the second-order perturbative correction to ground-state energies is decomposed and the scaling properties of the underlying tensor network are discussed. The employed tensor formats are found to lead to an efficient data compression of two-body matrix elements of the nuclear Hamiltonian. In particular, the sophisticated tensor hypercontraction scheme yields low tensor ranks with respect to both harmonic-oscillator and Hartree-Fock single-particle bases. It is found that the tensor rank depends on the two-body total angular momentum J for which one performs the decomposition, which is itself directly related to the sparsity of the corresponding tensor. Furthermore, including normal-ordered two-body contributions originating from three-body interactions does not compromise the efficient data compression. Ultimately, the use of factorized matrix elements authorizes controlled approximations of the exact second-order ground-state energy corrections. In particular, a small enough error is obtained from low-rank factorizations in He4,O16, and Ca40. It is presently demonstrated that tensor-decomposition techniques can be efficiently applied to systematically approximate the nuclear many-body Hamiltonian in terms of lower-rank tensors. Beyond the input Hamiltonian, tensor-decomposition techniques can be envisioned to scale down the cost of state-of-the-art nonperturbative many-body methods in order to extend ab initio studies to (i) higher precisions, (ii) larger masses, and (iii) nuclei of doubly open-shell character.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Assessing MP2 frozen natural orbitals in relativistic correlated electronic structure calculations

The high computational scaling with the basis set size and the number of correlated electrons is a bottleneck limiting applications of coupled cluster algorithms, in particular for calculations based on two- or four-component relativistic Hamiltonians, which often employ uncontracted basis sets. This problem may be alleviated by replacing canonical Hartree–Fock virtual orbitals by natural orbitals (NOs). Here, in this paper, we describe the implementation of a module for generating NOs for correlated wavefunctions and, in particular, second order Møller–Plesset perturbation frozen natural orbitals (MP2FNOs) as a component of our novel implementation of relativistic coupled cluster theory for massively parallel architectures [Pototschnig et al. J. Chem. Theory Comput. 17, 5509, (2021)]. Our implementation can manipulate complex or quaternion density matrices, thus allowing for the generation of both Kramers-restricted and Kramers-unrestricted MP2FNOs. Furthermore, NOs are re-expressed in the parent atomic orbital (AO) basis, allowing for generating coupled cluster singles and doubles NOs in the AO basis for further analysis. By investigating the truncation errors of MP2FNOs for both the correlation energy and molecular properties—electric field gradients at the nuclei, electric dipole and quadrupole moments for hydrogen halides HX (X = F–Ts), and parity-violating energy differences for H 2 Z 2 (Z = O–Se)—we find MP2FNOs accelerate the convergence of the correlation energy in a roughly uniform manner across the Periodic Table. It is possible to obtain reliable estimates for both energies and the molecular properties considered with virtual molecular orbital spaces truncated to about half the size of the full spaces.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1994

Constraint based scheduling for the Goddard Space Flight Center distributed Active Archive Center's data archive and distribution system

The Goddard Space Flight Center (GSFC) Distributed Active Archive Center (DAAC) has been operational since October 1, 1993. Its mission is to support the Earth Observing System (EOS) by providing rapid access to EOS data and analysis products, and to test Earth Observing System Data and Information System (EOSDIS) design concepts. One of the challenges is to ensure quick and easy retrieval of any data archived within the DAAC's Data Archive and Distributed System (DADS). Over the 15-year life of EOS project, an estimated several Petabytes (10(exp 15)) of data will be permanently stored. Accessing that amount of information is a formidable task that will require innovative approaches. As a precursor of the full EOS system, the GSFC DAAC with a few Terabits of storage, has implemented a prototype of a constraint-based task and resource scheduler to improve the performance of the DADS. This Honeywell Task and Resource Scheduler (HTRS), developed by Honeywell Technology Center in cooperation the Information Science and Technology Branch/935, the Code X Operations Technology Program, and the GSFC DAAC, makes better use of limited resources, prevents backlog of data, provides information about resources bottlenecks and performance characteristics. The prototype which is developed concurrently with the GSFC Version 0 (V0) DADS, models DADS activities such as ingestion and distribution with priority, precedence, resource requirements (disk and network bandwidth) and temporal constraints. HTRS supports schedule updates, insertions, and retrieval of task information via an Application Program Interface (API). The prototype has demonstrated with a few examples, the substantial advantages of using HTRS over scheduling algorithms such as a First In First Out (FIFO) queue. The kernel scheduling engine for HTRS, called Kronos, has been successfully applied to several other domains such as space shuttle mission scheduling, demand flow manufacturing, and avionics communications scheduling.

Short, Nick, Jr.↗
NASA NTRS2014

Controller Inhibition of Automated Conflict Resolutions in a Maximum NextGen Condition

With the continued projection of increases in air traffic density, operations in the National Airspace System are expected to exceed human capabilities in the near future. In order to address the bottleneck of human workload capacity, highly automated safety-critical systems are under development to support air traffic controllers. A human-in-the-loop experiment examined controllers transition through four NextGen automation stages: Current-Day, Minimum, Moderate, and Maximum. Maximum NextGen simulated a fully automated environment where the automation was responsible for detecting and resolving conflicts within simulation parameters in high-density airspace. By allocating these tasks to the automation, the controllers task changed. The human moved to primarily a supervisory position- typically only regaining control over separation assurance tasks during conflict situations deferred by the automation. While tasks were allocated a-priori between the controller and automated agent, controllers maintained authority to inhibit the automation from interacting with particular aircraft. Preliminary work is complete, where significant differences were found in inhibition frequency between simulation participants. However, the contexts in which the controllers inhibited the automation, and their reasons for doing so, remain unclear. This analysis attempts to identity factors contributing to human controllers inhibition of the automation in the Maximum NextGen condition.

adaptable automation↗
DOE OSTI2016

Population genomic analysis reveals differential evolutionary histories and patterns of diversity across subgenomes and subpopulations of Brassica napus L.

The allotetraploid species Brassica napus L. is a global crop of major economic importance, providing canola oil (seed) and vegetables for human consumption and fodder and meal for livestock feed. Characterizing the genetic diversity present in the extant germplasm pool of B. napus is fundamental to better conserve, manage and utilize the genetic resources of this species. We used sequence-based genotyping to identify and genotype 30,881 SNPs in a diversity panel of 782 B. napus accessions, representing samples of winter and spring growth habits originating from 33 countries across Europe, Asia, and America. We detected strong population structure broadly concordant with growth habit and geography, and identified three major genetic groups: spring (SP), winter Europe (WE), and winter Asia (WA). Subpopulation-specific polymorphism patterns suggest enriched genetic diversity within the WA group and a smaller effective breeding population for the SP group compared to WE. Interestingly, the two subgenomes of B. napus appear to have different geographic origins, with phylogenetic analysis placing WE and WA as basal clades for the other subpopulations in the C and A subgenomes, respectively. Finally, we identified 16 genomic regions where the patterns of diversity differed markedly from the genome-wide average, several of which are suggestive of genomic inversions. The results obtained in this study constitute a valuable resource for worldwide breeding efforts and the genetic dissection and prediction of complex B. napus traits.

58 GEOSCIENCES↗
DOE OSTI2019

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Data I/O poses a significant bottleneck in large-scale CFD simulations; thus, practitioners would like to significantly reduce the number of times the solution is saved to disk, yet retain the ability to recover any field quantity (at any time instance) a posteriori . The objective of this work is therefore to accurately recover missing CFD data a posteriori at any time instance, given that the solution has been written to disk at only a relatively small number of time instances. We consider in particular high-order discretizations (e.g., discontinuous Galerkin), as such techniques are becoming increasingly popular for the simulation of highly separated flows. To satisfy this objective, this work proposes a methodology consisting of two stages: 1) dimensionality reduction and 2) dynamics learning. For dimensionality reduction, we propose a novel hierarchical approach. First, the method reduces the number of degrees of freedom within each element of the high-order discretization by applying autoencoders from deep learning. Second, the methodology applies principal component analysis to compress the global vector of encodings. This leads to a low-dimensional state, which associates with a nonlinear embedding of the original CFD data. For dynamics learning, we propose to apply regression techniques (e.g., kernel methods) to learn the discrete-time velocity characterizing the time evolution of this low-dimensional state. In conclusion, a numerical example on a large-scale CFD example characterized by nearly 13 million degrees of freedom illustrates the suitability of the proposed method in an industrial setting.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

An AI-driven framework for evaluating local and state authorities’ permitting processes

The demand for new energy infrastructure is increasing across the United States, but heterogenous permitting processes and embedded requirements across different local jurisdictions can cause project delays, increase “soft costs,” and hinder developer expansion. This study analyzes the variability in local permitting requirements across the U.S. and develops a quantitative approach to describe their clarity and effectiveness in enabling infrastructure project development. By using an Energy Language Model (ELM), a large language model (LLM) for energy technologies, we systematically gathered permitting information from nearly 300 state-, county-, and city-level documents, creating a structured dataset of requirements and procedures on an unprecedented scale and speed. Our analysis revealed that local (city and county) permitting requirement documents are underrepresented compared to state-level guidance documents, which can impede timely and cost-effective installation of new electric infrastructure. Our validation process showed that the final database has an accuracy of approximately 95%. We, further, created a new quantitative method to score permitting requirements for clarity and efficiency, with electric vehicle supply equipment as an initial use case. The average local permitting document scored a 1.8 out of 5, which we interpret as meaning that half of the requirements developers face when installing electric infrastructure are ambiguous, increasing both cost and time. We also created a “Generalized Permit Process”, highlighting common procedural steps and identifying specific opportunities for municipalities to improve their documentation. This research establishes a systematic and scalable framework for evaluating the complexities of local infrastructure permitting processes by combining LLM-powered data collection and quantitative scoring. The framework enables policymakers and developers to identify and mitigate procedural bottlenecks, with the expectation that these improvements can accelerate application review and approval, reduce project costs, and expedite connection to utility distribution grids. As a foundational approach for streamlining local project development processes, this study’s methods are intended to be extended to a wide range of energy applications.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

LatticeAnalytics: Strut-Level Visualization and Inspection of Additively Manufactured Lattice Structures

Additive manufacturing (AM) is revolutionizing the production of custom components with complex internal geometries, essential for high-performance applications in diverse fields such as medicine and defense. These AM parts optimize strength while minimizing weight by utilizing internal lattice structures consisting of large quantities of small interconnected struts. However, the complexity of these structures, combined with the challenges of using X-ray Computed Tomography (XCT) data, makes validation of part reliability difficult. This ultimately inhibits the development of novel parts for our collaborating material scientists. Here, we introduce LatticeAnalytics, a novel framework specifically designed for visual inspection of defects in these lattice structures. Our framework offers an end-to-end solution that includes the data management of XCT scans, enables remote access for geographically dispersed teams through a web-based dashboard, and incorporates novel visualizations. Our analysis is facilitated by a coarse alignment between the lattice’s nominal model, a spatial graph, and the XCT data. We employ a simple VR-based approach for fast and rough alignment, followed by an offline registration and identification of the struts. With the nodes and struts aligned and identified in the volume, our framework allows querying of subvolumes containing a single strut at multiple resolutions. This avoids computation over the entire lattice and also allow for easy parallelization of down-stream computations, such as strut-specific metrics. To depict a fast overview of the strut quality, we introduce two innovative visual encodings, crucial for our collaborators’ research in creating novel AM parts: the Contour View and the Roughness Map, which depict critical geometrical and surface features of individual struts in standardized two 2D views. We evaluated the integrated system through expert interviews. The feedback confirms the framework’s practicality and its effectiveness in enhancing current inspection workflows. It solves major bottlenecks for our collaborators, ultimately helping them create novel parts with advanced properties.

Miao, Haichao [Lawrence Livermore National Laborat↗
DOE OSTI2020

Toward Large-Scale Image Segmentation on Summit

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (~ 10 2 × 10 2 ), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (~10 4 × 10 4 ). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ~ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.

Seal, Sudip↗
DOE OSTI2022

Installation and imaging of thousands of minirhizotrons to phenotype root systems of field-grown plants

Roots are vital to plant performance because they acquire resources from the soil and provide anchorage. However, it remains difficult to assess root system size and distribution because roots are inaccessible in the soil. Existing methods to phenotype entire root systems range from slow, often destructive, methods applied to relatively small numbers of plants in the field to rapid methods that can be applied to large numbers of plants in controlled environment conditions. Much has been learned recently by extensive sampling of the root crown portion of field-grown plants. But, information on large-scale genetic and environmental variation in the size and distribution of root systems in the field remains a key knowledge gap. Minirhizotrons are the only established, non-destructive technology that can address this need in a standard field trial. Prior experiments have used only modest numbers of minirhizotrons, which has limited testing to small numbers of genotypes or environmental conditions. This study addressed the need for methods to install and collect images from thousands of minirhizotrons and thereby help break the phenotyping bottleneck in the field. Over three growing seasons, methods were developed and refined to install and collect images from up to 3038 minirhizotrons per experiment. Modifications were made to four tractors and hydraulic soil corers mounted to them. High quality installation was achieved at an average rate of up to 84.4 minirhizotron tubes per tractor per day. A set of four commercially available minirhizotron camera systems were each transported by wheelbarrow to allow collection of images of mature maize root systems at an average rate of up to 65.3 tubes per day per camera. This resulted in over 300,000 images being collected in as little as 11 days for a single experiment. The scale of minirhizotron installation was increased by two orders of magnitude by simultaneously using four tractor-mounted, hydraulic soil corers with modifications to ensure high quality, rapid operation. Image collection can be achieved at the corresponding scale using commercially available minirhizotron camera systems. Along with recent advances in image analysis, these advances will allow use of minirhizotrons at unprecedented scale to address key knowledge gaps regarding genetic and environmental effects on root system size and distribution in the field.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Enabling Intelligent Data Downlink Prioritization of In-Situ Observations through Generalizable and Computationally Inexpensive Anomaly Detection

High-fidelity measurements of magnetic fields and other observed properties, such as energetic particle fluxes, are a necessary component to our understanding of the highly dynamic near-Earth space environment. As our desire to study smaller-scale phenomena such as shocks and dipolorizations has increased, we have been driven to take and telemeter measurements at higher cadences. Unfortunately, many missions are unable to downlink all their captured data due to the well-known data transmission bottleneck at the DSN. These missions must then prioritize their high-cadence data such that the most scientifically useful intervals are transmitted. One simple prioritization technique uses the spacecraft position to telemeter data from only the region of interest. Although easy to implement, this method does not leverage the available scientific data and can omit intervals of useful scientific data when they lie outside the region of interest. The Magnetospheric Multiscale Mission (MMS) uses mission-specific parameterization of several data products to automatically prioritize scientifically useful intervals. Then, MMS verifies the automatically selected intervals by having a domain expert manually select intervals for downlink. The overall complexity required by this technique make it prohibitive for deployment on low-cost platforms (i.e., CubeSats) or on future missions featuring large constellations of satellites such as the Geospace Dynamics Constellation (GDC). We present preliminary results for a simple, generic, and data-driven method of downlink prioritization for magnetic field (and other) measurements. Specifically, Principal Components Analysis (PCA) and One-Class Support Vector Machines (OC-SVMs) are used to detect intervals containing anomalous activity, which can then be prioritized for subsequent downlink. The computational simplicity of this algorithm makes it an excellent candidate for implementation on spaceflight hardware, as well as provide generalizability to a broad range of missions and data products. Initial analysis of this technique has been performed using magnetic field measurements from the Magnetospheric Multiscale Mission and CASSIOP, where it automatically identified scientifically interesting intervals containing Alfvén waves and EMIC activity.

Matthew G. Finley↗
NASA NTRSDate not supplied

Enabling Intelligent Data Downlink Prioritization of In-Situ Observations through Generalizable and Computationally Inexpensive Anomaly Detection

High-fidelity measurements of magnetic fields and other observed properties, such as energetic particle fluxes, are a necessary component to our understanding of the highly dynamic near-Earth space environment. As our desire to study smaller-scale phenomena such as shocks and dipolorizations has increased, we have been driven to take and telemeter measurements at higher cadences. Unfortunately, many missions are unable to downlink all their captured data due to the well-known data transmission bottleneck at the DSN. These missions must then prioritize their high-cadence data such that the most scientifically useful intervals are transmitted. One simple prioritization technique uses the spacecraft position to telemeter data from only the region of interest. Although easy to implement, this method does not leverage the available scientific data and can omit intervals of useful scientific data when they lie outside the region of interest. The Magnetospheric Multiscale Mission (MMS) uses mission-specific parameterization of several data products to automatically prioritize scientifically useful intervals. Then, MMS verifies the automatically selected intervals by having a domain expert manually select intervals for downlink. The overall complexity required by this technique make it prohibitive for deployment on low-cost platforms (i.e., CubeSats) or on future missions featuring large constellations of satellites such as the Geospace Dynamics Constellation (GDC). We present preliminary results for a simple, generic, and data-driven method of downlink prioritization for magnetic field (and other) measurements. Specifically, Principal Components Analysis (PCA) and One-Class Support Vector Machines (OC-SVMs) are used to detect intervals containing anomalous activity, which can then be prioritized for subsequent downlink. The computational simplicity of this algorithm makes it an excellent candidate for implementation on spaceflight hardware, as well as provide generalizability to a broad range of missions and data products. Initial analysis of this technique has been performed using magnetic field measurements from the Magnetospheric Multiscale Mission and CASSIOP, where it automatically identified scientifically interesting intervals containing Alfvén waves and EMIC activity.

Matthew G. Finley↗
DOE OSTI2015

A Phylogeographic Investigation of African Monkeypox

Monkeypox is a zoonotic disease caused by a virus member of the genus Orthopoxvirus and is endemic to Central and Western African countries. Previous work has identified two geographically disjuct clades of monkeypox virus based on the analysis of a few genomes coupled with epidemiological and clinical analyses; however, environmental and geographic causes of this differentiation have not been explored. Here, we expand previous phylogenetic studies by analyzing a larger set of monkeypox virus genomes originating throughout Sub-Saharan Africa to identify possible biogeographic barriers associated with genetic differentiation; and projected ecological niche models onto environmental conditions at three periods in the past to explore the potential role of climate oscillations in the evolution of the two primary clades. Analyses supported the separation of the Congo Basin and West Africa clades; the Congo Basin clade shows much shorter branches, which likely indicate a more recent diversification of isolates within this clade. The area between the Sanaga and Cross Rivers divides the two clades and the Dahomey Gap seems to have also served as a barrier within the West African clade. Contraction of areas with suitable environments for monkeypox virus during the Last Glacial Maximum, suggests that the Congo Basin clade of monkeypox virus experienced a severe bottleneck and has since expanded its geographic range.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2016

A Survey of Solver-Related Geometry and Meshing Issues

There is a concern in the computational fluid dynamics community that mesh generation is a significant bottleneck in the CFD workflow. This is one of several papers that will help set the stage for a moderated panel discussion addressing this issue. Although certain general "rules of thumb" and a priori mesh metrics can be used to ensure that some base level of mesh quality is achieved, inadequate consideration is often given to the type of solver or particular flow regime on which the mesh will be utilized. This paper explores how an analyst may want to think differently about a mesh based on considerations such as if a flow is compressible vs. incompressible or hypersonic vs. subsonic or if the solver is node-centered vs. cell-centered. This paper is a high-level investigation intended to provide general insight into how considering the nature of the solver or flow when performing mesh generation has the potential to increase the accuracy and/or robustness of the solution and drive the mesh generation process to a state where it is no longer a hindrance to the analysis process.

Masters, James↗
NASA NTRS2016

Using Docker Containers to Extend Reproducibility Architecture for the NASA Earth Exchange (NEX)

NASA Earth Exchange (NEX) is a data, supercomputing and knowledge collaboratory that houses NASA satellite, climate and ancillary data where a focused community can come together to address large-scale challenges in Earth sciences. As NEX has been growing into a petabyte-size platform for analysis, experiments and data production, it has been increasingly important to enable users to easily retrace their steps, identify what datasets were produced by which process chains, and give them ability to readily reproduce their results. This can be a tedious and difficult task even for a small project, but is almost impossible on large processing pipelines. We have developed an initial reproducibility and knowledge capture solution for the NEX, however, if users want to move the code to another system, whether it is their home institution cluster, laptop or the cloud, they have to find, build and install all the required dependencies that would run their code. This can be a very tedious and tricky process and is a big impediment to moving code to data and reproducibility outside the original system. The NEX team has tried to assist users who wanted to move their code into OpenNEX on Amazon cloud by creating custom virtual machines with all the software and dependencies installed, but this, while solving some of the issues, creates a new bottleneck that requires the NEX team to be involved with any new request, updates to virtual machines and general maintenance support. In this presentation, we will describe a solution that integrates NEX and Docker to bridge the gap in code-to-data migration. The core of the solution is saemi-automatic conversion of science codes, tools and services that are already tracked and described in the NEX provenance system, to Docker - an open-source Linux container software. Docker is available on most computer platforms, easy to install and capable of seamlessly creating and/or executing any application packaged in the appropriate format. We believe this is an important step towards seamless process deployment in heterogeneous environments that will enhance community access to NASA data and tools in a scalable way, promote software reuse, and improve reproducibility of scientific results.

earth exchange↗
DOE OSTI2014

Cosmological constraints from Sunyaev-Zeldovich cluster counts: An approach to account for missing redshifts

The accumulation of redshifts provides a significant observational bottleneck when using galaxy cluster surveys to constrain cosmological parameters. We propose a simple method to allow the use of samples where there is a fraction of the redshifts that are not known. The simplest assumption is that the missing redshifts are randomly extracted from the catalog, but the method also allows one to take into account known selection effects in the accumulation of redshifts. We quantify the reduction in statistical precision of cosmological parameter constraints as a function of the fraction of missing redshifts for simulated surveys, and also investigate the impact of making an incorrect assumption for the distribution of missing redshifts.

79 ASTRONOMY AND ASTROPHYSICS↗