SEARCH · Engineering Papers

Results for “parallel programming patterns”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2016

High-performance epistasis detection in quantitative trait GWAS

epiSNP is a program for identifying pairwise single nucleotide polymorphism (SNP) interactions (epistasis) in quantitative-trait genome-wide association studies (GWAS). A parallel MPI version (EPISNPmpi) was created in 2008 to address this computationally expensive analysis on large data sets with many quantitative traits and SNP markers. However, the falling cost of genotyping has led to an explosion of large-scale GWAS data sets that challenge EPISNPmpi’s ability to compute results in a reasonable amount of time. Therefore, we optimized epiSNP for modern multi-core and highly parallel many-core processors to efficiently handle these large data sets. This paper describes the serial optimizations, dynamic load balancing using MPI-3 RMA operations, and shared-memory parallelization with OpenMP to further enhance load balancing and allow execution on the Intel Xeon Phi coprocessor (MIC). For a large GWAS data set, our optimizations provided a 38.43× speedup over EPISNPmpi on 126 nodes using 2 MICs on TACC’s Stampede Supercomputer. We also describe a Coarray Fortran (CAF) version that demonstrates the suitability of PGAS languages for problems with this computational pattern. We show that the Coarray version performs competitively with the MPI version on the NERSC Edison Cray XC30 supercomputer. Finally, the performance benefits of hyper-threading for this application on Edison (average 1.35× speedup) are demonstrated.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Distributed non-negative matrix factorization with determination of the number of latent features

The holistic analysis and understanding of the latent (that is, not directly observable) variables and patterns buried in large datasets is crucial for data-driven science, decision making and emergency response. Such exploratory analyses require devising unsupervised learning methods for data mining and extraction of the latent features, and non-negative matrix factorization (NMF) is one of the prominent such methods. NMF is based on compute-intense non-convex constrained minimization, which, for large datasets requires fast and distributed algorithms. However, current parallel implementations of NMF fail to estimate the number of latent features. In practice, identifying these features is both difficult and significant for pattern recognition and latent feature analysis, especially for large dense matrices. Here, we introduce a distributed NMF algorithm coupled with distributed custom clustering followed by a stability analysis on dense data, which we call DnMFk, to determine the number of latent variables. The results on synthetic data and the classical Swimmer data set demonstrate the accuracy of model determination while scaling nearly linearly across multiple processors for large data. Further, we employ DnMFk to determine the number of hidden features from a terabyte matrix.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2013

North Pacific Mesoscale Coupled Air-Ocean Simulations Compared with Observations

Executive summary The main objective of the study was to investigate atmospheric and ocean interaction processes in the western Pacific and, in particular, effects of significant ocean heat loss in the Kuroshio and Kuroshio Extension regions on the lower and upper atmosphere. It is yet to be determined how significant are these processes are on climate scales. The understanding of these processes led us also to development of the methodology of coupling the Weather and Research Forecasting model with the Parallel Ocean Program model for western Pacific regional weather and climate simulations. We tested NCAR-developed research software Coupler 7 for coupling of the WRF and POP models and assessed its usability for regional-scale applications. We completed test simulations using the Coupler 7 framework, but implemented a standard WRF model code with options for both one- and two-way mode coupling. This type of coupling will allow us to seamlessly incorporate new WRF updates and versions in the future. We also performed a long-term WRF simulation (15 years) covering the entire North Pacific as well as high-resolution simulations of a case study which included extreme ocean heat losses in the Kuroshio and Kuroshio Extension regions. Since the extreme ocean heat loss occurs during winter cold air outbreaks (CAO), we simulated and analyzed a case study of a severe CAO event in January 2000 in detail. We found that the ocean heat loss induced by CAOs is amplified by additional advection from mesocyclones forming on the southern part of the Japan Sea. Large scale synoptic patterns with anomalously strong anticyclone over Siberia and Mongolia, deep Aleutian Low, and the Pacific subtropical ridge are a crucial setup for the CAO. It was found that the onset of the CAO is related to the breaking of atmospheric Rossby waves and vertical transport of vorticity that facilitates meridional advection. The study also indicates that intrinsic parameterization of the surface fluxes within the WRF model needs more evaluation and analysis.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Cabana: A Performance Portable Library for Particle-Based Simulations

Particle-based simulations are ubiquitous throughout many fields of computational science and engineering, spanning the atomistic level with molecular dynamics (MD), to mesoscale particle-in-cell (PIC) simulations for solid mechanics, device-scale modeling with PIC methods for plasma physics, and massive N-body cosmology simulations of galaxy structures, with many other methods in between (Hockney & Eastwood, 1989). While these methods use particles to represent significantly different entities with completely different physical models, many low-level details are shared including performant algorithms for short- and/or long-range particle interactions, multi-node particle communication patterns, and other data management tasks such as particle sorting and neighbor list construction. Cabana is a performance portable library for particle-based simulations, developed as part of the Co-Design Center for Particle Applications (CoPA) within the Exascale Computing Project (ECP) (Alexander et al., 2020). The CoPA project and its full development scope, including ECP partner applications, algorithm development, and similar software libraries for quantum MD, is described in (Mniszewski et al., 2021). Cabana uses the Kokkos library for on-node parallelism (Edwards et al., 2014; Trott et al., 2022), enabling simulation on multi-core CPU and GPU architectures, and MPI for GPU-aware, multi-node communication. Cabana provides particle simulation capabilities on almost all current Kokkos backends, including serial execution, OpenMP (including OpenMP-Target for GPUs), CUDA (NVIDIA GPUs), HIP (AMD GPUs), and SYCL (Intel GPUs), providing a clear path for the coming generation of accelerator-based exascale hardware. Cabana builds on Kokkos by providing new particle data structures and particle algorithms resulting in a similar execution policy-based, node-level programming model that is intended to be used in addition to the core Kokkos library within an application. Cabana is designed as an application and physics agnostic, but particle-specific toolkit which can either be used to generate a new application, or to be used as needed in existing applications at various levels of invasiveness including through interfaces that wrap user memory in existing data structures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

MILK : a Python scripting interface to MAUD for automation of Rietveld analysis

Modern diffraction experiments ( e.g. in situ parametric studies) present scientists with many diffraction patterns to analyze. Interactive analyses via graphical user interfaces tend to slow down obtaining quantitative results such as lattice parameters and phase fractions. Furthermore, Rietveld refinement strategies ( i.e. the parameter turn-on-off sequences) tend to be instrument specific or even specific to a given dataset, such that selection of strategies can become a bottleneck for efficient data analysis. Managing multi-histogram datasets such as from multi-bank neutron diffractometers or caked 2D synchrotron data presents additional challenges due to the large number of histogram-specific parameters. To overcome these challenges in the Rietveld software Material Analysis Using Diffraction ( MAUD ), the MAUD Interface Language Kit ( MILK ) is developed along with an updated text batch interface for MAUD . The open-source software MILK is computer-platform independent and is packaged as a Python library that interfaces with MAUD . Using MILK , model selection ( e.g. various texture or peak-broadening models), Rietveld parameter manipulation and distributed parallel batch computing can be performed through a high-level Python interface. A high-level interface enables analysis workflows to be easily programmed, shared and applied to large datasets, and external tools to be integrated with MAUD . Through modification to the MAUD batch interface, plot and data exports have been improved. The resulting hierarchical folders from Rietveld refinements with MILK are compatible with Cinema: Debye–Scherrer , a tool for visualizing and inspecting the results of multi-parameter analyses of large quantities of diffraction data. In this manuscript, the combined Python scripting and visualization capability of MILK is demonstrated with a quantitative texture and phase analysis of data collected at the HIPPO neutron diffractometer.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Fault Tolerant Frequent Pattern Mining

FP-Growth algorithm is a Frequent Pattern Mining (FPM) algorithm that has been extensively used to study correlations and patterns in large scale datasets. While several researchers have designed distributed memory FP-Growth algorithms, it is pivotal to consider fault tolerant FP-Growth, which can address the increasing fault rates in large scale systems. In this work, we propose a novel parallel, algorithm-level fault-tolerant FP-Growth algorithm. We leverage algorithmic properties and MPI advanced features to guarantee an O(1) space complexity, achieved by using the dataset memory space itself for checkpointing. We also propose a recovery algorithm that can use in-memory and disk-based checkpointing, though in many cases the recovery can be completed without any disk access, and incurring no memory overhead for checkpointing. We evaluate our FT algorithm on a large scale InfiniBand cluster with several large datasets using up to 2K cores. Our evaluation demonstrates excellent efficiency for checkpointing and recovery in comparison to the disk-based approach. We have also observed 20x average speed-up in comparison to Spark, establishing that a well designed algorithm can easily outperform a solution based on a general fault-tolerant programming model.

Shohdy, Sameh↗

DOE OSTI2017

Data Structures for Extreme Scale Computing

As computing problems of national importance grow, the government meets the increased demand by funding the development of ever larger systems. The overarching goal of the work supported in part by this grant is to increase efficiency of programming and performing computations on these large computing systems. In past work, we have demonstrated that some of these computations once thought to require expensive hardware designs and/or complex, special-purpose programming may be executed efficiently on low-cost commodity cluster computing systems using a general-purpose “latency-tolerant” programming framework. One important developed application of the ideas underlying this framework is graph database technology supporting social network pattern matching used by US intelligence agencies to more quickly identify potential terrorist threats. This database application has been spun out by the Pacific Northwest National Laboratory, a Department of Energy Laboratory, into a commercial start-up, Trovares Inc. We explore an alternative application of the same underlying ideas to a well-studied challenge arising in engineering: solving unstructured sparse linear equations. Solving these equations is key to predicting the behavior of large electronic circuits before they are fabricated. Predicting that behavior ahead of fabrication means that designs can optimized and errors corrected ahead of the expense of manufacture.

42 ENGINEERING↗

DOE OSTI2024

Computational Analysis of Hydraulic Efficiency of Michigan DOT Cover C

Drainage structures are used to capture stormwater runoff in streets and highways in urban environments. These drainage structures, which typically consist of catch basins with grates, inlets, or combination grates/inlets, collect stormwater runoff and discharge through buried conveyance systems. They are strategically placed for public safety in curb and gutter systems to provide efficient drainage of water from roadways and thus reduce the risk of hydroplaning. The performance of drainage structures is measured in terms of hydraulic efficiency, which is defined as the percentage of flow captured by the basin as compared to the total flow drainage to the structure. Understanding of the performance of these drainage structures helps designers properly space inlets to promote an economic design that ensures the safety of the traveling public. The current design methodology used to determine drainage structure follows guidelines established in the current Michigan Department of Transportation (MDOT) Drainage Manual (2006). The guidelines in the MDOT Drainage Manual were modeled after the Federal Highway Administration’s (FHWA) Hydraulic Engineering Circular 22 “Urban Drainage Design” (HEC-22). HEC-22 includes empirically derived equations to calculate the interception capacity of drainage structures for several commonly used grate configurations, such as the parallel bar, curved vane and tilt bar grates, which are based on a research study performed by Burgi et al. in the 1970s. MDOT uses several drainage structures to capture runoff that are detailed as Standard Plans. Many of these drainage structures utilize sinusoidal type grates that are not described in HEC-22. Physical modeling of these structures has been limited, posing the need to have them analyzed to verify their capture efficiency. Current MDOT practice is to assume a similar sized reticuline grate, as described in HEC-22, for capture efficiencies. Until recently, evaluating the hydraulic performance of drainage structures was limited to physical modeling in a hydraulics laboratory. With advances in engineering software and computing power, computational fluid dynamics (CFD) modeling has become a more cost-effective alternative. The Federal Highway Administration (FHWA) provides states the option to evaluate their drainage structures using CFD through the Transportation Pooled Fund Program. This study, “Computational Analysis of Hydraulic Efficiency of Michigan DOT Cover C,” was carried out using the pooled fund. MDOT’s Cover C was chosen as the first test candidate, given its similar sinusoidal pattern to other MDOT grates, but it is typically used for high-volume, higher speed applications. A similar version, Cover CX, is used on interstate highways but does not have traverse bars for bicycle safety. Additional grates may be considered for evaluation in the future.

42 ENGINEERING↗

DOE OSTI2024

Fusion Materials Research at Oak Ridge National Laboratory in Fiscal Year 2023

The materials science challenge of providing a suite of suitable materials to satisfy the technology to achieve fusion energy is addressed in this ORNL program. The inability of currently available materials and components to withstand the harsh fusion nuclear environment requires development of new materials, and an understanding of their response to the fusion environment. The overarching goal of the ORNL Fusion Materials program is to provide the applied materials science support and materials understanding to underpin the ongoing DOE Office of Science—Fusion Energy Sciences program, in parallel with developing the materials for fusion power systems. In this effort the program continues to be integrated both with the larger U.S. and international fusion materials communities and with the U.S. and international fusion design and technology communities. The excitement of this program comes from the priorities given to this subject in the two recent fusion reviews, by the FESAC and NAS committees. An important element of those recommendations is the support for pivoting the national R&D emphasis to the Fusion Materials and Technologies (FM&T), the long-advocated Fusion Prototypic Neutron Source, and for the Fusion Pilot Plant study that will help focus program direction and efforts. Furthermore, the surge of venture capital investment into the private fusion industry start-ups over the last few years is anticipated to help accelerate all aspects of the fusion energy development. This twelfth annual report of the ORNL (Oak Ridge National Laboratory) Fusion Reactor Materials Program summarizes the accomplishments in Fiscal Year 2023 (FY2023). The year was the first to return to full post-COVID-restriction operations, with students and international assignees no longer impacted by COVID restrictions, as in FY20-21-22. Following the pattern of planning used in this program, work for the year FY2023 focused on having the data and productivity to support a strong presence at the International Conference on Fusion Reactor Materials (ICFRM) 21, organized by Spain and occurred in October 2023. Twenty-nine ORNL-led abstracts were submitted, with all accepted. Four were invited presentations, nine contributed oral, fourteen posters, and two withdrawn due to unforeseen circumstances. Additionally, nine external abstracts with ORNL contributing authors were presented. These will be reported in the FY24 report next year.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

Computing for the DUNE Long-Baseline Neutrino Oscillation Experiment

This is a talk given at Computers in High Energy Physics in Adelaide, South Australia, Australia in November 2019. It is partially intended to explain the context of DUNE Computing for computing specialists. The DUNE collaboration consists of over 180 institutions from 33 countries. The experiment is in preparation now with commissioning of the first 10kT fiducial volume Liquid Argon TPC expected over the period 2025-2028 and a long data taking run with 4 modules expected from 2029 and beyond. An active prototyping program is already in place with a short test beam run with a 700T, 15,360 channel prototype of single-phase readout at the neutrino platform at CERN in late 2018 and tests of a similar sized dual-phase detector scheduled for mid-2019. The 2018 test beam run was a valuable live test of our computing model. The detector produced raw data at rates of up to ~2GB/s. These data were stored at full rate on tape at CERN and Fermilab and replicated at sites in the UK and Czech Republic. In total 1.2 PB of raw data from beam and cosmic triggers were produced and reconstructed during the six week test beam run. Baseline predictions for the full DUNE detector data, starting in the late 2020's are 30-60 PB of raw data per year. In contrast to traditional HEP computational problems, DUNE's Liquid Argon TPC data consist of simple but very large (many GB) 2D data objects which share many characteristics with astrophysical images. This presents opportunities to use advances in machine learning and pattern recognition as a frontier user of High Performance Computing facilities capable of massively parallel processing.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2017

FleCSPH notes

FleCSI is a compile-time con gurable framework designed to support multi-physics application development. As such, FleCSI provides a very general set of infrastructure design patterns that can be specialized and extended to suit the needs of a broad variety of solver and data requirements. FleCSI currently supports multi-dimensional mesh topology, geometry, and adjacency information, as well as n-dimensional hashed-tree data structures, graph partitioning interfaces, and dependency closures. FleCSI introduces a functional programming model with control, execution, and data abstractions that are consistent both with MPI and with state-of-the-art, task-based runtimes such as Legion and Charm++. The abstraction layer insulates developers from the underlying runtime, while allowing support for multiple runtime systems including conventional models like asynchronous MPI. The intent is to provide developers with a concrete set of user-friendly programming tools that can be used now, while allowing exibility in choosing runtime implementations and optimization that can be applied to future architectures and runtimes. FleCSI's control and execution models provide formal nomenclature for describing poorly understood concepts such as kernels and tasks. FleCSI's data model provides a low-buy-in approach that makes it an attractive option for many application projects, as developers are not locked into particular layouts or data structure representations. FleCSI currently provides a parallel but not distributed implementation of Binary, Quad and Oct-tree topology. This implementation is base on space lling curves domain decomposition, the Morton order. The current FleCSI version requires the implementation of a driver and a specialization driver. The role of the specialization driver is to provide the data distribution. This feature is not complete in FleCSI code and we provide it. The next step will be to incorporate it directly from FleCSPH to FleCSI as we reach a good level of performance. Then the driver represent the general execution of the resolution without worrying of the data locality and communications. As FleCSI is an On-Development code the structure may change in the future and we keep track of these changes in FleCSPH.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2013

Environmental Impacts of Wind Power Development on the Population Biology of Greater Prairie-Chickens

1. We investigated the impacts of wind power development on the demography, movements, and population genetics of Greater Prairie-Chickens (Tympanuchus cupido) at three sites in northcentral and eastern Kansas for a 7-year period. Only 1 of 3 sites was developed for wind power, the 201MW Meridan Way Wind Power Facility at the Smoky Hills site in northcentral Kansas. Our project report is based on population data for prairie chickens collected during a 2-year preconstruction period (2007-2008), a 3-year postconstruction period (2009-2011) and one final year of lek surveys (2012). Where relevant, we present preconstruction data from our field studies at reference sites in the northern Flint Hills (2007-2009) and southern Flint Hills (2006-2008). 2. We addressed seven potential impacts of wind power development on prairie chickens: lek attendance, mating behavior, use of breeding habitat, fecundity rates, natal dispersal, survival rates, and population numbers. Our analyses of pre- and postconstruction impacts are based on an analysis of covariance design where we modeled population performance as a function of treatment period, distance to eventual or actual site of the nearest wind turbine, and the interaction of these factors. Our demographic and movement data from the 6-year study period at the Smoky Hills site included 23 lek sites, 251 radio-marked females monitored for 287 bird-years, and 264 nesting attempts. Our genetic data were based on genotypes of 1,760 females, males and chicks that were screened with a set of 27 microsatellite markers that were optimized in the lab. 3. In our analyses of lek attendance, the annual probability of lek persistence during the preconstruction period was ~0.9. During the postconstruction period, distance to nearest turbine did not have a significant effect on the probability of lek persistence. However, the probability of lek persistence increased from 0.69 at 0 m to 0.89 at 30 km from turbines, and most abandoned lek sites were located <5 km from turbines. Probability of lek persistence was significantly related to habitat and number of males. Leks had a higher probability of persistence in grasslands than agricultural fields, and increased from ~0.2 for leks of 5 males, to >0.9 for leks of 10 or more males. Large leks in grasslands should be a higher priority for conservation. Overall, wind power development had a weak effect on the annual probability of lek persistence. 3. We used molecular methods to investigate the mating behavior of prairie chickens. The prevailing view for lek-mating grouse is that females mate once to fertilize the clutch and that conspecific nest parasitism is rare. We found evidence that females mate multiple times to fertilize the clutch (8-18% of broods, 4-38% of chicks) and will parasitize nests of other females during egg-laying (~17% of nests). Variable rates of parentage were highest in the fragmented landscapes at the Smoky Hills field site, and were lower at the Flint Hills field site. Comparisons of the pre- and postconstruction periods showed that wind energy development did not affect the mating behaviors of prairie chickens. 4. We examined use of breeding habitats by radio-marked females and conducted separate analyses for nest site selection, and movements of females not attending nests or broods. The landscape was a mix of native prairie and agricultural habitats, and nest site selection was not random because females preferred to nest in grasslands. Nests tended to be closer to turbines during the postconstruction period and there was no evidence of behavioral avoidance of turbines by females during nest site selection. Movements of females not attending nests or broods showed that females crossed the site of the wind power development at higher rates during the preconstruction period (20%) than the postconstruction period (11%), and that movements away from turbines were more frequent during the postconstruction period. Thus, wind power development appears to affect movements in breeding habitats but not nest site selection of female prairie chickens during the breeding season. 5. We tested the effects of wind power development on five components of female fecundity: timing of clutch initiation, clutch size of first nests and renests, nest survival, and hatchability of eggs. Average date of clutch initiation was 26 April, clutch size was 12.7 and 10.6 eggs for first nests and renests, probability of nest survival was low at 0.18, but egg hatchability was high at 0.79. Wind power development had no impact on reproductive effort or nesting success, and all five components of fecundity were not related to treatment period or distance to turbine. Nest survival was the main factor limiting reproductive output of female prairie chickens and most losses were due to predation. Daily nest survival was strongly related to vegetative cover at the nest. Changes to rangeland management practices that would double nesting cover from 2.5 to 5 dm would triple the probability of nest survival from 0.17 to 0.52. Grass and forb cover had weak positive effects on daily nest survival whereas shrub cover, proximity to woodlands, and recent rainfall had negative effects. Reproductive performance of prairie chickens is low in managed rangelands in northcentral Kansas and efforts to improve range conditions and reduce predator activity would aid recovery of prairie chicken populations. 6. We used molecular methods to investigate patterns of natal dispersal in prairie chickens. High rates of nest failure limited the number of young that we could sample. Direct detections of natal dispersal were limited because survival of newly hatched chicks to become adults were low and because we were unable to detect dispersal distances outside of our study area. Direct observations of natal movements were limited and were inadequate to make conclusions about the potential impacts of wind energy development on natal dispersal. Spatial correlograms of genetic distance among males at leks were a more sensitive measure of population structure, and indicated a weak effect of wind energy development on the spatial genetic structure of prairie chickens. 7. We tested the effects of wind power development on female survival with time-to-event models, and on residual body mass of males with analyses of covariance. Distance to turbine and the interaction of distance and treatment period had no effect on female survival. Contrary to predictions of negative impacts of wind power development, the probability of female was lowest during the preconstruction period (0.274) and increased significantly during the postconstruction period (0.543). Inspection of hazard functions indicated that the difference in annual survival could be attributed to a higher risk of mortality during the lekking season in the preconstruction period. We suggest that wind power development may have improved ecological conditions for prairie chickens by disrupting the foraging behavior of diurnal raptors that kill prairie chickens at lek sites. In support of this idea, raptor kills tended to be farther from turbines during the postconstruction period whereas mammalian kills were closer. Analyses of the major causes of mortality did not support our hypothesis because the odds of raptor predation were greater after development was completed. Most mortality losses of radio-marked females were due to predation, and losses to collision mortality or harvest were rare events. Low rates of natural mortality during fall and winter imply that harvest is likely to be additive mortality in prairie chickens. Wind energy development reduced the residual body mass of male Greater Prairie-Chickens at lek sites near turbines. Low values of residual body mass could have a negative impact on individual survival or fecundity rates, or may be related to predation risk and flight performance of males displaying at open lek sites. 8. We tested for impacts of wind power development on population numbers of prairie chickens with monitoring of male numbers at leks, and with genetic measures of population structure. Lek counts indicated that wind power development did not affect the population size of prairie chickens. Peak counts of males at leks were recorded the first year after construction was completed and the highest rates of population change were observed during the interval when the wind power facility was constructed. Population numbers of prairie chickens near and distant from turbines appeared to covary in parallel, probably because bird numbers were linked by dispersal movements. Estimates of population viability based on genetic diversity, effective population size and rates of population exchange did not show annual changes and were unaffected by wind development during our study. Estimates of relatedness among males at the same and different leks suggested that wind power development has either reduced dispersal rates or changed settlement patterns, leading to higher rates of relatedness among males displaying at the same lek site. 9. Greater Prairie-Chickens were not strongly affected by wind power development in Kansas. Negative impacts of wind power development included a trend for reductions in lek persistence near turbines, behavioral avoidance of turbines by females during their breeding season movements, and changes in the genetic structure of males at leks that were consistent with reduced dispersal or recruitment rates. We found no impacts of wind power development on nest site selection, female reproductive effort or nesting success, or population numbers. Positive impacts of wind power development included an increase in female survival rates. We hypothesized that the unexpected increase in female survival was related to changes in trophic interactions and disruption of the foraging behavior of raptors that kill prairie chickens at lek site. 10. Research funding for this project included a grant from the 20% Wind by 2030 Program of the Department of Energy (this final report), and grants from the Kansas Department of Wildlife, Parks and Tourism, the National Fish and Wildlife Federation, and initial funding from the National Wind Coordinating Collaborative. Research products from data collected during the preconstruction period have included five peer-reviewed research articles and two PhD dissertations at Kansas State University. Additional manuscripts are in review for possible publication in 2013-2014.

17 WIND ENERGY↗

DOE OSTI2021

Restoration Strategy for Active Distribution Systems Considering Endogenous Uncertainty in Cold Load Pickup

Cold load pickup (CLPU) phenomenon is identified as the persistent power inrush upon a sudden load pickup after an outage. Under the active distribution system (ADS) paradigm, where distributed energy resources (DERs) are extensively installed, the decreased outage duration can induce a strong interdependence between CLPU pattern and load pickup decisions. In this paper, we propose a novel modelling technique to tractably capture the decision-dependent uncertainty (DDU) inherent in the CLPU process. Subsequently, a two-stage stochastic decision-dependent service restoration (SDDSR) model is constructed, where first stage searches for the optimal switching sequences to decide step-wise network topology, and the second stage optimizes the detailed generation schedule of DERs as well as the energization of switchable loads. Further, to tackle the computational burdens introduced by mixed-integer recourse, the progressive hedging algorithm (PHA) is utilized to decompose the original model into scenario-wise subproblems that can be solved in parallel. The numerical test on modified IEEE 123-node test feeders has verified the efficiency of our proposed SDDSR model and provided fresh insights into the monetary and secure values of DDU quantification.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2019

Production Application Performance Data Streaming for System Monitoring

In this article, we present an approach to streaming collection of application performance data. Practical application performance tuning and troubleshooting in production high-performance computing (HPC) environments requires an understanding of how applications interact with the platform, including (but not limited to) parallel programming libraries such as Message Passing Interface (MPI). Several profiling and tracing tools exist that collect heavy runtime data traces either in memory (released only at application exit) or on a file system (imposing an I/O load that may interfere with the performance being measured). Although these approaches are beneficial in development stages and post-run analysis, a systemwide and low-overhead method is required to monitor deployed applications continuously. This method must be able to collect information at both the application and system levels to yield a complete performance picture. In our approach, an application profiler collects application event counters. A sampler uses an efficient inter-process communication method to periodically extract the application counters and stream them into an infrastructure for performance data collection. We implement a tool-set based on our approach and integrate it with the Lightweight Distributed Metric Service (LDMS) system, a monitoring system used on large-scale computational platforms. LDMS provides the infrastructure to create and gather streams of performance data in a low overhead manner. We demonstrate our approach using applications implemented with MPI, as it is one of the most common standards for the development of large-scale scientific applications. We utilize our tool-set to study the impact of our approach on an open source HPC application, Nalu. Our tool-set enables us to efficiently identify patterns in the behavior of the application without source-level knowledge. We leverage LDMS to collect system-level performance data and explore the correlation between the system and application events. Finally, we demonstrate how our tool-set can help detect anomalies with a low latency. We run tests on two different architectures: a system enabled with Intel Xeon Phi and another system equipped with Intel Xeon processor. Our overhead study shows our method imposes at most 0.5% CPU usage overhead on the application in realistic deployment scenarios.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

An Integrated View of Hydraulic Induced Fracture Geometry in Hydraulic Fracture Test Site 2

Until relatively recently, most of our understanding about the geometry of hydraulically induced fractures in unconventional reservoirs has been inferred by patterns observed via microseismic monitoring. Important information, such as fracture-azimuth, fracture-length, height-growth and other geometric data has been determined from microseismic “event clouds”, which in turn has been used to interpret a stimulated reservoir volume (SRV). It is important to note that relatively limited microseismic results have been acquired in conjunction with other diagnostics and validated in a consistent manner. The Hydraulic Fracture Test Site 2 (HFTS2) in the Permian Delaware Basin provides a unique opportunity to compare the frac geometry interpretations derived from multiple frac diagnostic tools and at multiple scales. The integrated view that emerges from all the frac diagnostic tools is that in HFTS2 the geometries of hydraulically induced fractures are not random and with a highly complex branching architecture. In fact, the fracs are mostly vertical, parallel plana domains, and both vertically and horizontally asymmetrical. The hydraulically induced fractures have a very consistent azimuth with a strike that matches SHmax. At the distance corresponding to the spacing between nearby wells in HFTS2, ~ 660 ft, the pattern of strain interceptions observed from Low Frequency Distributed Acoustic Sensing (LF-DAS) shows far-field dimensions that are consistent with the stage length dimensions at the stimulated well. In general, larger stages with higher number of clusters create larger number of interceptions and wider stimulated intervals than that of shorter stages with the same cluster to cluster spacing and fewer clusters. The hydraulically induced fractures also show a clear tendency to preferentially grow upwards as shown by the microseismic data, by the relative lower number of LF-DAS interceptions in nearby deeper wells and finally by the observed LF-DAS strain patterns in the vertical observation well. In the near field, the geometries of the fracture domains are also consistent. As shown by the distributed strain measurements obtained during production via the monitoring of Rayleigh Frequency Shift (RFS), each cluster has a separate, non-overlapping, frac-zone-domain that are generally centered around the locations of each perforation cluster. Overall, the hydraulically induced fractures in HFTS2 are interpreted as occurring in swarms associated with individual clusters, each with its own unique geometry but with similar predictable geometric and propagation tendencies. The comprehensive frac diagnostic program in HFTS2, the well-pad layout that permits the investigation of the Downloaded from http://onepetro.org/URTECONF/proceedings-pdf/21URTC/1-21URTC/D011S018R003/2477511/urtec-2021-5396-ms.pdf/1 by Carol Worster on 28 February 2022 URTeC 5396 2 impact that nearby producing wells has on frac geometry, together with an innovative Design of Experiment (DofE) that includes sequential-fracing, consistent cluster-to-cluster distances and few single-perforation cluster stages, has allowed us to gain a unique perspective about the geometry of the induced hydraulic fractures on this pad. Finally, the HFTS2 dataset has also provided us with a better understanding of the applicability and limitations of the different diagnostic tools.

58 GEOSCIENCES↗

DOE OSTI2014

Damsel: A Data Model Storage Library for Exascale Science

Computational science applications have been described as having one of seven motifs (the “seven dwarfs”), each having a particular pattern of computation and communication. From a storage and I/O perspective, these applications can also be grouped into a number of data model motifs describing the way data is organized and accessed during simulation, analysis, and visualization. Major storage data models developed in the 1990s, such as Network Common Data Format (netCDF) and Hierarchical Data Format (HDF) projects, created support for more complex data models. Development of both netCDF and HDF5 was influenced by multi-dimensional dataset storage requirements, but their access models and formats were designed with sequential storage in mind (e.g., a POSIX I/O model). Although these and other high-level I/O libraries have had a beneficial impact on large parallel applications, they do not always attain a high percentage of peak I/O performance due to fundamental design limitations, and they do not address the full range of current and future computational science data models. The goal of this project is to enable exascale computational science applications to interact conveniently and efficiently with storage through abstractions that match their data models. The project consists of three major activities: (1) identifying major data model motifs in computational science applications and developing representative benchmarks; (2) developing a data model storage library, called Damsel, that supports these motifs, provides efficient storage data layouts, incorporates optimizations to enable exascale operation, and is tolerant to failures; and (3) productizing Damsel and working with computational scientists to encourage adoption of this library by the scientific community. The product of this project, Damsel library, is openly available for download from http://cucis.ece.northwestern.edu/projects/DAMSEL. Several case studies and application programming interface reference are also available to assist new users to learn to use the library.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Association Mapping of Cell Wall Synthesis Regulatory Genes and Cell Wall Quality in Switchgrass

Inefficient conversion of biomass to biofuels is one of the main barriers for biofuel production from such materials. Approximately half of polysaccharides in biomass remain unused by typical biochemical conversion methods. Conversion efficiency is influenced by the composition and structure of cell walls of biomass. Grasses such as wheat, maize, and rice, as well as dedicated perennial bioenergy crops, like switchgrass, make up ~55% of biomass that can be produced in the United States. Grass cell walls have a different composition and patterning compared with dicotyledonous plants, including the well-studied model plant, Arabidopsis. This project identified genetic determinants of cell wall composition in grasses using both naturally occurring genetic variation of switchgrass and gene network reconstruction and functional assays in rice. In addition, the project linked functional data in rice and other species to switchgrass improvement efforts through curation of the most abundant class of regulators in the switchgrass genome. Characterizing natural diversity of switchgrass for variation in cell wall composition and properties, also known as quality, provides an unbiased avenue for identifying biologically viable diversity in switchgrass cell walls. To characterizing natural diversity, this project generated cell wall composition and enzymatic deconstruction data for ~450 genotypes of the Switchgrass Southern Association Collection (SSAC), a diverse collection composed of 36 switchgrass accessions from the southern U.S. distribution of switchgrass. Comparing these data with other measures of cell wall quality for the same samples demonstrated the complementary nature of the diverse characterization platforms now being used for biomass characterization. Association of the composition data with ~3.2K single nucleotide variant markers identified six significant single nucleotide variant markers co-associated with digestibility and another compositional trait. These markers might be used to select switchgrass genotypes with improved composition in breeding programs for biofuel and forage production. Because the SSAC continues to be characterized by collaborators in the bioenergy community, the data generated will be used to identify additional markers in higher resolution genotyping data to approach identifying the genes and alleles that cause natural variation in switchgrass cell wall quality. For example, these markers can be surveyed in the 2100-member Oklahoma Southern and Northern Lowland switchgrass collections that this project also characterized. An orthogonal approach to biodiversity studies, using comparative functional genomics permits systematic querying of how much regulatory information is likely to be transferable from dicots to grasses and use of accumulated functional genomics resources for better-characterized grass species, such as rice, itself a biomass source in global agriculture and in certain regions. The project generated and tested a number of specific hypotheses regarding cell wall transcription factors and enzymes of grasses. To aid identification of cell wall regulators, the project assembled a novel, highdepth and -quality gene association network using a general linearized model scoring system to combine rice gene network data. Using known or putative orthologs of Arabidopsis cell wall biosynthesis genes and regulators, the project pulled from this network a cell wall sub-network that includes 96 transcription factors. Reverse genetics of a co-ortholog of the Arabidopsis MYB61 transcription factor in rice revealed that this regulatory node has evolved the ability to regulate grass-specific cell wall synthesis enzymes. A transcription factor with such activity has not been previously characterized to our knowledge, representing a major conclusion of this work. Changes in gene expression in a protoplast-based assay demonstrated positive or negative roles in cell wall regulation for eleven other transcription factors from the rice gene network. Eight of fifteen (53%) of these have not previously been examined for this function. Some of these may represent novel grass-diverged cell wall regulators, while others are likely to have this function across angiosperms. A parallel effort of this project to expand knowledge of enzymes that have evolved to function in grass cell wall synthesis, revealed that a grass-diverged enzyme in rice, OsAT 5 , ferulates monolignols that are naturally incorporated into grass cell walls. This finding opens potential natural selection avenues for improving biomass composition for downstream processing by weak base pretreatment. Thus, this project has significantly expanded knowledge of cell wall synthesis and regulation in rice, information that can be used in reverse genetics and synthetic biology approaches to re-engineer cell walls for improved production of biofuel and high-value products. To lay the foundation for translating these results directly for switchgrass improvement, the project employed a comparative phylogenetic analysis of the major group of cell wall transcription factors that have been found to function in cell wall regulation, the R 2 R 3 MYBs. This analysis concluded that known cell wall regulators are largely conserved across switchgrass, rice, maize, poplar, and Arabidopsis. This interpretation is also largely consistent with the gene network analysis described above, though both approaches provide evidence that some co-orthologs of Arabidopsis regulators have diminished or increased in importance based on gene expression patterns. Also, several clades containing dicot cell wall regulators have expanded, consistent with the evolution of new cell wall regulators. This latter result is supported by functional analysis of the R 2 R 3 MYB protein SWAM 1 in a collaboration between this project and the DOE-funded group of Dr. S. Hazen at the University of Massachusettes. The curation of the switchgrass genome through this project provides specific targets for future engineering of switchgrass cell wall regulation and may also facilitate identification of regulators that underlie the molecular markers that are genetically linked to differences in cell wall quality. With the goal of spurring further research and technological developments in lignocellulosic biofuel production, this work has been communicated to the bioenergy and cell wall communities though various presentations and publications. To date, three manuscripts have been published, two others are near to publication, three others are in an advanced state, and two to four more are likely to be written based on analyses still in progress. In addition, project participants have presented thirteen posters and talks at regional, national, and international meetings about aspects of this project. In sum, the work supported by this funding has made and communicated significant progress in identifying the genes that grasses use for cell wall synthesis and regulation, information that will be used by project participants and others to improve the efficiency of conversion of lignocellulosic biomass to biofuels.

09 BIOMASS FUELS↗

DOE OSTI2014

Parallel-scanning tomosynthesis using a slot scanning technique: Fixed-focus reconstruction and the resulting image quality

Purpose: Parallel-scanning tomosynthesis (PS-TS) is a novel technique that fuses the slot scanning technique and the conventional tomosynthesis (TS) technique. This approach allows one to obtain long-view tomosynthesis images in addition to normally sized tomosynthesis images, even when using a system that has no linear tomographic scanning function. The reconstruction technique and an evaluation of the resulting image quality for PS-TS are described in this paper. Methods: The PS-TS image-reconstruction technique consists of several steps (1) the projection images are divided into strips, (2) the strips are stitched together to construct images corresponding to the reconstruction plane, (3) the stitched images are filtered, and (4) the filtered stitched images are back-projected. In the case of PS-TS using the fixed-focus reconstruction method (PS-TS-F), one set of stitched images is used for the reconstruction planes at all heights, thus avoiding the necessity of repeating steps (1)–(3). A physical evaluation of the image quality of PS-TS-F compared with that of the conventional linear TS was performed using a R/F table (Sonialvision safire, Shimadzu Corp., Kyoto, Japan). The tomographic plane with the best theoretical spatial resolution (the in-focus plane, IFP) was set at a height of 100 mm from the table top by adjusting the reconstruction program. First, the spatial frequency response was evaluated at heights of −100, −50, 0, 50, 100, and 150 mm from the IFP using the edge of a 0.3-mm-thick copper plate. Second, the spatial resolution at each height was visually evaluated using an x-ray test pattern (Model No. 38, PTW Freiburg, Germany). Third, the slice sensitivity at each height was evaluated via the wire method using a 0.1-mm-diameter tungsten wire. Phantom studies using a knee phantom and a whole-body phantom were also performed. Results: The spatial frequency response of PS-TS-F yielded the best results at the IFP and degraded slightly as the distance from the IFP increased. A visual evaluation of the spatial resolution using the x-ray test pattern indicated that the resolution was 1.8 lp/mm at the IFP and 1.2 lp/mm at heights of −100 and 100 mm from the IFP. The authors demonstrated that a spatial resolution of 1.2–1.8 lp/mm could be obtained within heights of 200 mm of the IFP. The slice sensitivity varied between 11.1 and 13.8 mm for heights between −50 and 100 mm, and there was no critical change in the slice sensitivity within a height range of 150 mm around the IFP. The phantom results demonstrated that tomosynthesis and long-view images could be reconstructed. Conclusions: PS-TS-F provides tomosynthesis images while using low-cost systems that have no tomographic scanning function, such as tableside-controlled universal R/F systems or universal radiographic systems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗