Defining Metrics to Distill Large-Scale HPC Platform and Application Performance Data into Actionable Quantities ? Resource Contention.
Abstract not provided.
Agelastos, Anthony Michael↗
SEARCH · Engineering Papers
Results for “resource data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Defining Metrics to Distill Large-Scale HPC Platform and Application Performance Data into Actionable Quantities ? Resource Contention of File System and Aries Interconnect.
Abstract not provided.
Agelastos, Anthony Michael↗
Defining Metrics to Distill Large-Scale HPC Platform and Application Performance Data into Actionable Quantities ? Resource Contention of File System and Aries Interconnect.
Abstract not provided.
Agelastos, Anthony Michael↗
Feasibility of Tidal and Ocean Current Energy in False Pass, Aleutian Islands, Alaska final report
The Aleutian Pribilof Islands Association was awarded a U.S. Department of Energy Tribal Energy Program grant (DE-EE0005624) for the Feasibility of Tidal and Ocean Current Energy in False Pass, Aleutian Islands, Alaska (Project). The goal of the Project was to perform a feasibility study to determine if a tidal energy project would be a viable means to generate electricity and heat to meet long-term fossil fuel use reduction goals, specifically to produce at least 30% of the electrical and heating needs of the tribally-owned buildings in False Pass. The Project Team included the Aleut Region organizations comprised of the Aleutian Pribilof Island Association (APIA), and Aleutian Pribilof Island Community Development Association (APICDA); the University of Alaska Anchorage, ORPC Alaska a wholly-owned subsidiary of Ocean Renewable Power Company (ORPC), City of False Pass, Benthic GeoScience, and the National Renewable Energy Laboratory (NREL). The following Project objectives were completed: collected existing bathymetric, tidal, and ocean current data to develop a basic model of current circulation at False Pass, measured current velocities at two sites for a full lunar cycle to establish the viability of the current resource, collected data on transmission infrastructure, electrical loads, and electrical generation at False Pass, performed economic analysis based on current costs of energy and amount of energy anticipated from and costs associated with the tidal energy project conceptual design and scoped environmental issues. Utilizing circulation modeling, the Project Team identified two target sites with strong potential for robust tidal energy resources in Isanotski Strait and another nearer the City of False Pass. In addition, the Project Team completed a survey of the electrical infrastructure, which identified likely sites of interconnection and clarified required transmission distances from the tidal energy resources. Based on resource and electrical data, the Project Team developed a conceptual tidal energy project design utilizing ORPC’s TidGen® Power System. While the Project Team has not committed to ORPC technology for future development of a False Pass project, this conceptual design was critical to informing the Project’s economic analysis. The results showed that power from a tidal energy project could be provided to the City of False at a rate at or below the cost of diesel generated electricity and sold to commercial customers at rates competitive with current market rates, providing a stable, flat priced, environmentally sound alternative to the diesel generation currently utilized for energy in the community. The Project Team concluded that with additional grants and private investment a tidal energy project at False Pass is well-positioned to be the first tidal energy project to be developed in Alaska, and the first tidal energy project to be interconnected to an isolated micro grid in the world. A viable project will be a model for similar projects in coastal Alaska.
16 TIDAL AND WAVE POWER↗
Data processing requirements for the Earth Resources Technology Satellite (ERTS)
Some of the data handling considerations that led to the formulation of the data processing requirements for the ERTS system are discussed. The significant parameters that influence these requirements are: (1) payload characteristics, (2) spacecraft characteristics such as orbit, attitude and attitude rates, (3) payload coverage, and (4) product types and quantities. Items 3 and 4 have had the most significant influence on the generation of the requirements.
Sos, J. Y.↗
Geographic analysis of multiple sensor data from the NASA/USGS earth resources program
Qualitative and quantitative analyses were made of multi-sensor data acquired during aircraft missions. While the principal analysis effort was concentrated on imagery taken over test sites in Southern California, data were also studied from records acquired on missions over test sites at Phoenix, Chicago, Asheville, and New Orleans. The objectives of the analyses were: (1) to determine the capabilities of ten remote sensors in identifying the elements of information necessary in conducting geographic investigations in land use analysis, urban problems, surface energy budget, and soil moisture; (2) to determine the feasibility of using these sensors for these purposes at orbital altitudes; and (3) to collate and analyze ground and air data previously collected and assemble it in a format useful in the accomplishment of cost effectiveness studies.
Pascucci, R. F.↗
Enforcing End-to-end I/O Policies for Scientific Workflows using Software-Defined Storage Resource Enclaves
Data-intensive knowledge discovery requires scientific applications to run concurrently with analytics and visualization codes executing in situ for timely output inspection and knowledge extraction. Consequently, I/O pipelines of scientific workflows can be long and complex because they comprise many stages of analytics across different layers of the I/O stack of high-performance computing systems. Performance limitations at any I/O layer or stage can cause an I/O bottleneck resulting in greater than expected end-to-end I/O latency. In this paper, we present the design and implementation of a novel data management infrastructure called Software-Defined Storage Resource Enclaves (SIREN) at system level to enforce end-to-end policies that dictate an I/O pipeline's performance. SIREN provides an I/O performance interface for users to specify the desired storage resources in the context of in-situ analytics. If suboptimal performance of analytics is caused by an I/O bottleneck when data are transferred between simulations and analytics, schedulers in different layers of the I/O stack automatically provide the guaranteed lower bounds on I/O throughput. Lastly, our experimental results demonstrate that SIREN provides performance isolation among scientific workflows sharing multiple storage servers across two I/O layers (burst buffer and parallel file systems) while maintaining high system scalability and resource utilization.
97 MATHEMATICS AND COMPUTING↗
Low Latency DESDynI Data Products for Disaster Response, Resource Management and Other Applications
We are developing onboard processor technology targeted at the L-band SAR instrument onboard the planned DESDynI mission to enable formation of SAR images onboard opening possibilities for near-real-time data products to augment full data streams. Several image processing and/or interpretation techniques are being explored as possible direct-broadcast products for use by agencies in need of low-latency data, responsible for disaster mitigation and assessment, resource management, agricultural development, shipping, etc. Data collected through UAVSAR (L-band) serves as surrogate to the future DESDynI instrument. We have explored surface water extent as a tool for flooding response, and disturbance images on polarimetric backscatter of repeat pass imagery potentially useful for structural collapse (earthquake), mud/land/debris-slides etc. We have also explored building vegetation and snow/ice classifiers, via support vector machines utilizing quad-pol backscatter, cross-pol phase, and a number of derivatives (radar vegetation index, dielectric estimates, etc.). We share our qualitative and quantitative results thus far.
applications↗
ELAS - A geobased information system that is transferable to several computers
In the early years of remote sensing, emphasis was placed on the processing and analysis of data from a single multispectral sensor, such as the Landsat Multispectral Scanner System (MSS). However, in connection with attempts to use the data for resource management, it was realized that many deficiencies existed in single data sets. A need was established to geographically reference the MSS data and to register with it data from disparate sources. Technological transfer activities have required systems concepts that can be easily transferred to computers of different types in other organizations. ELAS (Earth Resources Laboratory Applications Software), a geographically based information system, was developed to meet the considered needs. ELAS accepts data from a variety of sources. It contains programs to geographically reference the data to the Universal Transverse Mercator grid. One of the primary functions of ELAS is to produce a surface cover map.
Whitley, S. L.↗
GPU Lossless Hyperspectral Data Compression System
Hyperspectral imaging systems onboard aircraft or spacecraft can acquire large amounts of data, putting a strain on limited downlink and storage resources. Onboard data compression can mitigate this problem but may require a system capable of a high throughput. In order to achieve a high throughput with a software compressor, a graphics processing unit (GPU) implementation of a compressor was developed targeting the current state-of-the-art GPUs from NVIDIA(R). The implementation is based on the fast lossless (FL) compression algorithm reported in "Fast Lossless Compression of Multispectral-Image Data" (NPO- 42517), NASA Tech Briefs, Vol. 30, No. 8 (August 2006), page 26, which operates on hyperspectral data and achieves excellent compression performance while having low complexity. The FL compressor uses an adaptive filtering method and achieves state-of-the-art performance in both compression effectiveness and low complexity. The new Consultative Committee for Space Data Systems (CCSDS) Standard for Lossless Multispectral & Hyperspectral image compression (CCSDS 123) is based on the FL compressor. The software makes use of the highly-parallel processing capability of GPUs to achieve a throughput at least six times higher than that of a software implementation running on a single-core CPU. This implementation provides a practical real-time solution for compression of data from airborne hyperspectral instruments.
Aranki, Nazeeh I.↗
Data communications in a parallel active messaging interface of a parallel computer
Data communications in a parallel active messaging interface (`PAMI`) of a parallel computer composed of compute nodes that execute a parallel application, each compute node including application processors that execute the parallel application and at least one management processor dedicated to gathering information regarding data communications. The PAMI is composed of data communications endpoints, each endpoint composed of a specification of data communications parameters for a thread of execution on a compute node, including specifications of a client, a context, and a task, the compute nodes and the endpoints coupled for data communications through the PAMI and through data communications resources. Embodiments function by gathering call site statistics describing data communications resulting from execution of data communications instructions and identifying in dependence upon the call cite statistics a data communications algorithm for use in executing a data communications instruction at a call site in the parallel application.
97 MATHEMATICS AND COMPUTING↗
Resource Discovery for Extreme Scale Collaboration (RDESC) Final Report - RPI/TWC - Year 3
The amount of data produced in the practice of science is growing rapidly. Despite the accumulation and demand for scientific data, relatively little is actually made available for the broader scientific community. We surmise that the root of the problem is the perceived difficulty to electronically publish scientific data and associated metadata in a way that makes it discoverable. We propose to exploit Semantic Web technologies and practices to make (meta)data discoverable and easy to publish. We share our experiences in curating metadata to illustrate both the flexibility of our approach and the pain of discovering data in the current research environment. We also make recommendations by concrete example of how data publishers can provide their (meta)data by adding some limited, additional markup to HTML pages on the Web. With little additional effort from data publishers, the difficulty of data discovery/access/sharing can be greatly reduced and the impact of research data greatly enhanced.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
Mission 73 - Summary and data catalog
Earth resources program geographic applications program summary, recommendations, and data catalog for remote sensor mission 73
North, G. W.↗
Improved LANDSAT to give better view of earth resources
The launch data of LANDSAT 3 is announced. The improved capability of the spacecrafts' remote sensors (the return beam vidicon and the multispectral scanner) and application of LANDSAT data to the study of energy supplies, food production, and global large-scale environmental monitoring are discussed along with the piggyback amateur radio communication satellite-OSCAR-D, the plasma Interaction Experiment, and the data collection system onboard LANDSAT 3. An assessment of the utility of LANDSAT multispectral data is given based on the research results to data from studies of LANDSAT 1 and 2 data. Areas studied include agriculture, rangelands, forestry, water resources, environmental and marine resources, environmental and marine resources, cartography, land use, demography, and geological surveys and mineral/petroleum exploration.
Source record↗
PvaPy streaming framework for real-time data processing
User facility upgrades, new measurement techniques, advances in data analysis algorithms as well as advances in detector capabilities result in an increasing amount of data collected at X-ray beamlines. Some of these data must be analyzed and reconstructed on demand to help execute experiments dynamically and modify them in real time. In turn, this requires a computing framework for real-time processing capable of moving data quickly from the detector to local or remote computing resources, processing data, and returning results to users. In this paper, we discuss the streaming framework built on top of PvaPy, a Python API for the EPICS pvAccess protocol. We describe the framework architecture and capabilities, and discuss scientific use cases and applications that benefit from streaming workflows implemented on top of this framework. We also illustrate the framework's performance in terms of achievable data-processing rates for various detector image sizes.
EPICS pvAccess↗
Performability modeling based on real data: A casestudy
Described is a measurement-based performability model based on error and resource usage data collected on a multiprocessor system. A method for identifying the model structure is introduced and the resulting model is validated against real data. Model development from the collection of raw data to the estimation of the expected reward is described. Both normal and error behavior of the system are characterized. The measured data show that the holding times in key operational and error states are not simple exponentials and that a semi-Markov process is necessary to model the system behavior. A reward function, based on the service rate and the error rate in each state, is then defined in order to estimate the performability of the system and to depict the cost of different types of errors.
Hsueh, M. C.↗
Prospective CO 2 saline resource estimation methodology: Refinement of existing US-DOE-NETL methods based on data availability
Carbon storage resource estimation in subsurface saline formations plays an important role in establishing the scale of carbon capture and storage activities for governmental policy and commercial project decision-making. Prospective CO 2 resource estimation of large regions or subregions, such as a basin, occurs at the initial screening stages of a project using only limited publicly available geophysical data, i.e. prior to project-specific site selection data generation. As the scale of investigation is narrowed and selected areas and formations are identified, prospective CO 2 resource estimation can be refined and uncertainty narrowed when site-specific geophysical data are available. Here, we refine the United States Department of Energy – National Energy Technology Laboratory (US-DOE-NETL) methodology as the scale of investigation is narrowed from very large regional assessments down to selected areas and formations that may be developed for commercial storage. In addition, we present a new notation that explicitly identifies differences between data availability and data sources used for geologic parameters and efficiency factors as the scale of investigation is narrowed. This CO 2 resource estimation method is available for screening formations in a tool called CO 2 -SCREEN.
54 ENVIRONMENTAL SCIENCES↗
DUNE Database Development
The DUNE experiment will produce vast amounts of metadata, which describe the data coming from the read-out of the primary DUNE detectors. Various databases will make up the overall DB architecture for this metadata. ProtoDUNE at CERN is the largest existing prototype for DUNE and serves as a testing ground for - among other things - possible database solutions for DUNE. The subset of all metadata that is accessed during offline data reconstruction and analysis is referred to as ‘conditions data’ and it is stored in a dedicated database. As offline data reconstruction and analysis will be deployed on HTC and HPC resources, conditions data is expected to be accessed at very high rates. It is therefore crucial to store it in a granularity that matches the expected access patterns allowing for extensive caching. This requires a good understanding of the sources and use cases of conditions data. This contribution will briefly summarize the database architecture deployed at ProtoDUNE and explain the various sources of conditions data. We will present how the conditions data is retrieved and streamed from the databases and how it is handled to match expected access patterns.
Vizcaya Hernandez, Ana Paula↗