Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA STORAGE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,729 records · Page 96

The Kepler Science Operations Center Pipeline Framework Extensions

The Kepler Science Operations Center (SOC) is responsible for several aspects of the Kepler Mission, including managing targets, generating on-board data compression tables, monitoring photometer health and status, processing the science data, and exporting the pipeline products to the mission archive. We describe how the generic pipeline framework software developed for Kepler is extended to achieve these goals, including pipeline configurations for processing science data and other support roles, and custom unit of work generators that control how the Kepler data are partitioned and distributed across the computing cluster. We describe the interface between the Java software that manages the retrieval and storage of the data for a given unit of work and the MATLAB algorithms that process these data. The data for each unit of work are packaged into a single file that contains everything needed by the science algorithms, allowing these files to be used to debug and evolve the algorithms offline.

Klaus, Todd C.↗

NASA POWER: Providing Present and Future Climate Services Based on NASA Data for the Energy, Agricultural, and Sustainable Buildings Communities

NASA’s Prediction of Worldwide Energy Resource (POWER) project facilitates the use of NASA Earth Science data holdings within the renewable energy, agricultural, and building heating/cooling design industries. POWER packages solar and meteorological data at various temporal levels from several NASA projects in a user friendly GIS-enabled web services system (https://power.larc.nasa.gov). Data users can access these data either through an intuitive data viewer, image services fully integrable with GIS analysis, connections in the cloud through an Amazon Web Services S3 Bucket, or fully customizable access through an API. Data provided by POWER has been successfully used by decision makers to support actions that address climate change. For example, POWER data has been used to remotely monitor solar array fields and integrated in a sizing tool for off-grid solar and storage systems. POWER data has also been coupled with key building decision tools to support design and retrofitting of building energy systems for energy efficiency and reduction of greenhouse gases. POWER is now developing climate services to provide time series of the projected future evolution of key quantities that interest our users, such as heating/cooling degree days, temperature, wind speed, and downwelling solar flux. We demonstrate the potential of the new climate services by presenting here a range of possible future changes in these quantities at different NASA centers across the continental United States. These data services are based on downscaled climate model data from the NASA Earth Exchange Global Daily Downscaled Projections (NEX-GDDP) data set. We highlight the important insights that new climate services can provide. Our climate services will help our user communities quantify the impacts of climate change to support their key decisions in planning for the future, both inside and outside the Federal Government, especially for decisions in renewable energy and in building heating and cooling.

Bradley Hegyi↗

Plans for the extreme ultraviolet explorer data base

The paper presents an approach for storage and fast access to data that will be obtained by the Extreme Ultraviolet Explorer (EUVE), a satellite payload scheduled for launch in 1991. The EUVE telescopes will be operated remotely from the EUVE Science Operation Center (SOC) located at the University of California, Berkeley. The EUVE science payload consists of three scanning telescope carrying out an all-sky survey in the 80-800 A spectral region and a Deep Survey/Spectrometer telescope performing a deep survey in the 80-250 A spectral region. Guest Observers will remotely access the EUVE spectrometer database at the SOC. The EUVE database will consist of about 2 X 10 to the 10th bytes of information in a very compact form, very similar to the raw telemetry data. A history file will be built concurrently giving telescope parameters, command history, attitude summaries, engineering summaries, anomalous events, and ephemeris summaries.

Marshall, Herman L.↗

Initial Mobility Analysis for ORNL VA-EDH Synthetic Populations

Travel burdens are a major barrier to healthcare access among US Veteran patient populations, particularly those residing in rural areas. Spatial accessibility to points of care for US Veteran populations is commonly assessed in two ways. The first approach uses open data from the US Census to represent collective travel burdens, for example the distance between population-weighted census tract centroids and VHA points of care. The second approach uses restricted-access VHA patient data to measure travel costs (e.g., distance, time) for accessing points of care with respect to geolocated patient addresses and real or approximated transportation networks. While the advantage of the open data approach lies in its reproducibility, it has notable limitations in its tendency to infer individual travel behavior from aggregate population characteristics, a problem known as ecological fallacy. Conversely, while the patient data approach is able to account for individual travel behavior, its ability to account for localized access disparities (e.g., a neighborhood with exceptionally high transportation costs) and patient demographics is limited as protecting individual patient data requires their storage in closed systems with limited capacity for adequately modeling real-world travel patterns or for supplementing patient attributes. Additionally, the patient data approach cannot account for veterans who are not enrolled in the VHA system but who may be eligible for care. These challenges limit the ability to perform “what if” analyses on the effects of place-specific interventions on veteran populations with high access barriers to healthcare. To address these challenges, we explore the application of realistic synthetic populations to examine travel burdens and spatial accessibility issues among veteran patient populations. Synthetic populations provide a virtual, individually-resolved and cross-sectional representation of the veteran patient population that enables investigation of spatial access to points of care in ways in which aggregate data and patient data do not. First, synthetic populations allow one to directly assess how individuals access points of care, from synthesized residential locations to outpatient facilities on real-world transportation networks. Modeling access to points of care at the individual scale addresses the ecological fallacy problem associated with using aggregated census data to represent veteran populations and patterns of movement. Second, synthetic populations provide a means of completely representing an area’s veteran population using only publicly available, anonymized census microdata from the American Community Survey (ACS) to ensure the privacy of real-world individuals. Generating synthetic populations from the ACS also expands descriptive characteristics beyond what patient data typically offers to include socio-demographic, economic, housing, and mobility attributes. More detailed profiles of both VHA patient populations and veterans not enrolled in the VA system will provide a comprehensive picture of groups that may benefit from interventions or outreach. As an initial exercise for using synthetic populations to measure veteran travel burdens to VA care, we apply Oak Ridge National Laboratory’s (ORNL) UrbanPop capability to generate a series of synthetic VHA patient populations for 9 Veterans Integrated Services Networks (VISN) market areas in 9 Census Divisions across the continental United States, which are listed in Table 1. We use UrbanPop to produce synthetic populations for the VISN markets selected for each US Census Division, then assign VA outpatient clinic destinations to synthetic VHA patients based on travel about each VISN market’s road network. To demonstrate using the synthetic populations to evaluate healthcare travel burdens, we compare the time-based impedance between simulated home locations and VA outpatient clinics in each VISN market. We then perform validation exercises on the synthetic populations with respect to neighborhood (block group) demographic composition as well as patient mobility, comparing aggregate origin-destination statistics for the synthetic population to outpatient visits available in restricted patient data from the VA’s Corporate Data Warehouse (CDW) database.

97 MATHEMATICS AND COMPUTING↗

Arboreal Epiphytes in the Soil-Atmosphere Interface: How Often Are the Biggest “Buckets” in the Canopy Empty?

Arboreal epiphytes (plants residing in forest canopies) are present across all major climate zones and play important roles in forest biogeochemistry. The substantial water storage capacity per unit area of the epiphyte “bucket” is a key attribute underlying their capability to influence forest hydrological processes and their related mass and energy flows. It is commonly assumed that the epiphyte bucket remains saturated, or near-saturated, most of the time; thus, epiphytes (particularly vascular epiphytes) can store little precipitation, limiting their impact on the forest canopy water budget. We present evidence that contradicts this common assumption from (i) an examination of past research; (ii) new datasets on vascular epiphyte and epi-soil water relations at a tropical montane cloud forest (Monteverde, Costa Rica); and (iii) a global evaluation of non-vascular epiphyte saturation state using a process-based vegetation model, LiBry. All analyses found that the external and internal water storage capacity of epiphyte communities is highly dynamic and frequently available to intercept precipitation. Globally, non-vascular epiphytes spend <20% of their time near saturation and regionally, including the humid tropics, model results found that non-vascular epiphytes spend ~1/3 of their time in the dry state (0–10% of water storage capacity). Even data from Costa Rican cloud forest sites found the epiphyte community was saturated only 1/3 of the time and that internal leaf water storage was temporally dynamic enough to aid in precipitation interception. Analysis of the epi-soils associated with epiphytes further revealed the extent to which the epiphyte bucket emptied—as even the canopy soils were often <50% saturated (29–53% of all days observed). Results clearly show that the epiphyte bucket is more dynamic than currently assumed, meriting further research on epiphyte roles in precipitation interception, redistribution to the surface and chemical composition of “net” precipitation waters reaching the surface.

Hargis, Hailey↗

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗

Serial data correlator/code translator

A system for analyzing asynchronous signals containing bits of information for ensuring the validity of said signals, by sampling each bit of information a plurality of times, and feeding the sampled pieces of bits of information into a sequence controlled is described. The sequence controller has a plurality of maps or programs through which the sampled pieces of bits are stepped so as to identify the particular bit of information and determine the validity and phase of the bit. The step in which the sequence controller is clocked is controlled by a storage register. A data decoder decodes the information fed out of the storage register and feeds such information to shift registers for storage.

Morgan, L. E.↗

h5bench: A unified benchmark suite for evaluating HDF5 I/O performance on pre‐exascale platforms

Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.

97 MATHEMATICS AND COMPUTING↗

File-Based Data Flow in the CMS Filter Farm

During the LHC Long Shutdown 1, the CMS Data Acquisition system underwent a partial redesign to replace obsolete network equipment, use more homogeneous switching technologies, and prepare the ground for future upgrades of the detector front-ends. The software and hardware infrastructure to provide input, execute the High Level Trigger (HLT) algorithms and deal with output data transport and storage has also been redesigned to be completely file- based. This approach provides additional decoupling between the HLT algorithms and the input and output data flow. All the metadata needed for bookkeeping of the data flow and the HLT process lifetimes are also generated in the form of small “documents” using the JSON encoding, by either services in the flow of the HLT execution (for rates etc.) or watchdog processes. These “files” can remain memory-resident or be written to disk if they are to be used in another part of the system (e.g. for aggregation of output data). We discuss how this redesign improves the robustness and flexibility of the CMS DAQ and the performance of the system currently being commissioned for the LHC Run 2.

Andre, J. M.↗

Autoresonant excitation of nonlinear beam motion in storage rings

Understanding and controlling of nonlinear beam dynamics are essential to the performance of storage ring light sources. In order to measure and correct limitations due to nonlinear beam dynamics, an effective method is needed to excite the beam oscillation to large amplitudes. Autoresonance is a method that enables the control of the amplitude of a driven nonlinear oscillator by sweeping the frequency of the driver. In this study, autoresonant excitation of beam transverse oscillations in storage rings is studied. The threshold of the drive amplitude for autoresonance is theoretically obtained, for cases with or without damping. The results are in good agreement with simulations for a simple storage ring model, as well as for models of actual storage rings. Application of the theory to experimental data on the SPEAR3 storage ring also validates the results

43 PARTICLE ACCELERATORS↗

Detection of Synchrophasor False Data Injection Attack using Feature Interactive Network

The synchrophasor data recorded by Phasor Measurement Units (PMUs) plays an increasingly critical role in the regulation and situational awareness of power systems. However, the widely installed PMUs are vulnerable to multiple malicious attacks from cyber hackers during data transmission and storage. To address this problem, a Modified Ensemble Empirical Mode Decomposition (MEEMD) is proposed first to extract the intrinsic mode functions of each Synchrophasor Data Attacks (SDA). The frequency-based adaptive screening criterion embedded in MEEMD is used to eliminate the false intrinsic mode functions. Next, a Multivariate Convolutional Neural Network (MCNN) is proposed to identify multiple SDA by utilizing the extracted intrinsic mode functions and original SDA as input vectors. A fusion block as the main structure of MCNN is also leveraged to increase the diversity of features and compress the model parameters. Integrating MEEMD and MCNN, a framework with automatic feature extraction and multi-source information fusion capability, referred to as Feature Interactive Network (FIN), is proposed to detect multiple SDA. Based on the proposed FIN framework, six types of SDA are explored for the first time using actual synchrophasor data in FNET/Grideye that was collected from different locations in the U.S. Eastern Interconnection. Finally, a large quantity of experiments with different attack strengths are used to evaluate the adaptability and classification performance of the proposed FIN.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Data acquisition system for the Baikal-GVD neutrino telescope

The objective of the Baikal-GVD project is the construction of a km{sup 3}-scale neutrino telescope in Lake Baikal. The Gigaton Volume Detector consists of a large three-dimensional array of photo-multiplier tubes. The first GVD-cluster has been deployed and commissioned in April 2015. The data acquisition system (DAQ) of the detector takes care of the digitization of the photo-multiplier tube signals, data transmission, filtering and storage. The design and the implementation of the data acquisition system are described.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Ultrafast Error-bounded Lossy Compression for Scientific Datasets

Today's scientific high-performance computing applications and advanced instruments are producing vast volumes of data across a wide range of domains, which impose a serious burden on data transfer and storage. Error-bounded lossy compression has been developed and widely used in the scientific community because it not only can significantly reduce the data volumes but also can strictly control the data distortion based on the user-specified error bound. Existing lossy compressors, however, cannot offer ultrafast compression speed, which is highly demanded by numerous applications or use cases (such as in-memory compression and online instrument data compression). In this paper, we propose a novel ultrafast error-bounded lossy compressor that can obtain fairly high compression performance on both CPUs and GPUs and with reasonably high compression ratios. The key contributions are threefold. (1) We propose a generic error-bounded lossy compression framework---called SZx---that achieves ultrafast performance through its novel design comprising only lightweight operations such as bitwise and addition/subtraction operations, while still keeping a high compression ratio. (2) We implement SZx on both CPUs and GPUs and optimize the performance according to their architectures. (3) We perform a comprehensive evaluation with six real-world production-level scientific datasets on both CPUs and GPUs. Experiments show that SZx is 2~16x faster than the second-fastest existing error-bounded lossy compressor (either SZ or ZFP) on CPUs and GPUs, with respect to both compression and decompression.

Yu, Xiaodong↗

DMS Capture Task Manager

The DMS Capture Task Manager is a part of PRISM, the Proteomics Research Information and Management System. The capture manager processes instrument data files, copying them from the instrument to storage servers, validating the data, creating QC graphics, and pushing the data into MyEMSL. Plugin DLLs implement the processing details for each tool.

Gibbons, Bryson↗

ATL Products Division's entries into the computer mass storage marketplace

The viewgraphs of a discussion on ATL Products Division's entries into the computer mass storage marketplace presented at the National Space Science Data Center (NSSDC) Mass Storage Workshop is included. Topics covered are product evolution including robotics; aperture storage module library; Broadcast Division's TCS2000 Video Cart and TCS90 Videocart System; high density systems business product lines; and storage and library management.

Zeiler, Fred↗

Data management and scientific integration within the Atmospheric Radiation Measurement Program

The Atmospheric Radiation Measurement (ARM) Program has been developed by the U.S. Department of Energy with the goal to improve the predictive capabilities of General Circulation Models (GCM's) in their treatment of clouds and radiative transfer effects. To achieve this goal, three experimental testbeds were designed for the deployment of instruments that will collect atmospheric data used to drive the GCM's. Each site, known as a Cloud and Radiation Testbed (CART), consists of a highly available, redundant data system for the collection of data from a variety of instrumentation. The first CART site was deployed in April 1992 in the Southern Great Plains (SGP), Lamont, Oklahoma, with the other two sites to follow in September 1995 in the Tropical Western Pacific and in 1997 on the North Slope of Alaska. Approximately 400 MB of data are transferred per day via the Internet from the SGP site to the ARM Experiment Center at Pacific Northwest Laboratory in Richland, Washington. The Experiment Center is central to the ARM data path and provides for the collection, processing, analysis, and delivery of ARM data. Data are received from the CART sites from a variety of instrumentation, observational systems, amd external data sources. The Experiment Center processes these data streams on a continuous basis to provide derived data products to the ARM Science Team in near real-time while providing a three-month running archive of data. A primary requirement of the ARM Program is to preserve and protect all data produced or acquired. This function is performed at Oak Ridge National Laboratory where leading edge technology is employed for the long-term storage of ARM data. The ARM Archive provides access to data for participation outside of the ARM Program. The ARM Program involves a collaborative effort by teams from various DOE National Laboratories, providing multi-disciplinary areas of expertise. This paper will discuss the collaborative methods in which the ARM teams translate the scientific goals of the Program into data products. By combining atmospheric scientists, systems engineers, and software engineers, the ARM Program has successfully designed and developed an environment where advances in understanding the parameterizations of GCM's can be made.

Gracio, Deborah K.↗

HPC s Pivot to Data

Computer centers such as NERSC and OLCF have traditionally focused on delivering computational capability that enables breakthrough innovation in a wide range of science domains. Accessing that computational power has required services and tools to move the data from input and output to computation and storage. A ''pivot to data'' is occurring in HPC. Data transfer tools and services that were previously peripheral are becoming integral to scientific workflows. Emerging requirements from high-bandwidth detectors, high-throughput screening techniques, highly concur- rent simulations, increased focus on uncertainty quantification, and an emerging open-data policy posture toward published research are among the data-drivers shaping the networks, file systems, databases, and overall compute and data environment. In this paper we explain the pivot to data in HPC through user requirements and the changing resources provided by HPC with particular focus on data movement. For WAN data transfers we present the results of a study of network performance between centers.

97 MATHEMATICS AND COMPUTING↗

Archiving and retrieval of data from the International Ultraviolet Explorer /IUE/ mission

The data base of IUE spectra available for dissemination to the scientific community presently contains approximately 22,000 separate images. Due to storage space and computer processing time limitations, it was decided to store the data on high density (6250 bpi) specially formatted tapes and to retrieve and reconvert them to the originally supplied format upon request. The archival/retrieval system for the data and an automatic data entry system for creation and maintenance of an indexing data base are described. Suggestion are made for the use of standard formats for future image data storage, retrieval and dissemination.

Warren, W. H., Jr.↗