SEARCH · Engineering Papers
Results for “data requests”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Municipal Adaptation to Changing Curbside Demands: Findings from Semi-Structured Interviews with Ten U.S. Cities
Emerging mobility services (e.g. ride-hailing, e-commerce, micro-mobility, etc.), are generating novel and rapidly growing demands to use curbside space, with potentially large impacts on mobility, energy consumption, and related outcomes. This presents both opportunities and challenges to municipal agencies responsible for managing this interface between the road network and adjacent land uses, as legacy practices require updating. In this study, we employ a semi-structured interviewing approach to establish how municipalities are adapting to these new pressures on their curbside. We interviewed senior staff responsible for curbside policy of ten large U.S. municipalities, with populations ranging from ~250,000 to ~5,000,000 (and the majority of which are the central city of their metropolitan region). We document a trend of organizational restructuring to more formally include curbside management teams, with the majority of interviewees also reporting increased staffing. Respondents reported that operational failures at their curbside (e.g. demand in excess of capacity) have impacts on safety, capacity, and emergency vehicle mobility, with impacts highly concentrated spatially and temporally (e.g. late evenings in nightlife districts). We document a diversity of data flows between ride-hailing operators (e.g. Uber, Lyft) and municipalities, with some cities reporting obtaining types of data that other cities report not receiving, despite requesting such data. Finally, respondents consistently expressed a desire for new data streams and methodologies to help manage the curbside of the future. It is hoped that establishing the state of practice in this rapidly changing context will be of use to practitioners facing similar pressures as those of our interviewees.
BASIN-3D Data Integration for Selected ARM Data Field Campaign Report
The purpose of this data services request was to demonstrate integration of the Atmospheric Radiation Measurement (ARM) User Facility’s “met” datastreams with time series data from other earth science data sources using the BASIN-3D data synthesis software tool. BASIN-3D is an open-source Python library that enables researchers to integrate data across configured public and private data sources. It provides a common query language for researchers to request measurement locations and time series data based on specified locations, variables, time period, statistics, aggregation, and data quality. BASIN-3D acquires the data that match the query from each configured data source and translates the results into harmonized vocabularies, thus reducing researchers' data-wrangling effort. In addition, because the queries are executed on demand, researchers can easily regenerate their synthesized data sets as new data and/or data updates become available, eliminating one-off data products. BASIN-3D can output data using a variety of different data structures for end-user applications including Python pandas data frames and hdf5 output formats.
SLOPE daily and 250 m gross primary productivity (GPP) for the CONUS, 2000-2019, Carbon Monitoring System (CMS)
The SatelLite Only Photosynthesis Estimation (SLOPE) GPP product is a daily, 250 m GPP dataset covering the Contiguous United States (CONUS) from 2000 to present with 1 day latency. Gross primary productivity (GPP) quantifies the amount of carbon dioxide (CO2) fixed by plants through photosynthesis. Although as a key quantity of terrestrial ecosystems, there is a lack of high-spatial-and-temporal-resolution, real-time, and observation-based GPP products. This product has been developed to address this critical gap, leveraging a number of MODIS land and atmosphere products. There are three distinct features of the SLOPE GPP production algorithm. (1) SLOPE couples machine learning models with MODIS atmosphere and land products to accurately estimate PAR. (2) SLOPE couples highly efficient and pragmatic gap-filling and filtering algorithms with surface reflectance acquired by both Terra and Aqua MODIS satellites to derive a soil-adjusted NIRv (SANIRv) dataset. (3) SLOPE couples a temporal pattern recognition approach with a long-term Crop Data Layer (CDL) product to predict dynamic C4 crop fraction. PAR, SANIRv and C4 fraction are used to drive a parsimonious model with only two slope parameters to estimate GPP along with a quantitative uncertainty on a per-pixel and daily basis. The slope GPP product has a R2 = 0.84 and a root-mean-square error (RMSE) of 1.65 gC m-2 d-1, evaluated against from 50 AmeriFlux eddy covariance sites (332 site-years). Archived data from 2000 to 2019 are publicly available at the NASA's Oak Ridge National Laboratory Distributed Active Archive Center (ORNL DAAC). Data from 2020 are available from the authors upon request. All data are projected in the standard MODIS Land Integerized Sinusoidal tile map projection. Each processing tile is in size of 4800 pixels by 4800 pixels, representing approximately 1200 km by 1200 km land region. In addition to the GPP product, SLOPE PAR, SANIRV, and C4 fraction, along with their uncertainties, are also released.
NDI Multigroup Radiochemistry Dosimetry Library hoffman2009
We have processed the modeled neutron induced nuclear reaction cross section data from Hoffman et al. into NDI 618 lanl multigroup neutron dosimetry tables for yttrium, zirconium, and rhodium, and created corresponding radiochemistry chain files. The resulting cross sections for 87g Y(n,γ) 88g Y and 88g Y(n,2n) 87g Y compare favorably to the data in the rc2004 618 library and are similar to the data in the crsllnl library. We have released these data and chains to facilitate the direct comparison of yttrium ratios for Los Alamos and Livermore data and request feedback from our user community on the usefulness of this data before we process more data.
Collective memory transfer devices and methods for multiple-core processors
This disclosure provides systems, methods, and apparatus for collective memory transfers. A control unit may be configured to coordinate a transfer of data between a memory and processor cores. For a read data transfer operation, the control unit may receive a trigger packet identifying a read data transfer operation and identifying a first plurality of data lines based on data values included in the trigger packet. The control unit may read the first plurality of data lines from the memory sequentially and send a second plurality of data lines to the processor cores. For a write data transfer operation, the control unit may send a request for at least one data line to a plurality of processor cores, may receive and reorder the requested data lines, and may write the data lines to a memory. The control unit may determine a mapping between processor cores and the memory.
FLYING SERVING: On-the-Fly Parallelism Switching for Large Language Model Serving
Production LLM serving must simultaneously deliver high throughput, low latency, and sufficient context capacity under non-stationary traffic and mixed request requirements. Data parallelism (DP) maximizes throughput by running independent replicas, while tensor parallelism (TP) reduces per-request latency and pools memory for long-context inference. However, existing serving stacks typically commit to a static parallelism configuration at deployment; adapting to bursts, priorities, or long-context requests is often disruptive and slow. We present Flying Serving, a vLLM-based system that enables online DP-TP switching without restarting engine workers. Flying Serving makes reconfiguration practical by virtualizing the state that would otherwise force data movement: (i) a zero-copy Model Weights Manager that exposes TP shard views on demand, (ii) a KV Cache Adaptor that preserves request KV state across DP/TP layouts, (iii) an eagerly initialized Communicator Pool to amortize collective setup, and (iv) a deadlock-free scheduler that coordinates safe transitions under execution skew. Across three popular LLMs and realistic serving scenarios, Flying Serving improves performance by up to 4.79 × under high load and 3.47 × under low load while supporting latency- and memory-driven requests.
Multi-registration of software library resources
Data communications, including issuing, by an application program to a high level data communications library, a request for initialization of a data communications service; issuing to a low level data communications library a request for registration of data communications functions; registering the data communications functions, including instantiating a factory object for each of the one or more data communications functions; issuing by the application program an instruction to execute a designated data communications function; issuing, to the low level data communications library, an instruction to execute the designated data communications function, including passing to the low level data communications library a call parameter that identifies a factory object; creating with the identified factory object the data communications object that implements the data communications function according to the protocol; and executing by the low level data communications library the designated data communications function.
RISE: Reducing I/O Contention in Staging-based Extreme-Scale In-situ Workflows
While in-situ workflow formulations have addressed some of the data-related challenges associated with extreme-scale scientific workflows, these workflows involve complex interactions and different modes of data exchange. In the context of increasing system complexity, such workflows present significant resource management challenges, requiring complex cost-performance tradeoffs. This paper presents RISE, an intelligent staging-based data management middleware, which builds on the DataSpaces framework and performs intelligent scheduling of data management operations to reduce I/O contention. In RISE, data are always written immediately to local buffers to reduce the effect of the transfer impact upon application performance. RISE identifies applications’ data access patterns and moves data towards data consumers only when the network is expected to be idle, reducing the impact of asynchronous background data movement upon critical data read/write requests. Here, we experimentally demonstrate that RISE can take advantage of staging nodes to offload data during writes without degrading application data movement performance.
New Mexico Small Business Assistance Program: Management Sciences Inc. (Final Report)
Sandia National Laboratories will, as time and budget allow, perform the following tasks as part of a New Mexico Small Business Assistance (NMSBA) Program project for Management Sciences, Inc. (MSI): 1. Set up a thermal radar in Sandia's Technical Area (TA) III test bed. 2. Collect alarm data from the thermal radar and a radio frequency (RF) radar during simulated intrusion tests to estimate detection performance, based on the Department of Energy (DOE) threat definition. 3. Collect nuisance alarm data caused by weather and non-intruder-related stimuli, to estimate nuisance alarm rate performance. 4. Provide data to the requester, allowing them to process the data. 5. Provide the requester, as a stretch goal, with live radar and thermal radar sensor feeds, allowing real-time processing of the RF radar and the thermal radar. A key technical issue that will influence the success of this activity is the range accuracy of the thermal radar. If issues are encountered, Sandia will work with the requester to correct the range issues.
Data transfer acceleration
Data transfer acceleration includes receiving, by a data transfer accelerator in a first node of a plurality of nodes, from a second node of the plurality of nodes, a request for data in a second state, wherein the second node stores an instance of the data in a first state; generating a message including one or more operations to transform the data from the first state to the second state; and sending the message to the second node in response to the request.
Combined guaranteed throughput and best effort network-on-chip
A first packet-switched reservation request is received. Data associated with the first packet-switched reservation request is communicated through a first circuit-switched channel according to a best effort communication scheme. A second packet-switched reservation request is received. Data associated with the second packet-switched reservation request is communicated through a second circuit-switched channel according to a guaranteed throughput communication scheme.
Recent Experience with the CMS Data Management System
The CMS[1] experiment manages a large-scale data infrastructure, currently handling over 200 PB of disk and 500 PB of tape storage and transferring more than 1 PB of data per day on average between various WLCG[2] sites. Utilizing Rucio[3] for high-level data management, FTS[4] for data transfers, and a variety of storage and network technologies at the sites, CMS confronts inevitable challenges due to the system’s growing scale and evolving nature. Key challenges include managing transfer and storage failures, optimizing data distribution across different storages based on production and analysis needs, implementing necessary technology upgrades and migrations, and efficiently handling user requests. The data management team has established comprehensive monitoring to supervise this system and has successfully addressed many of these challenges. The team’s efforts aim to ensure data availability and protection, minimize failures and manual interventions, maximize transfer throughput and resource utilization, and provide reliable user support. This paper details the operational experience of CMS with its data management system in recent years, focusing on the encountered challenges, the effective strategies employed to overcome them and the ongoing challenges as we prepare for future demands.
Light Duty Fuel Cell Electric Vehicle Validation Data (Final Technical Report)
Electricore, along with partners from Quong & Associates, Inc., Honda R&D Americas (Honda), Nissan Technical Center North America (Nissan), and Toyota Motor Engineering & Manufacturing North America, Inc. (Toyota), participated in the Light Duty Fuel Cell Electric Vehicle (FCEV) Validation Data program sponsored by the Department of Energy (DOE) Office of Energy Efficiency and Renewable Energy (EERE) (Cooperative Agreement No. DE-EE0005968). The goal of this program was to provide real world data from the operation of past and current FCEVs, in order to measure their performance and improvements over time. The program was successful; 85% of the data fields requested were provided and not restricted due to proprietary reasons. Overall, the team from Electricore provided at least 4.8 GB of data to DOE, which was combined with data from other participants to produce over 33 key data products. These products included vehicle performance and fuel cell stack performance/durability. The data were submitted to the National Renewable Energy Laboratory’s National Fuel Cell Technology Evaluation Center (NREL NFCTEC) and combined with input from other participants. NREL then produced composite data products (CDP) which anonymized the data in order to maintain confidentiality. The results were compared with past data, which showed a measurable improvement in FCEVs over the past several years. The results were presented by NREL at the 2014 Fuel Cell Seminar, and 2014 and 2015 (planned) DOE Annual Merit Review. The project was successful. The team provided all of the data agreed upon and met all of its goals. The project finished on time and within budget. In addition, an extra $62,911 of cost sharing was provided by the Electricore team. All participants believed that the method used to collect, combine, anonymize, and present the data was technically and economically effective. This project helped EERE meet its mission of ensuring America’s security and prosperity by documenting progress in addressing energy and environmental challenges. Information from this project will be used by the hydrogen and vehicle industries to help advance the introduction of FCEVs and associated hydrogen infrastructure.
SDS: A Framework for Scientific Data Services
Large-scale scientific applications typically write their data to parallel file systems with organizations designed to achieve fast write speeds. Analysis tasks frequently read the data in a pattern that is different from the write pattern, and therefore experience poor I/O performance. In this paper, we introduce a prototype framework for bridging the performance gap between write and read stages of data access from parallel file systems. We call this framework Scientific Data Services, or SDS for short. This initial implementation of SDS focuses on reorganizing previously written files into data layouts that benefit read patterns, and transparently directs read calls to the reorganized data. SDS follows a client-server architecture. The SDS Server manages partial or full replicas of reorganized datasets and serves SDS Clients' requests for data. The current version of the SDS client library supports HDF5 programming interface for reading data. The client library intercepts HDF5 calls and transparently redirects them to the reorganized data. The SDS client library also provides a querying interface for reading part of the data based on user-specified selective criteria. We describe the design and implementation of the SDS client-server architecture, and evaluate the response time of the SDS Server and the performance benefits of SDS.
End-to-End Network QoS via Scheduling of Flexible Resource Reservation Requests
Modern data-intensive applications move vast amounts of data between multiple locations around the world. To enable predictable and reliable data transfer, next generation networks allow such applications to reserve network resources for exclusive use. In this paper, we solve an important problem (called SMR3) to accommodate multiple and concurrent network reservation requests between a pair of end-sites. Given the varying availability of bandwidth within the network, our goal is to accommodate as many reservation requests as possible while minimizing the total time needed to complete the data transfers. We first prove that SMR3 is an NP-hard problem. Then we solve it by developing a polynomial-time heuristic, called RRA. The RRA algorithm hinges on an efficient mechanism to accommodate large number of requests by minimizing the bandwidth wastage. Finally, via numerical results, we show that RRA constructs schedules that accommodate significantly larger number of requests compared to other, seemingly efficient, heuristics.
Hanford 200 West Area Flowsheet Data to Support Waste Treatment and Disposal Request for Proposal
This report documents the 200 West Area (200W) flowsheet supplemental data needed to support the Request for Proposal (RFP) to procure onsite and/or offsite treatment and disposal capabilities for West Area pretreated1 tank waste (PTW). The data provided in this document is based on the results of a 200W flowsheet model run evaluating single-shell tank (SST) retrievals for all S, SX, and U Farms, except for Tank S-112, which was retrieved in March of 2007 (HNF-EP-0182, Waste Tank Summary Report for Month End August 31, 2024). The evaluation also includes the waste inventory in double-shell tanks (DSTs) in SY Farm.
Accessing Data Federations with CVMFS
Data federations have become an increasingly common tool for large collaborations such as CMS and Atlas to efficiently distribute large data files. Unfortunately, these typically are implemented with weak namespace semantics and a non-POSIX API. On the other hand, CVMFS has provided a POSIX-compliant read-only interface for use cases with a small working set size (such as software distribution). The metadata required for the CVMFS POSIX interface is distributed through a caching hierarchy, allowing it to scale to the level of about a hundred thousand hosts. In this paper, we will describe our contributions to CVMFS that merges the data scalability of XRootD-based data federations (such as AAA) with metadata scalability and POSIX interface of CVMFS. We modified CVMFS so it can serve unmodified files without copying them to the repository server. CVMFS 2.2.0 is also able to redirect requests for data files to servers outside of the CVMFS content distribution network. Finally, we added the ability to manage authorization and authentication using security credentials such as X509 proxy certificates. We combined these modifications with the OSGs StashCache regional XRootD caching infrastructure to create a cached data distribution network. Here, we will show performance metrics accessing the data federation through CVMFS compared to direct data federation access. Additionally, we will discuss the improved user experience of providing access to a data federation through a POSIX filesystem.