Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59

Neutron Science TeraGrid Gateway

The unique contributions of the Neutron Science TeraGrid Gateway (NSTG) are the connection of national user facility instrument data sources to the integrated cyberinfrastructure of the National Science FoundationTeraGrid and the development of a neutron science gateway that allows neutron scientists to use TeraGrid resources to analyze their data, including comparison of experiment with simulation. The NSTG is working in close collaboration with the Spallation Neutron Source (SNS) at Oak Ridge as their principal facility partner. The SNS is a next-generation neutron source. It has completed construction at a cost of $1.4 billion and is ramping up operations. The SNS will provide an order of magnitude greater flux than any previous facility in the world and will be available to all of the nation's scientists, independent of funding source, on a peer-reviewed merit basis. With this new capability, the neutron science community is facing orders of magnitude larger data sets and is at a critical point for data analysis and simulation. There is a recognized need for new ways to manage and analyze data to optimize both beam time and scientific output. The TeraGrid is providing new capabilities in the gateway for simulations using McStas and a fitting service on distributed TeraGrid resources to improved turnaround. NSTG staff are also exploring replicating experimental data in archival storage. As part of the SNS partnership, the NSTG provides access to gateway support, cyberinfrastructure outreach, community development, and user support for the neutron science community. This community includes not only SNS staff and users but extends to all the major worldwide neutron scattering centers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Accessing Wi-Fi Data for Occupancy Sensing

A key issue for saving energy in buildings is to assure that delivery of energy services matches building occupancy as closely as possible, to assure that energy is not wasted providing the services to empty rooms or buildings, and to assure that needed services are provided during all times when desired. Accomplishing this to date has been significantly hampered by a lack of inexpensive mechanisms to obtain and use building-wide and more granular data about occupancy. In recent years, the concept of inferential (or implicit) sensing has been proposed and explored to use data from IT systems that already exists in most buildings, to obtain data that are not perfect, but are nearly free to obtain. Past work by LBNL and others has primarily demonstrated the principle and potential for this, with a primary focus on data from Wi-Fi networks as the best near-term opportunity for inferential sensing from IT networks. This is due to its near-ubiquity, ease of explanation to many audiences, relative uniformness in deployment, low latency of detection, clear ways to mitigate privacy and security, and other benefits (Price et al., 2015). While the potential and value are clear, researchers or building managers who want to obtain the data lack a source of information to understand what data might exist, what devices may have it, and what mechanisms are available to obtain the data. The purpose of this report is to fill that gap. The report begins with a review of institutional issues in collecting such data, including very real concerns about privacy and IT security. It then describes the various system architectures used in Wi-Fi systems in commercial buildings today, and a variety of mechanisms that can be used to obtain information from them, including examples of how to use them, and sample output. Next is a summary of how to use such data for several different purposes, from retrospective analysis to dynamic building operation, and a conclusion. An appendix provides additional detail on specific mechanisms available from major equipment manufacturers. The mechanisms specified in this report can be used by building owners and operators to confirm proper operation and uncover any issues or unexpected conditions as a resource for building owners and researchers interesting in utilizing inferential sensing data.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Big data analytics in the building industry

Catalyzed by recent market, technology, and policy trends, energy data collection in the building industry is becoming more widespread. This wealth of information allows more data-driven decision-making by designers, commissioning agents, facilities staff, and energy service providers during the course of building design, operation and retrofit. The U.S. Department of Energy’s Building Performance Database (BPD) has taken advantage of this wealth of building asset- and energy-related data by collecting, cleansing, and standardizing data from across the U.S. on over 870,00 buildings, and is designed to support building benchmarking, energy efficiency project design, and buildings-related policy development with real-world data. Here, this article explores the promises and perils energy professionals are faced with when leveraging such tools, presenting example analyses for commercial and residential buildings, highlighting potential issues, and discussing solutions and best practices that will enable designers, operators and commissioning agents to make the most of ‘big data’ resources such as the BPD.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Vulcan Test Platform: Demonstrating the Data Center as a Flexible Grid Asset

Explosive data center demand is outpacing grid infrastructure development. AI workloads and hyperscale cloud growth are creating unprecedented power requirements, while traditional grid expansion faces multi-year development timelines, regulatory hurdles, and decarbonization challenges. Sidewalk Infrastructure Partners recognized this impending crisis years ago and founded Verrus to develop an innovative solution: data centers that function as grid assets rather than passive loads. The Verrus approach integrates proprietary grid-aware controls with battery energy storage systems (BESS) in a medium-voltage architecture that delivers three critical capabilities: Fast-responding demand flexibility that can service requests from the utility within 10 seconds, Uninterrupted transition to islanded operation during grid disturbances, Continuous uptime assurance while maintaining all customer service level commitments Through Verrus' strategic partnership with the National Renewable Energy Laboratory (NREL), we validated these capabilities on Vulcan, a 70 MW utility-scale test platform powered by NREL's ARIES Virtual Emulation Environment. This deployment-ready technology has successfully demonstrated that Verrus data centers can deliver meaningful grid services while maintaining mission-critical reliability. This technical report outlines the design, methodology, and results of this emulated deployment, demonstrating that data centers can play a pivotal role in enhancing grid flexibility and reliability, without sacrificing service level guarantees.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Harmonizing model organism data in the Alliance of Genome Resources

The Alliance of Genome Resources (the Alliance) is a combined effort of 7 knowledgebase projects: Saccharomyces Genome Database, WormBase, FlyBase, Mouse Genome Database, the Zebrafish Information Network, Rat Genome Database, and the Gene Ontology Resource. The Alliance seeks to provide several benefits: better service to the various communities served by these projects; a harmonized view of data for all biomedical researchers, bioinformaticians, clinicians, and students; and a more sustainable infrastructure. The Alliance has harmonized cross-organism data to provide useful comparative views of gene function, gene expression, and human disease relevance. The basis of the comparative views is shared calls of orthology relationships and the use of common ontologies. The key types of data are alleles and variants, gene function based on gene ontology annotations, phenotypes, association to human disease, gene expression, protein–protein and genetic interactions, and participation in pathways. The information is presented on uniform gene pages that allow facile summarization of information about each gene in each of the 7 organisms covered (budding yeast, roundworm Caenorhabditis elegans, fruit fly, house mouse, zebrafish, brown rat, and human). The harmonized knowledge is freely available on the alliancegenome.org portal, as downloadable files, and by APIs. We expect other existing and emerging knowledge bases to join in the effort to provide the union of useful data and features that each knowledge base currently provides.

59 BASIC BIOLOGICAL SCIENCES↗

Portable Acceleration of CMS Computing Workflows with Coprocessors as a Service

Computing demands for large scientific experiments, such as the CMS experiment at the CERN LHC, will increase dramatically in the next decades. To complement the future performance increases of software running on central processing units (CPUs), explorations of coprocessor usage in data processing hold great potential and interest. Coprocessors are a class of computer processors that supplement CPUs, often improving the execution of certain functions due to architectural design choices. We explore the approach of Services for Optimized Network Inference on Coprocessors (SONIC) and study the deployment of this as-a-service approach in large-scale data processing. In the studies, we take a data processing workflow of the CMS experiment and run the main workflow on CPUs, while offloading several machine learning (ML) inference tasks onto either remote or local coprocessors, specifically graphics processing units (GPUs). With experiments performed at Google Cloud, the Purdue Tier-2 computing center, and combinations of the two, we demonstrate the acceleration of these ML algorithms individually on coprocessors and the corresponding throughput improvement for the entire workflow. This approach can be easily generalized to different types of coprocessors and deployed on local CPUs without decreasing the throughput performance. We emphasize that the SONIC approach enables high coprocessor usage and enables the portability to run workflows on different types of coprocessors.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DR Resources for Energy and Ancillary Services in the West (Presentation)

Demand response (DR) resources present a potentially important source of grid flexibility however, DR in grid models is limited by data availability and modeling complexity. This presentation focuses on the co-optimization of DR resources to provide energy and ancillary services in a production cost model of the Colorado "test system". We assume each DR resource can provide energy services by either shedding load or shifting its use between different times, as well as operating reserves: frequency regulation, contingency reserve, and flexibility (or ramping) reserve. There are significant variations in the availabilities of different types of DR resources, which affect both the operational savings as well as the revenue for each DR resource. The results presented include the system-wide avoided fuel and generator start-up costs as well as the composite revenue for each DR resource by energy and operating reserves.

24 POWER TRANSMISSION AND DISTRIBUTION↗

CLAS12 remote data-stream processing using ERSAP framework

Implementing a physics data processing application is relatively straightforward with the use of current containerization technologies and container image runtime services, which are prevalent in most high-performance computing (HPC) environments. However, the process is complicated by the challenges associated with data provisioning and migration, impacting the ease of workflow migration and deployment. Transitioning from traditional file-based batch processing to data-stream processing workflows is suggested as a method to streamline these workflows. This transition not only simplifies file provisioning and migration but also significantly reduces the necessity for extensive disk space. Data-stream processing is particularly effective for real-time processing during data acquisition, thereby enhancing data quality assurance. This paper introduces the integration of the JLAB CLAS12 event reconstruction application within the ERSAP data-stream processing framework that facilitates the execution of streaming event reconstruction at a remote data center and enables the return streaming of reconstructed events to JLAB while circumventing the need for temporary data storage throughout the process.

Gyurjyan, Vardan↗

Comparing cropland net primary production estimates from inventory, a satellite-based model, and a process-based model in the Midwest of the United States

Accurately quantifying the spatial and temporal variability of net primary production (NPP) for croplands is essential to understand regional cropland carbon dynamics. We compared three NPP estimates for croplands in the Midwestern United States: inventory-based estimates using crop yield data from the U.S. Department of Agriculture (USDA) National Agricultural Statistics Service (NASS); estimates from the satellite-based Moderate Resolution Imaging Spectroradiometer (MODIS) NPP product; and estimates from the General Ensemble biogeochemical Modeling System (GEMS) process-based model. The three methods estimated mean NPP in the range of 469–687 g C m -2 yr -1 and total NPP in the range of 318–490 Tg C yr -1 for croplands in the Midwest in 2007 and 2008. The NPP estimates from crop yield data and the GEMS model showed the mean NPP for croplands was over 650 g C m -2 yr -1 while the MODIS NPP product estimated the mean NPP was less than 500 g C m -2 yr -1 . MODIS NPP also showed very different spatial variability of the cropland NPP from the other two methods. We found these differences were mainly caused by the difference in the land cover data and the crop specific information used in the methods. Our study demonstrated that the detailed mapping of the temporal and spatial change of crop species is critical for estimating the spatial and temporal variability of cropland NPP. Finally, we suggest that high resolution land cover data with species–specific crop information should be used in satellite-based and process-based models to improve carbon estimates for croplands.

Biogeochemical model↗

Quality control of meteorological data for the chemical stockpile emergency preparedness program.

The Chemical Stockpile Emergency Preparedness Program Meteorological Support Project ensures the accuracy and reliability of data acquired by meteorological monitoring stations located at seven U.S. Army chemical weapons depots where storage and weapons destruction (demilitarization) activities are ongoing. The data are delivered in real time to U.S. Army plume dispersion models, which are used to plan for and respond to a potential accidental release of a chemical weapons agent. The project provides maintenance, calibration, and audit services for the instrumentation; collection, automated screening, visual inspection, and analysis of the data; and problem reporting and tracking to carefully control the data quality. The resulting high-quality meteorological data enhance emergency response modeling and public safety.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

The Firegoose: two-way integration of diverse data from different bioinformatics web resources with desktop applications

Background: Information resources on the World Wide Web play an indispensable role in modern biology. But integrating data from multiple sources is often encumbered by the need to reformat data files, convert between naming systems, or perform ongoing maintenance of local copies of public databases. Opportunities for new ways of combining and re-using data are arising as a result of the increasing use of web protocols to transmit structured data. Results: The Firegoose, an extension to the Mozilla Firefox web browser, enables data transfer between web sites and desktop tools. As a component of the Gaggle integration framework, Firegoose can also exchange data with Cytoscape, the R statistical package, Multiexperiment Viewer (MeV), and several other popular desktop software tools. Firegoose adds the capability to easily use local data to query KEGG, EMBL STRING, DAVID, and other widely-used bioinformatics web sites. Query results from these web sites can be transferred to desktop tools for further analysis with a few clicks. Firegoose acquires data from the web by screen scraping, microformats, embedded XML, or web services. We define a microformat, which allows structured information compatible with the Gaggle to be embedded in HTML documents. We demonstrate the capabilities of this software by performing an analysis of the genes activated in the microbe Halobacterium salinarum NRC-1 in response to anaerobic environments. Starting with microarray data, we explore functions of differentially expressed genes by combining data from several public web resources and construct an integrated view of the cellular processes involved. Conclusion: The Firegoose incorporates Mozilla Firefox into the Gaggle environment and enables interactive sharing of data between diverse web resources and desktop software tools without maintaining local copies. Additional web sites can be incorporated easily into the framework using the scripting platform of the Firefox browser. Performing data integration in the browser allows the excellent search and navigation capabilities of the browser to be used in combination with powerful desktop tools.

59 BASIC BIOLOGICAL SCIENCES↗

Sandia Cloud Forensics and Incident Response Platform v 1.0

As systems and devices become virtualized and deployed in the cloud, the hypervisor becomes an increasingly appropriate place to collect performance data, system state, system landscape, function calls, transaction traces, and other characteristics. We developed a method by which an introspection application may be coupled with a hypervisor to “reach into” the VM with minimal intrusiveness to collect data critical to the reconstruction of events, files, and operations. Such a capability is required to take advantage of the hypervisor as an instrumentation platform and to integrate that data with more traditional collection mechanisms. The concept of a VM serviced by a lightweight hypervisor is a relatively new paradigm for forensic practitioners. Traditional forensic techniques, based on assumptions that the filesystem was directly interacting with the hardware through an abstraction, afforded the forensic practitioner the assumption that there was nothing controlling the application below the filesystem. This is not the case when using virtualized technologies. Hypervisors can covertly monitor, introspect and interact with the guest in a transparent fashion. The problems of storage and collection of actionable data are exhausting. The current challenge is most hypervisors do not expose a useful application programming interface (API) at a sufficient level to do transparent, fine-grained and customizable introspection. Scalable VM instrumentation and introspection at an in-depth level requires fast handling of events, as well as direct access to VM state. Furthermore, deep introspection benefits greatly from the ability to gather data from the hardware during the VM’s exit to the hypervisor. All of this requires identical access to the system as the hypervisor itself; improper use of this ability could easily cause system instability. It is for this reason we believe that the hypervisor developers have been hesitant to grant this much control through their APIs. However, our approach leverages other means to collect and monitor the guests in a targeted fashion. Virtual machine introspection (VMI) is a technique used to monitor the runtime state of a system-level virtual machine. The runtime state can be may include processor registers, memory, disk, network, and any other hardware-level events A review of research literature and current VMI technologies exposed several limitations and tradeoffs in VMI approaches, including: the use of in-guest agents; kernel to user space transitions (dramatically slowing down processing); VMI tool pre-configuration requirements; hypervisor version lock-in or source code patching; reliance on operating system symbols; limited processor features due to hypervisor (even if the hardware could do more). To address these constraints, a VMI tool was envisioned to provide the cloud forensic capabilities while having as few of these limitations as possible. The Kernel-based Virtual Machine Introspection (KVMi) tool was developed for hypervisors on Intel’s x86-64 architecture. To meet performance, scoping and use-case demands, the follow criteria were applied to KVMi to support live, forensic data collection: • Shall not require in-guest agents. • Shall work with any recent version of modern hypervisors. • Shall introspect upon any guest (VM) running modern versions of Windows, Linux and OS X. • Shall have the ability to find and track operating system artifacts • Shall not require operating system symbol files. • Shall be able to fully handle VM-exits, bypassing execution of the hypervisor if necessary, to facilitate new features the hypervisor may not support. • Shall be compilable/loadable on a running system with standard build tools.

Stout, WilliamM.S.↗

HexWeather: Hexagonal Spatial Data Aggregation for Weather-Driven Grid Resilience Analysis

Extreme weather accounts for over 8 0 % of major U.S. power outages since 2000, highlighting the need for spatial tools that align weather data with the irregular boundaries of electric infrastructure. This paper introduces HexWeather, a modular, resolution-aware framework for aggregating historical and forecasted weather data using Uber's H3 hexagonal spatial indexing system. Unlike traditional methods that rely on state or county-level grids, HexWeather enables weather analysis across custom geographies such as utility service areas where public datasets are often unavailable or misaligned. Using Open-Meteo data, we evaluate how H3 resolution affects anomaly detection, spatial variability, and forecast uncertainty across three scales: state, county, and utility. Results show that while coarse resolutions suffice for broad trend tracking, finer resolutions are essential for identifying localized variability and operational risks. By applying metrics like Z-score standard deviation and interquartile range, HexWeather quantifies the spatial spread of both historical anomalies and forecasted conditions, allowing users to assess resolution adequacy for each analysis. This framework supports rapid weather data reuse, reproducible anomaly detection, and predictive modeling for infrastructure resilience. By bridging spatial misalignment in traditional datasets and enabling retrospective and forward-looking analysis within the same pipeline, HexWeather lays the groundwork for better post event analysis, outage prediction, and resilience planning.

Morris, Jacob [ORNL]↗

Sectoral approaches to improve regional carbon budgets

Humans utilise about 40% of the earth s net primary production (NPP) but the products of this NPP are often managed by different sectors, with timber and forest products managed by the forestry sector and food and fibre products from croplands and grasslands managed by the agricultural sector. Other significant anthropogenic impacts on the global carbon cycle include human utilization of fossil fuels and impacts on less intensively managed systems such as peatlands, wetlands and permafrost. A great deal of knowledge, expertise and data is available within each sector. We describe the contribution of sectoral carbon budgets to our understanding of the global carbon cycle. Whilst many sectors exhibit similarities for carbon budgeting, some key differences arise due to differences in goods and services provided, ecology, management practices used, landmanagement personnel responsible, policies affecting land management, data types and availability, and the drivers of change. We review the methods and data sources available for assessing sectoral carbon budgets, and describe some of key data limitations and uncertainties for each sector in different regions of the world. We identify the main gaps in our knowledge/data, show that coverage is better for the developed world for most sectors, and suggest how sectoral carbon budgets could be improved in the future. Research priorities include the development of shared protocols through site networks, a move to full carbon accounting within sectors, and the assessment of full greenhouse gas budgets.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Recovery Act: Geothermal Data Aggregation: Submission of Information into the National Geothermal Data System, (Final Report, June 24, 2014)

The National Geothermal Data System (NGDS) is a Department of Energy funded effort to create a single cataloged source for a variety of geothermal information through a distributed network of databases made available via web services. The NGDS will help identify regions suitable for potential development and further scientific data collection and analysis of geothermal resources as a source for clean, renewable energy.

15 GEOTHERMAL ENERGY↗

BioHackathon 2015: Semantics of data for life sciences and reproducible research

We report on the activities of the 2015 edition of the BioHackathon, an annual event that brings together researchers and developers from around the world to develop tools and technologies that promote the reusability of biological data. We discuss issues surrounding the representation, publication, integration, mining and reuse of biological data and metadata across a wide range of biomedical data types of relevance for the life sciences, including chemistry, genotypes and phenotypes, orthology and phylogeny, proteomics, genomics, glycomics, and metabolomics. We describe our progress to address ongoing challenges to the reusability and reproducibility of research results, and identify outstanding issues that continue to impede the progress of bioinformatics research. We share our perspective on the state of the art, continued challenges, and goals for future research and development for the life sciences Semantic Web.

97 MATHEMATICS AND COMPUTING↗

CoREC: Scalable and Resilient In-memory Data Staging for In-situ Workflows

The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.

97 MATHEMATICS AND COMPUTING↗