Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Use of the ERD for administrative monitoring of Theta

Monitoring the state of an HPC cluster in a timely and accurate fashion is critical to most system administration functions. For many Cray users, the first step in monitoring is ingestion of log files. Unfortunately, log parsing is an inherently inefficient process, requiring multiple software components to read and write from files on disk. Cray's own utilities use a message bus, the Event Router Daemon (ERD), for a wide variety of purposes. At the Argonne Leadership Computing Facility (ALCF), we have begun to use this message bus for monitoring via a client library written in Go, allowing us to read in structured data directly from Cray's services and, in many instances, bypass log files entirely. In this paper, we will examine the implementation and utilization of this approach on our 4392 node XC40, Theta, as well as the overall benefits and drawbacks to using the ERD for real-time monitoring.

Cray↗

Transparent in Situ Data Transformations in ADIOS

Though an abundance of novel "data transformation" technologies have been developed (such as compression, level-of-detail, layout optimization, and indexing), there remains a notable gap in the adoption of such services by scientific applications. In response, we develop an in situ data transformation framework in the ADIOS I/O middleware with a "plug in" interface, thus greatly simplifying both the deployment and use of data transform services in scientific applications. Our approach ensures user-transparency, runtime-configurability, compatibility with existing I/O optimizations, and the potential for exploiting read-optimizing transforms (such as level-of-detail) to achieve I/O reduction. We demonstrate use of our framework with the QLG simulation at up to 8,192 cores on the leadership-class Titan supercomputer, showing negligible overhead. We also explore the read performance implications of data transforms with respect to parameters such as chunk size, access pattern, and the "opacity" of different transform methods including compression and level-of-detail.

Computer Science↗

Sync Matrix Enterprise

SME is an emergency response planning and exercise design and management software. SME implements an innovative approach of hosting its data in a customized Windows SharPoint Services (WSS) 3.0 site, and uses a Microsoft Win Forms technology as front-end to access the backend SharePoint list data as a client-server application. The utilization of WSS 3.0 allowed for a light weight application with the pwoer or project, data, document and user management tools that can link with everyday application such as Microsoft Office products, especially Outlook. The WinForms front-end application programmatically accesses the SharePoint List data through the exposed SharePoint Web Services application programming inerface (API). The SharePoint environment includes customized Web Parts that programmatically create new SharePoint sites with custom lists. The application also takes advantage of AJAX and Silverlight technologies to create a richer user experience for the SharePoint users.

Source record↗

Commercial Building Planning and Retrofitting Strategy for Grid Services

The increasing integration of distributed energy resources (DERs) plays an important role in improving energy consumption efficiency. In September 2020, the Federal Energy Regulatory Commission (FERC) approved Order 2222 which opens wholesale electricity markets to small capacity DERs. The benefit of this new FERC Order 2222 is that DERs, such as rooftop solar panels and batteries, will be able to participate in regional electricity markets and provide grid services. Meanwhile, the planning and operation strategies of DERs are facing new challenges to account for the impact of the wholesale market with numerous uncertainty factors. Therefore, in this paper, we propose a new planning and retrofitting model for long-term commercial buildings that considers both DER investment and market participation. Specifically, we explore the capability of implementing DERs for grid services. The effectiveness of the proposed model is validated using real-world data. Simulation results also validate that participating in grid services can significantly increase revenues through appropriate building energy management and shorten the payback period of DER investments.

building energy management↗

A National Collaboratory to Advance the Science of High Temperature Plasma Physics for Magnetic Fusion

This report summarizes the work of the National Fusion Collaboratory (NFC) Project to develop a persistent infrastructure to enable scientific collaboration for magnetic fusion research. The original objective of the NFC project was to develop and deploy a national FES Grid (FusionGrid) that would be a system for secure sharing of computation, visualization, and data resources over the Internet. The goal of FusionGrid was to allow scientists at remote sites to participate as fully in experiments and computational activities as if they were working on site thereby creating a unified virtual organization of the geographically dispersed U.S. fusion community. The vision for FusionGrid was that experimental and simulation data, computer codes, analysis routines, visualization tools, and remote collaboration tools are to be thought of as network services. In this model, an application service provider (ASP provides and maintains software resources as well as the necessary hardware resources. The project would create a robust, user-friendly collaborative software environment and make it available to the US FES community. This Grid's resources would be protected by a shared security infrastructure including strong authentication to identify users and authorization to allow stakeholders to control their own resources. In this environment, access to services is stressed rather than data or software portability.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

High Energy Physics and Nuclear Physics Network Requirements

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the U.S. Department of Energy (DOE) Office of Science (SC), the single largest supporter of basic research in the physical sciences in the United States. In support of SC programs, ESnet regularly updates and refreshes its understanding of the networking requirements needed by instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 25 years. In August 2013, ESnet and the DOE SC Offices of High Energy Physics (HEP) and Nuclear Physics (NP) organized a review to characterize the networking requirements of the programs funded by the HEP and NP program offices. Several key findings resulted from the review. Among them: 1. The Large Hadron Collider's ATLAS (A Toroidal LHC Apparatus) and CMS (Compact Muon Solenoid) experiments are adopting remote input/output (I/O) as a core component of their data analysis infrastructure. This will significantly increase their demands on the network from both a reliability perspective and a performance perspective. 2. The Large Hadron Collider (LHC) experiments (particularly ATLAS and CMS) are working to integrate network awareness into the workflow systems that manage the large number of daily analysis jobs (1 million analysis jobs per day for ATLAS), which are an integral part of the experiments. Collaboration with networking organizations such as ESnet, and the consumption of performance data (e.g., from perfSONAR [PERformance Service Oriented Network monitoring Architecture]) are critical to the success of these efforts. 3. The international aspects of HEP and NP collaborations continue to expand. This includes the LHC experiments, the Relativistic Heavy Ion Collider (RHIC) experiments, the Belle II Collaboration, the Large Synoptic Survey Telescope (LSST), and others. The international nature of these collaborations makes them heavily reliant on transoceanic connectivity, which is subject to longer term service disruptions than terrestrial connectivity. The network engineering aspects of undersea connectivity will continue to be a significant part of the planning, deployment, and operation of the data analysis infrastructure for HEP and NP experiments for the foreseeable future. Given their critical dependency on networking services, the experiments have expressed the need for tight integration (both technically and operationally) of the domestic and the transoceanic parts of the network infrastructure that supports the experiments. 4. The datasets associated with simulations continue to increase in size, and the need to move these datasets between analysis centers is placing ever-increasing demands on networks and on data management systems at the supercomputing centers. In addition, there is a need to harmonize cybersecurity practice with the data transfer performance requirements of the science. This report expands on these points, and addresses others as well. The report contains a findings section in addition to the text of the case studies discussed during the review.

43 PARTICLE ACCELERATORS↗

Data-Driven Linear Parameter-Varying Modeling and Control of Flexible Loads for Grid Services: Preprint

Flexible loads have great potential to improve the power grid's flexibility and stability. To effectively control large ensembles of heterogeneous loads, reliable models thereof are required. The paper presents a data-driven modeling and control approach to manage flexible loads for providing grid services. We leverage a linear parameter-varying autoregressive moving average (LPV-ARMA) model to describe the aggregate load response, where the parameters in the model are used to capture external environment impacts (e.g., weather). A gain-scheduling feedback controller is then developed to adapt to environmental variations. This data-driven approach can be easily applied to different types of loads in various environmental conditions. In addition to the ensemble controller, distributed load controllers are designed to deliver grid services, while maintaining the quality of service of inherent load tasks. We demonstrate the work on the IEEE 37-node distribution system for real-time power regulation services through control of thermostatically controlled loads.

control↗

An On-Demand Electric Transit Case Study of New Rochelle, New York

Here, this article characterizes the performance and ridership patterns of an on-demand transit (ODT) service utilizing lightweight electric vehicles (EVs) in New Rochelle, New York. Ridership sociodemographics, travel patterns (both temporal and spatiotemporal), and energy use from the service were explored using travel and survey data from September 2019 through December 2023. The ODT service was found to be used more by women (nearly 60%) and younger demographics (>65% under the age of 42), with peak use in the middle of the day and a grocery store as a top origin and destination. The service was utilized primarily for short trips (86% under 2 mi), with approximately one-third of riders using the ODT service to connect to a train or bus. The costs associated with fueling/charging were compared for different types of fleet vehicles, and the small, right-sized EVs were found to have annual charging costs that were roughly half of the refueling costs for conventional hybrid vans, and 24 times lower than a fleet of diesel buses. Evaluating the vehicle fleet and mapping current socio-spatial travel demand can inform system performance, guide service area development, and support future planning such as expansion to nearby communities and transit hubs. The findings in this case study suggest that on-demand electric transit may be a significant and growing space for advancing highly valued public mobility services. Public transport interventions that consider right-sized, electric, on-demand vehicles can help improve mobility access and reduce energy use and refueling costs.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Utah FORGE: Neubrex Well 16B(78)-32 Circulation Test Fiber Optics Monitoring Data and Reports - July, 2023

This dataset features Distributed Acoustic Sensing (DAS) and fiber optics monitoring data acquired by Neubrex Energy Services during the Utah FORGE Well 16B(78)-32 circulation test in July 2023. DAS and fiber optic monitoring data include absolute strain, strain change, strain change rate, distributed temperature sensing (DTS), and frequency band extraction (FBE). Data processing was completed between September and October 2023. The dataset includes the fiber optic sensing data collected during these operations, along with a data collection report, descriptions of the data, and instructions for reading the data files.

15 GEOTHERMAL ENERGY↗

Doing Your Science While You're in Orbit

Abstract. Large-scale neutron facilities such as the Spallation Neutron Source (SNS) located at Oak Ridge National Laboratory need easy-to-use access to Department of Energy Leadership Computing Facilities and experiment repository data. The Orbiter thick- and thinclient and its supporting Service Oriented Architecture (SOA) based services (available at https://nucleus.sns.gov) consist of standards-based components that are reusable and extensible for accessing high performance computing, data and computational grid infrastructure, and cluster-based resources easily from a user configurable interface. The primary Orbiter system goals consist of (1) developing infrastructure for the creation and automation of virtual instrumentation experiment optimization, (2) developing user interfaces for thin- and thickclient access, (3) provide a prototype incorporating major instrument simulation packages, and (4) facilitate neutron science community access and collaboration. The secure Orbiter SOA authentication and authorization is achieved through the developed Virtual File System (VFS) services, which use Role-Based Access Control (RBAC) for data repository file access, thinand thick-client functionality and application access, and computational job workflow management. The VFS Relational Database Management System (RDMS) consists of approximately 43 database tables describing 429 user accounts with 437 groups over 451,000 directories with 739,618 repository files. Over 39 million NeXus file metadata records are associated to the 12,794 unique NeXus file field/class names generated from the 42,894 repository NeXus files. Services that enable (a) summary dashboards of data repository status with Quality of Service (QoS) metrics, (b) data repository NeXus file field/class name full text search capabilities within a Google like interface, (c) fully functional RBAC browser for the read-only data repository and shared areas, (d) user/group defined and shared metadata for data repository files, (e) user, group, repository, and web 2.0 based global positioning with additional service capabilities are currently available. The SNS based Orbiter SOA integration progress with the Distributed Data Analysis for Neutron Scattering Experiments (DANSE) software development project is summarized with an emphasis on DANSE Central Services and the Virtual Neutron Facility (VNF). Additionally, the DANSE utilization of the Orbiter SOA authentication, authorization, and data transfer services best practice implementations are presented.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Cryptonite: A Secure and Performant Data Repository on Public Clouds

Cloud storage has become immensely popular for maintaining synchronized copies of files and for sharing documents with collaborators. However, there is heightened concern about the security and privacy of Cloud-hosted data due to the shared infrastructure model and an implicit trust in the service providers. Emerging needs of secure data storage and sharing for domains like Smart Power Grids, which deal with sensitive consumer data, require the persistence and availability of Cloud storage but with client-controlled security and encryption, low key management overhead, and minimal performance costs. Cryptonite is a secure Cloud storage repository that addresses these requirements using a StrongBox model for shared key management.We describe the Cryptonite service and desktop client, discuss performance optimizations, and provide an empirical analysis of the improvements. Our experiments shows that Cryptonite clients achieve a 40% improvement in file upload bandwidth over plaintext storage using the Azure Storage Client API despite the added security benefits, while our file download performance is 5 times faster than the baseline for files greater than 100MB.

Cloud data storage↗

Reactive transport benchmarks for subsurface environmental simulation

Over the last 20 years, we have seen firsthand the evolution of multicomponent reactive transport modeling and the expanding range and increasing complexity of subsurface applications it is being used to address. There is a growing reliance on reactive transport modeling (RTM) to address some of the most compelling issues facing our planet: climate change, nuclear waste management, contaminant remediation, and pollution prevention. While these issues are motivating the development of new and improved capabilities for subsurface environmental modeling using RTM (e.g., biogeochemistry from cell-scale physiology to continental-scale terrestrial ecosystems, nonisothermal multiphase conditions, coupled geomechanics), there remain longstanding challenges in characterizing the natural variability of hydrological, biological, and geochemical properties in subsurface environments and limited success in transferring models between sites and across scales. An equally important trend over the last 20 years is the evolution of modeling from a service sought out after data has been collected to a multifaceted research approach that provides (1) an organizing principle for characterization and monitoring activities; (2) a systematic framework for identifying knowledge gaps, developing and integrating new knowledge; and (3) a mechanistic understanding that represents the collective wisdom of the participating scientists and engineers. There are now large multidisciplinary projects where the research approach is model-driven, and the principal product is a holistic predictive simulation capability that can be used as a test bed for alternative conceptualizations of processes, properties, and conditions. Much of the future growth and expanded role for RTM will depend on its continued ability to exploit technological advancements in the earth and environmental sciences. Advances in measurement technology, particularly in molecular biology (genomics), isotope fractionation, and high-resolution X-ray spectroscopy, have created new lines of research that can be used to inform the conceptualization of reactions and rate laws and validate mechanistic models. For example, spectroscopy has identified the oxidation states of key components and elemental distributions at increasingly smaller scales and lower concentrations; molecular biology has progressed from identifying the presence of microbes to characterization of which microbial communities are active and what they are doing (i.e., microbial function), which has led in turn to the identification of active processes under conditions beyond what analytical chemistry can discern; isotope ratios in pore water and solid phases that can be used to distinguish between biotic from abiotic processes, sorption from precipitation, and origin and age of groundwater. The other noteworthy development that is expanding the role of RTM in subsurface environmental modeling is he advance in computational technology that is enabling the simulation of more coupled processes with increasing mechanistic detail. In some cases, this involves the inclusion of more reactive species and/or microbial populations in the simulations; in other cases, the impact is through the ability to achieve high resolution of property distributions over longer simulated times. To achieve these ambitious objectives for subsurface reactive transport simulation, the subsurface science and engineering community is being driven to provide accurate assessments of engineering performance and risk for important issues with far-reaching consequences. As a result, the complexity and detail of subsurface processes, properties, and conditions that can be simulated have significantly expanded. This expansion was enabled, in part, by advances in measurement technology, computing technology, and numerical techniques.

benchmarking↗

Visualizing feedstock siting in biomass production: Tradeoffs between economic and water quality objectives

New domestic, renewable energy resources must be considered to increase energy security in the U.S. Ethanol production through second-generation (cellulosic) feedstocks will help the U.S. meet the legislative Renewable Fuel Standard, which mandates 36 billion gallons of renewable fuels by 2022. Yet, conversion of cropland to meet the cellulosic feedstock production goals may have unforeseen environmental consequences. Here, using Soil Water Assessment Tool (SWAT) outputs and National Agricultural Statistics Service (USDA NASS) economic data, we conducted a spatial optimization of bioenergy feedstock introduction into the Arkansas White-Red River Basin based on water quality and economic objectives, subject to constraints on total land conversion. Results displayed tradeoffs between bioenergy yield for three crops (switchgrass, sorghum and poplar) and land rent objectives. Optimal solutions tended to prioritize conversion of land in eastern AWR subbasins where yield and water quality objective improvements were greatest. A small number of subbasins contributed to basin-wide water quality improvements, whereas subbasins contributing to economic benefits were more spatially dispersed, indicating that water quality responses are more likely to constrain feedstock placement. Biomass production targets can be met vianumerous spatial arrangements, whereas marginal improvement in water quality objectives can best be achieved by selectively siting perennial feedstocks in the eastern half of the region.

09 BIOMASS FUELS↗

Situational Awareness as a Measure of Performance in Cyber Security Collaborative Work

Cyber defense competitions arising from U.S. service academy exercises, offer a platform for collecting data that can inform research that ranges from characterizing the ideal cyber warrior to describing behaviors during certain challenging cyber defense situations. This knowledge in turn could lead to better preparation of cyber defenders in both military and civilian settings. We conducted proof of concept experimentation to collect data during the Pacific-rim Regional Collegiate Cyber Defense Competition (PRCCDC) and analyzed it to study the behavior of cyber defenders. We propose that situational awareness predicts performance of cyber security professionals, and in this paper we focus on our collection and analysis of competition data to determine whether it supports our hypothesis. In addition to normal cyber data, we collected situational awareness and workload data and compared it against the performance of cyber defenders as indicated by their competition score. We conclude that there is a weak correlation between our measure of situational awareness and performance that we hope to exploit in further studies.

99 GENERAL AND MISCELLANEOUS↗

Mahanaxar: quality of service guarantees in high-bandwidth, real-time streaming data storage

Large radio telescopes, cyber-security systems monitoring real-time network traffic, and others have specialized data storage needs: guaranteed capture of an ultra-high-bandwidth data stream, retention of the data long enough to determine what is 'interesting,' retention of interesting data indefinitely, and concurrent read/write access to determine what data is interesting, without interrupting the ongoing capture of incoming data. Mahanaxar addresses this problem. Mahanaxar guarantees streaming real-time data capture at (nearly) the full rate of the raw device, allows concurrent read and write access to the device on a best-effort basis without interrupting the data capture, and retains data as long as possible given the available storage. It has built in mechanisms for reliability and indexing, can scale to meet arbitrary bandwidth requirements, and handles both small and large data elements equally well. Results from our prototype implementation shows that Mahanaxar provides both better guarantees and better performance than traditional file systems.

97 MATHEMATICS AND COMPUTING↗

Deep Factorization Machine Learning for Disaggregation of Transmission Load Profiles with High Penetration of Behind-The-Meter Solar

The ever-growing integration of distributed energy resources (DERs), especially behind-the-meter (BTM) solar generations, poses imperative operational challenges to system operators such as regional transmission organizations (RTOs). It is important for RTOs to effectively and accurately extract actual load profiles at the transmission level for a single node with significant BTM solar injection. This paper first illustrates the necessity of disaggregating the daily actual load profile of a single node. Furthermore, by segmenting nodes with selected timeseries features, nodes with significant BTM solar generation are identified. Lastly, a bi-level framework is proposed, comprising reference node disaggregation and DeepFM nodal disaggregation, aimed at disaggregating the nodal load profiles from which system operators require more information. By adopting a hybrid Deep Factorization Machine (DeepFM) model, the model achieve accurate results by extracting both linear and nonlinear relations between nodes in the same region and the zonal load and nodal load profile. To overcome the lack of ground truth, this paper segments the load profile into daytime, nighttime, and zero-crossing points and utilizes the latter two for evaluation purposes. The proposed disaggregation procedure is validated using real world, minute-level, normalized, and anonymized nodal data in the PJM service territory.

42 ENGINEERING↗

The safety of pranlukast and montelukast during the first trimester of pregnancy: A prospective, two‐centered cohort study in Japan

Abstract For leukotriene receptor antagonists (LTRAs), especially pranlukast, safety data during pregnancy is limited. Therefore, we conducted a prospective, two‐centered cohort study using data from teratogen information services in Japan to clarify the effects of LTRA exposure during pregnancy on maternal and fetal outcomes. Pregnant women who being counseled on drug use during pregnancy at two facilities were enrolled. The primary outcome of this study was major congenital anomalies. The incidence of major congenital anomalies in women exposed to montelukast or pranlukast during the first trimester of pregnancy was compared with that of controls. Logistic regression analysis was performed to analyze the effects of maternal LTRA use during the first trimester of pregnancy on major congenital anomalies. The outcomes of 231 pregnant women exposed to LTRAs (montelukast n = 122; pranlukast n = 106; both n = 3) and 212 live births were compared with those of controls. The rate of major congenital anomalies in the LTRA group was 1.9%. Multivariable logistic regression analysis revealed that LTRA exposure was not a risk factor for major congenital anomalies (adjusted odds ratio, 0.78; 95% confidence interval, 0.23–2.05; p = 0.653). In addition, no significant difference was detected in stillbirth, spontaneous abortion, preterm birth, and low birth weight between the two groups. The present study revealed that montelukast and pranlukast were not associated with the risk of major congenital anomalies. Our findings suggest that LTRAs could be safely employed for asthma therapy during pregnancy.

Hatakeyama, Shiro↗