Semantic Technologies in Practical Demand Response: An Information Requirement-based Roadmap
Explore the source record for details and available documents.
Mulayim, Ozan Baris↗
SEARCH · Engineering Papers
Results for “semantic technology”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models
Semantic schemas like Haystack 4, Brick and ASHRAE standard 223 enable the structured, standardized, and machine-readable representation of building data, facilitating interoperability, data integration, and advanced analytics. However, extracting information from these models requires specialized expertise in SPARQL and other programming languages, skills that are not commonly found among building professionals. Recent advancements in Large Language Models (LLMs), such as ChatGPT, enable the construction of queries using natural language, making it easier for individuals to interact with these systems in a manner that resembles everyday speech. However, these methods have not yet been tested on building semantic ontologies. This paper introduces a novel workflow and tool for enabling users to ask questions about a specific building's data, using natural language and receive answers automatically generated by GPT-4o. Our approach integrates semantic ontologies with advanced LLM capabilities to automate three critical steps: (1) generating SPARQL queries to retrieve time series references from ontological models, (2) extracting the corresponding time series data from the Building Automation System, and (3) performing computations and visualizations tailored to the user's query. The proposed method simplifies access to BAS data, allowing both domain experts and non-specialists to conduct sophisticated analyses without needing extensive technical knowledge of semantic web technologies. By demonstrating this pipeline, we facilitate more accessible and scalable data-driven decision-making in building operations and management.
Mulayim, Ozan Baris↗
Extraction and Analysis of Time Series Data from Building Automation Systems Using Large Language Models
Semantic schemas like Haystack 4, Brick and ASHRAE standard 223 enable the structured, standardized, and machine-readable representation of building data, facilitating interoperability, data integration, and advanced analytics. However, extracting information from these models requires specialized expertise in SPARQL and other programming languages, skills that are not commonly found among building professionals. Recent advancements in Large Language Models (LLMs), such as ChatGPT, enable the construction of queries using natural language, making it easier for individuals to interact with these systems in a manner that resembles everyday speech. However, these methods have not yet been tested on building semantic ontologies. This paper introduces a novel workflow and tool for enabling users to ask questions about a specific building's data, using natural language and receive answers automatically generated by GPT-4o. Our approach integrates semantic ontologies with advanced LLM capabilities to automate three critical steps: (1) generating SPARQL queries to retrieve time series references from ontological models, (2) extracting the corresponding time series data from the Building Automation System, and (3) performing computations and visualizations tailored to the user's query. The proposed method simplifies access to BAS data, allowing both domain experts and non-specialists to conduct sophisticated analyses without needing extensive technical knowledge of semantic web technologies. By demonstrating this pipeline, we facilitate more accessible and scalable data-driven decision-making in building operations and management.
Mulayim, Ozan Baris↗
A semantics-driven framework to enable demand flexibility control applications in real buildings
Decarbonising and digitalising the energy sector requires scalable and interoperable Demand Flexibility (DF) applications. Semantic models are promising technologies for achieving these goals, but existing studies focused on DF applications exhibit limitations. These include dependence on bespoke ontologies, lack of computational methods to generate semantic models, ineffective temporal data management and absence of platforms that use these models to easily develop, configure and deploy controls in real buildings. This paper introduces a semantics-driven framework to enable DF control applications in real buildings. The framework supports the generation of semantic models that adhere to Brick and SAREF while using metadata from Building Information Models (BIM) and Building Automation Systems (BAS). The work also introduces a web platform that leverages these models and an actor and microservices architecture to streamline the development, configuration and deployment of DF controls. The paper demonstrates the framework through a case study, illustrating its ability to integrate diverse data sources, execute DF actuation in a real building, and promote modularity for easy reuse, extension, and customisation of applications. The paper also discusses the alignment between Brick and SAREF, the value of leveraging BIM data sources, and the framework's benefits over existing approaches, demonstrating a 75% reduction in effort for developing, configuring, and deploying building controls.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
SAM-I-Am: Semantic boosting for zero-shot atomic-scale electron micrograph segmentation
Image segmentation is a critical enabler for tasks ranging from medical diagnostics to autonomous driving. However, the correct segmentation semantics — where are boundaries located? what segments are logically similar? — change depending on the domain, such that state-of-the-art foundation models can generate meaningless and incorrect results. Moreover, in certain domains, fine-tuning and retraining techniques are infeasible: obtaining labels is costly and time-consuming; domain images (micrographs) can be exponentially diverse; and data sharing (for third-party retraining) is restricted. To enable rapid adaptation of the best segmentation technology, we propose the concept of semantic boosting: given a zero-shot foundation model, guide its segmentation and adjust results to match domain expectations. Here, we apply semantic boosting to the Segment Anything Model (SAM) to obtain microstructure segmentation for transmission electron microscopy. Our booster, SAM-I-Am, serves as a post-processing engine that extracts geometric and textural features of various intermediate masks to perform mask removal and mask merging operations. We demonstrate a zero-shot performance increase of (absolute) +21.35%, +12.6%, +5.27% in mean IoU, and a -9.91%, -18.42%, -4.06% drop in mean false positive masks across images of three difficulty classes over vanilla SAM (ViT-L).
36 MATERIALS SCIENCE↗
BOPTest As a Platform for Building Controls and Grid-Interactive Buildings Workforce Training
Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.
Paul, Lazlo↗
BOPTEST as a Platform for Building Controls and Grid-Interactive Buildings Workforce Training
Building automation and controls are becoming increasingly complex with the emergence of Grid Integrated Efficient Buildings (GEBs) as well as new highly efficient sequences of operation and data-driven control schemes. However, there remains a significant gap in hands-on training opportunities for building operators and technicians to gain practical experience with advanced control systems in a low-risk environment. This paper presents BOPTEST (Building Optimization Performance Test) as a suitable platform for workforce training in building controls and GEB technologies. BOPTEST provides a suite of standardized building simulation test cases with a REST API, real-time control interfaces through BACnet, semantic models connecting users to building data, and built-in calculation of control metrics and performance indicators. The platform enables trainees to interact with virtual buildings using industry-standard protocols while learning how to implement and innovate control strategies. The training platform is designed to offer a structured and interactive learning experience for building engineers, helping them effectively develop, learn, and retain skills in fault identification, troubleshooting, and correction. The workflow is divided into three main phases: 1) Setup, 2) Exercise, and 3) Review, each comprising specific activities performed by either the instructor or the student. Initial pilot training sessions have yielded positive feedback from instructors and participants and demonstrates that BOPTEST effectively fills an industry need for a low-risk training resource via simulation of real building control systems, allowing trainees to gain practical experience before working in the field. The platform's ability to provide immediate performance feedback while maintaining familiar industry interfaces makes it particularly suitable for workforce development programs. This work provides a replicable model for leveraging building simulation in control education and training.
Paul, Lazlo↗
Concept Lens: Visual Comparison and Evaluation of Generative Model Manipulations
Generative models are becoming a transformative technology for the creation and editing of images. However, it remains challenging to harness these models for precise image manipulation. These challenges often manifest as inconsistency in the editing process, where both the type and amount of semantic change, depend on the image being manipulated. Moreover, there exist many methods for computing image manipulations, whose development is hindered by the matter of inconsistency. This paper aims to address these challenges by improving how we evaluate, compare, and explore the space of manipulations offered by a generative model. We present Concept Lens, a visual interface that is designed to aid users in understanding semantic concepts carried in image manipulations, and how these manipulations vary over generated images. Given the large space of possible images produced by a generative model, Concept Lens is designed to support the exploration of both generated images, and their manipulations, at multiple levels of detail. To this end, the layout of Concept Lens is informed by two hierarchies: a hierarchical organization of (1) original images, grouped by their similarities, and (2) image manipulations, where manipulations that induce similar changes are grouped together. This layout allows one to discover the types of images that consistently respond to a group of manipulations, and vice versa, manipulations that consistently respond to a group of codes. We show the benefits of this design across multiple use cases, specifically, studying the quality of manipulations for a single method, and offering a means of comparing different methods.
clustering↗
Graph neural network for neutrino physics event reconstruction
Liquid argon time projection chamber (LArTPC) detector technology offers a wealth of high-resolution information on particle interactions, and leveraging that information to its full potential requires sophisticated automated reconstruction techniques. Here, this article describes NUGRAPH 2, a graph neural network for low-level reconstruction of simulated neutrino interactions in a LArTPC detector. Simulated neutrino interactions in the MicroBooNE detector geometry are described as heterogeneous graphs, with energy depositions on each detector plane forming nodes on planar subgraphs. The network utilizes a multihead attention message-passing mechanism to perform background filtering and semantic labeling on these graph nodes, identifying those associated with the primary physics interaction with 98.0% efficiency and labeling them according to particle type with 94.9% efficiency. The network operates directly on detector observables across multiple two-dimensional representations but utilizes a three-dimensional-context-aware mechanism to encourage consistency between these representations. Model inference takes 0.12 s / event on a CPU and 0.005 s / event batched on a GPU. This architecture is designed to be a general-purpose solution for particle reconstruction in neutrino physics, with the potential for deployment across a broad range of detector technologies, and offers a core convolution engine that can be leveraged for a variety of tasks beyond the two described in this paper.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Bridging semantics, control specifications and assessment: A library for scalable demand flexibility controls
There is growing recognition that Demand Flexibility (DF) can play a major role in enhancing grid reliability, with building control applications emerging as key enablers for DF. However, the traditional approach to deploying new control applications in buildings, including those for DF, remains largely manual and tailored to individual buildings, making it difficult to scale. While research efforts have explored semantics-driven portability, DF controls specification, and assessment approaches, these initiatives are fragmented and limited in scope. This paper proposes a novel methodology, grounded in design science research, to integrate these elements and create a comprehensive DF controls library for both industry and academia. This approach is applied to develop the Demand FLEXibility controls LIBrary using Semantics (DFLEXLIBS), an extensible open-source library that provides DF controls for HVAC systems in Python. DFLEXLIBS enables portable, easy-to-deploy controls that abstract building-specific data points, facilitating assessment across diverse buildings. DFLEXLIBS features nine different control applications, and it is successfully implemented and tested across four virtual and two real buildings, bridging the gap between semantics-driven portability, DF controls specification, and rigorous performance assessment. Its benefits are measured by a reusability ratio greater than 90% and a functional overlap ratio of around 70% for the most common functions used in the library, significantly reducing time for deploying new controls.
Controls library↗
Arbitrary Polynomial Separations in Trainable Quantum Machine Learning
Recent theoretical results in quantum machine learning have demonstrated a general trade-off between the expressive power of quantum neural networks (QNNs) and their trainability; as a corollary of these results, practical exponential separations in expressive power over classical machine learning models are believed to be infeasible as such QNNs take a time to train that is exponential in the model size. We here circumvent these negative results by constructing a hierarchy of efficiently trainable QNNs that exhibit unconditionally provable, polynomial memory separations of arbitrary constant degree over classical neural networks—including state-of-the-art models, such as Transformers—in performing a classical sequence modeling task. This construction is also computationally efficient, as each unit cell of the introduced class of QNNs only has constant gate complexity. We show that contextuality—informally, a quantitative notion of semantic ambiguity—is the source of the expressivity separation, suggesting that other learning tasks with this property may be a natural setting for the use of quantum learning algorithms.
Anschuetz, Eric R. [California Institute of Techno↗
The Unified Phenotype Ontology : a framework for cross-species integrative phenomics
Phenotypic data are critical for understanding biological mechanisms and consequences of genomic variation, and are pivotal for clinical use cases such as disease diagnostics and treatment development. For over a century, vast quantities of phenotype data have been collected in many different contexts covering a variety of organisms. The emerging field of phenomics focuses on integrating and interpreting these data to inform biological hypotheses. A major impediment in phenomics is the wide range of distinct and disconnected approaches to recording the observable characteristics of an organism. Phenotype data are collected and curated using free text, single terms or combinations of terms, using multiple vocabularies, terminologies, or ontologies. Integrating these heterogeneous and often siloed data enables the application of biological knowledge both within and across species. Existing integration efforts are typically limited to mappings between pairs of terminologies; a generic knowledge representation that captures the full range of cross-species phenomics data is much needed. We have developed the Unified Phenotype Ontology (uPheno) framework, a community effort to provide an integration layer over domain-specific phenotype ontologies, as a single, unified, logical representation. uPheno comprises (1) a system for consistent computational definition of phenotype terms using ontology design patterns, maintained as a community library; (2) a hierarchical vocabulary of species-neutral phenotype terms under which their species-specific counterparts are grouped; and (3) mapping tables between species-specific ontologies. This harmonized representation supports use cases such as cross-species integration of genotype-phenotype associations from different organisms and cross-species informed variant prioritization.
59 BASIC BIOLOGICAL SCIENCES↗
Molecular Vision - Multimodal, multitask retrieval of molecular structure from measured signatures for reference-free compound identification
We are currently at risk of generating false conclusions based on limited methods to identify small molecules in biological systems and in chemical forensics. By definition, the chemical structures of novel small molecules have not been determined, let alone measured or synthesized. Currently, unambiguous structure determination of small molecules is constrained by the time and effort needed to isolate compounds and perform de novo structure elucidation using laboratory-based methods, significantly extending the time to inform mitigation strategies. To address this gap, we have developed a deep learning approach to directly map molecular structure to experimental signatures. We aim to unify measurement technologies employed in untargeted small molecule identification studies—such as infrared (IR) spectrometry, tandem mass spectrometry (MS/MS), ion mobility spectrometry-derived collision cross section (CCS)—through use of a multimodal, multitask deep learning architecture. Where existing methods require direct generation of information-rich spectra and/or properties, an inherently difficult task, we will simplify molecular signature-based identification by posing the problem as a recognition or retrieval task. The model is thus presented with relevant endpoints – structure and one or more molecular signatures – and need only determine whether they are semantically related. Thus, our approach offers the following advantages over existing techniques: (i) circumvents difficulties associated with direct generation of molecular signatures from structure and structure from signatures; (ii) incorporates multiple molecular signatures simultaneously, as available, to support identification; and (iii) enables rapid computation of structural embeddings toward broad coverage of known chemical space. Taken together, the approach removes the need to explicitly obtain or compute reference spectra, representing a powerful method for compound identification that requires only experimentally observed signatures.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decoding
Decoding visual experiences from fMRI offers a powerful avenue to understand human perception and develop advanced brain-computer interfaces. However, current progress often prioritizes maximizing reconstruction fidelity while overlooking interpretability, an essential aspect for deriving neuroscientific insight. To address this gap, we propose MoRE-Brain, a neuro-inspired framework designed for high-fidelity, adaptable, and interpretable visual reconstruction. MoRE-Brain uniquely employs a hierarchical Mixture-of-Experts architecture where distinct experts process fMRI signals from functionally related voxel groups, mimicking specialized brain networks. The experts are first trained to encode fMRI into the frozen CLIP space. A finetuned diffusion model then synthesizes images, guided by expert outputs through a novel dual-stage routing mechanism that dynamically weighs expert contributions across the diffusion process. MoRE-Brain offers three main advancements: First, it introduces a novel Mixture-of-Experts architecture grounded in brain network principles for neuro-decoding. Second, it achieves efficient cross-subject generalization by sharing core expert networks while adapting only subject-specific routers. Third, it provides enhanced mechanistic insight, as the explicit routing reveals precisely how different modeled brain regions shape the semantic and spatial attributes of the reconstructed image. Extensive experiments validate MoRE-Brain’s high reconstruction fidelity, with bottleneck analyses further demonstrating its effective utilization of fMRI signals, distinguishing genuine neural decoding from over-reliance on generative priors. Consequently, MoRE-Brain marks a substantial advance towards more generalizable and interpretable fMRI-based visual decoding.
Wei, Yuxiang [Georgia Institute of Technology]↗
Generalizable Web User Interface for Scalable and Streamlined Deployment of Building Energy Management Systems in Small and Medium-Sized Commercial Buildings
Small and medium-sized commercial buildings (SMCBs) comprise 94% of US commercial buildings yet face significant barriers to implementing building energy management systems despite advances in smart device technology. Existing solutions present critical limitations: cloud-based API solutions simplify deployment but create vendor lock-in constraints; commercial integrated software solutions ensure compatibility via standardized protocols but require substantial cost and technical expertise; open-source IoT platforms offer cost-effective vendor independence but provide insufficient standardized protocol support for commercial building automation. This research presents a generalizable web user interface framework that bridges the gap between evolving smart device capabilities and lagging software infrastructure for SMCBs. The proposed system integrates VOLTTRON open-source middleware with an automated configuration converter that transforms unified specifications written in YAML, a human-readable data-serialization format, into system-specific files, streamlining manual setup processes. The vendor-agnostic architecture supports industry-standard protocols (BACnet and Modbus) and semantic building models while providing adaptive web interfaces that dynamically adjust to various building configurations. Demonstrations through simulation-based testing and a field deployment show automatic interface adaptation across heterogeneous HVAC systems and multizone monitoring. The automated configuration converter also substantially reduces labor-intensive setup.
Chung, Jihoon [ORNL] (ORCID:0000000184880815)↗
NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction
Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction
Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction
Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments [https://arxiv.org/abs/2403.11872]. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗