Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Programming Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Expandable Log Analyzing Framework

Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.

Clark, Dylan [Unlisted, IL]↗
DOE OSTI2026

Database-Agnostic Log Analysis and Monitoring Framework

Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.

Clark, Dylan [Unlisted, US, IL; Fermilab]↗
DOE OSTI2023

Ciel

Compiler optimizations can alter the numerical results of scientific computing applications. When numerical results differ significantly between compilers, optimization levels, and floating-point hardware, these numerical inconsistencies can impact programming productivity. Ciel is a framework that helps programmers identify locations in the source code that are affected by compiler optimizations in CPU and GPU code. Ciel uses a floating-point precision enhancement strategy, guided by a recursive bisection search algorithm with increasing search granularity, to identify the program expressions that induce numerical inconsistencies due to compiler optimizations.

Miao, Wenjun↗
DOE OSTI2020

Optimization Framework to Assess the Demand Response Capacity of a Water Distribution System

As large electricity consumers, water distribution system (WDS) pumping stations have the potential to become meaningful participants in demand response (DR) programs. The authors propose an optimization framework for assessing the DR capacity of a WDS and identifying the optimal bidding strategy for maximizing WDS revenue in the DR spot market. The proposed mixed integer linear programming (MILP) model overcomes computational constraints of previous DR optimization models by adopting a preprocessing procedure to minimize the number of binary variables and implementing a convex relaxation technique to linearize the hydraulic equations. The proposed MILP model also explicitly accounts for varying levels of risk tolerance of WDS operators by varying the recovery period over which pumping returns to business-as-usual operation. The optimization framework is implemented on a skeletonized 48-node WDS model that includes 7 pumps, 6 tanks, and 39 pipes. Using a simulated DR event and water consumption profile, the authors derive the optimal DR supply curves (i.e., compensation price versus load curtailment quantity) and revenue potential of the WDS under six scenarios for DR participation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Data-Driven Energy Resilience Assessment and Enhancement in Urban Communities: A Case Study in Detroit

This paper presents a data-driven framework for assessing and enhancing energy resilience in urban communities. The resilience assessment is based on two datasets: 1) annual aggregated power outage data and 2) 15-minute interval outage data. High-impact, low-probability (HILP) events are identified within these datasets to evaluate community resilience under extreme conditions. To enhance resilience, an optimization framework utilizing mixed integer linear programming is developed to determine the optimal sizing and placement of solar photovoltaic (PV) systems and battery energy storage systems (BESS). This method offers a cost-effective and practical solution for improving energy resilience in vulnerable communities. Furthermore, a case study of the City of Detroit in Michigan demonstrates the effectiveness of the framework through simulation and validation.

Energy resilience assessment↗
DOE OSTI2026

Navigating Large Chemical Spaces Using Graph Theory and Integer Programming

Navigating and analyzing large chemical spaces are necessary to accelerate the design and discovery of new molecules and chemical processes. In this work, we introduce a computational framework that integrates graph theory and integer programming to enable the efficient navigation of large chemical spaces. Our framework represents the chemical space as a graph, wherein nodes represent molecules and edges represent the degree of similarity or connectivity based on domain-specific information. Using the graph representation, we identify representative molecules by computing the so-called minimum dominating set (MDS), which in our context is the minimum set of molecules that is connected to all other molecules. We present a suite of solution strategies for the MDS problem including heuristic and rigorous integer programming (IP) approaches. We show that these approaches allow us to capture physicochemical properties and domain-specific logic and constraints, facilitating the identification of molecules with the target properties. We demonstrate the effectiveness of the proposed approach by navigating the chemical space of per- and polyfluoroalkyl substances (PFAS); this comprises approximately 15,000 molecular structures. We compare our framework against traditional dimensionality reduction and clustering methods such as t-SNE and K-means clustering.

Chemical structure↗
DOE OSTI2022

Model Predictive Control of Discrete-Continuous Energy Systems via Generalized Disjunctive Programming

Generalized Disjunctive Programming (GDP) provides an alternative framework to model optimization problems with both discrete and continuous variables. The key idea behind GDP involves the use of logical disjunctions to represent discrete decisions in the continuous space, and logical propositions to denote algebraic constraints in the discrete space. Compared to traditional mixed-integer programming (MIP), the inherent logic structure in GDP yields tighter relaxations that are exploited by global branch and bound algorithms to improve solution quality. In this paper, we present a general GDP model for optimal control of hybrid systems that exhibit both discrete and continuous dynamics. Specifically, we use GDP to formulate a model predictive control (MPC) model for piecewise-affine systems with implicit switching logic. As an example, the GDP-based MPC approach is used as a supervisory control to improve energy efficiency in residential buildings with binary on/off, relay-based thermostats. A simulation study is used to demonstrate the validity of the proposed approach, and the improved solution quality compared to existing MIPbased control approaches.

Bhattacharya, Arnab↗
DOE OSTI2023

SeeQ: A Programming Model for Portable Data-Driven Building Applications

This paper introduces SeeQ, a programming model and an abstraction framework that facilitates the development of portable data- driven building applications. Data-driven approaches can provide insights into building operations and guide decision-making to achieve operational objectives. Yet the configuration of such applications per building requires extensive effort and tacit knowledge. In SeeQ, we propose a portable programming model and build a software system that enables self-configuration and execution across diverse buildings. The configuration of each building is captured in a unified data model - in this paper, we work with the Brick ontology without loss of generality. SeeQ focuses on the distinction between the application logic and the configuration of an application against building-specific data inputs and systems. We test the proposed approach by configuring and deploying a diverse range of applications across five heterogeneous real-world buildings. The analysis shows the potential of SeeQ to significantly reduce the efforts associated with the delivery of building analytics.

analytics↗
DOE OSTI2023

Jay : A software framework for prototyping and evaluating offloading applications in hybrid edge clouds

Abstract We present Jay , a software framework for offloading applications in hybrid edge clouds. Jay provides an API, services, and tools that enable mobile application developers to implement, instrument, and evaluate offloading applications using configurable cloud topologies, offloading strategies, and job types. We start by presenting Jay 's job model and the concrete architecture of the framework. We then present the programming API with several examples of customization. Then, we turn to the description of the internal implementation of Jay instances and their components. Finally, we describe the Jay Workbench, a tool that allows the setup, execution, and reproduction of experiments with networks of hosts with different resource capabilities organized with specific topologies. The complete source code for the framework and workbench is provided in a GitHub repository.

Silva, Joaquim↗
DOE OSTI2023

GAHLS: an optimized graph analytics based high level synthesis framework

The urgent need for low latency, high-compute and low power on-board intelligence in autonomous systems, cyber-physical systems, robotics, edge computing, evolvable computing, and complex data science calls for determining the optimal amount and type of specialized hardware together with reconfigurability capabilities. With these goals in mind, we propose a novel comprehensive graph analytics based high level synthesis (GAHLS) framework that efficiently analyzes complex high level programs through a combined compiler-based approach and graph theoretic optimization and synthesizes them into message passing domain-specific accelerators. This GAHLS framework first constructs a compiler-assisted dependency graph (CaDG) from low level virtual machine (LLVM) intermediate representation (IR) of high level programs and converts it into a hardware friendly description representation. Next, the GAHLS framework performs a memory design space exploration while account for the identified computational properties from the CaDG and optimizing the system performance for higher bandwidth. The GAHLS framework also performs a robust optimization to identify the CaDG subgraphs with similar computational structures and aggregate them into intelligent processing clusters in order to optimize the usage of underlying hardware resources. Finally, the GAHLS framework synthesizes this compressed specialized CaDG into processing elements while optimizing the system performance and area metrics. Evaluations of the GAHLS framework on several real-life applications (e.g., deep learning, brain machine interfaces) demonstrate that it provides 14.27× performance improvements compared to state-of-the-art approaches such as LegUp 6.2.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

MDDC Multi-Length Scale Data Architecture Contribution Report – PNNL, INL, ANL, LANL and ORNL

This report offers a comprehensive view of data streams currently generated at Pacific Northwest National Laboratory, Idaho National Laboratory, Argonne National Laboratory, Los Alamos National Laboratory, and Oak Ridge National Laboratory set to integrate into the evolving Multi-Dimensional Data Correlation framework at Oak Ridge National Laboratory. Developed by the Advanced Materials and Manufacturing Technologies program, the Multi-Dimensional Data Correlation framework serves as a cutting-edge software to manage data relevant to advanced manufacturing and material behavior in advanced reactors. The report defines data streams, highlights their generation methods and visualization methods both for experimental and computational aspects relevant to the Advanced Materials and Manufacturing Technologies project. A logical next step for this work is to integrate the MDDC framework into PNNL’s, INL’s, ANL’s, LANL’s and ORNL’s fabrication, experimentation, and modelling workflows. This would require setting up the MDDC framework at PNNL, INL, ANL, and LANL and integrating it into the data collection and storage for these different activities.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Co-design optimization of combined heat and power-based microgrids

With the emergent need for clean and reliable energy resources, hybrid energy systems, such as the microgrid, are widely adopted in the United States. A microgrid can consist of various distributed energy resources, for instance, combined heat and power (CHP) systems. Here, the CHP module is a distributed cogeneration technology that produces electricity and recaptures heat generated as a by-product. It is an energy-efficient technology converting heat that would otherwise be wasted to valuable thermal energy. For an optimal system configuration, this study develops a novel co-design optimization framework for CHP-based cogeneration microgrids. The framework provides the stakeholder with a method to optimize investments and attain resilient operations. The proposed co-design framework has a mixed integer programming (MIP) model that outputs decisions for both plant designs and operating controls. The microgrid considered in this study contains six components: the CHP, boiler, heat recovery unit, thermal storage system, power storage system, and photovoltaic plant. After solving the MIP model, the optimal design parameters of each component can be found to minimize the total installation cost of all components in the microgrid. Furthermore, the online costs from energy production, operation, maintenance, machine startup, and disruption-induced unsatisfied loads are minimized by solving the optimal control decisions for operations. Case studies based on designing a CHP-based microgrid with empirical data are conducted. Moreover, we consider both nominal and disruptive operational scenarios to validate the performance of the proposed co-design framework in terms of a cost-effective, resilient system.

42 ENGINEERING↗
DOE OSTI2022

DFSynthesizer: Dataflow-based Synthesis of Spiking Neural Networks to Neuromorphic Hardware

Spiking Neural Networks (SNNs) are an emerging computation model that uses event-driven activation and bio-inspired learning algorithms. SNN-based machine learning programs are typically executed on tile-based neuromorphic hardware platforms, where each tile consists of a computation unit called a crossbar, which maps neurons and synapses of the program. However, synthesizing such programs on an off-the-shelf neuromorphic hardware is challenging. This is because of the inherent resource and latency limitations of the hardware, which impact both model performance, e.g., accuracy, and hardware performance, e.g., throughput. We propose DFSynthesizer, an end-to-end framework for synthesizing SNN-based machine learning programs to neuromorphic hardware. The proposed framework works in four steps. First, it analyzes a machine learning program and generates SNN workload using representative data. Second, it partitions the SNN workload and generates clusters that fit on crossbars of the target neuromorphic hardware. Third, it exploits the rich semantics of the Synchronous Dataflow Graph (SDFG) to represent a clustered SNN program, allowing for performance analysis in terms of key hardware constraints such as number of crossbars, dimension of each crossbar, buffer space on tiles, and tile communication bandwidth. Finally, it uses a novel scheduling algorithm to execute clusters on crossbars of the hardware, guaranteeing hardware performance. We evaluate DFSynthesizer with 10 commonly used machine learning programs. Our results demonstrate that DFSynthesizer provides a much tighter performance guarantee compared to current mapping approaches.

Computer Science↗
DOE OSTI2020

Collaborative Exploration of Scientific Datasets Using Immersive and Statistical Visualization: Preprint

We discuss the value of collaborative, immersive visualization for the exploration of scientific datasets and review techniques and tools that have been developed and deployed at the National Renewable Energy Laboratory (NREL). We believe that collaborative visualizations linking statistical interfaces and graphics on laptops and high-performance computing (HPC) with 3D visualizations on immersive displays (head-mounted displays and large-scale immersive environments) enable scientific workflows that further rapid exploration of large, high-dimensional datasets by teams of analysts. We present a framework, PlottyVR, that blends statistical tools, general-purpose programming environments, and simulation with 3D visualizations. To contextualize this framework, we propose a categorization and loose taxonomy of collaborative visualization and analysis techniques. Finally, we describe how scientists and engineers have adopted this framework to investigate large, complex datasets.

collaborative visualization↗
DOE OSTI2021

Collaborative Exploration of Scientific Datasets Using Immersive and Statistical Visualization

We discuss the value of collaborative, immersive visualization for the exploration of scientific datasets and review techniques and tools that have been developed and deployed at the National Renewable Energy Laboratory (NREL). We believe that collaborative visualizations linking statistical interfaces and graphics on laptops and high-performance computing (HPC) with 3D visualizations on immersive displays (head-mounted displays and large-scale immersive environments) enable scientific workflows that further rapid exploration of large, high-dimensional datasets by teams of analysts. We present a framework, PlottyVR, that blends statistical tools, general-purpose programming environments, and simulation with 3D visualizations. To contextualize this framework, we propose a categorization and loose taxonomy of collaborative visualization and analysis techniques. Finally, we describe how scientists and engineers have adopted this framework to investigate large, complex datasets.

collaborative visualization↗
DOE OSTI2025

Machine Learning for superconducting magnets application (2023 Italian Summer Students program at FNAL Final Report)

This report documents the work I conducted during my internship as part of the Italian Summer Students program at the Fermilab National Accelerator Laboratory (FNAL). Throughout my internship, I was stationed in the Technical Division within the Fermilab laboratory, where I was under the guidance of Emanuela Barzi and co-supervised by Reed Teyber from the Lawrence Berkeley National Laboratory (LBNL). The aim of the project is the study and characterization of quench antenna signals in order gain understanding on the mechanical and electromagnetic phenomena happening inside superconductive magnets. The results presented here identify some potential areas of investigation and potential directions for improving the construction of magnetic superconductors, particularly within the framework of the CCT subscale program at LBNL.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

DFTT report for Signal Analysis (2023)

The Detection Framework Testbed and Toolkit (DFTT) is a database and associated java programs intended to facilitate the development and testing of algorithms for operating suites of correlation and subspace detectors. This framework is a generalization of the system described in Harris and Dodge (2011) and Dodge and Harris (2016). DFTT allows retrospective processing of sequences of data using various system configurations. Results are saved in a database, so it is easy to compare the results obtained using different configurations of the system. The principal software components of DFTT are the ConfigCreator, the framework_runner and the Builder program. ConfigCreator assembles the files necessary to define a particular configuration used for processing. The framework_runner operates suites of detectors and saves the results in an Oracle™ database. Builder allows visual examination of templates and detected signals and allows editing and creation of detectors. In addition, there are tools for importing continuous data into the database. DFTT is licensed under the MIT license and has LLNL release number LLNL-CODE-801881. In the current fiscal year DFTT has been used in two different projects. The first (NTL-funded) project investigates the feasibility of screening nuisance detections using correlation detectors. The second (GNEM-funded) project seeks to extend the results of Harris and Dodge (2021) to 3- D sources and multiple stations. Each of these efforts required that additional functionality be added to DFTT. In this report I will summarize the enhancements to DFTT in the context of the relevant research effort.

58 GEOSCIENCES↗
DOE OSTI2022

A symbolic framework to obtain mid-fidelity models of flexible multibody systems with application to horizontal-axis wind turbines

Abstract. The article presents a symbolic framework (also called computer algebra program) that is used to obtain, in symbolic mathematical form, the linear and nonlinear equations of motion of a mid-fidelity multibody system including rigid and flexible bodies. Our approach is based on Kane's method and a nonlinear shape function representation for flexible bodies. The shape function approach does not represent the state of the art for flexible multibody dynamics but is an effective trade-off to obtain mid-fidelity models with few degrees of freedom, taking advantage of the separation of space and time. The method yields compact symbolic equations of motion with implicit account of the constraints. The general and automatic framework facilitates the creation and manipulation of models with various levels of complexity by adding or removing degrees of freedom. The symbolic treatment allows for analytical gradients and linearized equations of motion. The linear and nonlinear equations can be exported to Python code or dedicated software. There are multiple applications, such as time domain simulation, stability analyses, frequency domain analyses, advanced controller design, state observers, and digital twins. In this article, we describe the method we used to systematically generate the equations of motion of multibody systems and present the implementation of the framework using the Python package SymPy. We apply the framework to generate illustrative land-based and offshore wind turbine models. We compare our results with OpenFAST simulations and discuss the advantages and limitations of the method. The Python implementation is provided as an open-source project.

Branlard, Emmanuel (ORCID:0000000277506128)↗