Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Data Visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Grid Modernization of Cooperatives and Municipal Utilities via Breakthrough System Monitoring, Control and Optimization (CRADA Final Report)

This project aims at developing and demonstrating successful implementation of breakthrough approaches in real-time data visualization as well as real-time distributed DER control and optimization to provide ample benefits to both utilities and end users. The National Renewable Energy Laboratory (NREL), Holy Cross Energy (HCE), National Rural Electric Cooperative Association (NRECA) and Survalent are collaborating to enable Cooperative and Municipal utilities to fully leverage DERs as part of their strategies for providing safe, reliable, and affordable electric services to their customers and help meet DOE Grid modernization goal of achieving at least 10% active devices to provide grid flexibility by 2035. This project will use novel real-time control algorithms and approaches for distributed control recently developed under DOE-funded projects, using the date from the Survalent’s basic SCADA engine, GIS and AMI engines deployed at HCE combined with NRECA’s globally-used MultiSpeak(R) software interoperability specification for seamless and real-time communications between electric utility enterprise software to embrace DER as part of their strategies for providing safe, reliable and affordable electric service to their customers.

24 POWER TRANSMISSION AND DISTRIBUTION

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) v1

SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training) is a comprehensive data visualization and analysis application focused on working with COLTRIMS (COLd Target Recoil Ion Momentum Spectroscopy) data, which is used in atomic and molecular physics experiments. The application offers several powerful features: - Data uploading and processing capabilities for COLTRIMS files - Multiple visualization methods using UMAP (Uniform Manifold Approximation and Projection) for dimensionality reduction - Interactive selection of data points across multiple views - Feature engineering through various methods: - Manual feature selection from calculated physics parameters - Deep autoencoder for dimension reduction - Genetic programming for discovering meaningful features - Mutual information-based feature selection - Multiple clustering approaches (DBSCAN, KMeans, Agglomerative) - Quality metrics for evaluating clustering results - Export capabilities for selections and generated features

Daoud, Hazem [Lawrence Berkeley National Laborator

Energy Consumption, Performance, and Cost Estimates for Medium and Heavy-Duty Vehicles Based on 2022 Assumptions

Assumptions for this work was collected and the analysis was completed in FY22. This contains information for more than 20 types of medium and heavy duty vehicles. Vehicles with various levels of hybridization, electric and fuel cell powertrains are considered in this work. More details are available in the report published by Argonne accessible from https://vms.taps.anl.gov/research-highlights/u-s-doe-vto-hfto-r-d-benefits/. TechScape, a convenient data visualization tool is also provided by Argonne for this data, accessible from [TechScape Web](https://vms.taps.anl.gov/data/techscape-web-2023/).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

Residential Solar-Adopter Income and Demographic Trends: 2024 Update [Slides]

The report describes income, demographic, and other socio-economic trends among U.S. residential rooftop solar adopters. The report is based on address-level data for roughly 4.1 million residential rooftop solar systems installed through 2023, representing 87% of all U.S. systems. With its unique size, geographic scope, and level of detail, this report is intended to serve as a foundational reference document for policy-makers, industry stakeholders, and researchers. Key findings include the following: -The median income of households that installed solar in 2023 was about $\$$115k/year, compared to a U.S. median of $\$$75k/year for all households and $\$$94k/year for all U.S. owner-occupied households. -Compared to owner-occupied households in the same state, 2023 solar-adopter incomes were 7% higher in the median case, and in 10 states, median solar-adopter incomes were below the corresponding median income for all owner-occupied households. -Roughly 49% of solar adopters in 2023 had incomes below 120% of their area median income (AMI), a threshold sometimes used to define “low-and-moderate income” (or LMI), while 26% were below 80% of AMI, often used to define “low-income”. -Solar adoption continues to shift toward less affluent households over time, with the median present-day income of solar adopters dropping from $\$$141k for households that installed systems in 2010 to $\$$115k in 2023. -PV systems installed in 2023 by households earning less than $\$$50k had a median size of 6.4 kW, 33% were third-party owned, and 6% included battery storage, compared to corresponding values of 8.0 kW, 18%, and 14% for households earning more than $\$$200k. -Compared to all households in their respective state, solar adopters in 2023 were slightly more likely to be college educated and to live in rural areas; had higher home values; and were more likely to live outside a disadvantaged community (DAC), be middle-aged, identify as non-Hispanic white, work in a business or financial occupation, and own a single-family home. In conjunction with the report, Berkeley Lab has published an updated accompanying set of online data visualizations that allow users to further explore the underlying data. Berkeley Lab is also offering related analytical support to states, local agencies, and other organizations on issues related to solar adoption among low-to-moderate income households; requests for analytical support may be submitted through this online form.

14 SOLAR ENERGY

Applications of visualization technology in the structural sciences

The structural sciences are undergoing a transformation driven by advancements in visualization technologies that aid researchers in understanding and communicating experimental data from complex molecular systems. New applications of integrative structural biological and biophysical approaches add a wide variety of complementary information from a broad range of scientific disciplines. These approaches extend structural biophysical methodologies to enable research by the incorporation of a variety of data streams and utilization of tools like molecular graphics, virtual reality, and machine learning. To redefine how structural data—particularly from cryo-electron microscopy and x-ray crystallography—are fed forward for scientific exploration and communication, the advances in tools for data visualization and interpretation have been critical. By bringing molecular systems into an interactive three-dimensional space, these novel technologies enhance research workflows, facilitate structure-based drug design, and create engaging educational experiences. Taken together, these visualization innovations are essential tools for advancing the field by making concepts more accessible and compelling.

Eng, Edward T. [New York Structural Biology Center

Peregrine Software Development: Report on the Code Conversion From Python to C++

This work package seeks to convert the Peregrine software tool from its original Python implementation to a production version based on the C++ language. Peregrine is a powerful research platform with a multitude of advanced data analytics and data visualization functionalities. Developed by scientists to explore multimodal and multidimensional data related to the production of components using powder bed additive manufacturing processes, the tool implements state-of-the-art algorithms to assist machine users in making build or part quality determinations. Given that Peregrine is data-intensive, the goal of this conversion is to enhance the tool’s flexibility and interactivity and reduce the number of code dependencies to facilitate its deployment as part of the ongoing technology transfer campaign. This brief document provides an overview of Peregrine’s functionalities and capabilities, along with a detailed description of the core functionalities that have been implemented to date in the new C++ version. This document serves as a development update at the end of the first year of the ongoing conversion and will be regularly updated as progress continues.

97 MATHEMATICS AND COMPUTING

System and method for transparent augmented reality

A system and method for displaying augmented reality information for a real object. A transparent display displays the information for the real object within a field of view, such that the information and the real object are simultaneously viewable by a user. Data-gathering peripheral devices gather data of the field of view, including a camera gathering visual data for tracking the real object. An articulating arm moveably supports and facilitates repositioning the display and redirecting the data-gathering peripheral devices in a particular direction. A security interlock disables the data-gathering peripherals when the articulating arm is moved beyond a pre-established threshold, thereby preventing the data-gathering peripherals from gathering data of a real environment outside of a limit of the field of view as determined by the pre-established threshold. The security interlock may include an adjustable contact switch which adjustably defines the pre-established threshold in at least one direction.

Runyan, Paul Ryan

Opportunities and Challenges in the Visualization of Energy Scenarios for Decision-Making: Preprint

Scenario studies are a technique for representing a range of possible complex decisions through time, and analyzing the impact of those decisions on future outcomes of interest. It is common to use scenarios as a way to study potential pathways towards future build-out and decarbonization of energy systems. The results of these studies are often used by diverse energy system stakeholders - such as community organizations, power system utilities, and policymakers - for decision-making using data visualization. However, the role of visualization in facilitating decision-making with energy scenario data is not well understood. In this work, we review visualization designs employed in energy scenario studies found in the literature and publicly accessible online sources. We discuss the effectiveness of existing techniques particularly in regards to decision-making, and present opportunities and challenges in the visualization of energy system scenario data.

decision-making

radkit base v1.6

The radkit (base) software suite (python) consists of three primary libraries: stark, trajan, and curie. The trajan library provides the tools to analyze and manipulate data from lidar and inertial measurement unit (IMU) devices, cameras, as well as trajectories from algorithms such as simultaneous localization and mapping (SLAM). These components allow reading and writing standard data formats, performing rigid affine transformations, discretizing three-dimensional space, and visualizing data products. The curie library comprises a standard set of object-oriented tools for radiation data and analysis in the following modules: (1) listmode and binmode data classes with methods for manipulation, plotting, slicing and file IO; (2) radiological/nuclear source detection/identification analysis results; (3) source encounters of correlated analyses and (4) energy-dependent angular detector response functions. The stark package provides low-level tools that are leveraged by both curie and trajan. The tools are flexible for offline analysis as well as performant for real-time integrations.

Salathe, Marco [Lawrence Berkeley National Laborat

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis

Top Research Challenges and Opportunities for Near Real-Time Extreme-Scale Visualization of Scientific Data

The rapid advancement in scientific simulations and experimental facilities has resulted in the generation of vast amounts of data at unprecedented scales. The analysis and visualization of large amounts of data is a challenge in and of itself, but the requirements for timeliness significantly magnify these difficulties. Near real-time visualization is critical to monitor and analyze the data produced by these large facilities, but current production tools are not well-suited to these requirements. In this position paper, we share our perspective on some of the challenges, and thus, opportunities for research that stand in the way of near-real-time visualization of large scientific data.

Pugmire, Dave

Remote operation of the DIII-D National Fusion Facility

Abstract Full remote scientific operation of the DIII-D National Fusion Facility is now possible through significant advances in the computer science hardware and software infrastructure made over the last decade. Capabilities around information visualization, data movement, and communication have all been enhanced. The level of capability deployed to remotely operate DIII-D required an infrastructure advancement over what had previously been achieved in the fusion community. The large quantity of real-time data that is automatically displayed on DIII-D’s control room screens can now be visualized by remote participants via web-based applications. New audio/video solutions using the VoIP and instant messaging application Discord have been implemented to mimic the dynamic and ad-hoc scientific conversations that are critical in successfully operating an experimental campaign. Discord’s ability for a user to rapidly move between audio channels, text with images, and share screens is a significant enhancement over traditional videoconferencing tools. In addition, multiple combinations of broadcast audio are made available via a web-based application to allow remote participants to simultaneously listen to general announcements/sounds while conducting their own specific conversations. Secure methodologies have been put into place to allow remote control of hardware including DIII-D’s plasma control system application. Secure methods also included the ability of the on-site team to closely coordinate their work with remote team members which has been enhanced through extensions to the wireless network and the use of tablet computers for audio/video/screen sharing. However, no amount of software can fully replace the need for ‘hands on hardware.’ This infrastructure was severely stress tested during the COVID-19 pandemic where occupancy of the DIII-D control room was restricted. Operational efficiency during the pandemic, measured in discharges per hour, remained high (3.8 ± 0.8) compared to values obtained pre-pandemic (3.7 ± 0.8).

70 PLASMA PHYSICS AND FUSION TECHNOLOGY

Designing a User Interface for Real-Time Magnetometer Data Acquisition

The Matter-wave Atomic Gradiometer Interferometric Sensor (MAGIS-100) is a next-generation quantum sensor designed to search for ultralight dark matter and explore new frontiers in quantum mechanics. Due to the experiment s sensitivity to magnetic interference, a magnetometer trolley system was developed to scan magnetic fields along a vacuum tube. Interacting with the system required command-line inputs, creating usability challenges. To improve accessibility and streamline data acquisition, I developed a graphical user interface (GUI) using Python and the customtkinter library. The GUI supports real-time data display, state/mode switching, command execution, and CSV file management. I collaborated with another intern to integrate data visualization features into the GUI, allowing users to generate 3D plots of post-acquisition magnetic field data. In the future, I aim to fix the real-time plotting feature as it results in an unresponsive GUI.

Mendez, Milagros [DuPage Coll.]

Designing a User Interface for Real-Time Magnetometer Data Acquisition

The Matter-wave Atomic Gradiometer Interferometric Sensor (MAGIS-100) is a next-generation quantum sensor designed to search for ultralight dark matter and explore new frontiers in quantum mechanics. Due to the experiment’s sensitivity to magnetic interference, a magnetometer trolley system was developed to scan magnetic fields along a vacuum tube. Interacting with the system required command-line inputs, creating usability challenges. To improve accessibility and streamline data acquisition, I developed a graphical user interface (GUI) using Python and the customtkinter library. The GUI supports real-time data display, state/mode switching, command execution, and CSV file management. I collaborated with another intern to integrate data visualization features into the GUI, allowing users to generate 3D plots of post-acquisition magnetic field data. In the future, I aim to fix the real-time plotting feature as it results in an unresponsive GUI.

Mendez, Milagros [DuPage Coll.]

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY

Enabling Command-and-Control in Advanced In Situ Workflows

Scientific discovery is progressing towards autonomous science with the combination of scientific instruments, high-performance computing, and artificial intelligence in complex workflows. This evolution introduces new requirements for managing scientific workflows, including feedback loops, near real-time constraints, and the ability to dynamically control workflow execution. In situ workflows that analyze and visualize data as it is generated are well-suited to satisfy stringent time constraints and their iterative nature offers greater opportunities for command-and-control. However, only a few of the many workflow management systems available have been specifically designed to manage in situ workflows and often lack support for automated feedback loops that allow analysis and visualization components to interact with the main scientific data producer. To address this need, we present in this paper how to add command-and-control capabilities to a workflow management system. We identify the functional design requirements of such a command-and-control system, detail its architecture, interface, and core mechanisms, and illustrate how advanced in situ workflows can leverage command-and-control in three use cases: graceful termination with checkpoint, dynamic and adaptive data reduction, and event-triggered analysis.

Mehta, Kshitij [ORNL] (ORCID:0000000297149981)

Faraday: A High-temperature Electrolysis Data Explorer

Faraday is a high-temperature electrolysis data visualization tool, which reveals the performance of various button cells under test conditions. These tests and the resulting analytics on their data constitute a state of the industry as the US Department of Energy pushes for the production of hydrogen. Faraday leverages the Idaho National Laboratory's DeepLynx data warehouse to standardize and query button cell data. Faraday programmatically accesses this data in DeepLynx by traversing the schema, represented by a custom ontology. The user interface queries DeepLynx for timeseries data associated with specific button cells in the warehouse, and renders them using JavaScript charts. Additional charting and data analysis techniques are made possible by an auxiliary Python server.

Woodruff, Nathan

Meteorological and Soil Data from Ecohydrology Sensor Towers at Pump House and Snodgrass Mountain in East River Watershed, Colorado, 2019-2025

This data package includes hourly meteorological and soil sensor data at eight ecohydrology monitoring sites in East River Watershed, Colorado as part of the Watershed Function Scientific Focus Area (WFSFA) research led by Lawrence Berkeley National Lab (LBNL). Four field sites were located on the hillslope of East River (ER) near Pump House (PH) at Mount Crested Butte (ER-PHS1 to 4), and the other four are in the Snodgrass Mountain (SG) area (SG-EHS5 to 8). In terms of vegetation cover, three sites are in montane grasslands (ER-PHS1, ER-PHS2, and SG-EHS5), three are below evergreen conifer canopy (ER-PHS3, SG-EHS6, and SG-EHS7), and two are below deciduous aspen canopy (ER-PHS4 and SG-EHS8). The monitoring period began in October 2019 at the East River sites, in October 2020 at SG-EHS5 and SG-EHS6, and in October 2021 at SG-EHS7 and SG-EHS8. In September 2024, all four East River sites were fully retired. The four Snodgrass Mountain sites remain active. Each site is equipped with a comprehensive suite of meteorological sensors on a tripod and soil sensors that measure weather, energy fluxes, and soil variables. This data package includes measurements from ten different types of sensors and up to thirteen individual sensors per site, including (1) a weather station (measurement height ranges from 2.8~3.8 meters (m) above ground), (2) a quantum sensor for photosynthetic active radiation (PAR) (2.4~3.3m), (3) a net radiometer (1.7~2.1m), (4) an infrared radiometer (1.6~2.2m), (5) a sonic distance sensor (1.5~1.9m), (6) a soil carbon dioxide (CO2) flux chamber (0m), (7) a soil heat flux plate (-0.05m below ground), (8) a soil oxygen sensor (-0.3m), (9) a soil water potential sensor (-0.3m), and (10) soil water content sensors at 3~4 depths (-1.15 ~ -0.1m). A total of twenty-three variables is reported in this data package, including (1) atmospheric variables: air temperature (TA), atmospheric pressure (PA), vapor pressure (VP), and vapor pressure deficit (VPD), (2) precipitation variables: rain precipitation (P) and snow depth (D_SNOW), (3) energy fluxes variables: four-component net radiation (NETRAD) (shortwave/longwave incoming/outgoing radiation, SW_IN, SW_OUT, LW_IN, LW_OUT), photosynthetic photon flux density (PPFD), and soil heat flux (G), (4) soil variables: soil water content (SWC), soil water potential (SWP), soil temperature (TS), soil bulk electrical conductivity (COND_SOIL), and soil gaseous oxygen concentration (O2_SOIL), (5) wind variables: two-dimensional wind speed (WS), gust speed (WS_MAX), and wind direction (WD), and (6) surface variables: surface infrared temperature (T_CANOPY) and soil CO2 flux (CO2_SOIL). Please see the Methods section for data processing and QA/QC steps taken to generate the hourly datasets. The following files are included in this data package (notes on version: v{x}-{y}, where x is the metadata version, and y is the data version, when applicable): (1) “metadata_site_v{x}-{y}.csv” - a site metadata file that summarizes location information of all sites, including site ID, description, coordinates, timeframe, elevation, and vegetation cover, (2) “metadata_instrument_v{x}-{y}.csv” - an instrument metadata file that summarizes sensor information of all sites, including sensor manufacturer and model, measurement height, and sampling and averaging interval of all variables, (3) "data_{SITE_ID}_v{x}-{y}.csv" - eight data files that contain hourly data of each site indicated by {SITE_ID} in the filename, (4) “/figure/data_{SITE_ID}_v{x}-{y}.png" - eight figures that help visualize data of each site indicated by {SITE_ID} in the filename, (5) “/photo/*” - photos of each site indicated by {SITE_ID} in the filename, and (6) four file level metadata (flmd.csv) and data dictionary (*_dd.csv) files that summarize file, header, column, and variable information of all files. Notes: (1) Measurement height: Each variable name is followed by conventional positional qualifiers “H_V_R”, where H indicates the relative horizontal positions of that specific variable, V the vertical positions, and R the replicates. In this data package, only the vertical qualifier V varies, and V increases from the highest vertical position (V=1) to the lowest. Variables with the same qualifier are not necessarily measured by the same sensor, and the same variable with the same qualifier across different sites are not necessarily measured at the same height. Please refer to “metadata_instrument.csv” for the sensor information and measurement heights, and whether a variable is measured below the canopy. (2) Variable availability: Snow depth is not available at ER-PHS3 and SG-EHS7. SWC, soil temperature, and soil bulk EC at the deepest depth (<-1m) are not available at SG-EHS6 and SG-EHS7. The missing value code for numeric variables is -9999, except for SWP. For SWP, the missing value code is +9999, because SWP values are negative. (3) Sampling frequency: Please refer to “metadata_instrument.csv” for the increase of sampling frequency of some variables from 30-min to 1-min at ER-PHS1 to 4 in July 2020. (4) Sensors: While the methods of each sensor are not detailed, all sensors are commercially available, and their methods can be found in their manuals. Please refer to “metadata_instrument.csv” for the sensor manufacturer and model information. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

54 ENVIRONMENTAL SCIENCES