Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software: data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

PIPES (Pipeline for Integrated Projects in Energy Systems) [SWR-24-89]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. https://github.com/nrel-pipes/pipes-api https://github.com/nrel-pipes/pipes-web https://github.com/nrel-pipes/nrel-pipes

Gu, Jianli↗
DOE OSTI2020

ASCENDS: Advanced data SCiENce toolkit for Non-Data Scientists

Recently, advances in machine learning and artificial intelligence have been playing more and more critical roles in a wide range of areas. For the last several years, industries have shown that how learning from data, identifying patterns and making decisions with minimal human intervention can be extremely useful to their business (e.g., image classification, recommending a product to a customer, finding friends in a social network, predicting customers actions, etc.). These success stories have been motivating scientists who study physics, chemistry, materials, medicine, and many other subjects, to explore a new pathway of utilizing machine learning techniques like regression and classification for their scientific activities. However, most existing machine learning tools, systems, and methodologies have been developed for programming experts but not for scientists (or any users) who have no or little knowledge of programming. ASCENDS is a toolkit that is developed to assist scientists (or any persons) who want to use their data for machine learning tasks, more specifically, correlation analysis, regression, and classification. ASCENDS does not require programming skills. Instead, it provides a set of simple but powerful CLI (Command Line Interface) and GUI (Graphic User Interface) tools for non-data scientists to be able to intuitively perform advanced data analysis and machine learning techniques. ASCENDS has been implemented by wrapping around opensource software including Keras, TensorFlow, and scikit-learn.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Pipeline for Integrated Projects in Energy Systems (PIPES): A Tool for Integrated System Planning [Slides]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. This presentation introduces PIPES a multi-model tool for integrated system planning; it describes the underlying architecture, deep dives into common user workflows, and outlines the upcoming development roadmap beyond its current alpha state.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2020

The parallelism motifs of genomic data analysis

Genomic datasets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share these data with the research community, but some of these genomic data analysis problems require large-scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high-end parallel systems today and place different requirements on programming support, software libraries and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high-performance genomics analysis, including alignment, profiling, clustering and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or ‘motifs’ that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Studying the hadron structure with PANDA and CLAS using machine learning techniques

The hadron spectroscopy and structure are currently very active fields of research to study the non-perturbative regime of quantum chronodynamics. The first one studies the complex structure of excited hadrons by looking at their decay products, while the latter uses lepton scattering on nucleons. Both methods require reconstruction algorithms with great efficiency and good particle identification and background rejection rates. This work aims to provide these by either improving the existing methods or developing new ones. The first part of this document presents a feasibility study of a predicted hybrid charmonium state for the PANDA experiment. Lattice QCD calculations predict the ground state hybrid charmonium to be a spin exotic with quantum numbers of JP C = 1?+ at a mass of around 4.3 GeV with a width to be around 20 MeV. A machine learning based data analysis scheme is proposed to further improve the signal efficiency and the background reduction, alongside with improvements of the analysis software (PandaRoot), that are vital for this study. These improvements include a reworked clustering algorithm for the electromagnetic calorimeter (EMC) and an optimized monte carlo matching for neutral particles. The second part of this document is about studying the proton structure. A multidimensional study of the structure function ratio Fsin(?)LU /FUU has been performed for K±, based on the measurement of beam-spin asymmetries. It uses the high statistics data recorded with the CLAS12 spectrometer at Jefferson Laboratory. Fsin(?)LU is a twist-3 quantity that provides information about the quark gluon correlations in the proton. This document will present for the first time a simultaneous analysis of two kaon channels over a large kinematic range of z, xB , PT and Q2 with virtualities Q2 ranging from 1 GeV2 up to 8 GeV2 using machine learning techniques for improved particle identification.

Kripko, Aron↗
DOE OSTI2022

A Systematic Review and Meta-Analysis of Tai Chi Training in Cardiorespiratory Fitness of Elderly People

Objectives. The purpose of this study was to investigate the influence of Tai Chi on cardiorespiratory fitness (CRF) in elderly people using meta-analysis. Methods. This study used seven electronic databases and data retrieved from randomized controlled trials (RCTs) investigating the role of Tai Chi on CRF in the elderly. All these 24 RCTs were screened and selected from 7 literature databases. The Stata 11.2 software (StataCorp, USA) was used for the meta-analysis, subgroup analysis, and bias test, while the Cochrane Collaboration’s tool was used for the assessment of the risk of bias (RoB). 4 researchers independently participated in sample selection, data extraction, and RoB assessment. Results. Following the inclusion criteria, 24 eligible studies were included in our analysis. The meta-analysis indicated that Tai Chi practice significantly increased the maximum rate of oxygen consumption (VO2 max) (weighted mean difference (WMD) = 3.76, 95% CI: 1.25 to 6.26, P < 0.1 ), leading to an overall reduction in the heart rate (HR) (WMD = −1.84, 95% CI: −2.04 to −1.63, P ≤ 0.001 ) and an increase in the O2 pulse (WMD = 0.94, 95% CI: 0.60 to 1.28, P ≤ 0.001 ) in individuals who practiced Tai Chi regularly compared with those who did not. The subgroup analysis suggested that overall in those who practiced Tai Chi, males (WMD = 1.48, 95% CI: 0.85 to 2.12, P ≤ 0.001 ) had higher O2 pulse than females (WMD = 0.73, 95% CI: 0.33 to 1.12, P ≤ 0.001 ). The subgroup analysis also showed an increase in the vital capacity (VC) (WMD = 316.05, 95% CI: 239.74 to 392.35, P ≤ 0.001 ) in individuals practicing Tai Chi. When the samples were further stratified by Tai Chi practicing time, the subgroup analysis suggested that individuals practicing Tai Chi over a period of 24 weeks showed no significant difference in VC (WMD = 82.95, 95% CI: -98.34 to 264.23, P = 0.370 ), while those practicing Tai Chi over a period of 48 weeks showed a significant increase (WMD = 416.62, 95% CI: 280.68 to 552.56, P ≤ 0.001 ). Furthermore, the subgroup analysis demonstrated that the increase in VC is significantly correlated with the Tai Chi practicing time (WMD = 344.97, 95% CI: 227.88 to 442.06, P ≤ 0.001 ). Conclusion. Regular Tai Chi practice could improve the CRF in the elderly, as indicated by significant improvement in indicators including VO2max, O2pulse, VC, and HR. However, gender and practice time might influence the overall beneficial outcomes.

Tan, Tianyang↗
DOE OSTI2025

Improving Cyber Situational Understanding

Effective cybersecurity operations require the ability to analyze large amounts of information to assess security risks and formulate defensive strategies against adversaries. This has become more complex in recent years as the sprawl and interconnectivity of devices grows through implementation of virtualization, cloud computing, and Internet of Things (IoT). The amount of data and analysis required for effective cybersecurity command and control decisions far exceeds humans’ capacity to perform manually. We characterize the analysis problem as cyber situational understanding. The research presented to improve cyber situational understanding focuses on vulnerability analysis and threat intelligence. Regarding vulnerabilities, entities must analyze and plan work for between thousands and tens of thousands of software vulnerabilities annually. Entities heavily use network firewalls to limit vulnerability exposure. As a result, some of these vulnerabilities permit exposure to adversarial exploitation, whereas others are inaccessible and therefore present negligible risk of exploitation. Distinguishing between high and low risk software vulnerabilities requires a deep understanding of the vulnerability, network firewall protection, and characteristics of the targeted device. This problem is solved by extracting network service features from vulnerability data features using both machine-learning and natural language processing. Then, the network firewall topology is parsed to determine which vulnerabilities are reachable by adversaries. Ultimately, a state-based safety analysis ascertains which vulnerabilities are unsafe. A related vulnerability analysis problem occurs in cybersecurity operations when associating an entity’s hardware and software assets to public vulnerability databases. Assets often reveal hardware and software through installation artifacts and network service identification, and entities store these artifacts in inventory databases. However, software and hardware vendors apply a standard Common Platform Enumeration (CPE) naming convention when publicly reporting vulnerabilities. Associating these two datasets often requires many hours to days of manual inspection. The proposed solution automates the mapping approach of human analysts using fuzzy matching techniques, natural language processing, and, ultimately, machine learning to present a small set of recommendations for mapping the two datasets. The result significantly reduces human analysis time and reduces the occurrence of false positives in vulnerability notifications. Finally, cyber threat intelligence (CTI) requires associating cyber observable artifacts, such as IP addresses, URIs, and file hashes, with cyber threat tactics, techniques, and procedures. Unfortunately, most CTI data is compartmentalized across multiple organizations and cannot be shared due to the legal and reputational risk with cyber threat being associated with the entity. The approach to solving this problem inovlves using a distributed ledger with anonymous token spending and authentication. This allows a consortium of semi-trusted entities to share the workload of curating CTI for a threat sharing community’s cooperative benefit.

Huff, Philip↗
DOE OSTI2026

MCP-eGridGPT (MCP-Enabled Chatbot with Electrical Power System Analysis and Interactive Visualization Tool) [SWR-25-126]

This software is an advanced chatbot system that integrates the Model Context Protocol (MCP) to provide intelligent electrical power system analysis and automated visualization generation. The system enables users to interact with complex electrical engineering tools through natural language, automatically analyzes power system data for voltage violations and grid health assessment, and generates professional interactive HTML dashboards and reports. Key features include dynamic tool discovery from MCP servers, multi-LLM provider support, intelligent data interpretation using large language models, automated chart generation, and a web-based interface for real-time analysis. The software bridges sophisticated electrical engineering analysis with user-friendly interfaces, making power system diagnostics accessible through conversational AI.

Choi, Seong [National Laboratory of the Rockies (N↗
DOE OSTI2023

Operational Energy Life Cycle Data Development for the National Institute of Standards And Technology (NIST) Building Industry Reporting and Design for Sustainability (BIRDS) Neutral Environmental Software Tool (NEST)

For this analysis, regionalized life cycle assessment (LCA) results for environmental impacts (using the Tool for Reduction and Assessment of Chemicals and Other Environmental Impacts [TRACI] 2.1) and cumulative energy demand (using the Federal Life Cycle Analysis Commons Elementary Flow List [FEDEFL] Inventory Methods v1.0.0) were evaluated for the production and utilization of electricity, natural gas, fuel oil, and propane as commodities within residential and commercial buildings. These results can used as a framework for future research into net zero, high-performance buildings, such as done here for the Building Industry Reporting and Design for Sustainability (BIRDS) database by the National Institute of Standards and Technology (NIST) Engineering Laboratory. The geographical results were assigned to each United States (U.S.) Zone Improvement Plan (ZIP) code based on the ZIP code location and corresponding Balancing Authority Area, natural gas basin, and Petroleum Administration for Defense Districts (PADDs). Additionally, previously developed models were utilized to develop future life cycle profiles. Projections were based on data available from the U.S. Energy Information Administration Annual Energy Outlook 2022 through 2050 (AEO 2022). Electricity LCA models were updated based on AEO 2022 projected annual generation mixes, while the natural gas baseline model was updated based on projected shares of natural gas types (conventional, shale, tight, and coalbed methane). Projections of crude oil production rates and export rates were applied to the petroleum baseline model in five-year increments to investigate their effects on the life cycle profile of fuel oil and propane. While only 100-year Global Warming Potential (GWP-100) with climate carbon feedback (CC-FB) and Cumulative Energy Demand are shown in Section 4: Results, the complete results, including Acidification Potential, Eutrophication Potential, Freshwater Ecotoxicity Potential, GWP-100 without inclusion of CC-FB, Human Health Impacts Potentials (Cancer, Non-Cancer), Ozone Depletion Potential, Particulate Matter Formation Potential, and Photochemical Smog Formation Potential, are tabulated for each ZIP code in the Excel worksheets that accompany this analysis. For the Excel spreadsheet tools associated with this report, please go to https://www.netl.doe.gov/energy-analysis/details?id=f8890fac-be55-44ac-aaa9-e2888bfabe93

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

KBase partners with community developers from several Department of Energy Science Focus Areas to increase functionality of tools and analysis

Several SFAs were selected for training on the KBase Software Development Kit (SDK) to integrate tools and data into the KBase platform. Overall, the collaborative effort was viewed as highly successful. Every SFA project was able to achieve their proposed targets for training and functionality, while also gaining an appreciation for the role KBase plays/could play in their future science plans. Both the SFA and KBase teams felt that continued collaboration would be beneficial, especially if the programs were able to jointly prioritize scope and readiness of future projects. This poster will outline the scope of functionality added by SFA community developers, and highlight points of intersection across these efforts.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Science of the Van Allen Probes Science Operations Centers

The Van Allen Probes mission operations materialized through a distributed model in which operational responsibility was divided between the Mission Operations Center (MOC) and separate instrument specific SOCs. The sole MOC handled all aspects of telemetering and receiving tasks as well as certain scientifically relevant ancillary tasks. Each instrument science team developed individual instrument specific SOCs proficient in unique capabilities in support of science data acquisition, data processing, instrument performance, and tools for the instrument team scientists. In parallel activities, project scientists took on the task of providing a significant modeling tool base usable by the instrument science teams and the larger scientific community. With a mission as complex as Van Allen Probes, scientific inquiry occurred due to constant and significant collaboration between the SOCs and in concert with the project science team. Planned cross-instrument coordinated observations resulted in critical discoveries during the seven-year mission. Instrument cross-calibration activities elucidated a more seamless set of data products. Specific topics include post-launch changes and enhancements to the SOCs, discussion of coordination activities between the SOCs, SOC specific analysis software, modeling software provided by the Van Allen Probes project, and a section on lessons learned. One of the most significant lessons learned was the importance of the original decision to implement individual team SOCs providing timely and well-documented instrument data for the NASA Van Allen Probes Mission scientists and the larger magnetospheric and radiation belt scientific community.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Analysis of X-ray images and spectra (aXis2000): A toolkit for the analysis of X-ray spectromicroscopy data

Spectromicroscopy refers to analytical methods that combine imaging and spectroscopy to provide detailed, spatially resolved analytical information about a sample, such as the type and quantitative spatial distributions of chemical components, geometric or magnetic alignment information, crystal structure, etc. The analysis of X-ray images and spectra (aXis2000) software described in this work provides a set of routines within a single, integrated, graphical-oriented package to read, display, manipulate and analyze spectromicroscopy data, with particular focus on soft X-ray spectromicroscopy methods such as scanning transmission X-ray microscopy (STXM), X-ray photoemission electron microscopy (XPEEM), scanning photoelectron X-ray microscopy (SPEM) and transmission X-ray microscopy (TXM). Here, this free software is described and compared to other software that can provide similar or complementary capabilities. Examples of spectromicroscopic analyses using advanced features of aXis2000 are provided.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Practical galaxy morphology tools from deep supervised representation learning

Astronomers have typically set out to solve supervised machine learning problems by creating their own representations from scratch. We show that deep learning models trained to answer every Galaxy Zoo DECaLS question learn meaningful semantic representations of galaxies that are useful for new tasks on which the models were never trained. We exploit these representations to outperform several recent approaches at practical tasks crucial for investigating large galaxy samples. The first task is identifying galaxies of similar morphology to a query galaxy. Given a single galaxy assigned a free text tag by humans (e.g. ‘#diffuse’), we can find galaxies matching that tag for most tags. The second task is identifying the most interesting anomalies to a particular researcher. Our approach is 100 per cent accurate at identifying the most interesting 100 anomalies (as judged by Galaxy Zoo 2 volunteers). The third task is adapting a model to solve a new task using only a small number of newly labelled galaxies. Models fine-tuned from our representation are better able to identify ring galaxies than models fine-tuned from terrestrial images (ImageNet) or trained from scratch. We solve each task with very few new labels; either one (for the similarity search) or several hundred (for anomaly detection or fine-tuning). This challenges the longstanding view that deep supervised methods require new large labelled data sets for practical use in astronomy. To help the community benefit from our pretrained models, we release our fine-tuning code zoobot. Zoobot is accessible to researchers with no prior experience in deep learning.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗
DOE OSTI2021

Exploring Explicit Uncertainty for Binary Analysis (EUBA)

Reverse engineering (RE) analysts struggle to address critical questions about the safety of binary code accurately and promptly, and their supporting program analysis tools are simply wrong sometimes. The analysis tools have to approximate in order to provide any information at all, but this means that they introduce uncertainty into their results. And those uncertainties chain from analysis to analysis. We hypothesize that exposing sources, impacts, and control of uncertainty to human binary analysts will allow the analysts to approach their hardest problems with high-powered analytic techniques that they know when to trust. Combining expertise in binary analysis algorithms, human cognition, uncertainty quantification, verification and validation, and visualization, we pursue research that should benefit binary software analysis efforts across the board. We find a strong analogy between RE and exploratory data analysis (EDA); we begin to characterize sources and types of uncertainty found in practice in RE (both in the process and in supporting analyses); we explore a domain-specific focus on uncertainty in pointer analysis, showing that more precise models do help analysts answer small information flow questions faster and more accurately; and we test a general population with domain-general sudoku problems, showing that adding "knobs" to an analysis does not significantly slow down performance. This document describes our explorations in uncertainty in binary analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sandia Toolkit Manual (V.5.21.1)

This report provides documentation for the Sandia Toolkit (STK) modules. STK modules are intended to provide infrastructure that assists the development of computational engineering software such as finite-element analysis applications. STK includes modules for unstructured-mesh data structures, reading/writing mesh files, geometric proximity search, transfers, MPMD coupling support, and various other utilities. This document contains a chapter for each module, and each chapter contains overview descriptions and usage examples. Usage examples are primarily code listings which are generated from working test programs that are included in the STK code-base. A goal of this approach is to ensure that the usage examples will not fall out of date.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Sandia Toolkit Manual Version 5.15.6

This report provides documentation for the Sandia Toolkit (STK) modules. STK modules are intended to provide infrastructure that assists the development of computational engineering software such as finite-element analysis applications. STK includes modules for unstructured-mesh data structures, reading/writing mesh files, geometric proximity search, and various utilities. This document contains a chapter for each module, and each chapter contains overview descriptions and usage examples. Usage examples are primarily code listings which are generated from working test programs that are included in the STK code-base. A goal of this approach is to ensure that the usage examples will not fall out of date.

97 MATHEMATICS AND COMPUTING↗