Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis, visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Landscape rules predict optimal superhighways for the first peopling of Sahul

Archaeological data and demographic modelling suggest that the peopling of Sahul required substantial populations, occurred rapidly within a few thousand years and encompassed environments ranging from hyper-arid deserts to temperate uplands and tropical rainforests. How this migration occurred and how humans responded to the physical environments they encountered have, however, remained largely speculative. By constructing a high-resolution digital elevation model for Sahul and coupling it with fine-scale viewshed analysis of landscape prominence, least-cost pedestrian travel modelling and high-performance computing, here we create over 125 billion potential migratory pathways, whereby the most parsimonious routes traversed emerge. Our analysis revealed several major pathways—superhighways—transecting the continent, that we evaluated using archaeological data. These results suggest that the earliest Australian ancestors adopted a set of fundamental rules shaped by physiological capacity, attraction to visually prominent landscape features and freshwater distribution to maximize survival, even without previous experience of the landscapes they encountered.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Root system architecture and environmental flux analysis in mature crops using 3D root mesocosms

Current methods of root sampling typically only obtain small or incomplete sections of root systems and do not capture their true complexity. To facilitate the visualization and analysis of full-sized plant root systems in 3-dimensions, we developed customized mesocosm growth containers. While highly scalable, the design presented here uses an internal volume of 45 ft 3 (1.27 m 3 ), suitable for large crop and bioenergy grass root systems to grow largely unconstrained. Furthermore, they allow for the excavation and preservation of 3-dimensional root system architecture (RSA), and facilitate the collection of time-resolved subterranean environmental data. Sensor arrays monitoring matric potential, temperature and CO 2 levels are buried in a grid formation at various depths to assess environmental fluxes at regular intervals. Methods of 3D data visualization of fluxes were developed to allow for comparison with root system architectural traits. Following harvest, the recovered root system can be digitally reconstructed in 3D through photogrammetry, which is an inexpensive method requiring only an appropriate studio space and a digital camera. We developed a pipeline to extract features from the 3D point clouds, or from derived skeletons that include point cloud voxel number as a proxy for biomass, total root system length, volume, depth, convex hull volume and solidity as a function of depth. Ground-truthing these features with biomass measurements from manually dissected root systems showed a high correlation. We evaluated switchgrass, maize, and sorghum root systems to highlight the capability for species wide comparisons. We focused on two switchgrass ecotypes, upland (VS16) and lowland (WBC3), in identical environments to demonstrate widely different root system architectures that may be indicative of core differences in their rhizoeconomic foraging strategies. Finally, we imposed a strong physiological water stress and manipulated the growth medium to demonstrate whole root system plasticity in response to environmental stimuli. Hence, these new “3D Root Mesocosms” and accompanying computational analysis provides a new paradigm for study of mature crop systems and the environmental fluxes that shape them.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Unified Language Frontend for Physic-Informed AI/ML

Artificial intelligence and machine learning (AI/ML) are becoming important tools for scientific modeling and simulation as in several other fields such as image analysis and natural language processing. ML techniques can leverage the computing power available in modern systems and reduce the human effort needed to configure experiments, interpret and visualize results, draw conclusions from huge quantities of raw data, and build surrogates for physics based models. Domain scientists in fields like fluid dynamics, microelectronics and chemistry can automate many of their most difficult and repetitive tasks or improve the design times by use of the faster ML-surrogates. However, modern ML and traditional scientific highperformance computing (HPC) tend to use completely different software ecosystems. While ML frameworks like PyTorch and TensorFlow provide Python APIs, most HPC applications and libraries are written in C++. Direct interoperability between the two languages is possible but is tedious and error-prone. In this work, we show that a compiler-based approach can bridge the gap between ML frameworks and scientific software with less developer effort and better efficiency. We use the MLIR (multi-level intermediate representation) ecosystem to compile a pre-trained convolutional neural network (CNN) in PyTorch to freestanding C++ source code in the Kokkos programming model. Kokkos is a programming model widely used in HPC to write portable, shared-memory parallel code that can natively target a variety of CPU and GPU architectures. Our compiler-generated source code can be directly integrated into any Kokkosbased application with no dependencies on Python or cross-language interfaces.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Supporting Greater Interactivity in the IPython Visualization Ecosystem

Interactive visualizations are invaluable tools for building intuition and supporting rapid exploration of datasets and models. Numerous libraries in Python support interactivity, and workflows that combine Jupyter and IPyWidgets in particular make it straightforward to build data analysis tools on the fly. However, the field is missing the ability to arbitrarily overlay widgets and plots on top of others to support more flexible details-on-demand techniques. This work discusses some limitations of the base IPyWidgets library, explains the benefits of IPyVuetify and how it addresses these limitations, and finally presents a new open-source solution that builds on IPyVuetify to provide easily integrated widget overlays in Jupyter.

Martindale, Nathan↗
DOE OSTI2024

An ontology-based knowledge graph for representing interactions involving RNA molecules

The "RNA world" represents a novel frontier for the study of fundamental biological processes and human diseases and is paving the way for the development of new drugs tailored to each patient's biomolecular characteristics. Although scientific data about coding and non-coding RNA molecules are constantly produced and available from public repositories, they are scattered across different databases and a centralized, uniform, and semantically consistent representation of the "RNA world" is still lacking. We propose RNA-KG, a knowledge graph (KG) encompassing biological knowledge about RNAs gathered from more than 60 public databases, integrating functional relationships with genes, proteins, and chemicals and ontologically grounded biomedical concepts. To develop RNA-KG, we first identified, pre-processed, and characterized each data source; next, we built a meta-graph that provides an ontological description of the KG by representing all the bio-molecular entities and medical concepts of interest in this domain, as well as the types of interactions connecting them. Finally, we leveraged an instance-based semantically abstracted knowledge model to specify the ontological alignment according to which RNA-KG was generated. RNA-KG can be downloaded in different formats and also queried by a SPARQL endpoint. A thorough topological analysis of the resulting heterogeneous graph provides further insights into the characteristics of the "RNA world". RNA-KG can be both directly explored and visualized, and/or analyzed by applying computational methods to infer bio-medical knowledge from its heterogeneous nodes and edges. The resource can be easily updated with new experimental data, and specific views of the overall KG can be extracted according to the bio-medical problem to be studied.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Automating Log Synthesis and Visualization with Python and Splunk

The goal of this project is to automate log analysis by utilizing Splunk, Bash, and Python together. Simplifying the monitoring and analysis of network traffic was the main goal. In order to accomplish this, a Bash script was created to use 'tcpdump' to automate network sniffing. It also included a 24-hour file rotation mechanism to effectively manage the pcap files that were generated. After that, a Python script was written to read these pcap files and retrieve pertinent data about network traffic. After processing the collected data, Splunk is used to summarize the important metrics and visualize said information with relevant graphs.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Quantification and visualization of uncertainties in reconstructed penumbral images of implosions at Omega

Penumbral imaging is a technique used in plasma diagnostics in which a radiation source shines through one or more large apertures onto a detector. To interpret a penumbral image, one must reconstruct it to recover the original source. The inferred source always has some error due to noise in the image and uncertainty in the instrument geometry. Interpreting the inferred source thus requires quantification of that inference’s uncertainty. Markov chain Monte Carlo algorithms have been used to quantify uncertainty for similar problems but have never been used for the inference of the shape of an image. Because of this, there are no commonly accepted ways of visualizing uncertainty in two-dimensional data. This paper demonstrates the application of the Hamiltonian Monte Carlo algorithm to the reconstruction of penumbral images of fusion implosions and presents ways to visualize the uncertainty in the reconstructed source. This methodology enables more rigorous analysis of penumbral images than has been done in the past.

Instruments & Instrumentation↗
DOE OSTI2020

ROBIN (RuciO/BIgData Express/SENSE) A Next-Generation High-Performance Data Service Platform

Increasingly, scientific research discoverydepends on the analysis of extremely large amounts of data. Thisdata is gathered through very large scientific instruments, verylarge numbers of small devices, and mid-scale instruments.These devices generate exponentially increasing volumes of data.Such data is often obtained, analyzed, stored, visualized, andtransferred at and among sites world-wide by largecollaborations of science communities. Managing and movingextremely large volumes of data across today’s networks is aspecial multidimensional challenge. Also, existing datamanagement and movement services and tools often areinadequate to address all specific requirements. This paperdescribes the various components of this challenge and howthose issues can be addressed by ROBIN, a uniquecomprehensive set of integrated services designed specificallyfor managing and moving extremely large amounts of data overlong distances, e.g., thousands of miles around the globe. Thispaper also describes the results of initial experiments using thatset of integrated services.

Wu, Wenji↗
DOE OSTI2024

helios: An R package to process heating and cooling degrees for GCAM

helios is an open-source R package that estimates population-weighted heating and cooling degree-hours (HDH and CDH) and degree-days (HDD and CDD) at various temporal (e.g., energy dispatch segments, monthly, yearly) and spatial scales (e.g., U.S. states, global political regions, countries). The degree hour and degree day outputs from helios are used to inform electricity demand load in the Global Change Analysis Model (GCAM) as well as in GCAM-USA (which is the version of GCAM with U.S. state-level details). helios uses a workflow with four steps: processing raw data; calculating heating and cooling degrees; visualizing performance diagnostics; and outputing results in various formats. There are two sources of widely-used climate data compatible with helios: (1) hourly climate data with 12-km resolution that are dynamically downscaled with the Weather Research and Forecasting (WRF) model and projected using a thermal global warming (TGW) approach; and (2) daily climate data with 0.5-degree resolution from the Coupled Model Intercomparison Project (CMIP) that is bias-adjusted and statistical downscaled by the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP). In summary, helios is a model that standardizes methodology of heating and cooling degrees-hours and degree-days using publicly available data and advance the understanding of the impact of spatial and temporal temperature variability on building energy services.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

Data Analysis Approach for Large Data Volumes in a Connected Community

Recent advancements within smart neighborhoods where utilities are enabling automatic control of appliances such as heating, ventilation, and air conditioning (HVAC) and water heater (WH) systems are providing new opportunities to minimize energy costs through reduced peak load. This requires systematic collection, storage, management, and in-memory processing of large volumes of streaming data for fast performance. In this paper, we propose a multi-tier layered IoT software framework that enables effective descriptive and predictive data analysis for understanding live operation of the neighborhood, fault identification, and future opportunities for further optimization of load curves. We then demonstrate how we achieve live situational awareness of the connected neighborhood through a suite of visualization components. Finally, we discuss a few analytic dashboards that address questions such as peak load reductions obtained due to optimization, customer preference for automatic control of appliances (do they override the automatic control of HVAC?, etc.). 1 1 This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Chinthavali, Supriya↗
DOE OSTI2020

Mirostructure Characterization of Friction Consolidated Copper-Nickel using a Machine Learning Approach: Developing Process to Microstructure Associations

Friction consolidation (FC) is a solid phase processing approach where discrete material forms such as powders, chips, nuggets, etc. are densified via shear deformation. The precursors are placed in a billet container and brought in contact with a rotating tool that applying the desirable amount of normal force. Under the combined action of the rotation and normal pressure, the discrete precursor is consolidated through porosity reduction and shear deformation. FC is increasingly being studied as an attractive approach to manufacturing fully dense parts from powder forms owing to its ability to mix, alloy and consolidate difficult-to-process precursors in minimal number of process steps. Material consolidation and deformation in shear consolidation processes have been studied extensively previously for different material combinations previously. However, despite the extensive research in this area, understanding of the mechanistic processes in pore consolidation, deformation-induced mixing and material solubility during FC is still evolving. Material development using solid phase processing approaches such as FC is often performed based on research experience/education, which can be biased. Conventional analysis and simulation tools in this area tend to be successful only when material thermodynamic pathways and microstructural evolution sequences resulting from processing are clearly defined or known. They are not as effective for emerging advanced manufacturing technologies where material evolution pathways are not well established. The ability to predict optimal process parameters based on material chemistry and bulk properties is essential to accelerate materials design and processing, as are an understanding of the relevant structure-processing-property relationships. These structure-processing-property-performance relationships are at the core of materials science research. Microstructure characterization provides the link to these four core areas, often through visualizing material microstructure using imaging techniques. However, linking microstructure image data (i.e., micrographs) to variables of interest (e.g., processing parameters, material chemistry) in a reproducible, generalizable, and quantitative manner is a significant challenge. Typically, quantitatively linking image data to processing history relies on significant domain knowledge and manual or subject matter expert (SME)-heuristic based image analysis. Such an approach to image analysis has the potential to be biased, inefficient, and difficult to replicate.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Positron emission tomography harmonization in the Alzheimer's Disease Neuroimaging Initiative: A scalable and rigorous approach to multisite amyloid and tau quantification

Abstract INTRODUCTION A key goal of the Alzheimer's Disease NeuroImaging Initiative (ADNI) positron emission tomography (PET) Core is to harmonize quantification of β‐amyloid (Aβ) and tau PET image data across multiple scanners and tracers. METHODS We developed an analysis pipeline (Berkeley PET Imaging Pipeline, B‐PIP) for ADNI Aβ and tau PET images and applied it to PET data from other multisite studies. Steps include image pre‐processing, refacing, magnetic resonance imaging (MRI)/PET co‐registration, visual quality control (QC), quantification of tracer uptake, and standardization of Aβ and tau standardized uptake value ratios (SUVrs) across tracers. RESULTS Measurements from 10,105 cross‐sectional and longitudinal Aβ and tau PET scans acquired in several studies between 2010 and 2024 can be processed, harmonized, and directly merged across tracers and cohorts. DISCUSSION The B‐PIP developed in ADNI is a scalable image harmonization approach used in several observational studies and clinical trials that facilitates rigorous Aβ and tau PET quantification and data sharing. Highlights Quantitative results from ADNI Aβ and tau PET data are generated using a rigorous, scalable image processing pipeline This pipeline has been applied to PET data from several other large, multisite studies and trials Quantitative outcomes are harmonizable across studies and are shared with the scientific community

Neurosciences & Neurology↗
DOE OSTI2020

Elevation distributed micro-climatology data in a coastal glaciated watershed

Air temperature, ground temperature and relative humidity data were collected in a longitudinal transect of the Nooksack watershed at varying elevations from 500 to 1800 m above sea level. Data were collected by anchoring sensors from trees above winter snow levels and shaded from direct solar radiation. Paired sensors were also buried 3 cm under ground near each air temperature sensor to determine snow absence or presence. Select sites included relative humidity sensors to indicate whether precipitation was occurring. Data were collected every 3-4 h from December 2015 to Sept 2018 (with ongoing collection). Code for analysis of daily mean, minimum, maximum, and temperature change with elevation (lapse rates) are available on Github (https://doi.org/10.5281/zenodo.3239539). The sensor download and intermediate data products are available on HydroShare at (http://www.hydroshare.org/resource/222e832d3df24dea9bae9bbeb6f4219d) with publicly accessible visualization available from the Nooksack Observatory at data.cuahsi.org. Hydrologic models are generally structured with a single annual average lapse rate parameter which assumes a linear temperature gradient with elevation. The daily data (2016-2018) is used as part of ongoing studies on the non-linear dynamics and temporal variability of temperature with elevation to improve assessments of watershed function and salmon habitat.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Inferring the shape of data: a probabilistic framework for analysing experiments in the natural sciences

A critical step in data analysis for many different types of experiments is the identification of features with theoretically defined shapes in N -dimensional datasets; examples of this process include finding peaks in multi-dimensional molecular spectra or emitters in fluorescence microscopy images. Identifying such features involves determining if the overall shape of the data is consistent with an expected shape; however, it is generally unclear how to quantitatively make this determination. In practice, many analysis methods employ subjective, heuristic approaches, which complicates the validation of any ensuing results—especially as the amount and dimensionality of the data increase. Here, we present a probabilistic solution to this problem by using Bayes’ rule to calculate the probability that the data have any one of several potential shapes. This probabilistic approach may be used to objectively compare how well different theories describe a dataset, identify changes between datasets and detect features within data using a corollary method called Bayesian Inference-based Template Search; several proof-of-principle examples are provided. Altogether, this mathematical framework serves as an automated ‘engine’ capable of computationally executing analysis decisions currently made by visual inspection across the sciences.

Science & Technology - Other Topics↗
DOE OSTI2023

Compilation of a Solar Mirror Materials Database and an Analysis of Natural and Accelerated Mirror Exposure and Degradation

The National Renewable Energy Laboratory (NREL) has been conducting exposure experiments on solar reflectors for over four decades. Thousands of mirror samples from over one hundred suppliers have been exposed to and monitored in a range of relevant environments. These test conditions include outdoor test settings and several controlled laboratory environments. These samples have been rigorously individually characterized using a series of reflectance measurements, visual inspections, and in some cases, in-depth composition analysis to identify degradation modes, reflectance losses, and other mirror properties integral to understanding the solar reflector's life cycle. Here, this paper compiles the decades of measurement data into a concise statistical analysis. It includes exposure and degradation data for numerous reflector types, including secondary-surface reflector permutations of polymer and glass superstrates with silver and aluminum reflectors as well as front-surface reflectors. The results herein are intended to analyze environmental stressors and degradation trends among various historical and state-of-the-art solar reflectors. It may be used to support solar reflector design, effective testing methodology, and inform manufacturing decisions moving forward. Presented are the results of the compiled database and an initial analysis for degradation rate modeling using full-spectrum and wavelength-dependent approaches. The database is a growing resource hosted on a live, publicly accessible website. In conjunction with the analysis presented here, it provides a valuable resource to the solar reflector manufacturing and testing industry.

14 SOLAR ENERGY↗
DOE OSTI2020

Characterizing Dislocation Defects in Dark Field X-ray Microscopy Images of Bulk Aluminum

Material defects play a large role in material response under shock loading, yet the process of how these defects initiate, propagate, and annihilate is not well understood. Using a newly developed diagnostic, dark field X-ray microscopy (DFXM), we can now visualize the behavior of dislocation defects in materials at the mesoscale under varying conditions. Using data from the European Synchrotron Radiation Facility, we apply a variety of image processing techniques to capture relevant features to locate and characterize size and orientation of dislocation defects in bulk aluminum DFXM images. Beyond simple visualization, this analysis drives statistical characterization of the defects and their dynamic response to further improve the relevant physics models.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Noodles: Cooking Up Collaborative Visualization

NOODLES is a new cross-domain collaborative visualization and analysis capability being developed at NREL. The NOODLES specification is a minimal protocol that can tie different software packages and platforms together. As an example, a simulation code that speaks this protocol could stream iso-surfaces to an immersive VR space and a web browser. Another example is a team (with some members across the country) classifying and discovering features in a statistical data plot, driven from a researcher's existing Python-based workflow. In this talk, we will discuss some background on the problems that this protocol intends to solve, basic principles of the specification, the current state of supporting libraries, and some demonstrations of the protocol in action.

3D↗