SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
System assessments typically yield large quantities of data from disparate sources for an analyst to scrutinize for issues. Netmeld is used to parse input from different file formats, store the data in a common format, allow users to easily query it, and enable analysts to tie different analysis tools together using a common back-end.
We characterize the GPU energy usage of two widely adopted exascale-ready applications representing two classes of particle and mesh solvers: (i) QMCPACK, a quantum Monte Carlo package, and (ii) AMReX-Castro, an adaptive mesh astrophysical code. We analyze power, temperature, utilization, and energy traces from double-/single (mixed)-precision benchmarks on NVIDIA’s A100 and H100 and AMD’s MI250X GPUs using queries in NVML and rocm_smi_lib, respectively. We explore application-specific metrics to provide insights on energy vs. performance trade-offs. Our results suggest that mixed-precision energy savings range between 6–25% on QMCPACK and 45% on AMReX-Castro. Also, we found gaps in the AMD tooling used on Frontier GPUs that need to be understood, while query resolutions on NVML have little variability between 1 ms-1 s. Overall, application level knowledge is crucial to define energy-cost/science-benefit opportunities for the codesign of future supercomputer architectures in the post-Moore era.
The Bioenergy Knowledge Discovery Framework (KDF) hosts a vast repository of specialized data, yet traditional keyword-based search methods often struggle to provide direct answers, requiring significant domain expertise and manual effort to filter through raw documents. To overcome these barriers, this software introduces a semantic search engine that enables both specialists and non-specialists to query the KDF using natural language. By shifting from rigid keyword matching to intent-based retrieval, the tool automatically identifies and ranks the most relevant sources within the database. The system functions by processing natural language queries to extract the most pertinent information, delivering an AI-generated plain-language summary alongside exact supporting quotes from retrieved documents. This integrated approach provides users with immediate, evidence-based answers while eliminating the need for exhaustive manual review. By surfacing direct insights and contextual evidence, the software enhances the usability of existing KDF resources and democratizes access to complex bioenergy data. Ultimately, this semantic search solution accelerates the discovery process and supports faster, more informed decision-making across the bioenergy sector.
The National Biosurveillance Integration Center (NBIC) is developing a scalable, flexible open source data collection, analysis, and dissemination tool to support biosurveillance operations by the U.S. Department of Homeland Security (DHS) and its federal interagency partners. The NBIC integrates, analyzes, and distributes key information about health and disease events to help ensure the nation’s responses are well-informed, save lives, and minimize economic impact. NBIC serves as a bridge between Federal, State, Local, Territorial, and Tribal entities to conduct biosurveillance across human, animal, plant, and environmental domains. The integration of information enables early warning and shared situational awareness of biological events to inform critical decisions directing response and recovery efforts. To meet its mission objectives, NBIC utilizes a variety of datasets, including open source information, to provide comprehensive coverage of biological events occurring across the globe. NBIC Biofeeds is a digital tool designed to improve the efficiency of reviewing and analyzing large volumes of open source reporting by biosurveillance analysts on a daily basis; moreover, the system provides a mechanism to disseminate tailored feeds allowing NBIC to better meet the specific information needs of individual, interagency partners. The tool is currently under development by the Department of Energy (DOE), Pacific Northwest National Laboratory (PNNL)and it is in a testing and evaluation phase supported by NBIC biosurveillance subject matter experts. Integration with the Defense Threat Reduction Agency (DTRA), Biosurveillance Ecosystem(BSVE) is also underway. NBIC Biofeeds Version 1 is expected to be fully operational in Fiscal Year 2017. The PNNL is applying agile methodology to streamline the build of NBIC Biofeeds to specifications required for operational use by NBIC and its federal interagency partners. Biosurveillance, analytics, and system engineering subject matter experts provide guidance on the implementation of features in the tool to ensure functionality aligns with operational workflows and production support. PNNL is leveraging software from a previous government effort to repurpose the technology to meet NBIC needs. NBIC Biofeeds incorporates the open source, document-orientated MongoDB database to capture user- and system-generated metadata on hundreds of thousands of records, in part, to establish baselines to aid prospective and retrospective analysis on emerging biological events. NBIC Biofeeds integrates a biosurveillance taxonomy (uniquely developed by NBIC),which includes input from interagency partners to recognize critical characteristics of a biological event. In NBIC Biofeeds Version1, metadata capture of reported events is done manually by NBIC analysts; however, moving forward in Version 2, the tool will be further automated to flag significant reporting on biological events with a human remaining in the loop to confirm the validity of the system-generated tags. To serve as a one-stop tool for open source biosurveillance, NBIC Biofeeds automatically harvests information from thousands of websites, utilizing third party aggregators, paid subscriptions to data feeds, and scraping of high priority sources. Users can develop desired queries for automatic updating, leverage a unique review and curation mechanism, and further analyze data from topical, geographic, and temporal visualization features in the tool. To meet NBIC’s information sharing needs, the tool allows for design of tailored RSS feeds and electronic message-based delivery of analysis on biological events, intended for recipients in the government with unique missions around human, animal, plant, and environmental health. Through current testing and evaluation – underway by biosurveillance subject matter experts – NBIC Biofeeds is demonstrating value in supporting open source biosurveillance by the Center for more rapid recognition and sharing of key event characteristics. Centralizing access and analysis of this data set into a single system is increasing the efficiency of daily, global biosurveillance, while enhancing the value of information identified through use of the querying, curation, and production support features in the tool.
The CS PlanIT provides stakeholders with easy access to geographically explore, query, and evaluate multiple data layers for a user-defined area of interest, with the goal to accelerate carbon storage resource, feasibility assessments, and planning efforts. The data within CS PlanIT are arranged and categorized to assess each stage of the Carbon Capture and Storage supply chain, from CO2 (Carbon Dioxide) emission sources, critical energy infrastructure (including transport), to CO2 geologic storage and sink locations. Furthermore, stakeholders can gain insights into community and environmental impacts, as CS PlanIT leverages Environmental Justice and Social Justice (EJSJ) and natural hazard datasets. Additionally, documentation in the form of an interactive ArcGIS story map and data catalog are included within the EDX publication to provide user’s information to better understand and explain CS PlanIT’s layout, functions, and associated data sets.
This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.
This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.
We describe the first public data release of the Dark Energy Survey, DES DR1, consisting of reduced single-epoch images, co-added images, co-added source catalogs, and associated products and services assembled over the first 3 yr of DES science operations. DES DR1 is based on optical/near-infrared imaging from 345 distinct nights (2013 August to 2016 February) by the Dark Energy Camera mounted on the 4 m Blanco telescope at the Cerro Tololo InterAmerican Observatory in Chile. We release data from the DES wide-area survey covering similar to 5000 deg(2) of the southern Galactic cap in five broad photometric bands, grizY. DES DR1 has a median delivered point-spread function of g = 1.12, r = 0.96, i = 0.88, z = 0.84, and Y = 0.'' 90 FWHM, a photometric precision of <1% in all bands, and an astrometric precision of 151 mas. The median co-added catalog depth for a 1.'' 95 diameter aperture at signal-to-noise ratio (S/N) = 10 is g = 24.33, r = 24.08, i = 23.44, z = 22.69, and Y = 21.44 mag. DES DR1 includes nearly 400 million distinct astronomical objects detected in similar to 10,000 co-add tiles of size 0.534 deg(2) produced from similar to 39,000 individual exposures. Benchmark galaxy and stellar samples contain similar to 310 million and similar to 80 million objects, respectively, following a basic object quality selection. These data are accessible through a range of interfaces, including query web clients, image cutout servers, jupyter notebooks, and an interactive co-add image visualization tool. DES DR1 constitutes the largest photometric data set to date at the achieved depth and photometric precision.
Identification of domains in protein sequences allows their assigning to biological functions. Several webservers exist for identification of protein domains using similarity searches against various databases of protein domain models. However, none of them provides comprehensive domain coverage while allowing bulk querying and their visualization schemes can be improved. To address these issues, we developed CDvist (a comprehensive domain visualization tool), which combines the best available search algorithms and databases into a user-friendly framework. First, a given protein sequence is matched to domain models using high-specificity tools and only then unmatched segments are subjected to more sensitive algorithms resulting in a best possible comprehensive coverage. In conclusion, bulk querying and rich visualization and download options provide improved functionality to domain architecture analysis.
The D and D Knowledge Management Information Tool (D and D KM- IT) is a web-based knowledge management information tool built for the D and D user community. Big data is a massive volume of structured and unstructured datasets which is so large that it's difficult to process using traditional database techniques. Web logs are the repository of files which are generated automatically for any operation on the web site. Web log files generated from the D and D KM-IT will be processed using the Apache Hadoop Framework to extract meaningful data. Hadoop is an open-source software framework for storing and processing big data in a distributed environment on large clusters of commodity hardware. Hadoop framework consists of two main layers: Hadoop distributed file system (HDFS) and the Execution engine (MapReduce). Automatic parallelization and distribution; clean and simple programming abstraction. Implementation: Web log files from the D and D KM-IT server are fetched using Apache Flume and loaded into the Hadoop distributed file system. Using the Apache Hcatalog tool, the web log data is parsed and converted into a structured table format. A Pig program is developed to query the tables to retrieve and store the significant information onto the HDFS. The extracted data is fed to business visualization tools such as Microsoft Excel for analysis and reporting. Results The reports generated through the web log data processing will be analyzed to improve the usability and performance of the D and D KM-IT. Conclusion/Future Work: User keyword information can be explored to add new trending topics to the D and D KM-IT application. Statistics generated can be compared to analysis reports from existing analytical tools.
Isosurface visualization is fundamental for exploring and analyzing 3D volumetric data. Marching cubes (MC) algorithms with linear interpolation are commonly used for isosurface extraction and visualization. Although linear interpolation is easy to implement, it has limitations when the underlying data is complex and high-order, which is the case for most real-world data. Linear interpolation can output vertices at the wrong location. Its inability to deal with sharp features and features smaller than grid cells can lead to an incorrect isosurface with holes and broken pieces. Despite these limitations, isosurface visualizations typically do not include insight into the spatial location and the magnitude of these errors. We utilize high-order interpolation methods with MC algorithms and interactive visualization to highlight these uncertainties. Our visualization tool helps identify the regions of high interpolation errors. It also allows users to query local areas for details and compare the differences between isosurfaces from different interpolation methods. In addition, we employ high-order methods to identify and reconstruct possible features that linear methods cannot detect. We showcase how our visualization tool helps explore and understand the extracted isosurface errors through synthetic and real-world data.
Indexing technique has become an efficient tool to enable scientists to directly access the most relevant data records. But, the time and space requirements of building and storing indexes are expensive in the traditional approaches, such as R-tree and bitmaps. Recently, we started to address this issue by using the idea of "block index", and our previous work has shown promising results from comparing it against other well-known solutions, including ADIOS, SciDB, and FastBit. In this work, we further improve the technique from both theoretical and implementation perspectives. Driven by an extensive effort in characterizing scientific datasets and modeling I/O systems, we presented a theoretical model to analyze its query performance with respect to a given block size configuration. We also introduced three optimization techniques to achieve a 2.3x query time reduction comparing to the original implementation.
Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.
Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.
Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.
Despite being information rich, the vast majority of untargeted mass spectrometry data are underutilized; most analytes are not used for downstream interpretation or reanalysis after publication. The inability to dive into these rich raw mass spectrometry datasets is due to the limited flexibility and scalability of existing software tools. Here, in this study, we introduce a new language, the Mass Spectrometry Query Language (MassQL), and an accompanying software ecosystem that addresses these issues by enabling the community to directly query mass spectrometry data with an expressive set of user-defined mass spectrometry patterns. Illustrated by real-world examples, MassQL provides a data-driven definition of chemical diversity by enabling the reanalysis of all public untargeted metabolomics data, empowering scientists across many disciplines to make new discoveries. MassQL has been widely implemented in multiple open-source and commercial mass spectrometry analysis tools, which enhances the ability, interoperability and reproducibility of mining of mass spectrometry data for the research community.
This work presents a projection-based multiphysics Model Order Reduction (MOR) framework for the analysis of nuclear systems and its application to parametric simulations of Molten Salt Reactors (MSR). The framework, named GeN-ROM, is developed using OpenFOAM® and employs a Proper Orthogonal Decomposition aided Reduced-Basis technique (POD-RB). It can be used to reduce steady-state and transient multiphysics problems involving parametric fluid dynamics, heat exchange, and neutronics phenomena. For the treatment of structural elements in the hydraulic systems, a porous medium approach has been adopted. The reduction process is data-driven and snapshot information is extracted via POD to learn the solution manifold and to build global spatial basis functions. At the data collection phase, GeN-ROM makes use of the solvers available in GeN-Foam, a similarly OpenFOAM®-based multiphysics framework developed for the analysis of nuclear reactors. The global bases are used both to approximate the solution fields and to project the full-order equations onto lower-dimensional subspaces, thus considerably reducing the number of unknowns in a numerical system. This reduction leads to significant computational speedups, which is ideal for multi-query applications such as uncertainty quantification or design optimization. The developed tool has been tested using a 2D multiphysics model of the Molten Salt Fast Reactor (MSFR) with steady-state and transient scenarios, with speedups on the order of 10 – 10 5 .
Sirepo , a browser-based GUI for X-ray source and optics simulations, is presented. Such calculations can be performed using SRW ( Synchrotron Radiation Workshop ), which is a physical optics computer code, allowing simulation of entire experimental beamlines using the concept of a `virtual beamline' with accurate treatment of synchrotron radiation generation and propagation through the X-ray optical system. SRW is interfaced with Sirepo by means of a Python application programming interface. Sirepo supports most of the optical elements currently used at beamlines, including recent developments in SRW . In particular, support is provided for the simulation of state-of-the-art X-ray beamlines, exploiting the high coherence and brightness of modern light source facilities. New scientific visualization and reporting capabilities have been recently implemented within Sirepo , as well as automatic determination of electron beam and undulator parameters. Publicly available community databases can be dynamically queried for error-free access to material characteristics. These computational tools can be used for the development and commissioning of new X-ray beamlines and for testing feasibility and optimization of experiments. The same interface can guide simulation on a local computer, a remote server or a high-performance cluster. Sirepo is available online and also within the NSLS-II firewall, with a growing number of users at other light source facilities. Our open source code is available on GitHub.