SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

All sources NASA NTRS DOE OSTI

DOE OSTI2025

WELLS Interactive Application

The Wellbore Exploration and Location Logistic System (WELLS) Interactive Application is an interactive tool to enable easy exploration and visualization of the living national wellbore database (WELLS Database (https://edx.netl.doe.gov/dataset/wells_database)). The tool and underlying database were created and are maintained by the National Energy Technology Laboratory (NETL), providing visualization of the more than six million public wellbore records from more than 65 authoritative state, federal, and tribal resources. The WELLS Interactive Application serves up wellbore data from oil, gas, underground injection, research, geothermal, geotechnical, groundwater, and other types of wells in a single, standardized, unified system. In addition to the surface location of these wells, the underlying database combines select key attributes for features such as well age, depth, and operating status. The system also provides users with references back to the original sources used in this unified platform. The underlying data can be accessed through the WELLS Database: https://edx.netl.doe.gov/dataset/wells_database Additional Information: The WELLS Interactive Application (formerly titled CO2-Locate) enables visualization and access to the public wellbore records through an intuitive web-based mapping tool. The WELLS Interactive Application was designed to help users visualize, query, analyze, and download wellbore records. Public wellbore points are included as a layer in the Map page, called Public Wells. Additionally, a multivariate hexagon grid summarizing well density from proprietary well data, called Well Density, is included to identify data gaps between the public and proprietary well data. Filtering functionalities in the tool allow these two layers to be spatially filtered by state, county, or basin as well as by status, type, true vertical depth, and spud year. The WELLS Interactive Application also contains a Near Me tool can be used to search and explore wellbore data within a user-defined distance of a specified location on the map, which can also be downloaded. The Query tool allows users to query the selected or filtered wells in the Public Wells layer and export the data. For additional information on these tool functionalities, see the help documentation on the About page of the tool. Notes for Consideration: The Well Density layer provided in this application is derived from proprietary wellbore data, the records of which do not always contain values for key features (status, type, true vertical depth, or spud year). Therefore, data might not be available when layers are queried for all filter combinations. Additionally, visualizing layers and applying filters may take additional time to load (i.e., draw on the map) due to the large size of the data.

ccs↗

DOE OSTI2022

Integrating very-high-resolution UAS data and airborne imaging spectroscopy to map the fractional composition of Arctic plant functional types in Western Alaska

Widespread changes in vegetation cover and composition are driving strong impacts on Arctic ecosystem functioning and global climate feedbacks. An accurate characterization of tundra vegetation composition is required to understand how the Arctic will respond to future climate change. However, quantifying tundra vegetation composition over large areas is challenging as commonly-used satellite observations are too coarse, spatially and spectrally, to differentiate low-lying tundra vegetation types. Recent airborne and spaceborne imaging spectroscopy platforms provide better data to characterize vegetation composition. Yet, our ability to characterize vegetation composition with imaging spectroscopy remains largely unexplored in the Arctic, particularly due to a lack of ground observations needed to train and test classification models. To address this problem, we collected very-high-resolution (VHR, ~5 cm) unoccupied aerial system (UAS) imagery at three low-Arctic tundra sites located on the Seward Peninsula, western Alaska. In this paper, we examine the feasibility of integrating imagery from the UAS and the hyperspectral Airborne Visible/Infrared Imaging Spectrometer, Next Generation (AVIRIS-NG) airborne instrument to map the fractional composition of 12 key Arctic plant functional types (PFTs). To this end, we first mapped the 12 PFTs from our VHR UAS imagery using random forest classification. We then used these UAS-derived PFT maps as ground truth to develop partial least squares regression (PLSR) models to predict the fractional cover (FCover) of each PFT from AVIRIS-NG imagery. Further, we evaluated the performance of our PLSR models using reserved UAS samples, as well as by mapping PFT FCover and dominant PFT for large tundra landscapes. Our results show that 1) Arctic PFTs can be effectively mapped using VHR UAS imagery, with overall accuracy between 86% and 92%, 2) when the UAS mapped PFTs were used to inform PLSR scaling models, the FCover of the 12 PFTs could be effectively estimated from AVIRIS-NG imagery with a mean absolute error (MAE) <0.13, and 3) our PLSR models outperformed traditional, fully constrained least-squares (FCLS) linear mixture analysis and produced high-quality, spatially contiguous PFT FCover and PFT maps that captured vegetation spatial patterns with similar accuracy to those developed from UAS imagery. The developed PLSR models have the potential to be broadly applied for quantifying vegetation composition with AVIRIS-NG images to help monitor tundra vegetation dynamics and improve process-based modeling of tundra ecosystems.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Advances in building data management for building performance standards using the SEED platform

Reducing energy consumption and greenhouse gas emissions in the built environment is a critical step in achieving emission goals to mitigate climate change impacts. Local, federal, and international jurisdictions are deploying several methods to reduce energy and emissions such as voluntary and mandatory benchmarking and building performance standards, requiring building owners to reach energy and emission targets. Jurisdictions leveraging benchmarking and building performance standards require knowledge of the buildings covered; which is a large task due to staffing constraints, limited information on building characteristics and tax parcel data, and the need for advanced data management techniques to align datasets. This paper describes an open-source platform's recent advances to create consistent taxonomies, identify erroneous data, enable auditability, and track building performance. The paper concludes with two use cases on how the platform has been used by jurisdictions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2025

Convergent Manufacturing of Large-Scale Components for Nuclear Applications, via Additive Manufacturing and Powder Metallurgy Hot Isostatic Pressing

Powder metallurgy (PM)–hot isostatic pressing (PM-HIP) has long been recognized as a powerful route for producing fully dense, near net shape metallic components. By consolidating powders under high temperature and pressure, HIP provides isotropic properties, uniform microstructures, and scalability to complex geometries that are vital for sectors such as aerospace, energy, and nuclear power. Yet despite these advantages, the technology has remained constrained by costly trial and error canister fabrication, limitations of conventional forging, and incomplete knowledge about how the canister design influences final part properties. Additive manufacturing (AM), by contrast, thrives on design freedom and geometric flexibility but struggles with speed, scalability, and cost when applied to very large structures. The research presented in this report investigated how a convergent manufacturing approach, combining AM with PM-HIP, can merge the strengths of both technologies, leveraging AM’s flexibility for canister design and HIP’s consolidation capability to deliver reliable, large, and complex parts. The work progressed through three case studies that built on one another in scale and complexity. Small cylindrical canisters fabricated by conventional methods, laser powder bed fusion, and directed energy deposition were filled with stainless steel powders and subjected to HIP. The resulting parts demonstrated near-full density and mechanical properties on par with wrought stainless steel, showing for the first time that AM canisters can be a direct substitute for conventional ones without sacrificing quality. The next step involved a medium-scale, noncentrosymmetric T-valve, which is an enclosed, multibranch geometry that tested the limits of AM + PM-HIP integration. The T-valve achieved predictable shrinkage and uniform densification, confirming feasibility for enclosed designs. However, this study also revealed oxide inclusions and interfacial challenges at the AM + HIP boundary, underscoring the critical importance of controlling interface chemistry and employing robust, in situ strategies, such as melt pool monitoring and thermal monitoring, coupled with nondestructive evaluation techniques such as x-ray computed tomography. Finally, the effort culminated in fabricating a large-scale impeller weighing nearly 2000 lb and spanning 5 ft in diameter. Produced via multirobot wire arc AM and hot isostatic pressed to near-full density, the impeller validated industrial-scale feasibility. Predictive models closely matched experimental shrinkage, tensile properties were spatially uniform across the component, and the AM + PM-HIP interface proved mechanically sound despite the presence of oxide-decorated prior particle boundaries. This large-scale demonstration is a major milestone, showing that hybrid AM + PM‑HIP can reliably deliver components at reactor-relevant scales. Collectively, these studies charted a logical pathway: small-scale work built scientific confidence, medium-scale work highlighted opportunities and challenges, and large-scale work proved industrial impact. The overarching conclusion of this report is that AM + PM-HIP should not be seen as a replacement for forging but as a complementary pathway that provides the US with flexibility, resilience, and new options for manufacturing nuclear-grade components. Looking ahead, several directions emerge as critical to sustaining progress. Predictive modeling must become faster, more accessible, and more accurate, with digital twins and machine learning reducing reliance on trial and error. Powders and alloys must be optimized for HIP, with improved cleanliness, reduced oxides, and tailored chemistries that enhance creep, fatigue, and irradiation resistance. Interfaces between AM and HIP regions must be better engineered through coatings, machining strategies, and surface treatments to mitigate oxide formation and ensure reliable bonding to explore opportunities for HIP of targeted compositional parts, as well as multimaterial HIP cladding applications. Monitoring and nondestructive evaluation need to expand, incorporating multimodal sensors, x-ray computed tomography, and real-time data integration through platforms such as Pelican. At the same time, the pathway to industrial adoption requires techno-economic analysis, machinability studies, and qualification frameworks aligned with industry and regulatory standards. Finally, workforce and academic engagement must be strengthened. Programs that train technicians and engineers for US Navy and US Department of Energy manufacturing challenges should be paired with academic partnerships to support fundamental research, with open sharing of non-export-controlled data to accelerate innovation and build the next generation of experts. In conclusion, this report demonstrates that hybrid AM + PM-HIP is scientifically viable and strategically important. By combining the design agility of AM with the consolidation strength of HIP and embedding modeling, monitoring, and workforce development, this approach provided a transformative new capability for US manufacturing. The path forward is clear: hybrid AM + PM-HIP is not just a promising research direction but is also potentially an industrially relevant pathway that can reshape how nuclear-grade components are designed, qualified, and deployed.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Assessments of data centers for provision of frequency regulation

There are numerous opportunities for data centers to participate in demand response programs considering their large energy capacities, flexible working environments and workloads, redundant design and operation, etc. As a type of demand response, frequency regulation requires fast response, and its potential is not fully explored by data centers yet. This paper proposes a synergistic control strategy for data center frequency regulation which uses both IT and cooling systems. It combines power management techniques at the server level with control of the chilled water supply temperature to track the regulation signal from the electrical market. A frequency regulation flexibility factor is also proposed to increase the IT capacity for frequency regulation. The performance of the control strategy is studied through numerical simulations using an equation based object-oriented Modelica platform designed for data centers. Simulation results show that with well-tuned control parameters, data centers can provide frequency regulation service in both regulation up and down. The performance of data centers in providing frequency regulation service is largely influenced by the regulation capacity bid, frequency regulation flexibility factor, workload condition, and cooling mode of the cooling system, and not significantly influenced by the time constant of chillers. In addition, compared with a server-only control strategy, the proposed synergistic control strategy can provide an extra regulation capacity of 3% of the design power when chillers are activated. Here, when chillers are deactivated, both strategies have a similar regulation capacity.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

Evaluating the Hybrid Modelling Competition: A Step Towards Developing Good Modelling Practice

Hybrid modelling, a combination of mechanistic and data-driven modelling, is a promis¬ing approach to advance current mathematical models towards improved deci¬sion support tools for today's water-related challenges. Researchers have been develop¬ing guidelines or references for good modelling practices in the water field for mecha¬nistic (Rieger et al., 2012) and data-driven (Zhu et al., 2023) modelling, respectively. However, good modelling practices for hybrid modelling are currently missing (Schneider et al., 2022). Therefore, the International Water Association’s (IWA) hybrid modelling working group initiated the first competition on a data science competition platform (i.e. Kaggle) for water resource recovery modelling at the Watermatex con¬ference in September 2023 in Quebec. The main objective of this competition was to gain insights and experience to create good modelling practices. Further goals were to motivate students, researchers, and practitioners model, foster a vibrant and engaged community, and evaluate the efficacy of com¬petitions in solving modelling challenges within the water domain. Our next goal is that facilities will measure and gather relevant data for future competitions to solve their challenges from a modeller’s perspective.

Schneider, Mariane↗

DOE OSTI2026

Cloud-Based Demonstration of the Eastern Interconnection Situational Awareness Monitoring System (ESAMS)

This report describes a cloud-based implementation and field demonstration of the Eastern Interconnection Situational Awareness and Monitoring System (ESAMS). ESAMS was developed to support the detection and source localization of forced oscillations using synchrophasor measurements from tie-lines connecting areas served by different reliability coordinators (RCs), so that RCs could better coordinate their response to wide-area events. A previous effort had identified deployment barriers associated with hosting shared situational awareness tools at a single RC. To address these barriers, ESAMS was migrated to Amazon Web Services and evaluated in a six-month field demonstration. ISO New England (ISO-NE) and PJM streamed data to the platform using AWS Direct Connect and a site-to-site VPN, respectively. The resulting multi-utility measurement footprint enabled regional source localization across major portions of the U.S. Eastern Interconnection and supported routine identification of oscillation events. During the final three months of the trial, 24 events above 2 MW/MVAR were detected. The largest detected oscillation approached a 25 MW peak-to-peak amplitude, and the longest persisted intermittently for more than 11 hours. The demonstration also assessed operational considerations—including data transfer volumes, end-to-end latency, and cloud computing costs—and found that network and compute requirements were modest relative to typical cloud capabilities while providing performance comparable to prior on-premises deployments. Overall, the results indicate that cloud hosting can provide a practical path to shared interconnection-wide oscillation monitoring. The cloud ESAMS demonstration establishes a foundation for broader utility participation and for building future wide-area analytics that leverage measurements across organizational boundaries.

Follum, James D.↗

DOE OSTI2016

Real-Time Discovery Services over Large, Heterogeneous and Complex Healthcare Datasets Using Schema-Less, Column-Oriented Methods

We present a service platform for schema-leess exploration of data and discovery of patient-related statistics from healthcare data sets. The architecture of this platform is motivated by the need for fast, schema-less, and flexible approaches to SQL-based exploration and discovery of information embedded in the common, heterogeneously structured healthcare data sets and supporting components (electronic health records, practice management systems, etc.) The motivating use cases described in the paper are clinical trials candidate discovery, and a treatment effectiveness analysis. Following the use cases, we discuss the key features and software architecture of the platform, the underlying core components (Apache Parquet, Drill, the web services server), and the runtime profiles and performance characteristics of the platform. We conclude by showing dramatic speedup with some approaches, and the performance tradeoffs and limitations of others.

Apache Drill Apache Parquet Column Oriented Stores↗

DOE OSTI2022

Microcam: A Low Power and Privacy Preserving Multi-modal Platform for Occupancy Detection (Final Report)

Heating, ventilation, and air conditioning (HVAC) consumes a significant portion of the energy used in buildings. Much of this is wasted energy, used when buildings are either not occupied at all, or occupied well under their maximum design conditions. This project has focused on residential occupancy detection to autonomously control HVAC systems and save energy. Limitations of existing occupancy sensors include one or more of the following: (i) they employ sensors or algorithms that are not able to detect stationary occupants; (ii) they cannot classify the source of the motion (such as a pet); (iii) depending on the camera resolution and employed algorithms, they do not allow for embedded or onboard computation, and require external or cloud-based processing; (iv) many algorithms developed for camera-based systems are sensitive to lighting changes, and thus prone to missed detections or false alarms; (v) Most existing systems depend on adjustment of settings for different scenarios, complicating self-commissioning; (vi) they cannot provide high enough accuracy; (vii) they are costly; (viii) they are not battery-powered, thus limiting ease of use and installation. In this project, Syracuse University and its partner SRI have developed a low-cost, high accuracy, standalone residential occupancy sensing platform, referred to as the MicroCam, to address all of the aforementioned challenges. MicroCam can operate on typical alkaline batteries without relying on the “cloud” or external computing resources, and consists of low-power, Artificial Intelligence (AI)-based, IoT platforms. Each platform has multi-modal sensors and can process motion, audio and video data, and send binary occupancy result to a lead platform. All sensor data is processed locally on platforms, and the only transmitted data is the binary occupancy state. In addition, preliminary work has been done on images wherein occupants are not discernable. Thus, MicroCam is a standalone solution preserving privacy of the occupants.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2016

Kernels for scalable data analysis in science: Towards an architecture-portable future

In this paper, we pose and address some of the unique challenges in the analysis of scientific Big Data on supercomputing platforms. Our approach identifies, implements and scales numerical kernels that are critical to the instantiation of theory-inspired analytic workflows on modern computing architectures. We present the benefits of scalable kernels towards constructing algorithms such as principal component analysis and non-negative matrix factorization on an image-analysis use case at the Oak Ridge Leadership Computing Facility (OLCF). Based on experience with the use-case, we conclude that piecing scalable analytic kernels into user-defined analytic workflows are a flexible, modular and agile way to enable architecture-portable productivity for the data-intensive sciences.

Sukumar, Sreenivas R.↗

DOE OSTI2012

Hydrothermal Exploration Best Practices and Geothermal Knowledge Exchange on OpenEI

Though exploring for hydrothermal resources is not new, advances in exploration technologies and the pursuit of less visible resources have created a need to outline exploration best practices. This multi-year study outlines 21 geothermal exploration regions in the Western United States. These regions were developed based on the U.S. Geological Survey (USGS) physiographic regions, then adjustedto fit geothermal parameters such as differences in geologic regime, structure, heat source, surface effects (weather, vegetation patterns, groundwater flow), and other relevant factors. Literature searches were conducted in each of these 21 regions for the application of field reconnaissance, geochemical, geophysical and remote sensing techniques. At this time, data from more than 250 referenceshave been cataloged in U.S. Department of Energy's (DOE) OpenEnergy Information (OpenEI, http://en.openei.org) website, which allows industry to access data information and run analyses on specific data attributes. The platform also allows updates, edits, and additions from the public so that the dataset can be expanded as industry's knowledge grows. In addition to the literature survey,interviews were conducted with exploration experts with both geothermal and oil-and-gas industry experience to identify the exploration challenges and best practices for the exploration regions. This paper defines and describes the geothermal regions developed for this study, describes how exploration techniques contribute to hydrothermal analyses (structural, petrologic, temperature, andhydrologic), and defines the exploration hurdles and best practices for each region. The results of the study and accompanying data sets on OpenEI will be available through the National Geothermal Data System.

15 GEOTHERMAL ENERGY↗

DOE OSTI2023

Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC)

Abstract Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory (Rubin) will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition that aimed to catalyze the development of robust classifiers under LSST-like conditions of a nonrepresentative training set for a large photometric test set of imbalanced classes. Over 1000 teams participated in PLAsTiCC, which was hosted in the Kaggle data science competition platform between 2018 September 28 and 2018 December 17, ultimately identifying three winners in 2019 February. Participants produced classifiers employing a diverse set of machine-learning techniques including hybrid combinations and ensemble averages of a range of approaches, among them boosted decision trees, neural networks, and multilayer perceptrons. The strong performance of the top three classifiers on Type Ia supernovae and kilonovae represent a major improvement over the current state of the art within astronomy. This paper summarizes the most promising methods and evaluates their results in detail, highlighting future directions both for classifier development and simulation needs for a next-generation PLAsTiCC data set.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2018

NBIC Biofeeds: Deploying a New, Digital Tool for Open Source Biosurveillance across Federal Agencies

The National Biosurveillance Integration Center (NBIC) is deploying a scalable, flexible open source data collection, analysis, and dissemination tool to support biosurveillance operations by the U.S. Department of Homeland Security (DHS) and its federal interagency partners.IntroductionNBIC integrates, analyzes, and distributes key information about health and disease events to help ensure the nation’s responses are well-informed, save lives, and minimize economic impact. To meet its mission objectives, NBIC utilizes a variety of data sets, including open source information, to provide comprehensive coverage of biological events occurring across the globe. NBIC Biofeeds is a digital tool designed to improve the efficiency of analyzing large volumes of open source reporting and increase the number of relevant insights gleaned from this dataset. Moreover, the tool provides a mechanism to disseminate tailored, electronic message notifications in near-real time so that NBIC can share specific information of interest to its interagency partners in a timely manner. NBIC intends to implement operational use of the capability in FY 2018. The core components of the system are data collection, curation, and dissemination of information deemed important by NBIC subject matter experts. NBIC Biofeeds has captured information from more than 70,000 unique sources published from around the globe and presents, on average, 9,000 new biosurveillance-relevant articles to users each day. NBIC leverages a variety of data feeds, including third party aggregators like Google and subscription-based feeds such as HealthMap, as well as Really Simple Syndication (RSS) feeds and web-scraping of highly relevant sources. The NBIC biosurveillance taxonomy imbedded in the tool consists of more than 600 metadata targets that cover key information for understanding the significance of an active biological event, including etiologic agents, impact to humans and animals (e.g., infection severity, healthcare workers involved, type of host), social disruption, infrastructure strain, countermeasures engaged, and ‘red flag’ characteristics (e.g., pathogen appearance in a new geographic area, unusual clinical signs). This taxonomy serves as a foundation for data curation and can be tailored by NBIC partners to more directly meet their own mission objectives. At this time, metadata is predominately captured by NBIC analysts, who manually tag information, which triggers the population of three automatically-disseminated products from the tool: 1) the NBIC Daily Biosurveillance Review, 2) immediate and daily summary email notifications, and 3) custom-designed RSS feeds. These products are meant for individual recipients in the federal interagency and for consumption by other biosurveillance information technology systems, such as the Department of Defense, Defense Threat Reduction Agency (DTRA) Biosurveillance Ecosystem (BSVE). NBIC is working in partnership with DTRA to integrate NBIC Biofeeds as an application directly into the BSVE and further develop the BSVE as an all-in-one platform for biosurveillance data analytics. To improve the efficiency and effectiveness of gaining insights using NBIC Biofeeds, developers of the tool at the Pacific Northwest National Laboratory (PNNL) are researching and testing a variety of advanced analytics techniques focused on: 1) article relevancy ratings to improve the review of queried data, 2) significance ratings to elucidate the perceived severity of an event based on reported characteristics, 3) full-text article retrieval and storage for improved machine-tagging, and 4) anomaly detection for emerging threats. Testing and implementation of new analytic capabilities in NBIC Biofeeds is planned for this fiscal year. NBIC Biofeeds was developed to serve as a sophisticated and powerful open source biosurveillance technology of value to the federal government by providing information to stakeholders conducting open source biosurveillance as well as those consuming biosurveillance information. In FY 2018, NBIC Biofeeds will begin operational use by NBIC and an initial set of users in various federal agencies. User accounts for testing purposes will be available to other federal partners, and a broad scope of federal stakeholders can receive products directly from NBIC Biofeeds based on their interests. NBIC Biofeeds is expected to enable more rapid recognition and enhanced analysis of emerging biological events by NBIC analysts. NBIC anticipates other federal agencies with biosurveillance missions will find this technology of value and intends to offer use of the platform to those federal partners that can benefit from access to the tool and information generated from NBIC Biofeeds.

NBIC↗

DOE OSTI2025

Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data

Zero-shot and prompt-based models have excelled at visual reasoning tasks by leveraging large-scale natural image corpora, but they often fail on sparse and domain-specific scientific image data. We introduce Zenesis, a no-code interactive computer vision platform designed to reduce data readiness bottlenecks in scientific imaging workflows. Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement. We validate our approach on Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) datasets of catalyst-loaded membranes. Zenesis outperforms baselines, achieving an average accuracy of 0.947, Intersection over Union (IoU) of 0.858, and Dice score of 0.923 on amorphous catalyst samples; and 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples. These results represent a significant performance gain over conventional methods such as Otsu thresholding and standalone models like the Segment Anything Model (SAM). Zenesis enables effective image segmentation in domains where annotated datasets are limited, offering a scalable solution for scientific discovery.

Mukherjee, Shubhabrata↗

DOE OSTI2024

Evaluating the potential of disaggregated memory systems for HPC applications

Summary Disaggregated memory is a promising approach that addresses the limitations of traditional memory architectures by enabling memory to be decoupled from compute nodes and shared across a data center. Cloud platforms have deployed such systems to improve overall system memory utilization, but performance can vary across workloads. High‐performance computing (HPC) is crucial in scientific and engineering applications, where HPC machines also face the issue of underutilized memory. As a result, improving system memory utilization while understanding workload performance is essential for HPC operators. Therefore, learning the potential of a disaggregated memory system before deployment is a critical step. This paper proposes a methodology for exploring the design space of a disaggregated memory system. It incorporates key metrics that affect performance on disaggregated memory systems: memory capacity, local and remote memory access ratio, injection bandwidth, and bisection bandwidth, providing an intuitive approach to guide machine configurations based on technology trends and workload characteristics. We apply our methodology to analyze thirteen diverse workloads, including AI training, data analysis, genomics, protein, fusion, atomic nuclei, and traditional HPC bookends. Our methodology demonstrates the ability to comprehend the potential and pitfalls of a disaggregated memory system and provides motivation for machine configurations. Our results show that eleven of our thirteen applications can leverage injection bandwidth disaggregated memory without affecting performance, while one pays a rack bisection bandwidth penalty and two pay the system‐wide bisection bandwidth penalty. In addition, we also show that intra‐rack memory disaggregation would meet the application's memory requirement and provide enough remote memory bandwidth.

Ding, Nan↗

DOE OSTI2020

Data-driven model construction for industrial asset decision boundary classification

In some embodiments, a system model construction platform may receive, from a system node data store, system node data associated with an industrial asset. The system model construction platform may automatically construct a data-driven, dynamic system model for the industrial asset based on the received system node data. A synthetic attack platform may then inject at least one synthetic attack into the data-driven, dynamic system model to create, for each of a plurality of monitoring nodes, a series of synthetic attack monitoring node values over time that represent simulated attacked operation of the industrial asset. The synthetic attack platform may store, in a synthetic attack space data source, the series of synthetic attack monitoring node values over time that represent simulated attacked operation of the industrial asset. This information may then be used, for example, along with normal operational data to construct a threat detection model for the industrial asset.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Surrogate Model of Flexible Research Platform EnergyPlus Models to Enable Sensitivity Analysis

This letter report describes the surrogate models developed from the EnergyPlus model of Oak Ridge National Laboratory’s Flexible Research Platform. Two data-driven black-box models were developed, and the outputs of the surrogate models were compared with the EnergyPlus model. The two models developed are a multilayer perceptron deep learning model, and a long short-term memory (LSTM) neural network model. The three factors for selecting the black-box models are scalability, computation time, and accuracy. A total of 107 input variables were the dominant variables in determining the outputs of building energy consumptions and thermal comfort. A total of 54 output variables were identified as the prediction targets, including the system- and zone-level outputs. The large set of the simulation cases were generated by integrating sensor errors into an emulator based on EnergyPlus and Python EMS, which includes advanced control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. The surrogate models were developed based on a set of large-scale simulation runs (i.e., 4,000 runs) on a cloud platform. The comparison analysis shows that the two black-box models had good accuracy for predicting new outputs for sensitivity analysis using the root mean square error metric. As a next step, the developed surrogate models will be used to perform sensitivity analysis for different sensor impacts (e.g., sensor types, sensor locations).

42 ENGINEERING↗

DOE OSTI2013

End-to-end data movement using MPI-IO over routed terabits infrastructures. In: NDM '13 Proceedings of the Third International Workshop on Network-Aware Data Management, Article No. 9

Scientific discovery is nowadays driven by large-scale simulations running on massively parallel high-performance computing (HPC) systems. These applications each generate a large amount of data, which then needs to be post-processed for example for data mining or visualization. Unfortunately, the computing platform used for post processing might be different from the one on which the data is initially generated, introducing the challenge of moving large amount of data between computing platforms. This is especially challenging when these two platforms are geographically separated since the data needs to be moved between computing facilities. This is even more critical when scientists tightly couple their domain specific applications with a post processing application. The paper presents a solution for the data transfer between MPI applications using a dedicated wide area network (WAN) terabit infrastructure. The proposed solution is based on parallel access to data files and the Message Passing Interface (MPI) over the Common Communication Infrastructure (CCI) for the data transfer over a routed infrastructure. In the context of this research, the Energy Sciences Network (ESnet) of the U.S. Department of Energy (DOE) is targeted for the transfer of data between DOE national laboratories.

Vallée, Geoffroy↗