Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “FOUNDATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Pretraining Billion-Scale Geospatial Foundational Models on Frontier

As AI workloads increase in scope, generalization capability becomes challenging for small task-specific models and their demand for large amounts of labeled training samples increases. On the contrary, Foundation Models (FMs) are trained with internet-scale unlabeled data via self-supervised learning and have been shown to adapt to various tasks with minimal fine-tuning. Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators. Current satellite constellations collect 100+TBs of data a day, resulting in images that are billions of pixels and multimodal in nature. Such geospatial data poses unique challenges opening up new opportunities to develop FMs. We investigate billion scale FMs and HPC training profiles for geospatial applications by pretraining on publicly available data. We studied from end-to-end the performance and impact in the solution by scaling the model size. Our larger 3B parameter size model achieves up to 30% improvement in top1 scene classification accuracy when comparing a 100M parameter model. Moreover, we detail performance experiments on the Frontier supercomputer, America's first exascale system, where we study different model and data parallel approaches using PyTorch's Fully Sharded Data Parallel library. Specifically, we study variants of the Vision Transformer architecture (ViT), conducting performance analysis for ViT models with size up to 15B parameters. By discussing throughput and performance bottlenecks under different parallelism configurations, we offer insights on how to leverage such leadership-class HPC resources when developing large models for geospatial imagery applications.

Tsaris, Aristeidis (aris)↗
DOE OSTI2025

Exploring Capability of Multimodal Foundation Model for Image-based Fault Detection of Photovoltaic Modules

Multimodal Foundation Model (MFM), like ChatGPT and Gemini, have emerged as powerful tools for their exceptional natural language processing capabilities and their emerging potential in image analysis. This paper investigates the application of MFMs for photovoltaic (PV) fault detection through image analysis, focusing on ChatGPT 4.0 and Gemini 1.5 Pro. Three types of PV images and the corresponding common PV faults are detected: bird droppings using visible images, cell cracks via electroluminescence (EL) images, and hotspots using infrared (IR) images. Among the two models, Gemini 1.5 Pro demonstrated superior performance, achieving near-perfect results with an average F1 score of 0.97, consistently outperforming ChatGPT 4.0 in accuracy and reliability. Unlike traditional machine learning (ML) models, MFMs can operate in a zero shot manner that does not require additional training by the user, and the input images are not limited by size, angle, scope, or PV technology. The strong adaptability and user-friendliness make MFM a promising tool for analyzing PV images and advancing health monitoring for PV modules.

Li, Baojie↗
DOE OSTI2015

Critical Assessment of the Foundations of Power Transmission and Distribution Reliability Metrics and Standards

The U.S. federal government regulates the reliability of bulk power systems, while the reliability of power distribution systems is regulated at a state level. In this article, we review the history of regulating electric service reliability and study the existing reliability metrics, indices, and standards for power transmission and distribution networks. We assess the foundations of the reliability standards and metrics, discuss how they are applied to outages caused by large exogenous disturbances such as natural disasters, and investigate whether the standards adequately internalize the impacts of these events. Our reflections shed light on how existing standards conceptualize reliability, question the basis for treating large‐scale hazard‐induced outages differently from normal daily outages, and discuss whether this conceptualization maps well onto customer expectations. We show that the risk indices for transmission systems used in regulating power system reliability do not adequately capture the risks that transmission systems are prone to, particularly when it comes to low‐probability high‐impact events. We also point out several shortcomings associated with the way in which regulators require utilities to calculate and report distribution system reliability indices. We offer several recommendations for improving the conceptualization of reliability metrics and standards. We conclude that while the approaches taken in reliability standards have made considerable advances in enhancing the reliability of power systems and may be logical from a utility perspective during normal operation, existing standards do not provide a sufficient incentive structure for the utilities to adequately ensure high levels of reliability for end‐users, particularly during large‐scale events.

Nateghi, Roshanak↗
DOE OSTI2015

Shear effects on energy dissipation from an elastic beam on a rigid foundation

This paper describes the energy dissipation arising from microslip for an elastic shell incorporating shear and longitudinal deformation resting on a rough-rigid foundation. This phenomenon is investigated using finite element (FE) analysis and nonlinear geometrically exact shell theory. Both approaches illustrate the effect of shear within the shell and observe a reduction in the energy dissipated from microslip as compared to a similar system neglecting shear deformation. In particular, it is found that the shear deformation allows for load to be transmitted beyond the region of slip so that the entire interface contributes to the load carrying capability of the shell. The energy dissipation resulting from the shell model is shown to agree well with that arising from the FE model, and this representation can be used as a basis for reduced order models that capture the microslip phenomenon.

42 ENGINEERING↗
DOE OSTI2018

Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources

Apache Calcite is a foundational software framework that provides query processing, optimization, and query language support to many popular open-source data processing systems such as Apache Hive, Apache Storm, Apache Flink, Druid, and MapD. The goal of this paper is to formally introduce Calcite to the broader research community, brie y present its history, and describe its architecture, features, functionality, and patterns for adoption. Calcite's architecture consists of a modular and extensible query optimizer with hundreds of built-in optimization rules, a query processor capable of processing a variety of query languages, an adapter architecture designed for extensibility, and support for heterogeneous data models and stores (relational, semi-structured, streaming, and geospatial). This exible, embeddable, and extensible architecture is what makes Calcite an attractive choice for adoption in big-data frameworks. It is an active project that continues to introduce support for the new types of data sources, query languages, and approaches to query processing and optimization.

Begoli, Edmon↗
DOE OSTI2024

OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery

While the pretraining of Foundation Models (FMs) for remote sensing (RS) imagery is on the rise, models remain restricted to a few hundred million parameters. Scaling models to billions of parameters has been shown to yield unprecedented benefits including emergent abilities, but requires data scaling and computing resources typically not available outside industry R&D labs. In this work, we pair high-performance computing resources including Frontier supercomputer, America's first exascale system, and high-resolution optical RS data to pretrain billion-scale FMs. Our study assesses performance of different pretrained variants of vision Transformers across image classification, semantic segmentation and object detection benchmarks, which highlight the importance of data scaling for effective model scaling. Moreover, we discuss construction of a novel TIU pretraining dataset, model initialization, with data and pretrained models intended for public release. By discussing technical challenges and details often lacking in the related literature, this work is intended to offer best practices to the geospatial community toward efficient training and benchmarking of larger FMs.

Ambrozio Dias, Philipe↗
DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2025

ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, hyper-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 65,536 GPUs, achieving up to 4.1 ExaFLOPS sustained throughput and 74–98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. On 7 km resolution benchmarks, ORBIT-2 achieves high accuracy with R2 scores in range of 0.98–0.99 against observation data.

Wang, Xiao [ORNL] (ORCID:0000000165451943)↗
DOE OSTI2026

Scalable Federated Learning for Scientific Foundation Models on Leadership-Class Systems

Federated learning (FL) at leadership-class HPC systems remains largely unexplored, despite growing interest in deploying federated workflows on modern HPC systems. This paper provides the first system-level empirical characterization of federated fine-tuning of pretrained foundation models on an exascale supercomputer under a multi-node deployment. Using up to 96 concurrent FL clients deployed across Frontier nodes, we study the impact of client scale, model size, data heterogeneity, partial participation, and differential privacy on runtime, communication overhead, and convergence stability. Our results show that pretrained transformer models remain robust to heterogeneity, client dropout, and privacy noise, while system efficiency degrades rapidly with scale as synchronizat and orchestration dominate runtime. We further demonstrate that system-aware execution strategies, including intra-node aggregation and early aggregation, significantly reduce wall-clock time without degrading model quality. These findings establish a practical performance baseline and inform the design of communication-efficient FL systems on leadership-class HPC platforms.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗
DOE OSTI2025

genomeocean: a pretrained microbial genome foundational model (genomeoceanLLM) v1.0

We present Genomeocean, a foundational genome language model that represents the microbial genome sequences from complex environmental samples. By training on a large, diverse metagenomic dataset, Genomeocean learns species-specific sequence composition and can generate long, realistic open reading frames (ORFs). Our model employs a Byte-pair-encoding (BPE) tokenization strategy, allowing it to efficiently process large genomic datasets and generate long sequences up to 50kb. We demonstrate that fine-tuning Genomeocean can generate novel gene clusters encoding biosynthetic pathways, showcasing its ability to model both fundamental and complex biological processes. Our work establishes Genomeocean as a powerful tool for understanding microbial genome biology and paves the way for its application in a range of fields, from synthetic biology to microbiome research.

Wang, Zhong [Lawrence Berkeley National Laboratory↗
DOE OSTI2021

Scaling trends for balance-of-system costs at land-based wind power plants: Opportunities for innovations in foundation and erection

Wind power plant sizes, hub heights, and turbine ratings have increased since 2008 to optimize the cost and performance of wind power; however, the limits of these economies of scale remain unclear. Here, we explore how the costs incurred to install turbines at a wind power plant—the balance-of-system (BOS) costs—scale with turbine rating, hub height, and plant size. We also investigate how these changes in BOS costs influence the levelized cost of energy (LCOE). We show that increasing the plant size from 150 to 400 MW could reduce the BOS costs by 21%. We also show that if the foundation costs decreased by 50%, building a wind power plant with 5-MW turbines (having rotor diameters of 166 m and hub heights of 120 m) could decrease the LCOE by 5%. These results could help inform future BOS cost-reduction opportunities and thereby reduce future capital costs for land-based wind power.

17 WIND ENERGY↗
DOE OSTI2021

A vision of immuno-oncology: the Siena think tank of the Italian Network for Tumor Biotherapy (NIBIT) foundation

The yearly Think Tank Meeting of the Italian Network for Tumor Biotherapy (NIBIT) Foundation, brings together in Siena, Tuscany (Italy), experts in immuno-oncology to review the learnings from current immunotherapy treatments, and to propose new pre-clinical and clinical investigations in selected research areas. While immunotherapies in non-small cell lung cancer and melanoma led to practice changing therapies, the same therapies had only modest benefit for patients with other malignancies, such as mesothelioma and glioblastoma. One way to improve on current immunotherapies is to alter the sequence of each combination agent. Matching the immunotherapy to the host’s immune response may thus improve the activity of the current treatments. A second approach is to combine current immunotherapies with novel agents targeting complementary mechanisms. Identifying the appropriate novel agents may require different approaches than the traditional laboratory-based discovery work. For example, artificial intelligence-based research may help focusing the search for innovative and most promising combination partners. Novel immunotherapies are needed in cancer patients with resistance to or relapse after current immunotherapeutic drugs. Such new treatments may include targeted agents or monoclonal antibodies to overcome the immune-suppressive tumor microenvironment. The mode of combining the novel treatments, including vaccines, needs to be matched to the patient’s immune status for achieving the maximum benefit. In this scenario, specific attention should be also paid nowadays to the immune intersection between COVID-19 and cancer.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Chapter 11: Renewable Microgrids as a Foundation of the Future Sustainable Electrical Energy System

Renewable microgrids are an integral part of the future sustainable energy system. This chapter focuses on the role of renewable energy-based microgrids in the electricity system transformation. It is seen as a foundation that complements large-scale generation plants and high-voltage transmission lines by providing additional flexibility and resilience. This flexibility and resilience are crucial for balancing the energy system and ensuring its long-term sustainability, given the changing supply and demand patterns due to rooftop solar photovoltaics (PV), battery storage, electric mobility, and heat pumps. It is also an opportunity for residential and commercial consumers to become active participants in the energy market. Innovation, new technologies and business models will act as the key enabler for this transformation. The regulatory restructuring of the energy market will be crucial for successful integration of prosumers and aggregated energy communities. The chapter presents the vision of a sustainable, decentralized, and digitized energy future including market potential, most innovative case study projects and start-up companies, and the shifts needed in the regulatory landscape.

battery storage↗
DOE OSTI2017

xSDK Foundations: Toward an Extreme-scale Scientific Software Development Kit

Here, extreme-scale computational science increasingly demands multiscale and multiphysics formulations. Combining software developed by independent groups is imperative: no single team has resources for all predictive science and decision support capabilities. Scientific libraries provide high-quality, reusable software components for constructing applications with improved robustness and portability. However, without coordination, many libraries cannot be easily composed. Namespace collisions, inconsistent arguments, lack of third-party software versioning, and additional difficulties make composition costly. The Extreme-scale Scientific Software Development Kit (xSDK) defines community policies to improve code quality and compatibility across independently developed packages (hypre, PETSc, SuperLU, Trilinos, and Alquimia) and provides a foundation for addressing broader issues in software interoperability, performance portability, and sustainability. The xSDK provides turnkey installation of member software and seamless combination of aggregate capabilities, and it marks first steps toward extreme-scale scientific software ecosystems from which future applications can be composed rapidly with assured quality and scalability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Perceived Costs and Benefits of ICON Science and Foundational Documents associated with “Integrated, Coordinated, Open, and Networked (ICON) Science to Advance the Geosciences: Introduction and Synthesis of a Special Collection of Commentary Articles"

This data package is associated with the publication "Integrated, Coordinated, Open, and Networked (ICON) Science to Advance the Geosciences: Introduction and Synthesis of a Special Collection of Commentary Articles" in Earth and Space Science (Goldman et al. 2022; https://doi.org/10.1029/2021EA002099). The manuscript is an introductory article for a special collection of commentary articles across 19 geoscience disciplines that explore the challenges and opportunities associated with the use of ICON science principles. These principles focus on research intentionally designed to be Integrated, Coordinated, Open, and Networked (ICON) with the goal of maximizing mutual benefit (among stakeholders) and cross-system transferability of science outcomes. This data package contains data, figures, and R scripts associated with the cost/benefit analysis presented in the manuscript. The writing teams involved in the special collection placed each letter of ICON on a plot with perceived cost on one axis and perceived benefit on the other to summarize their perceptions of pursuing each principle of ICON science. These data were subsequently quantified and analyzed. Files are saved as .csv, .R, and .pdf. This data package also contains (1) the public foundational and instructional documents that enabled the crowdsourced creation of the special collection; (2) file-level metadata (flmd) that lists each file in the data package with a description; (3) data dictionary (dd) that defines column headers that appear in csv files. Files are saved as .pdf and .csv.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

The progeria research foundation 10 th international scientific workshop; researching possibilities, ExTENding lives – webinar version scientific summary

Progeria is an ultra-rare (prevalence 1 in 20 million), fatal, pediatric autosomal dominant premature aging disease caused by a mutation in the LMNA gene. This mutation results in accumulation of a high level of an aberrant form of the nuclear membrane protein, Lamin A. This aberrant protein, termed progerin, accumulates in many tissues and is responsible for the diverse array of disease phenotypes. Children die predominantly from premature atherosclerotic cardiovascular disease. The Progeria Research Foundation’s 10th International Scientific Workshop took place via webinar on November 2 and 3, 2020. Participants from 30 countries joined in this new, virtual meeting format. Patient family presentations led the program, followed by updates on Progeria’s first-ever application for FDA drug approval as well as initial results from the only current Progeria clinical trial. This was followed by presentations of unpublished preclinical data on drugs in development targeting the disease-causing DNA mutation, the aberrant mRNA, progerin protein, and its downstream effector proteins. Tying bench to bedside, clinicians presented new discoveries on the natural history of disease to inform future clinical trial development and new Progeria aortic valve replacement procedures. The program engaged the Progeria research community as a single unit with a common goal – to treat and cure children with Progeria worldwide.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2012

Foundation Heat Exchanger Final Report: Demonstration, Measured Performance, and Validated Model and Design Tool

Geothermal heat pumps, sometimes called ground-source heat pumps (GSHPs), have been proven capable of significantly reducing energy use and peak demand in buildings. Conventional equipment for controlling the temperature and humidity of a building, or supplying hot water and fresh outdoor air, must exchange energy (or heat) with the building's outdoor environment. Equipment using the ground as a heat source and heat sink consumes less non-renewable energy (electricity and fossil fuels) because the earth is cooler than outdoor air in summer and warmer in winter. The most important barrier to rapid growth of the GSHP industry is high first cost of GSHP systems to consumers. The most common GSHP system utilizes a closed-loop ground heat exchanger. This type of GSHP system can be used almost anywhere. There is reason to believe that reducing the cost of closed-loop systems is the strategy that would achieve the greatest energy savings with GSHP technology. The cost premium of closed-loop GSHP systems over conventional space conditioning and water heating systems is primarily associated with drilling boreholes or excavating trenches, installing vertical or horizontal ground heat exchangers, and backfilling the excavations. This project investigates reducing the cost of horizontal closed-loop ground heat exchangers by installing them in the construction excavations, augmented when necessary with additional trenches. This approach applies only to new construction of residential and light commercial buildings or additions to such buildings. In the business-as-usual scenario, construction excavations are not used for the horizontal ground heat exchanger (HGHX); instead the HGHX is installed entirely in trenches dug specifically for that purpose. The potential cost savings comes from using the construction excavations for the installation of ground heat exchangers, thereby minimizing the need and expense of digging additional trenches. The term foundation heat exchanger (FHX) has been coined to refer exclusively to ground heat exchangers installed in the overcut around the basement walls. The primary technical challenge undertaken by this project was the development and validation of energy performance models and design tools for FHX. In terms of performance modeling and design, ground heat exchangers in other construction excavations (e.g., utility trenches) are no different from conventional HGHX, and models and design tools for HGHX already exist. This project successfully developed and validated energy performance models and design tools so that FHX or hybrid FHX/HGHX systems can be engineered with confidence, enabling this technology to be applied in residential and light commercial buildings. The validated energy performance model also addresses and solves another problem, the longstanding inadequacy in the way ground-building thermal interaction is represented in building energy models, whether or not there is a ground heat exchanger nearby. Two side-by-side, three-level, unoccupied research houses with walkout basements, identical 3,700 ft{sup 2} floor plans, and hybrid FHX/HGHX systems were constructed to provide validation data sets for the energy performance model and design tool. The envelopes of both houses are very energy efficient and airtight, and the HERS ratings of the homes are 44 and 45 respectively. Both houses are mechanically ventilated with energy recovery ventilators, with space conditioning provided by water-to-air heat pumps with 2 ton nominal capacities. Separate water-to-water heat pumps with 1.5 ton nominal capacities were used for water heating. In these unoccupied research houses, human impact on energy use (hot water draw, etc.) is simulated to match the national average. At House 1 the hybrid FHX/HGHX system was installed in 300 linear feet of excavation, and 60% of that was construction excavation (needed to construct the home). At House 2 the hybrid FHX/HGHX system was installed in 360 feet of excavation, 50% of which was construction excavation. There are six pipes in all excavations (three parallel circuits - out and back), and the multiple instances of FHX and/or HGHX are all connected in series. The working fluid is 20% by weight propylene glycol in water. Model and design tool development was undertaken in parallel with constructing the houses, installing instrumentation, and monitoring performance for a year. Several detailed numerical models for FHX were developed as part of the project. Essentially the project team was searching for an energy performance model accurate enough to achieve project objectives while also having sufficient computational efficiency for practical use in EnergyPlus. A 3-dimensional, dual-coordinate-system, finite-volume model satisfied these criteria and was included in the October 2011 EnergyPlus Version 7 public release after being validated against measured data.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗