Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA BASE MANAGEMENT”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Data for Identifying the best high-biomass sorghum hybrids based on biomass yield potential and feedstock quality affected by nitrogen fertility management under various environments

Data were collected from agronomy fields in Urbana and Ewing, IL, during the 2022 and 2023 growing seasons. The dataset includes dry biomass yield, nitrogen, phosphorus, and potassium concentrations and removals, and chemical composition elements (cellulose, hemicellulose, lignin, and soluble fractions) for 13 high-biomass sorghum hybrids. data_sharing.xlsx contains 20 columns and 104 rows. Below is the explanation of all variables in the file: Year: 2022; 2023 Location: Urbana, IL; Ewing, IL N rate (kg-N/ha): 0; 112 Hybrid #: H1-H13 Pedigree: Pedigree for 13 hybrids Dry biomass yield (Mg/ha): Aboveground dry biomass yield N (g/kg): Nitrogen concentration in plant tissue P (g/kg): Phosphorus concentration in plant tissue K (g/kg): Potassium concentration in plant tissue N (kg/ha): Nitrogen removal by aboveground biomass P (kg/ha): Phosphorus removal by aboveground biomass K (kg/ha): Potassium removal by aboveground biomass Cellulose (g/kg): Cellulose concentration in plant tissue Hemicellulose (g/kg): Hemicellulose concentration in plant tissue Lignin (g/kg): Lignin concentration in plant tissue Soluble (g/kg): Soluble concentration in plant tissue Cellulose (Mg/ha): Cellulose content in aboveground biomass Hemicellulose (Mg/ha): Hemicellulose content in aboveground biomass Lignin (Mg/ha): Lignin content in aboveground biomass Soluble (Mg/ha): Soluble content in aboveground biomass

environmental adaptability↗
DOE OSTI2022

Accelerating Scientific Workflows on HPC Platforms with In Situ Processing

Scientific workflows drive most modern large-scale science breakthroughs by allowing scientists to define their computations as a set of jobs executed in a given order based on their data dependencies. Workflow management systems (WMSs) have become key to automating scientific workflows-executing computational jobs and orchestrating data transfers between those jobs running on complex high-performance computing (HPC) platforms. Traditionally, WMSs use files to communicate between jobs: a job writes out files that are read by other jobs. However, HPC machines face a growing gap between their storage and compute capabilities. To address that concern, the scientific community has adopted a new approach called in situ, which bypasses costly parallel filesystem I/O operations with faster in-memory or in-network communications. When using in situ approaches, communication and computations can be interleaved. In this work, we leverage the Decaf in situ dataflow framework to accelerate task-based scientific workflows managed by the Pegasus WMS, by replacing file communications with faster MPI messaging. We propose a new execution engine that uses Decaf to manage communications within a sub-workflow (i.e., set of jobs) to optimize inter-job communications. We consider two workflows in this study: (i) a synthetic workflow that benchmarks and compares file- and MPI-based communication; and (ii) a realistic bioinformatics workflow that computes mu-tational overlaps in the human genome. Experiments show that in situ communication can improve the bioinformatics workflow execution time by 22% to 30% compared with file communication. Our results motivate further opportunities and challenges for bridging traditional WMSs with in situ frameworks.

Decaf↗
DOE OSTI2021

Oak Ridge National Laboratory Roane and Anderson Counties, Tennessee Historic Architectural Resource Survey June 2018

From February through September 2017, Cultural Resource Analysts, Inc. personnel completed a historic architectural resource survey of the Oak Ridge National Laboratory in Roane and Anderson Counties, Tennessee. The survey was completed at the request of UT-Battelle, LLC, the private not-for-profit company that manages and operates Oak Ridge National Laboratory for the U.S. Department of Energy. The purpose of the survey was to assist the Department of Energy in complying with the National Historic Preservation Act of 1966, as amended, by providing updated recommendations regarding the National Register of Historic Places eligibility of historic architectural resources at Oak Ridge National Laboratory. The survey included the entirety of Oak Ridge National Laboratory’s main campus that occupies approximately 1,100 acres in Bethel and Melton Valleys and on Chestnut Ridge. Based on Facilities Information Management System data and guidance provided by UT-Battelle, LLC, preservation and modernization staff, the survey team developed a list of resources for inclusion in the survey. The list included all substantial buildings constructed in or before 1980 in order to provide sufficient coverage of buildings that will turn 50 years of age within the next decade. Only those buildings with recorded area over 100 sq ft were included in the survey. Additionally, the UT-Battelle team identified 15 buildings for more intensive survey because such buildings are central to Oak Ridge National Laboratory’s future mission needs and are expected to undergo renovations in the foreseeable future. These building were subject to both exterior and interior documentation with particular focus on identifying character-defining features, and thus were subject to more detailed analysis than the other resources included in the survey. The findings of this historic resource survey build on the conclusions of the 1994 survey by DuVall & Associates, Inc., as well as the survey updates completed by Thomason and Associates in 2004 and 2015 in support of the Oak Ridge National Laboratory Historic Preservation Plan. Within 12 distinct areas of Oak Ridge National Laboratory, as well as outlying areas, the survey recorded 195 buildings and structures constructed in or before 1980. In consideration of the role and significance of each area and each individual resource within the context of Oak Ridge National Laboratory, and in light of their current integrity and condition, each recorded resource was evaluated to determine if it is eligible for listing in the National Register of Historic Places.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

AmeriFlux BASE data pipeline to support network growth and data sharing

Abstract AmeriFlux is a network of research sites that measure carbon, water, and energy fluxes between ecosystems and the atmosphere using the eddy covariance technique to study a variety of Earth science questions. AmeriFlux’s diversity of ecosystems, instruments, and data-processing routines create challenges for data standardization, quality assurance, and sharing across the network. To address these challenges, the AmeriFlux Management Project (AMP) designed and implemented the BASE data-processing pipeline. The pipeline begins with data uploaded by the site teams, followed by the AMP team’s quality assurance and quality control (QA/QC), ingestion of site metadata, and publication of the BASE data product. The semi-automated pipeline enables us to keep pace with the rapid growth of the network. As of 2022, the AmeriFlux BASE data product contains 3,130 site years of data from 444 sites, with standardized units and variable names of more than 60 common variables, representing the largest long-term data repository for flux-met data in the world. The standardized, quality-ensured data product facilitates multisite comparisons, model evaluations, and data syntheses.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

GenomeDepot: data management system for microbial comparative genomics

Summary GenomeDepot is an open-source web-based platform for annotation, management, and comparative analysis of microbial genomic sequences and associated data including ortholog families, protein domains, operons, regulatory interactions, strain taxonomy, and sample metadata. GenomeDepot supports rapid creation of websites for user-defined genome collections that include bioinformatic tools for interactive genome browsing, Basic Local Alignment Search Tool (BLAST) search, annotation search, comparative genomic neighborhood visualization, and sequence download. Gene function annotations are generated by a customizable annotation pipeline. The pipeline runs annotation tools in Conda environments and can be easily extended with additional user-specified tools. Availability and implementation GenomeDepot is open source and distributed under the GNU General Public License via GitHub (https://github.com/aekazakov/genome-depot). GenomeDepot is implemented in Python and was tested in Ubuntu Linux. Full installation instructions and documentation are available at https://aekazakov.github.io/genome-depot/. GenomeDepot demo server is freely accessible at https://iseq.lbl.gov/demogd/.

Kazakov, Alexey [Lawrence Berkeley National Labora↗
DOE OSTI2020

Performance Potential of Mixed Data Management Modes for Heterogeneous Memory Systems

Many high-performance systems now include different types of memory devices within the same compute platform to meet strict performance and cost constraints. Such heterogeneous memory systems often include an upper-level tier with better performance, but limited capacity, and lower-level tiers with higher capacity, but less bandwidth and longer latencies for reads and writes. To utilize the different memory layers efficiently, current systems rely on hardware-directed, memory -side caching or they provide facilities in the operating system (OS) that allow applications to make their own data-tier assignments. Since these data management options each come with their own set of trade-offs, many systems also include mixed data management configurations that allow applications to employ hardware- and software-directed management simultaneously, but for different portions of their address space. Despite the opportunity to address limitations of stand-alone data management options, such mixed management modes are under-utilized in practice, and have not been evaluated in prior studies of complex memory hardware. In this work, we develop custom program profiling, configurations, and policies to study the potential of mixed data management modes to outperform hardware- or software-based management schemes alone. Our experiments, conducted on an Intel ® Knights Landing platform with high-bandwidth memory, demonstrate that the mixed data management mode achieves the same or better performance than the best stand-alone option for five memory intensive benchmark applications (run separately and in isolation), resulting in an average speedup compared to the best stand-alone policy of over 10 %, on average.

Effler, Chad↗
DOE OSTI2025

Next-Level Energy Management in Manufacturing: Facility-Level Energy Digital Twin Framework Based on Machine Learning and Automated Data Collection

This research introduces an energy prediction framework at the facility level supported by automated data collection and machine learning models. It investigates whether reducing the prediction time scale allows for applying more complex machine learning techniques and if those techniques improve the prediction accuracy. The primary advantages of this framework lie in its automation of the energy prediction process and its provision of real-time energy data suitable for use in energy dashboards or digital twins. A sitewide dataset was created by combining 15 min energy and daily production data of five shops—assembly, battery, body (electric), body (gas), and paint—from a globally recognized electric vehicle manufacturer. Various machine learning models were evaluated on daily, weekly, and monthly datasets, including, in increasingly complex order: naïve, simple linear regression, net regularized generalized linear regression, principal component regression, k-nearest neighbor, random forest, and Bayesian regularized neural network. Compared to the current state-of-the-art energy consumption prediction for the industrial facility level, this research investigates more complex models and smaller time intervals for higher accuracy. The findings revealed that the more complex monthly models require a minimum of a year and a half of data to operate, while weekly models demand a year of data to achieve improved accuracy. Daily models can operate with only six months of data but exhibit poor performance due to reduced prediction accuracy of production. Key challenges identified include access to reliable, high-quality energy and production data and the initial demand for human labor.

digital twin↗
DOE OSTI2024

Scalable Implicit Solvers with Dynamic Mesh Adaptation for a Relativistic Drift-Kinetic Fokker–Planck–Boltzmann Model

In this work we consider a relativistic drift-kinetic model for runaway electrons along with a Fokker–Planck operator for small-angle Coulomb collisions, a radiation damping operator, and a secondary knock-on (Boltzmann) collision source. Here, we develop a new scalable fully implicit solver utilizing finite volume and conservative finite difference schemes and dynamic mesh adaptivity. A new data management framework in the PETSc library based on the p4est library is developed to enable simulations with dynamic adaptive mesh refinement (AMR), distributed memory parallelization, and dynamic load balancing of computational work. This framework and the runaway electron solver building on the framework are able to dynamically capture both bulk Maxwellian at the low-energy region and a runaway tail at the high-energy region. To effectively capture features via the AMR algorithm, a new AMR indicator prediction strategy is proposed that is performed alongside the implicit time evolution of the solution. This strategy is complemented by the introduction of computationally cheap feature-based AMR indicators that are analyzed theoretically. Numerical results quantify the advantages of the prediction strategy in better capturing features compared with nonpredictive strategies; and we demonstrate trade-offs regarding computational costs. The robustness with respect to model parameters, algorithmic scalability, and parallel scalability are demonstrated through several benchmark problems including manufactured solutions and solutions of different physics models. We focus on demonstrating the advantages of using implicit time stepping and AMR for runaway electron simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

SlimIO: Lightweight I/O Path Design for Write Isolation in FDP-backed In-Memory Databases

In-Memory Databases (IMDBs) are widely used with HPC applications to manage transient data, often using snapshot-based persistence for backups. Redis, a representative IMDB, employs both snapshot and Write-Ahead Log (WAL) mechanisms, storing data on persistent devices via the traditional kernel I/O path. This method incurs syscall overhead, I/O contention between processes, and SSD garbage collection (GC) delays. To address these issues, we propose SlimIO, which adopts I/O passthru to minimize syscall overhead and inter-process I/O interference. Additionally, it leverages Flexible Data Placement (FDP) SSDs as backup storage to avoid performance degradation from SSD GC. Experimental results show that SlimIO reduces snapshot time by up to 25%, increases query throughput by up to 30% during non-snapshot periods, and lowers 99.9%-ile latency by up to 50%. Furthermore, it achieves a write amplification factor (WAF) of 1.00, indicating no redundant internal writes, thus extending SSD lifespan.

Lee, Sangyun [Sogang University]↗
DOE OSTI2021

Energy Efficient Data Centers

Energy efficient data center strategies include thermal/environmental guidelines, air management, free cooling, and liquid-based cooling. A case study highlighting the data center cooling system at the National Renewable Energy Laboratory Energy Systems Integration Facility illustrates many of these principles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Energy Efficient Data Centers

Energy efficient data center strategies include thermal/environmental guidelines, air management, free cooling, and liquid-based cooling. A case study highlighting the data center cooling system at the National Renewable Energy Laboratory Energy Systems Integration Facility illustrates many of these principles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Digital Twin for Hydropower System Object Modeling: Alder Dam (FY2023)

Hydropower is the world's largest source of renewable electricity, and hydropower plants are distributed all over the world. Typical major components of a hydropower plant are the governor, excitation, generator, thrust bearing, hydraulic turbine, transformer, the main lead, metering and control, tailwater depression, and dissolved oxygen. For each component, various measures are taken. The measurements are acquired by various heterogeneous systems, including standalone sensors, programmable logic controllers (PLC), Supervisory control and data acquisition (SCADA), Internet of Things (IoT), and data acquisition and integration platforms such as OSI/PI. The measured data are often archived within the plant by a data management platform, and many institutions have cloud-based archive systems, such as Hydropower Research Institution (HRI), U.S. Army Corps of Engineers (USACE), and Columbia River Data Access in Real Time (DART). Object Modeling is a general framework for designing information systems. It focuses on objects, the actions they perform, and the messages they send to one another to cause those actions to be taken. The major differences among object modeling, network modeling, data modeling, and process modeling are that in the first we focus on the actions in response to information, objects which form the system, the actions they perform, and how they pass information to one another, while in the second we concentrate on where, when and how much information is moved, while in the third we focus on what information is moved and where it is moved, while in the last we focus on how it is moved and when it is moved. Object modeling was developed basically as a method to develop object-oriented systems and to support object-oriented programming. It describes the static structure of the system. The object Modeling Technique is easy to draw and use. That is why we choose object modeling to connect physical hydropower plants to Digital Twin. It recognizes the objects and the relationship between them. It identifies the attributes and functions of each class. Dynamic Modeling: It explains how objects respond to events. Functional Modeling indicates the processes executed in an object and how data changes when it moves to objects. It has been used in many applications like telecommunication, transportation, etc.

13 HYDRO ENERGY↗
DOE OSTI2021

Interactive Web Application for Traffic Simulation Data Management and Visualization

As traffic simulation software becomes more effective for realistically simulating and analyzing traffic dynamics and vehicle interactions on the mesoscopic and microscopic level, the management, dissemination, and collaborative visualization of traffic simulation results produced by individual transportation planners presents a significant challenge. Existing online content management systems have a very limited capability in allowing users to query specific traffic simulation scenarios and geospatially visualize simulation results through shareable and interactive web interfaces. This paper presents a web-based application for promoting the archiving, sharing, and visualization of large-scale traffic simulation outputs. The application is developed to enhance cyber-physical controls, communications, and public education for collaborative transportation planning. Unique features of the web application include: (a) allowing users to upload their new traffic simulation scenarios (parameters and outputs), as well as search existing scenarios using easily accessible interfaces; (b) optimizing simulation output files with heterogeneous data formats and projected coordinate systems for web-based storage and management using a scalable and searchable data/metadata standard; (c) standardizing user-uploaded simulation outputs using web interfaces and data processing libraries with parallel computing capacity; and (d) providing shareable web visual interfaces for visualizing the traffic flow and signal information stored in simulation outputs (e.g., regional traffic patterns and individual vehicle interactions) and visually comparing multiple simulation outputs both spatially and temporally. Furthermore, the paper presents the conceptual design and implementation of this application, and demonstrates the application’s performance for sharing, comparing, and visualizing simulation outputs from VISSIM and SUMO, two commonly used traffic simulation software programs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

LANL Contributions to pyiron: An integrated development environment for computational materials science

To support and accelerate the development of simulation protocols in atomistic modelling, we introduce an integrated development environment (IDE) for computational materials science called pyiron (http://pyiron.org). The pyiron IDE combines a web based source code editor, a job management system for build automation, and a hierarchical data management solution. The core components of the pyiron IDE are pyiron objects based on an abstract class, which links application structures such as atomistic structures, projects, jobs, simulation protocols and computing resources with persistent storage and an interactive user environment. The simulation protocols within the pyiron IDE are constructed using the Python programming language. To highlight key concepts of this tool as well as to demonstrate its ability to simplify the implementation and testing of simulation protocols we discuss two applications. In these examples we show how pyiron supports the whole life cycle of a typical simulation, seamlessly combines ab initio with empirical potential calculations, and how complex feedback loops can be implemented. While originally developed with focus on ab initio thermodynamics simulations, the concepts and implementation of pyiron are general thus allowing to employ it for a wide range of simulation topics.

Janssen, Jan↗
DOE OSTI2025

Recent Experience with the CMS Data Management System

The CMS[1] experiment manages a large-scale data infrastructure, currently handling over 200 PB of disk and 500 PB of tape storage and transferring more than 1 PB of data per day on average between various WLCG[2] sites. Utilizing Rucio[3] for high-level data management, FTS[4] for data transfers, and a variety of storage and network technologies at the sites, CMS confronts inevitable challenges due to the system’s growing scale and evolving nature. Key challenges include managing transfer and storage failures, optimizing data distribution across different storages based on production and analysis needs, implementing necessary technology upgrades and migrations, and efficiently handling user requests. The data management team has established comprehensive monitoring to supervise this system and has successfully addressed many of these challenges. The team’s efforts aim to ensure data availability and protection, minimize failures and manual interventions, maximize transfer throughput and resource utilization, and provide reliable user support. This paper details the operational experience of CMS with its data management system in recent years, focusing on the encountered challenges, the effective strategies employed to overcome them and the ongoing challenges as we prepare for future demands.

Öztürk, Hasan [CERN]↗
DOE OSTI2025

Active learning path-dependent properties using a cloud-based materials acceleration platform

Solid state materials are central to many modern technologies in which a given material may be exposed to a variety of environments. The material properties often vary with the sequence of environments in an irreversible manner, resulting in a quintessential path-dependency in experimental observables. While sequential learning techniques have been effectively deployed for accelerating learning of state properties of materials, they often use a consistent environment path in all experiments. To elevate such techniques for making optimal decisions in experimental investigations of path-dependent properties, we introduce an iterated expected information gain acquisition function that optimizes over entire experimental trajectories. This approach is implemented within a cloud-based Materials Acceleration Platform architecture utilizing an event-driven stateful broker coupled with remote HELAO (Hierarchical Experimental Laboratory Automation and Orchestration) instances and an AI science manager. The platform's efficacy was demonstrated through a case study optimizing multi-step spectro-electrochemical experiments to identify optically stable potential windows in (Co–Ni–Sb)O z metal oxides. The system successfully integrated AI-driven experiment design, remote laboratory automation, and cloud-based data infrastructure, validating the platform's capability for managing complex, adaptive, path-dependent workflows in materials discovery.

Guevarra, Dan [California Institute of Technology ↗