SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

DOE OSTI2015

A Client/Server Architecture for Supporting Science Data Using EPICS Version 4

The Phase 1 grant that serves as a precursor to this proposal, prototyped complex storage techniques for high speed structured data that is being produced in accelerator diagnostics and beam line experiments. It demonstrates the technologies that can be used to archive and retrieve complex data structures and provide the performance required by our new accelerators, instrumentations, and detectors. Phase 2 is proposed to develop a high-performance platform for data acquisition and analysis to provide physicists and operators a better understanding of the beam dynamics. This proposal includes developing a platform for reading 109 MHz data at 10 KHz rates through a multicore front end processor, archiving the data to an archive repository that is then indexed for fast retrieval. The data is then retrieved from this data archive, integrated with the scalar data, to provide data sets to client applications for analysis, use in feedback, and to aid in identifying problem with the instrumentation, plant, beam steering, or model. This development is built on EPICS version 4 , which is being successfully deployed to implement physics applications. Through prior SBIR grants, EPICS version 4 has a solid communication protocol for middle layer services (PVAccess), structured data representation and methods for efficient transportation and access (PVData), an operational hierarchical record environment (JAVA IOC), and prototypes for standard structured data (Normative Types). This work was further developed through project funding to successfully deploy the first service based physics application environment with demonstrated services that provide arbitrary object views, save sets, model, lattice, and unit conversion. Thin client physics applications have been developed in Python that implement quad centering, orbit display, bump control, and slow orbit feedback. This service based architecture has provided a very modular and robust environment that enables commissioning teams to rapidly develop and deploy small scripts that build on powerful services. These services are all built on relational database data stores and scalar data. The work proposed herein, builds on these previous successes to provide data acquisition of high speed data for online analysis clients.

43 PARTICLE ACCELERATORS↗

DOE OSTI2019

GRACC: GRid ACcounting Collector

The OSG has long maintained a central accounting system called Gratia. It uses small probes on each computing and storage resource in order to collect resource usage. The probes report to a central collector which stores the usage in a database. The database is then queried to generate reports. As the OSG aged, the size of the database grew very large. It became too large for the database technology to efficiently query to generate detailed reports. The design of a replacement requires data storage that could be queried efficiently to generate multi-year reports. Additionally, it requires flexibilityto add new attributes to the collected data.In this paper we will describe updates to the GRACC architecture in the last 18 months. GRACC uses modern web technologies that were designed for large data storage, query, and visualization. That includes the open source database Elasticsearch, message broker software RabbitMQ, and Grafana and Kibana as data visualization platforms. It uses multiple agents that perform operations on the data to transform it for easier querying and summarization.

Weitzel, Derek↗

DOE OSTI2017

Echo

This document is a white paper marketing proposal for Echo™ is a data analysis platform designed for efficient, robust, and scalable creation and execution of complex workflows. Echo’s analysis management system refers to the ability to track, understand, and reproduce workflows used for arriving at results and decisions. Echo improves on traditional scripted data analysis in MATLAB, Python, R, and other languages to allow analysts to make better use of their time. Additionally, the Echo platform provides a powerful data management and curation solution allowing analysts to quickly find, access, and consume datasets. After two years of development and a first release in early 2016, Echo is now available for use with many data types in a wide range of application domains. Echo provides tools that allow users to focus on data analysis and decisions with confidence that results are reported accurately.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Navigation and Meteorological Data from Multiple Sensors on Airborne Platform (NAVMET-AIR) Value-Added Product Report

The Navigation and Meteorological Data from Multiple Sensors on Airborne Platform (NAVMET-AIR) value-added product encompasses the aafnaviwg data set in the ARM Data Center (ADC), known as the IWG file, named for the Inter-Agency Working Group for Airborne Data and Telemetry Systems (IWGADTS). Its purpose is to produce a suite of tools to promote standardization of instrument interface, data format, and data processing. This data set’s contents are navigational and meteorological state variables at 1 Hz. It also contains higher-order data flags created by ARM Aerial Facility (AAF) scientists to aid analysis such as periods when the aircraft is flying level, operating in maneuvers, or flying through cloud. Due to the nature of research flights, the payload of the aircraft consists of many duplicate and overlapping instruments to ensure, in case of instrument failure, there is a backup to record the data. Measurements from multiple instruments are consolidated into a single file. Each variable of this data set is carefully chosen from the onboard instrumentation and quality checked by AAF scientists to create this wholistic and most accurate data set for airborne research. This document is intended to be a hub towards the individual “read me” files produced for each campaign. For reference about what instrument was used for calculations on specific days, the read me files are located in the ARM IOP archive at the addresses listed, or access the IOP (intensive operational period) database at https://www.archive.arm.gov/.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2015

Edge Bioinformatics

Edge Bioinformatics is a developmental bioinformatics and data management platform which seeks to supply laboratories with bioinformatics pipelines for analyzing data associated with common samples case goals. Edge Bioinformatics enables sequencing as a solution and forward-deployed situations where human-resources, space, bandwidth, and time are limited. The Edge bioinformatics pipeline was designed based on following USE CASES and specific to illumina sequencing reads. 1. Assay performance adjudication (PCR): Analysis of an existing PCR assay in a genomic context, and automated design of a new assay to resolve conflicting results; 2. Clinical presentation with extreme symptoms: Characterization of a known pathogen or co-infection with a. Novel emerging disease outbreak or b. Environmental surveillance

Lo, Chien-Chi↗

DOE OSTI2025

Hazard Detection Detector Cards

This report presents a comprehensive summary of five advanced anomaly detection tools developed and deployed by Oak Ridge National Laboratory in support of the VA’s Health Information Technology modernization. These detectors—Order Path Tracker, Trend Watcher, Pain Pointer, Performance Monitor, and Patient Record Flag Detector—leverage statistical and machine learning methods to monitor workflow disruptions, detect anomalies in care sequences and volumes, identify bottlenecks, and track system-level performance metrics across VistA and Millennium systems. All detectors have been integrated into the Health Data Analytics Platform (HDAP), with most having completed deployment and testing using live data from targeted stations in cardiology and oncology domains. This work enhances VA’s capacity for proactive system surveillance, promotes patient safety, and informs data-driven operational improvements across the EHR ecosystem.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Outcomes of PAX sapiens-Supported Global Wildlife Data Sharing Conferences for Enhanced One Health Security (GWDSC)

Across two consecutive Global Wildlife Data Sharing Conferences supported by PAX sapiens—Year 1 (May 2024) at Pacific Northwest National Laboratory and Year 2 (2025) in Ciudad Real, Spain—the initiative converted wildlife data sharing from aspiration into operational reality, producing measurable impacts in platform development, data mobilization, standards harmonization, and international partnership formation. The conferences addressed a critical gap in global health security: while 75% of emerging infectious diseases affect both humans and animals and over 60% originate in wildlife, wildlife health surveillance has historically lagged behind human and agricultural sectors due to fragmented databases, inconsistent terminology, uneven capacity, and limited cross-border coordination. By convening practitioners, government agencies, international organizations, academic institutions, and NGOs, the GWDSC catalyzed trust-based relationships and practical workflows that enable earlier detection, better risk assessment, and more effective prevention of threats at the wildlife–domestic animal–human–environment interface.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗

DOE OSTI2014

Optimization and Parallelization of the Thermal-Hydraulic Sub-channel Code CTF for High-Fidelity Multi-physics Applications

This paper describes major improvements to the computational infrastructure of the CTF sub-channel code so that full-core sub-channel-resolved simulations can now be performed in much shorter run-times, either in stand-alone mode or as part of coupled-code multi-physics calculations. These improvements support the goals of the Department Of Energy (DOE) Consortium for Advanced Simulations of Light Water (CASL) Energy Innovation Hub to develop high fidelity multi-physics simulation tools for nuclear energy design and analysis. A set of serial code optimizations--including fixing computational inefficiencies, optimizing the numerical approach, and making smarter data storage choices--are first described and shown to reduce both execution time and memory usage by about a factor of ten. Next, a Single Program Multiple Data (SPMD) parallelization strategy targeting distributed memory Multiple Instruction Multiple Data (MIMD) platforms and utilizing domain-decomposition is presented. In this approach, data communication between processors is accomplished by inserting standard MPI calls at strategic points in the code. The domain decomposition approach implemented assigns one MPI process to each fuel assembly, with each domain being represented by its own CTF input file. The creation of CTF input files, both for serial and parallel runs, is also fully automated through use of a pre-processor utility that takes a greatly reduced set of user input over the traditional CTF input file. To run CTF in parallel, two additional libraries are currently needed; MPI, for inter-processor message passing, and the Parallel Extensible Toolkit for Scientific Computation (PETSc), which is leveraged to solve the global pressure matrix in parallel. Results presented include a set of testing and verification calculations and performance tests assessing parallel scaling characteristics up to a full core, sub-channel-resolved model of Watts Bar Unit 1 under hot full-power conditions (193 17x17 assemblies, ~56,000 pins, ~59,000 sub-channels, ~2.8 million thermal-hydraulic (TH) control volumes). Results demonstrate that CTF can now perform full core analysis (not previously possible due to excessively long runtimes and memory requirements) on the order of 20 minutes. This new capability is not only useful to standalone CTF users, but is also being leveraged in support of coupled code multi-physics calculations being done in the CASL program.

COBRA-TF↗

DOE OSTI2015

Edge Bioinformatics

Lo, Chien-Chi↗

DOE OSTI2022

High-Fidelity Solar Irradiance Data: Simple Access to State-of-the-Art Information Accelerates Southeast Asia's Clean Energy Economic Transformation

High quality, robust, and reliable renewable energy resource data is foundational to climate-smart decision making, evidence-based policy planning, and clean energy investment mobilization. USAID and NREL, through the Advanced Energy Partnership for Asia, are expanding access to this critical resource data by providing free, high-fidelity solar resource data for Southeast Asia through the RE Data Explorer platform. This brief highlights several ways the Southeast Asia solar resource data has been used for power system planning and project development in the region.

Advanced Energy Partnership for Asia↗

DOE OSTI2022

Digitalization, Management and Synthesis of Thermal Hydraulic Legacy Data Using Dynamical System Scaling

A unique attribute of the nuclear industry is the extent by which designers, developers, operators, and regulators pay attention to demonstrating the safety case. The safety case for nuclear systems is a written, traceable expression of safety, demonstrating the system’s protective measures against uncontrolled radiological releases. The nuclear installation resiliency or safety takes overriding priority over function and performance. The preparation of the safety case heavily relies on extensive experimental campaigns that challenge the target design under a spectrum of postulated accident scenarios. For safety reasons, these experiments are typically conducted in sub-scale test apparatuses intended to reproduce the postulated accident scenarios and event sequences realistically but subject to proven similarity criteria, often referred to as scaling analyses. Since the dawn of the nuclear industry, significant resources have been committed to the construction and operation of such test facilities and a massive amount of data has been generated. Such test data have been and are still used to validate the fundamental assumptions of nuclear power plant phenomena. As we move into the digital world, capturing this multi-decades worth of information in a transparent, easily accessible, and usable manner for the public and industry stakeholders is of paramount importance. Over the last few years FPoliSolutions has been actively developing an enterprise digital data ontology platform (FPoliDON) intended to do just that. The concept was introduced in a paper at the previous NURETH edition and has been significantly enhanced since. With the vision of helping designers use the data to assess their safety case evaluation models, the platform was designed not only to ontologically organize the experimental data, but also to guide the user in the complex analytics associated with the interpretation and use of such test results. A new feature which has been recently added is the ability for the user to access a library of sophisticated mathematical procedures for performing scaling analyses. For this purpose, the Dynamical System Scaling (DSS) procedure invented by Dr. Jose’ Reyes was formulated as one of the services in the digital platform. These methods have been demonstrated to provide an excelled mathematical apparatus to identify similarity criteria or quantify distortions between measured data and the target prototypical conditions. Moreover, DSS can provide a level of synthesis across SETs and IETs which has the promise to yield efficiency in the evaluation code assessment activities. This is often a daunting and expensive task to set up evaluation models that support the safety case of current and future nuclear power plants.

Reyes, Jose↗

DOE OSTI2021

Bioinformatic teaching resources - for educators, by educators - using KBase, a free, user-friendly, open source platform

Over the past year, biology educators and staff at the Department of Energy Systems Biology Knowledgebase (KBase) initiated a collaborative effort to develop a curriculum for bioinformatics education. KBase is a free and easily accessible data science platform that integrates many bioinformatics resources into a graphical user interface built upon reproducible analysis notebooks. KBase held conversations with college and high school instructors to understand how KBase could potentially support their educational goals. These conversations morphed into a working group of biological and data science instructors that adapted the KBase platform to their curriculum needs, specifically around concepts in Genomics, Metagenomics, Pangenomics, and Phylogenetics. The KBase Educators Working Group developed modular, adaptable, and customizable instructional units. Each instructional module contains teaching resources, publicly available data, analysis tools, and markdown capability to tailor instructions and learning goals for each class. The online user interface enables students to conduct hands-on data science research and analyses without requiring programming skills or their own computational resources (these are provided by KBase). Alongside these resources, KBase continues to work with instructors, supporting the development of additional curriculum modules. For anyone new to the platform, KBase, and the growing KBase Educators Organization, provides a community network, accompanied by community-sourced guidelines, instructional templates, and peer support to use KBase within a classroom whether virtual or in-person.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2019

Reimagining the potential of Earth observations for ecosystem service assessments

The benefits nature provides to people, called ecosystem services, are increasingly recognized and accounted for in assessments of infrastructure development, agricultural management, conservation prioritization, and sustainable sourcing. These assessments are often limited by data, however, a gap with tremendous potential to be filled through Earth observations (EO), which produce a variety of data across spatial and temporal extents and resolutions. Despite widespread recognition of this potential, in practice few ecosystem service studies use EO. Here, we identify challenges and opportunities to using EO in ecosystem service modeling and assessment. Some challenges are technical, related to data awareness, processing, and access. These challenges require systematic investment in model platforms and data management. Other challenges are more conceptual but still systemic; they are byproducts of the structure of existing ecosystem service models and addressing them requires scientific investment in solutions and tools applicable to a wide range of models and approaches. We also highlight new ways in which EO can be leveraged for ecosystem service assessments, identifying promising new areas of research. More widespread use of EO for ecosystem service assessment will only be achieved if all of these types of challenges are addressed. Furthermore, this will require non-traditional funding and partnering opportunities from private and public agencies to promote data exploration, sharing, and archiving. Investing in this integration will be reflected in better and more accurate ecosystem service assessments worldwide.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2019

DOE Data Day 2019 Report

The Department of Energy (DOE) has joined the larger scientific community in the promotion of data management as a means to higher quality, more efficient research and analysis, and as a critical component of data science. Tools and platforms to support data management and analysis are rapidly evolving and provide enormous opportunities. They also pose challenges that can be specific to DOE but are common across DOE mission areas and organizations. The DOE Data Day (D3) workshop was held on September 25–26, 2019, and organized by a team at Lawrence Livermore National Laboratory (LLNL) in an effort to gather the data management practitioners at the DOE labs together to share their work and results, facilitating knowledge transfers and best practices across project teams. Over 100 attendees participated from multiple national labs—LLNL, Sandia (SNL), Lawrence Berkeley (LBNL), Los Alamos (LANL), Oak Ridge (ORNL), Argonne (ANL), Pacific Northwest (PNNL), Stanford Linear Accelerator (SLAC)—along with Mission Support and Test Services (MSTS), National Energy Technology Laboratory (NETL), Pantex, National Security Complex (Y-12), National Nuclear Security Administration (NNSA), the DOE Office of Scientific and Technical Information (OSTI), and elsewhere in the DOE complex.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2023

High-Resolution Southeast Asia Wind Resource Data Set

Well informed decision-making is a key part of integrating variable renewable energy into the global energy marketplace. USAID and NREL, through the Advanced Energy Partnership for Asia, are expanding access to critical resource data by providing free, high-fidelity time-series wind resource data for Southeast Asia through the RE Data Explorer platform. This brief highlights the development of the Southeast Asia wind resource data set and discusses the impacts of this data.

Advanced Energy Partnership for Asia↗

DOE OSTI2014

ORBiT - The Oak Ridge Biosurveillance Toolkit

We describe ORBiT, the Oak Ridge Biosurveillance Toolkit, the objective of which is to provide 1) forecasting and early warning, and 2) an extensible data analytics platform for biosurveillance by enabling the use of traditional and non-traditional datasets, consisting of heterogeneous data types and modalities. ORBiT includes a data collection interface, analytic components, higher-order statistical tools, and a visual front-end with visual analytics. ORBiT is implemented as a distributed analytic platform consisting of a software stack atop of Hadoop and uses Titan, a distributed graph database as a backend for data storage.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

SEED Platform for Building Performance Standards Implementation Guide (Spanish Translation)

This guide provides an overview of the Standard Energy Efficiency Data (SEED) Platform. The SEED Platform developed by the U.S. Department of Energy (DOE) to provide a low-cost, user-friendly tool for jurisdictions to launch and manage energy benchmarking and Building Performance Standard (BPS) programs. It has been translated into Spanish. This is the Spanish translation of NREL/FS-5500-90691.

benchmarking↗