Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Digitalization of an experimental electrochemical reactor via the smart manufacturing innovation platform

The exponential increase in data produced over the last two decades has revolutionized the way we collect, store, process, analyze, model, and interpret information to improve profitability. Manufacturing is no exception. How- ever, Smart Manufacturing, the digital practice, organization, workforce, and infrastructure transformation for collection and deployment of data and models at scale and at all levels of manufacturing, is a complex, costly, and labor-intensive journey that is still seeing slow adoption. The Clean Energy Smart Manufacturing Innovation Institute (CESMII), a national Manufacturing USA public-private partnership sponsored by the Department of Energy, is addressing this scaled use of data and modeling in manufacturing. CESMII has focused on how to col- lect and use operating data for numerous applications that improve productivity, precision, and performance of manufacturing operations from factory floor to supply chain using process simulation, predictive analytics, mon- itoring and control, and real-time optimization. Because contextualized data are key, CESMII has developed the Smart Manufacturing Innovation Platform (SMIP) to lower the barriers to the data that are needed to accelerate data-based model building, improve data visualization, and more quickly gain insights. Reusable, standards-based ways of doing data collection, ingestion, and contextualization are particularly important for scaling access and use of data. The SMIP uses a standards-based definition and construct for reusable information models called an SM Profile. When an SM Profile is used in conjunction with the SMIP, the SMIP ensures the availability of contextualized, operational data for model building. The present work demonstrates Smart Manufacturing and the application of the SMIP for building several data-centered models for the operation and control of an ex- perimental electrochemical reactor that reduces carbon dioxide (CO 2 ) gas to valuable liquid and gas chemicals, such as alcohols, olefins, and syngas. We describe how the SMIP plays a central role in more effective model building and we demonstrate how the electochemical reactor can be controlled and optimized for the desired products. Use of the SMIP involves the transmission of real-time sensor measurements to a cloud resource so that the operating data are available to all model building experts. The data collection and transmission process is fully automated to greatly reduce the need for manual manipulation of the data. Data-driven machine learning models are used for advanced real-time state estimation, real-time optimization, and model-based feedback control for the reactor. The application models are implemented as a system to monitor the data flow and control the electrochemical reactor with a single visualization interface. SM Profiles are used to demonstrate reusability of the information models for the reactor and the instrumentation. The application packages, algorithms, and user interfaces developed are cast as Docker images in a library to facilitate reusability of the application models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Relating Aerial Infrared Thermography Defects to Photovoltaic Performance: Preprint

In this research, we examine the relationship between aerial IR defect analysis and photovoltaic (PV) performance data for twelve utility- and commercial-scale solar sites in the United States. To do this, we fuse the site diagram geoJSON's, aerial infrared thermography (aIRT) defect analyses, and associated inverter time series, allowing for a direct comparison between site defects and time series data. Defect analyses were provided by Zeitview, under its Solar Insights platform. Following the data fusion process, we look at the relationship between system performance and aIRT defects. We investigate the relationship between degradation and hotspot defects, as well as the relationship between AC power data and offline strings and misaligned modules. In general, system degradation was not affected by long-term or balance-of-system (BoS) defects as they occurred infrequently in the data set. However, for one system, a near statistically significant relationship (p-value=0.057) was found when comparing the degradation of inverter blocks with several multi-hotspot defects to all other inverter blocks without this particular defect. There was strong alignment when comparing short-term recoverable module defects such as stuck trackers and offline strings to time series data. In general, we found that when an inverter block has more than 80% of modules flagged for one of these defects, its AC power time data is flat-lined and the inverter block is not producing.

aerial inspection↗
DOE OSTI2019

Open Humans: A platform for participant-centered research and personal data exploration

Background: Many aspects of our lives are now digitized and connected to the internet. As a result, individuals are now creating and collecting more personal data than ever before. This offers an unprecedented chance for human-participant research ranging from the social sciences to precision medicine. With this potential wealth of data comes practical problems (e.g., how to merge data streams from various sources), as well as ethical problems (e.g., how best to balance risks and benefits when enabling personal data sharing by individuals). Results: To begin to address these problems in real time, we present Open Humans, a community-based platform that enables personal data collections across data streams, giving individuals more personal data access and control of sharing authorizations, and enabling academic research as well as patient-led projects. We showcase data streams that Open Humans combines (e.g., personal genetic data, wearable activity monitors, GPS location records, and continuous glucose monitor data), along with use cases of how the data facilitate various projects. Conclusions: Open Humans highlights how a community-centric ecosystem can be used to aggregate personal data from various sources, as well as how these data can be used by academic and citizen scientists through practical, iterative approaches to sharing that strive to balance considerations with participant autonomy, inclusion, and privacy.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Advanced Health Information Technology Analytic Framework and Application to Hazard Detection

Health Information Technology (HIT) aims to improve healthcare outcomes by organizing and analyzing various health-related data. With data accumulating at a staggering rate, the importance of real-time analytics has been increasing dramatically, shifting the focus of informatics from batch processing to streaming analytics. HIT is also facing unprecedented challenges in adapting to this new requirement and leveraging advanced IT technologies. This paper introduces a HIT data and compute platform that supports multi-granularity real-time analytics from heterogeneous data sources. The paper first identifies functional requirements and proposes a framework that satisfies the requirements using state-of-the-art big data technologies including Apache Kafka, Spark Structured Streaming Engine, and Delta Lake. To demonstrate its capability to support data analytics in multiple time granularities analytics, a statistical process control-based hazard detection algorithm has been implemented on top of the framework to detect unexpected hazards from order cancellation data of the Department of US Veterans Affairs (VA) in near real-time.

Kumar, Mohit↗
DOE OSTI2021

Preparation of Nuclear Data Libraries for Web Release [Slides] and Tutorial for Generating Correlated Random Samples and Propagation of Uncertainty [Slides]

The first presentation discuses moving distribution of nuclear data to an online platform which allows ore frequent nuclear data updates, greater ease of acquiring nuclear data, and user flexibility in what nuclear data to download. The second presentation illustrates uncertainties without correlation, uncertainties without correlation with negative samples, uncertainties with correlation, uncertainties with a χ-like constraint, uncertainties with a Σ tot -like constraint, sampling using different distributions, and dealing with negative eigenvalues in a covariance matrix.

42 ENGINEERING↗
DOE OSTI2025

Livewire: Automatic Annotations

Diogenes processes datasets to provide data quality metrics for the Livewire platform and creates standardized data dictionaries from data annotations. Diogenes needs data annotations that clearly outline thenformat and organization of the data. It also relies on the type, class, and unit of each data piece for comprehensive analysis, which it cannot determine independently. The Annotation Tool significantly reduces the time needed to create annotations for Diogenes by generating data annotations with the correct formatting and content. It also employs machine learning and hard-coded models to automatically annotate data class, quality type, and data units.

33 - ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2026

Toward the Neutrino Discovery Platform: An Auditable, Uncertainty-Bearing Toolchain for MINERvA Open-Data Cross-Section Analysis

The Neutrino Discovery Platform (NDP) aims to accelerate DUNE-era science by making the neutrino program's existing datasets analyzable through fast, reproducible, and auditable workflows. We report a working version of two of its layers, data curation and agentic orchestration, built and tested end to end on MINERvA open data. The guiding lesson throughout is that a cross section is a measurement, and not just a plotted shape, only if it carries a defensible systematic-uncertainty budget, a trustworthy unfolding, and a reproducible record. Using a single medium-energy playlist pair from the MINERvA open-data release (about $2.05\times10^{17}$ protons on target of data), we first reproduced the shapes of two published charged-current inclusive $\nu_\mu$ measurements through a complete extraction ladder: selection, background subtraction, D'Agostini unfolding, efficiency correction, and flux normalization. These shape-level reproductions ran and tracked the published results, but they lacked the systematic-uncertainty machinery that defines a MINERvA cross section. To supply it, we vendored and built the MINERvA Analysis Toolkit and developed a many-universe systematic-uncertainty tool that produces a portable covariance artifact, a parallel event-loop runner, and a per-run auditability harness. Validated against a published covariance release, the toolchain reproduces the released statistical, flux, and muon-energy-scale terms and shows that they account for roughly 63\% of the total variance, with the remainder unreleased. Using this same infrastructure, we then performed a measurement of our own design, the hadronic recoil-energy distribution of low-energy ($E_\nu<2.5$~GeV) charged-current inclusive events, and found data/simulation shape agreement of $\chi^2/\mathrm{ndf}=1.26$. Together these results show that the platform supports original physics and not only reproductions.

Breaux, Auto [Tulane U. (main)]↗
DOE OSTI2020

An Intelligent Adaptable Monitoring Package. Final Report

The “Intelligent Adaptable Monitoring Package” project was a four-year effort that demonstrated the feasibility of integrated sensing packages at tidal and wave energy sites. Such integration is generally required by the breadth of sensors required to understand environmental effects at marine energy sites and the operational difficulty of deploying, maintaining, and recovering such sensors. Over the course of the project, the Adaptable Monitoring Package (AMP) was deployed in multiple settings, each corresponding to a project budget period: - Budget Period 1: Demonstration of cabled deployment at Pacific Northwest National Laboratory’s Marine Science Laboratory. The deployment highlighted AMP hardware endurance over a 4-month deployment in a tidally-dominated environment and laid the groundwork for machine learning algorithms to detect and classify targets present in active sonar data. - Budget Period 2: Demonstration of an autonomous deployment at PacWave South off the coast of Newport, Oregon. The deployment highlighted the stability of AMP hardware and software, with the autonomous package collecting data on a duty cycle over a 1.5-month deployment. - Budget Period 3: Demonstration of an autonomous deployment powered by a wave energy converter at the U.S. Navy’s Wave Energy Test Site. The deployment highlighted the potential of wave energy to power ocean observatories and led to the development of machine learning algorithms to detect and classify targets in optical camera data. In aggregate, this project’s greatest success was demonstrating the AMP’s flexibility in a range of deployment scenarios. Each budget period represented a “first of a kind” demonstration of integrated instrumentation – cabled AMP, autonomous AMP, wave-energy powered AMP – and each deployment helped to identify and set goals for the next. Further, despite the exploratory nature of these deployments, each one achieved high system up-time and proved that flexible integration of multiple sensors in a single package represents a viable strategy for marine energy environmental monitoring. The key lessons learned from the project are: - Without continuous power, either from a shore cable or in situ source, many of the benefits of integration are lost (If continuous power is not available, the ability to detect rare events is lost, as is the ability to minimize the risk of behavioral changes through adaptive sensing. However, even on a duty cycle, there is still value in being able to acquire synchronous data from multiple sensors.); and - Observations from a moving platform present substantially greater data processing challenges than those from stationary platforms. Finally, these deployments also demonstrate an important truth: successful integration alone does not guarantee that relevant data are collected. To grow the knowledge base about environmental interactions with marine energy converters, integrated systems, like the AMP, need to include the right sensor mix and connect the data pipelines to effective processing algorithms. These deployments establish a strong foundation for future collaborations with the environmental research community: not only to understand the environmental effects of marine energy, but also to improve our general ability to study life in the sea.

16 TIDAL AND WAVE POWER↗
DOE OSTI2014

An integrated security framework for GOSS power grid analytics platform

In power grid operations, security is an essential component for any middleware platform. Security protects data against unwanted access as well as cyber attacks. GridOpticsTM Software System (GOSS) is an open source power grid analytics platform that facilitates ease of access between applications and data sources and promotes development of advanced analytical applications. GOSS contains an API that abstracts many of the difficulties in connecting to various heterogeneous data sources. A number of applications and data sources have already been implemented to demonstrate functionality and ease of use. A security framework has been implemented which leverages widely accepted, robust JavaTM security tools in a way such that they can be interchanged as needed. This framework supports the complex fine-grained, access control rules identified for the diverse data sources already in GOSS. Performance and reliability are also important considerations in any power grid architecture. An evaluation is done to determine the overhead cost caused by security within GOSS and ensure minimal impact to performance.

Gibson, Tara D.↗
DOE OSTI2020

Artificial Intelligence and Computer Vision for Cost-Effective Benthic Habitat Characterizations

Machine learning applied to computer vision and pattern recognition is a type of artificial intelligence that has advanced rapidly in the last 10 to 15 years, spurred forward by breakthroughs in deep convolutional neural networks. These state-of-the-art methods are poised to become widely used in environmental monitoring applications as a result of the increasing abundance of data available from different imaging platforms (e.g., fixed-point cameras, drone surveys, high-resolution satellite data) that can be analyzed to observe, model, and understand environmental conditions. Computer vision and pattern recognition (CVPR) tools advance our ability to use imagery and camera-based tools in cost-effective ways for environmental monitoring. Although these techniques offer great potential, some challenges remain, such as the need for large sets of labeled images for model training and validation and optimized hardware and software to ensure that the models can be trained effectively and in a reasonable amount of time. Here, we have overcome many of these rate-limiting challenges by using a diverse image library built across multiple projects coupled with staff expertise and onsite computing resources. We are working toward a fully automated SPI processing system and also are moving to develop CVPR analytical tools for other imaging platforms and data sets.

02 PETROLEUM↗
DOE OSTI2022

GROWdb US River Systems - Samples

GROW Overview We developed the Genome Resolved Open Watersheds database (GROWdb), which aims to increase genomic sampling and understanding of global river microbiomes. An emphasis of GROWdb is to create a publicly available and ever-expanding microbial genome database that is focused on rivers while being interoperable with databases from other ecosystems. GROWdb is based on a network-of-networks approach to move beyond a small collection of well-studied rivers, towards a spatially distributed, global network of systematic observations. GROWdb represents the first microbial, river-focused resource parsed at various scales from genes to MAGs to community level including expression and potential based measurements that will be of interest to microbiologists, ecologists, geochemists, hydrologists, and modelers. Dataset Acknowledgement GROWdb contains data from various research campaigns, please acknowledge the following data generators, as appropriate: WHONDRS derived genomes or samples - include this statement in your acknowledgements: “This study used data from the Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) under the River Corridor Science Focus Area (SFA) at the Pacific Northwest National Laboratory (PNNL) that was generated at the U.S. Department of Energy (DOE) Joint Genome Institute User Facility. PNNL is operated by Battelle Memorial Institute for the U.S. DOE under Contract No. DE-AC05-76RL01830. The SFA is supported by the U.S. DOE, Office of Biological and Environmental Research (BER), Environmental System Science (ESS) Program.” Total Samples loaded onto this Narrative: 178 Note: Not all GROW samples may be loaded into KBase Data Availability The data underlying GROWdb are accessible across various platforms to ensure all levels of data structure are widely available. First, all reads and MAGs are publicly hosted on National Center for Biotechnology (NCBI) under Bioproject PRJNA946291. Second, all data related data presented here including MAG annotations, extended data tables, phylogenetic tree files, antibiotic resistance gene database files, and MAG abundance tables are available in Zenodo (link). Beyond the flat database files listed above, our aim for GROWdb was to maximize data use by making the data available in searchable and interactive platforms including the National Microbiome Data Collaborative (NMDC) data portal, the Department of Energy’s Systems Biology Knowledgebase (KBase), and a GROW specific user interface released here, GROWdb Explorer. Each platform provides different ways to interact with GROWdb: NMDC GROWdb formed a pilot project for the NMDC. Specifically, individual GROWdb datasets (metagenomes, metatranscriptomes, etc) are easily accessible and searchable through the NMDC data portal, where they are systematically connected to each other and to a rich suite of sample information and standard analysis results, following Findable, Accessible, Interoperable, and Reusable (FAIR) data practices. KBase GROWdb is publicly available within KBase, including samples (this Narrative), MAGs, and corresponding genome scale metabolic models. Access within KBase allows for immediate access and reuse of data, including comparison to private data using KBase’s 500+ analysis tools. Other linked narratives in KBase: GROW Metagenome Assembled Genomes (MAGs) GROW Metabolic Models GROWdb Explorer GROWdb data is also explorable through a graphical user interface built through the Colorado State University Geospatial Centroid (https://geocentroid.shinyapps.io/GROWdatabase/), allowing users to search and graph microbial and spatial data simultaneously. In summary, this microbial genome resource represents the first publicly available genome collection from rivers and offers data that can be leveraged across microbiome studies. GROWdb is an expanding repository to incorporate and unify global river multi-omic data for the future.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

A hybrid computational strategy to address WGS variant analysis in >5000 samples

Abstract Background The decreasing costs of sequencing are driving the need for cost effective and real time variant calling of whole genome sequencing data. The scale of these projects are far beyond the capacity of typical computing resources available with most research labs. Other infrastructures like the cloud AWS environment and supercomputers also have limitations due to which large scale joint variant calling becomes infeasible, and infrastructure specific variant calling strategies either fail to scale up to large datasets or abandon joint calling strategies. Results We present a high throughput framework including multiple variant callers for single nucleotide variant (SNV) calling, which leverages hybrid computing infrastructure consisting of cloud AWS, supercomputers and local high performance computing infrastructures. We present a novel binning approach for large scale joint variant calling and imputation which can scale up to over 10,000 samples while producing SNV callsets with high sensitivity and specificity. As a proof of principle, we present results of analysis on Cohorts for Heart And Aging Research in Genomic Epidemiology (CHARGE) WGS freeze 3 dataset in which joint calling, imputation and phasing of over 5300 whole genome samples was produced in under 6 weeks using four state-of-the-art callers. The callers used were SNPTools, GATK-HaplotypeCaller, GATK-UnifiedGenotyper and GotCloud. We used Amazon AWS, a 4000-core in-house cluster at Baylor College of Medicine, IBM power PC Blue BioU at Rice and Rhea at Oak Ridge National Laboratory (ORNL) for the computation. AWS was used for joint calling of 180 TB of BAM files, and ORNL and Rice supercomputers were used for the imputation and phasing step. All other steps were carried out on the local compute cluster. The entire operation used 5.2 million core hours and only transferred a total of 6 TB of data across the platforms. Conclusions Even with increasing sizes of whole genome datasets, ensemble joint calling of SNVs for low coverage data can be accomplished in a scalable, cost effective and fast manner by using heterogeneous computing platforms without compromising on the quality of variants.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

House Advantage or House of Cards? Stacking the Deck for Data Videos Leads to Null Results: Preprint

Videos are becoming a ubiquitous means of sharing information on social media platforms. In response, data videos - short clips combining visualization with dynamic storytelling, audio descriptions, and spatial referencing - have gained popularity for communicating data. These affordances suggest that data videos might communicate data patterns, trends, and concepts more effectively than static visualizations, enhancing comprehension. However, existing research has not systematically tested this claim. To address this gap, we conducted three controlled studies to measure comprehension differences between data videos and static visualizations. Despite leveraging visual cues and audio explanations, no data video led to significantly better comprehension than an analogous static visualization. Our results suggest data videos are not categorically better and that future research should examine the tradeoffs between their engagement benefits and costs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Instantiation of the Damara Tern Platform for Advanced Materials and Manufacturing Technologies (AMMT) Program Collaborative Data Management

This work package focused on deploying an instance of the Damara Tern platform to support AMMT collaborative research activities. The objectives were to provide selected AMMT collaborators with access to a shared environment for capturing operations, trackables, and associated metadata, and to implement data entry functionalities that reflect site-specific procedures. Key activities included creating configurable, schema-driven entry forms and validating the data collection process. The report details the deployment process, the platform infrastructure, and the implemented data entry workflows, providing a reference for end users and establishing a foundation for future production-scale deployments.

36 MATERIALS SCIENCE↗
DOE OSTI2020

The Ontologies Community of Practice: A CGIAR Initiative for Big Data in Agrifood Systems

Heterogeneous and multidisciplinary data generated by research on sustainable global agriculture and agrifood systems requires quality data labeling or annotation in order to be interoperable. As recommended by the FAIR principles, data, labels, and metadata must use controlled vocabularies and ontologies that are popular in the knowledge domain and commonly used by the community. Despite the existence of robust ontologies in the Life Sciences, there is currently no comprehensive full set of ontologies recommended for data annotation across agricultural research disciplines. In this paper, we discuss the added value of the Ontologies Community of Practice (CoP) of the CGIAR Platform for Big Data in Agriculture for harnessing relevant expertise in ontology development and identifying innovative solutions that support quality data annotation. The Ontologies CoP stimulates knowledge sharing among stakeholders, such as researchers, data managers, domain experts, experts in ontology design, and platform development teams.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗