Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,531 records · Page 85
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Combinatorial Synthesis and High-Throughput Characterization of Microstructure and Phase Transformation in Ni–Ti–Cu–V Quaternary Thin-Film Library

Ni–Ti–based shape memory alloys (SMAs) have found widespread use in the last 70 years, but improving their functional stability remains a key quest for more robust and advanced applications. Named for their ability to retain their processed shape as a result of a reversible martensitic transformation, SMAs are highly sensitive to compositional variations. Alloying with ternary and quaternary elements to fine-tune the lattice parameters and the thermal hysteresis of an SMA, therefore, becomes a challenge in materials exploration. Combinatorial materials science allows streamlining of the synthesis process and data management from multiple characterization techniques. In this study, a composition spread of Ni–Ti–Cu–V thin-film library was synthesized by magnetron co-sputtering on a thermally oxidized Si wafer. Composition-dependent phase transformation temperature and microstructure were investigated and determined using high-throughput wavelength dispersive spectroscopy, synchrotron X-ray diffraction, and temperature-dependent resistance measurements. Of the 177 compositions in the materials library, 32 were observed to have shape memory effect, of which five had zero or near-zero thermal hysteresis. These compositions provide flexibility in the operating temperature regimes that they can be used in. A phase map for the quaternary system and correlations of functional properties are discussed with respect to the local microstructure and composition of the thin-film library.

42 ENGINEERING↗
DOE OSTI2022

Moving small files in a networked environment

Globally distributed computing infrastructures, such as clouds and supercomputers, are currently used to manage data that is generated with an unprecedented speed from a variety of resources. Coping with this trend, the volume of data exchanged across distant sites increases substantially. To accelerate data transfer, high-speed networks are provided to connect remote sites. Most existing data movement solutions are optimized for moving large files. However, it is still challenging to transfer a large number of small files across networks. This disadvantage not only lowers data transfer performance, but also decreases overall system utilization. Here, we identify that moving small files is mainly constrained by degraded file system throughput, not just network performance as might be suspected. We have built a data transfer pipeline model to analyze the impact of small network I/O and storage I/O on data movement. Extending one of the widely used open source data movement solutions, GridFTP, we demonstrate several appropriate engineering approaches that mitigate the bottleneck and increase data transfer efficiency. We show optimizations that improve data transfer performance more than 5 times. In comparison to existing solutions, our approaches can save a significant amount of system resources for moving lots of small files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

The Importance of Imaging in Radiation Oncology for National Clinical Trials Network Protocols

Imaging is essential in successfully executing radiation therapy (RT) in oncology clinical trials. As technically sophisticated diagnostic imaging and RT were incorporated into trials, quality assurance in the National Clinical Trials Network groups entered a new era promoting image acquisition and review. Most trials involving RT require pre- and post-therapy imaging for target validation and outcome assessment. The increasing real-time (before and during therapy) imaging and RT object reviews are to ensure compliance with trial objectives. Objects easily transmit digitally for review from anywhere in the world. Physician interpretation of imaging and image application to RT treatment plans is essential for optimal trial execution. Imaging and RT data sets are used to credential RT sites to confirm investigator and institutional ability to meet trial target volume delineation and delivery requirements. Real-time imaging and RT object reviews can be performed multiple times during a trial to assess response to therapy and application of RT objects. This process has matured into an effective data management mechanism. When necessary, site and study investigators review objects together through web media technologies to ensure the patient is enrolled on the appropriate trial and the intended RT is planned and executed in a trial-compliant manner. Real-time imaging review makes sure: (1) the patient is entered and eligible for the trial, (2) the patient meets trial-specific adaptive therapy requirements, if applicable, and (3) the intended RT is according to trial guidelines. This review ensures the study population is uniform and the results are believable and can be applied to clinical practice.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2024

Scalable Implicit Solvers with Dynamic Mesh Adaptation for a Relativistic Drift-Kinetic Fokker–Planck–Boltzmann Model

In this work we consider a relativistic drift-kinetic model for runaway electrons along with a Fokker–Planck operator for small-angle Coulomb collisions, a radiation damping operator, and a secondary knock-on (Boltzmann) collision source. Here, we develop a new scalable fully implicit solver utilizing finite volume and conservative finite difference schemes and dynamic mesh adaptivity. A new data management framework in the PETSc library based on the p4est library is developed to enable simulations with dynamic adaptive mesh refinement (AMR), distributed memory parallelization, and dynamic load balancing of computational work. This framework and the runaway electron solver building on the framework are able to dynamically capture both bulk Maxwellian at the low-energy region and a runaway tail at the high-energy region. To effectively capture features via the AMR algorithm, a new AMR indicator prediction strategy is proposed that is performed alongside the implicit time evolution of the solution. This strategy is complemented by the introduction of computationally cheap feature-based AMR indicators that are analyzed theoretically. Numerical results quantify the advantages of the prediction strategy in better capturing features compared with nonpredictive strategies; and we demonstrate trade-offs regarding computational costs. The robustness with respect to model parameters, algorithmic scalability, and parallel scalability are demonstrated through several benchmark problems including manufactured solutions and solutions of different physics models. We focus on demonstrating the advantages of using implicit time stepping and AMR for runaway electron simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

PDBx/mmCIF Ecosystem: Foundational Semantic Tools for Structural Biology

PDBx/mmCIF, Protein Data Bank Exchange (PDBx) macromolecular Crystallographic Information Framework (mmCIF), has become the data standard for structural biology. With its early roots in the domain of small-molecule crystallography, PDBx/mmCIF provides an extensible data representation that is used for deposition, archiving, remediation, and public dissemination of experimentally determined three-dimensional (3D) structures of biological macromolecules by the Worldwide Protein Data Bank (wwPDB, wwpdb.org). Extensions of PDBx/mmCIF are similarly used for computed structure models by ModelArchive (modelarchive.org), integrative/hybrid structures by PDB-Dev (pdb-dev.wwpdb.org), small angle scattering data by Small Angle Scattering Biological Data Bank SASBDB (sasbdb.org), and for models computed generated with the AlphaFold 2.0 deep learning software suite (alphafold.ebi.ac.uk). Community-driven development of PDBx/mmCIF spans three decades, involving contributions from researchers, software and methods developers in structural sciences, data repository providers, scientific publishers, and professional societies. Having a semantically rich and extensible data framework for representing a wide range of structural biology experimental and computational results, combined with expertly curated 3D biostructure data sets in public repositories, accelerates the pace of scientific discovery. Herein, we describe the architecture of the PDBx/mmCIF data standard, tools used to maintain representations of the data standard, governance, and processes by which data content standards are extended, plus community tools/software libraries available for processing and checking the integrity of PDBx/mmCIF data. Use cases exemplify how the members of the Worldwide Protein Data Bank have used PDBx/mmCIF as the foundation for its pipeline for delivering Findable, Accessible, Interoperable, and Reusable (FAIR) data to many millions of users worldwide.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

An overview of data tools for representing and managing building information and performance data

Building information modeling (BIM) has been widely adopted for representing and exchanging building data across disciplines during building design and construction. However, BIM's use in the building operation phase is limited. With the increasing deployment of low-cost sensors and meters, as well as affordable digital storage and computing technologies, growing volumes of data have been collected from buildings, their energy services systems, and occupants. Such data are crucial to help decision makers understand what, how, and when energy is consumed in buildings—a critical step to improving building performance for energy efficiency, demand flexibility, and resilience. However, practical analyses and use of the collected data are very limited due to various reasons, including poor data quality, ad-hoc representation of data, and lack of data science skills. To unlock value from building data, there is a strong need for a toolchain to curate and represent building information and performance data in common standardized terminologies and schemas, to enable interoperability between tools and applications. This study selected and reviewed 24 data tools based on common use cases of data across the building life cycle, from design to construction, commissioning, operation, and retrofits. The selected data tools are grouped into three categories: (1) data dictionary or terminology, (2) data ontology and schemas, and (3) data platforms. The data are grouped into ten typologies covering most types of data collected in buildings. This study resulted in five main findings: (1) most data representation tools can represent their intended data typologies well, such as Green Button for smart meter data and Brick schema for metadata of sensors in buildings and HVAC systems, but none of the tools cover all ten types of data; (2) there is a need for data schemas to represent the basis of design data and metadata of occupant data; (3) standard terminologies such as those defined in BEDES are only adopted in a few data tools; (4) integrating data across various stages in the building life cycle remains a challenge; and (5) most data tools were developed and maintained by different parties for different purposes, their flexibility and interoperability can be improved to support broader use cases. Finally, recommendations for future research on building data tools are provided for the data and buildings community based on the FAIR principles to make data Findable, Accessible, Interoperable, and Reusable.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2019

Balancing Open Science and Data Privacy in the Water Sciences

Open science practices such as publishing data and code are transforming water science by enabling synthesis and enhancing reproducibility. However, as research increasingly bridges the physical and social science domains (e.g., socio-hydrology), there is the potential for well-meaning researchers to unintentionally violate the privacy and security of individuals or communities by sharing sensitive information. Here we identify the contexts in which privacy violations are most likely to occur, such as working with high-resolution spatial data (e.g., from remote sensing), consumer data (e.g., from smart meters), and/or digital trace data (e.g., from social media). We also suggest practices for identifying and addressing privacy concerns at the individual, institutional, and disciplinary levels. We strongly advocate that the water science community continue moving toward open science and socio-environmental research and that progress toward these goals be rooted in open and ethical data management.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Machine learning enabling high-throughput and remote operations at large-scale user facilities

Imaging, scattering, and spectroscopy are fundamental in understanding and discovering new functional materials. Contemporary innovations in automation and experimental techniques have led to these measurements being performed much faster and with higher resolution, thus producing vast amounts of data for analysis. These innovations are particularly pronounced at user facilities and synchrotron light sources. Machine learning (ML) methods are regularly developed to process and interpret large datasets in real-time with measurements. However, there remain conceptual barriers to entry for the facility general user community, whom often lack expertise in ML, and technical barriers for deploying ML models. Herein, we demonstrate a variety of archetypal ML models for on-the-fly analysis at multiple beamlines at the National Synchrotron Light Source II (NSLS-II). We describe these examples instructively, with a focus on integrating the models into existing experimental workflows, such that the reader can easily include their own ML techniques into experiments at NSLS-II or facilities with a common infrastructure. The framework presented here shows how with little effort, diverse ML models operate in conjunction with feedback loops via integration into the existing Bluesky Suite for experimental orchestration and data management.

36 MATERIALS SCIENCE↗
DOE OSTI2023

High throughput discovery of ternary Cu-Fe-Ru alloy catalysts for photo-driven hydrogen production

Light driven hydrogen production from the water splitting reaction has the ability to reduce dependence on fossil fuels in a green energy future. Here, we highlight the discovery of Cu x Ru y Fe 1-x-y nanoparticle catalysts for photo-driven hydrogen production. Through a high throughput experimental setup, robust data management pipelines and intentional experimental design, this study uncovered three highly active bimetallic systems for photo-driven hydrogen and identified a new trimetallic catalyst for this system. In most cases, the multimetallic catalysts outperformed the monometallics. Furthermore, this study highlights the expansive catalytic screening capabilities of this system in contrast to traditional catalytic selection processes through the discovery of distributions of particle compositions in binary and ternary mixtures of metals with high activity for hydrogen evolution.

08 HYDROGEN↗
DOE OSTI2024

Adoption of a token-based authentication model for the CMS Submission Infrastructure

The CMS Submission Infrastructure (SI) is the main computing resource provisioning system for CMS workloads. A number of HTCondor pools are employed to manage this infrastructure, which aggregates geographically distributed resources from the WLCG and other providers. Historically, the model of authentication among the diverse components of this infrastructure has relied on the Grid Security Infrastructure (GSI), based on identities and X509 certificates. In contrast, commonly used modern authentication standards are based on capabilities and tokens. The WLCG has identified this trend and aims at a transparent replacement of GSI for all its workload management, data transfer and storage access operations, to be completed during the current LHC Run 3. As part of this effort, and within the context of CMS computing, the Submission Infrastructure group is in the process of phasing out the GSI part of its authentication layers, in favor of IDTokens and Scitokens. The use of tokens is already well integrated into the HTCondor Software Suite, which has allowed us to fully migrate the authentication between internal components of SI. Additionally, recent versions of the HTCondor-CE support tokens as well, enabling CMS resource requests to Grid sites employing this CE technology to be granted by means of token exchange. After a rollout campaign to sites, successfully completed by the third quarter of 2022, the totality of HTCondor CEs in use by CMS are already receiving Scitoken-based pilot jobs. On the ARC CE side, a parallel campaign was launched to foster the adoption of the REST interface at CMS sites (required to enable token-based job submission via HTCondor-G), which is nearing completion as well. In this contribution, the newly adopted authentication model will be described. We will then report on the migration status and final steps towards complete GSI phase out in the CMS SI.

Pérez-Calero Yzquierdo, Antonio↗
DOE OSTI2024

cclib 2.0: An updated architecture for interoperable computational chemistry

Interoperability in computational chemistry is elusive, impeded by the independent development of software packages and idiosyncratic nature of their output files. The cclib library was introduced in 2006 as an attempt to improve this situation by providing a consistent interface to the results of various quantum chemistry programs. The shared API across programs enabled by cclib has allowed users to focus on results as opposed to output and to combine data from multiple programs or develop generic downstream tools. Initial development, however, did not anticipate the rapid progress of computational capabilities, novel methods, and new programs; nor did it foresee the growing need for customizability. Here, we recount this history and present cclib 2, focused on extensibility and modularity. We also introduce recent design pivots—the formalization of cclib’s intermediate data representation as a tree-based structure, a new combinator-based parser organization, and parsed chemical properties as extensible objects.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Emergency Radiation Dose Rate Monitoring During Prolonged Armed Conflict

The 2022 Russian full-scale invasion of Ukraine has introduced unprecedented challenges for the nuclear power generation and radiological safety communities, including occupation and disturbance of highly-contaminated areas, occupation of a nuclear power plant, and strikes near and within boundaries of nuclear sites. The war has necessitated the implementation of a supplementary dose rate sensor network to provide resilient measurement data for public protection and leadership awareness. This paper discusses the implementation of such a system, the factors determining what equipment is best suited for the purpose, and practical factors regarding deployment of the system and data management. The crucial factors for operating a supplementary dose rate sensing network are backup power and communications options for dose rate sensors to make the network resilient to the effects of military operations. The most important implementation factor is to plan for extended operations beyond those typically considered for emergency response given the unpredictable nature of warfare.

resilience↗
DOE OSTI2023

Foundations of machine learning for low-temperature plasmas: methods and case studies

Abstract Machine learning (ML) and artificial intelligence have proven to be an invaluable tool in tackling a vast array of scientific, engineering, and societal problems. The main drivers behind the recent proliferation of ML in practically all aspects of science and technology can be attributed to: (a) improved data acquisition and inexpensive data storage; (b) exponential growth in computing power; and (c) availability of open-source software and resources that have made the use of state-of-the-art ML algorithms widely accessible. The impact of ML on the field of low-temperature plasmas (LTPs) could be particularly significant in the emerging applications that involve plasma treatment of complex interfaces in areas ranging from the manufacture of microelectronics and processing of quantum materials, to the LTP-driven electrification of the chemical industry, and to medicine and biotechnology. This is primarily due to the complex and poorly-understood nature of the plasma-surface interactions in these applications that pose unique challenges to the modeling, diagnostics, and predictive control of LTPs. As the use of ML is becoming more prevalent, it is increasingly paramount for the LTP community to be able to critically analyze and assess the concepts and techniques behind data-driven approaches. To this end, the goal of this paper is to provide a tutorial overview of some of the widely-used ML methods that can be useful, amongst others, for discovering and correlating patterns in the data that may be otherwise impractical to decipher by human intuition alone, for learning multivariable nonlinear data-driven prediction models that are capable of describing the complex behavior of plasma interacting with interfaces, and for guiding the design of experiments to explore the parameter space of plasma-assisted processes in a systematic and resource-efficient manner. We illustrate the utility of various supervised, unsupervised and active learning methods using LTP datasets consisting of commonly-available, information-rich measurements (e.g. optical emission spectra, current–voltage characteristics, scanning electron microscope images, infrared surface temperature measurements, Fourier transform infrared spectra). All the ML demonstrations presented in this paper are carried out using open-source software; the datasets and codes are made publicly available. The FAIR guiding principles for scientific data management and stewardship can accelerate the adoption and development of ML in the LTP community.

Physics↗
DOE OSTI2017

Experience in using commercial clouds in CMS

Historically high energy physics computing has been performed on large purpose-built computing systems. In the beginning there were single site computing facilities, which evolved into the Worldwide LHC Computing Grid (WLCG) used today. The vast majority of the WLCG resources are used for LHC computing and the resources are scheduled to be continuously used throughout the year. In the last several years there has been an explosion in capacity and capability of commercial and academic computing clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest amongst the cloud providers to demonstrate the capability to perform large scale scientific computing. In this presentation we will discuss results from the CMS experiment using the Fermilab HEPCloud Facility, which utilized both local Fermilab resources and Amazon Web Services (AWS). The goal was to work with AWS through a matching grant to demonstrate a sustained scale approximately equal to half of the worldwide processing resources available to CMS. We will discuss the planning and technical challenges involved in organizing the most IO intensive CMS workflows on a large-scale set of virtualized resource provisioned by the Fermilab HEPCloud. We will describe the data handling and data management challenges. Also, we will discuss the economic issues and cost and operational efficiency comparison to our dedicated resources. At the end we will consider the changes in the working model of HEP computing in a domain with the availability of large scale resources scheduled at peak times.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Publishing unbinned differential cross section results

Machine learning tools have empowered a qualitatively new way to perform differential cross section measurements whereby the data are unbinned, possibly in many dimensions. Unbinned measurements can enable, improve, or at least simplify comparisons between experiments and with theoretical predictions. Furthermore, many-dimensional measurements can be used to define observables after the measurement instead of before. There is currently no community standard for publishing unbinned data. While there are also essentially no measurements of this type public, unbinned measurements are expected in the near future given recent methodological advances. The purpose of this paper is to propose a scheme for presenting and using unbinned results, which can hopefully form the basis for a community standard to allow for integration into analysis workflows. This is foreseen to be the start of an evolving community dialogue, in order to accommodate future developments in this field that is rapidly evolving.

47 OTHER INSTRUMENTATION↗
DOE OSTI2016

Physical properties of star clusters in the outer LMC as observed by the DES

The Large Magellanic Cloud (LMC) harbors a rich and diverse system of star clusters, whose ages, chemical abundances, and positions provide information about the LMC history of star formation. We use Science Verification imaging data from the Dark Energy Survey to increase the census of known star clusters in the outer LMC and to derive physical parameters for a large sample of such objects using a spatially and photometrically homogeneous data set. Our sample contains 255 visually identified cluster candidates, of which 109 were not listed in any previous catalog. We quantify the crowding effect for the stellar sample produced by the DES Data Management pipeline and conclude that the stellar completeness is < 10% inside typical LMC cluster cores. We therefore develop a pipeline to sample and measure stellar magnitudes and positions around the cluster candidates using DAOPHOT. We also implement a maximum-likelihood method to fit individual density profiles and colour-magnitude diagrams. For 117 (from a total of 255) of the cluster candidates (28 uncatalogued clusters), we obtain reliable ages, metallicities, distance moduli and structural parameters, confirming their nature as physical systems. The distribution of cluster metallicities shows a radial dependence, with no clusters more metal-rich than [Fe/H] ~ -0.7 beyond 8 kpc from the LMC center. Furthermore, the age distribution has two peaks at ≃ 1.2 Gyr and ≃ 2.7 Gyr.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Mol* Viewer: modern web app for 3D visualization and analysis of large biomolecular structures

Abstract Large biomolecular structures are being determined experimentally on a daily basis using established techniques such as crystallography and electron microscopy. In addition, emerging integrative or hybrid methods (I/HM) are producing structural models of huge macromolecular machines and assemblies, sometimes containing 100s of millions of non-hydrogen atoms. The performance requirements for visualization and analysis tools delivering these data are increasing rapidly. Significant progress in developing online, web-native three-dimensional (3D) visualization tools was previously accomplished with the introduction of the LiteMol suite and NGL Viewers. Thereafter, Mol* development was jointly initiated by PDBe and RCSB PDB to combine and build on the strengths of LiteMol (developed by PDBe) and NGL (developed by RCSB PDB). The web-native Mol* Viewer enables 3D visualization and streaming of macromolecular coordinate and experimental data, together with capabilities for displaying structure quality, functional, or biological context annotations. High-performance graphics and data management allows users to simultaneously visualise up to hundreds of (superimposed) protein structures, stream molecular dynamics simulation trajectories, render cell-level models, or display huge I/HM structures. It is the primary 3D structure viewer used by PDBe and RCSB PDB. It can be easily integrated into third-party services. Mol* Viewer is open source and freely available at https://molstar.org/.

Sehnal, David↗
DOE OSTI2017

Extending Skel to Support the Development and Optimization of Next Generation I/O Systems

As the memory and storage hierarchy get deeper and more complex, it is important to have new benchmarks and evaluation tools that allow us to explore the emerging middleware solutions to use this hierarchy. Skel is a tool aimed at automating and refining this process of studying HPC I/O performance. It works by generating application I/O kernel/benchmarks as determined by a domain-specific model. This paper provides some techniques for extending Skel to address new situations and to answer new research questions. For example, we document use cases as diverse as using Skel to troubleshoot I/O performance issues for remote users, refining an I/O system model, and facilitating the development and testing of a mechanism for runtime monitoring and performance analytics. We also discuss data oriented extensions to Skel to support the study of compression techniques for Exascale scientific data management.

97 MATHEMATICS AND COMPUTING↗