Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Open data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,567 records · Page 87

End-to-end Analytics for Grid Arch Design & All-hazard Assessment

Resiliency, reliability, and security of the next-generation smart grid depend upon leveraging advanced communication and computing technologies, integrating them with physical power systems, and developing real-time, fast, data-based applications to help in wide-area monitoring and control of the grid. Using a high sampling data rate from phasor measurement units (PMUs) to develop applications has opened the door to achieving the next-generation grid requirements. The North American Synchrophasor Initiative Network (NASPlnet) was developed in 2007-09 to create a standard and guide for PMU data exchanges. With the advancement in both networking and grid requirements, it is necessary to evaluate the performance of different NASPInet versions and their impact on applications. Therefore, we need a cyber-power cosimulation framework that supports very large-scale co-simulation capable of running in parallel, high-performance computing platforms and capturing real-life network behavior. This work presents a cyber-physical co-simulation testbed using NS3 to model the communication network, GridPACK to model the power grid, and HELICS as a co-simulation engine. Comparative analysis of latency in synchrophasor networks and a performance evaluation of a power system stabilizer application based on PMU data in an Institute of Electrical and Electronics Engineers 39-bus test system is presented using this co-simulation testbed.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A Review of the Resuspension of Radioactively Contaminated Particles by Vehicle and Pedestrian Traffic—Current Theory, Practice, Gaps, and Needs

Here, the resuspension of radioactively contaminated particles in a built environment, such as from urban surfaces like foliage, building exteriors, and roadways, is described empirically by current plume and dosimetry models used for hazard assessment and long-term risk purposes. When applying these models to radiological contamination emergencies affecting urban areas, the accuracy of the results for recent contamination deposition is impacted in two main ways. First, the data supporting the underlying resuspension equations was acquired for open, quiescent conditions with no vehicle traffic or human activities, so it is not necessarily representative of the urban environment. Second, mechanical disturbance by winds in urban canyons and during emergency operations caused by vehicle traffic and human activities are not directly considered by the equations. Accordingly, plume and dosimetry models allow the user to input certain compensating values, but the models do not necessarily supply users instructions on what values to use. This manuscript reviews the available literature to comprehensively and consistently pool data for resuspension due to mechanically induced resuspension applicable to urban contamination. Because there are few studies that directly measured radioactive resuspension due to vehicles and pedestrians, this review novelly draws on a range of other studies involving non-radioactive particles, ranging from outdoor air pollution emissions to indoor allergen transport. The results lead to tabulated, recommended values for specific conditions in the emergency phase to help users of plume and dosimetry models maintain the conservativeness needed to properly capture the potential radiation dose posed by mechanically induced resuspension. These values are of benefit to model users until better data are available. The results also suggest the types of data that may result in improved plume and dose modeling.

61 RADIATION PROTECTION AND DOSIMETRY↗

WBS 1.2.3.405 - Life Cycle Assessment of Storage Technologies

Recent commitments by the Biden administration have established targets to achieve a net-zero energy system by 2050. Meeting these targets will spur a rapid transition to clean energy technologies and a commensurate need to develop and deploy energy storage technologies at scale. Pumped Storage Hydro (PSH) is expected to be part of this solution because its ability to provide grid flexibility and stability and enable the dispatching of disparate variable renewable energy technologies. Despite PSH being a mature technology with a history of deployment dating back several decades, there is very little information on the greenhouse gas (GHG) implications of PSH as compared to other storage technologies. The objective of this project is to perform a full lifecycle assessment (LCA) of new PSH projects in the U.S. This LCA includes all project phases (resource extraction, construction, operation, maintenance, end-of-life). The functional unit for this study is 1 kWh electricity delivered by system to grid substation connection point and the estimated lifetime for our base case is 80 years. Data used in this study are based on over 30 potential PSH projects that are in preliminary planning phases and are represent a wide range of potential closed-loop PSH systems in terms of location, technology, and capacity. The project approach, data sources, and modeling assumptions have been informed by a technical review committee of stakeholders that include experts from academia, national and international government, industry, and utilities. The GHGs and energy return on investment (EROI) from PSH will be compared to other storage technologies (e.g., stationary battery storage). Results from this project will improve the PSH community's understanding of the environmental impacts and sustainability of new PSH projects and how PSH compares to other storage technologies. The approach used in this project relies on open-source programming. The analysis framework (source code and data) and will be made publicly available at the end of the project. In addition to reporting results for the base case, we will perform rigorous sensitivity analysis to identify the major drivers, understand impacts of different configurations, and future energy markets. Results from this project will be published in a suitable journal.

ENERGY PLANNING, POLICY, AND ECONOMY,HYDRO ENERGY↗

openPMD-viewer

Many simulation software produce data in the form of a set of field values or of a set of particle positions. (one such example is that of particle-in-cell codes, which produce data on the electromagnetic fields that they simulate.) However, each particular software uses its own particular format and layout, for the output data. This makes it difficult to compare the results of different simulation software, or to have a common visualization tool for these results. However, a standardized layout for fields and particles has recently been developed: the openPMD format ( HYPERLINK "http://www.openpmd.org/"www.openpmd.org) This format is open- source, and specifies a standard way in which field data and particle data should be written. The openPMD format is already implemented in the particle-in-cell code Warp (developed at LBL) and in PIConGPU (developed at HZDR, Germany). In this context, the proposed software (openPMD-viewer) is a Python package, which allows to access and visualize any data which has been formatted according to the openPMD standard. This package contains two main components: - a Python API, which allows to read and extract the data from a openPMD file, so as to be able to work with it within the Python environment. (e.g. plot the data and reprocess it with particular Python functions) - a graphical interface, which works with the ipython notebook, and allows to quickly visualize the data and browse through a set of openPMD files. The proposed software will be typically used when analyzing the results of numerical simulations. It will be useful to quickly extract scientific meaning from a set of numerical data.

Lehe, Remi↗

openPMD-viewer

Many simulation software produce data in the form of a set of field values or of a set of particle positions. (one such example is that of particle-in-cell codes, which produce data on the electromagnetic fields that they simulate.) However, each particular software uses its own particular format and layout, for the output data. This makes it difficult to compare the results of different simulation software, or to have a common visualization tool for these results. However, a standardized layout for fields and particles has recently been developed: the openPMD format ( HYPERLINK "http://www.openpmd.org/"www.openpmd.org) This format is open- source, and specifies a standard way in which field data and particle data should be written. The openPMD format is already implemented in the particle-in-cell code Warp (developed at LBL) and in PIConGPU (developed at HZDR, Germany). In this context, the proposed software (openPMD-viewer) is a Python package, which allows to access and visualize any data which has been formatted according to the openPMD standard. This package contains two main components: - a Python API, which allows to read and extract the data from a openPMD file, so as to be able to work with it within the Python environment. (e.g. plot the data and reprocess it with particular Python functions) - a graphical interface, which works with the ipython notebook, and allows to quickly visualize the data and browse through a set of openPMD files. The proposed software will be typically used when analyzing the results of numerical simulations. It will be useful to quickly extract scientific meaning from a set of numerical data.

Lehe, Remi↗

How Many Kilonovae Can Be Found in Past, Present, and Future Survey Data Sets?

The discovery of a kilonova (KN) associated with the Advanced LIGO (aLIGO)/Virgo event GW170817 opens up new avenues of multi-messenger astrophysics. Here, using realistic simulations, we provide estimates of the number of KNe that could be found in data from past, present, and future surveys without a gravitational-wave trigger. For the simulation, we construct a spectral time-series model based on the DES-GW multi-band light curve from the single known KN event, and we use an average of BNS rates from past studies of ${10}^{3}\,{\mathrm{Gpc}}^{-3}\,{\mathrm{yr}}^{-1}$, consistent with the one event found so far. Examining past and current data sets from transient surveys, the number of KNe we expect to find for ASAS-SN, SDSS, PS1, SNLS, DES, and SMT is between 0 and 0.3. We predict the number of detections per future survey to be 8.3 from ATLAS, 10.6 from ZTF, 5.5/69 from LSST (the Deep Drilling/Wide Fast Deep), and 16.0 from WFIRST. The maximum redshift of KNe discovered for each survey is $z=0.8$ for WFIRST, $z=0.25$ for LSST, and $z=0.04$ for ZTF and ATLAS. This maximum redshift for WFIRST is well beyond the sensitivity of aLIGO and some future GW missions. For the LSST survey, we also provide contamination estimates from Type Ia and core-collapse supernovae: after light curve and template-matching requirements, we estimate a background of just two events. Finally, more broadly, we stress that future transient surveys should consider how to optimize their search strategies to improve their detection efficiency and to consider similar analyses for GW follow-up programs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

WREF Field Profiling Lidar-773 / Raw Data

The field profiling lidar #ZX300-773 was deployed on the ground in an open field 880 m south of the Wind River NEON tower in summer 2025. It was co-located with LLNL's Halo XR+ scanning lidar (z05) and Argonne's Waggle Node to capture mountain-valley flows during the warm summer season. The zip files contain high frequency data. The excel files are 10-min averages.

17 WIND ENERGY↗

A National Infrastructure for Artificial Intelligence on the Grid (NI4AI) (Final Scientific/Technical Report)

Electric utilities have traditionally taken a very pragmatic yet myopic approach with grid sensors and the resulting collected data. Sensors are purchased and deployed to solve a specific, known problem that has risen to sufficient awareness as to justify the effort of deploying sensors and the needed capital investment. This sensor data flows into proprietary software packages with limited functionality intended only to address the initial problem. This approach aligns with the financial incentives of the utility to deploy capital into fixed hardware assets for which the corporations earn a rate of return. This mentality stands in stark contrast to the big data revolution that started nearly 25 years ago with the rise of Google. In this worldview, data is a fundamental business asset; successful organizations collect, store, explore, merge, and exploit as much data as possible to not only solve problems well understood today but also to tackle new problems that will inevitably rise tomorrow. The ARPA-E Open Innovation 2018 project entitled A National Infrastructure for Artificial Intelligence on the Grid or NI4AI for short was designed to demonstrate this alternative paradigm for using data. To do this, the project was composed of three key thrust areas. The first major component deployed a variety of high-frequency grid sensors and captured terabytes of both wide-scale and localized grid measurements, generating high-value datasets for grid research and algorithm development. The second aspect made available PingThings’ PredictiveGridTM, a horizontally scalable, cloud-based data management and AI platform built for time series data to explore and exploit the collected data. Finally, the project fostered a diverse and open research community composed of experts from numerous fields through focused educational content, code sharing, and data science competitions. Shifting away from “single use” sensors and closed data silos within electric utilities is a major benefit to the public at large. This legacy approach to data is incredibly (1) capital intensive (new sensors must be deployed for each new problem and problems tend to arise continuously) and (2) painfully slow (new problems must be identified first and then new sensors must be deployed to collect data to begin to address the issue). The transition to a carbon neutral grid requires a massive transformation of the existing grid infrastructure and will continue to challenge the legacy grid in unforeseen ways. The only way to make the energy transition cost effective is for utilities to abandon this dated data paradigm and adopt more contemporary approaches. NI4AI has shown that it is technically possible and economically feasible to ingest, explore, and exploit grid data collected from even very high frequency sensing, such as continuous point on wave sensors collecting measurements 10,000 times a second. In fact, the PredictiveGrid platform used is commercially available and deployed at several utilities in the United States. Project accomplishments were numerous and included (1) making available a state of the art time series platform to the community, (2) collecting over 520 streams of time series data from grid sensors totaling over 1 trillion grid measurements, and (3) developing and nurturing a community within the industry focused on the use of data to create value for utilities and, ultimately, end consumers.

97 MATHEMATICS AND COMPUTING↗

On a Simplified Approach to Achieve Parallel Performance and Portability Across CPU and GPU Architectures

This paper presents software advances to easily exploit computer architectures consisting of a multi-core CPU and CPU+GPU to accelerate diverse types of high-performance computing (HPC) applications using a single code implementation. The paper describes and demonstrates the performance of the open-source C++ matrix and array (MATAR) library that uniquely offers: (1) a straightforward syntax for programming productivity, (2) usable data structures for data-oriented programming (DOP) for performance, and (3) a simple interface to the open-source C++ Kokkos library for portability and memory management across CPUs and GPUs. The portability across architectures with a single code implementation is achieved by automatically switching between diverse fine-grained parallelism backends (e.g., CUDA, HIP, OpenMP, pthreads, etc.) at compile time. The MATAR library solves many longstanding challenges associated with easily writing software that can run in parallel on any computer architecture. This work benefits projects seeking to write new C++ codes while also addressing the challenges of quickly making existing Fortran codes performant and portable over modern computer architectures with minimal syntactical changes from Fortran to C++. We demonstrate the feasibility of readily writing new C++ codes and modernizing existing codes with MATAR to be performant, parallel, and portable across diverse computer architectures.

97 MATHEMATICS AND COMPUTING↗

Data-intensive management and analysis for scientifc simulations.

Scientific simulations can produce enormous amounts of data, making the analysis of results and management of files a difficult task for scientists. The simulation management and analysis system (Smaash) described here is designed to allow scientists to easily capture, store, organize, monitor, and analyze simulation results. The system is automatic, standardized, and secure. Smaash was built using open-source tools and modularized to be independent of the scientific simulation. The web-based front-end allows the scientist to easily interact with the data, and has proved its usefulness in improving the efficiency of a scientific team's workflow. High performance parallel computing allows scientists to solve complex physical problems through computer simulation. However, the massive amounts of data generated and the complex computing environment can create additional complications. A recent review by Ludaescher et al.(2009) describes how scientific workflows can assist scientists in extracting knowledge from these data-intensive operations by automating components within pipelines. Within the fusion community, Klasky et al.(2008) and colleagues have developed a system that handles the storage management, data movement, metadata generation and management, and a means to analyze the results. In response to scientists needs, a simulation management and analysis system (Smaash) was developed at the University of Chicago and Argonne National Laboratory (USA). Smaash provides an integrated way to monitor simulations and analyze computational results; catalog, store, and retrieve simulations; and prepare output for publications. The system is independent of the particular simulation code, accessible from many HPC and browser-based platforms, and built around open-source software tools. Data security and provenance is considered throughout. The analysis components are hidden behind a web-based front end, enabling scientists to focus on their results and not get bogged down by information overload.

99 GENERAL AND MISCELLANEOUS↗

COVID19 Disease Map, a computational knowledge repository of virus–host interaction mechanisms

We need to effectively combine the knowledge from surging literature with complex datasets to propose mechanistic models of SARS-CoV-2 infection, improving data interpretation and predicting key targets of intervention. Here, we describe a large-scale community effort to build an open access, interoperable and computable repository of COVID-19 molecular mechanisms. The COVID-19 Disease Map (C19DMap) is a graphical, interactive representation of disease-relevant molecular mechanisms linking many knowledge sources. Notably, it is a computational resource for graph-based analyses and disease modelling. To this end, we established a framework of tools, platforms and guidelines necessary for a multifaceted community of biocurators, domain experts, bioinformaticians and computational biologists. The diagrams of the C19DMap, curated from the literature, are integrated with relevant interaction and text mining databases. We demonstrate the application of network analysis and modelling approaches by concrete examples to highlight new testable hypotheses. This framework helps to find signatures of SARS-CoV-2 predisposition, treatment response or prioritisation of drug candidates. Such an approach may help deal with new waves of COVID-19 or similar pandemics in the long-term perspective.

59 BASIC BIOLOGICAL SCIENCES↗

BioenergyKDF: Enabling Spatiotemporal Data Synthesis and Research Collaboration

The Bioenergy Knowledge Discovery Framework (BioenergyKDF) is a scalable, web-based collaborative environment for scientists working on bioenergy related research in which the connections between data, literature, and models can be explored and more clearly understood. The fully-operational and deployed system, built on multiple open source libraries and architectures, stores contributions from the community of practice and makes them easy to find, but that is just its base functionality. The BioenergyKDF provides a national spatiotemporal decision support capability that enables data sharing, analysis, modeling, and visualization as well as fosters the development and management of the U.S. bioenergy infrastructure, which is an essential component of the national energy infrastructure. The BioenergyKDF is built on a flexible, customizable platform that can be extended to support the requirements of any user community especially those that work with spatiotemporal data. While there are several community data-sharing software platforms available, some developed and distributed by national governments, none of them have the full suite of capabilities available in BioenergyKDF. For example, this component-based platform and database independent architecture allows it to be quickly deployed to existing infrastructure and to connect to existing data repositories (spatial or otherwise). As new data, analysis, and features are added; the BioenergyKDF will help lead research and support decisions concerning bioenergy into the future, but will also enable the development and growth of additional communities of practice both inside and outside of the Department of Energy. These communities will be able to leverage the substantial investment the agency has made in the KDF platform to quickly stand up systems that are customized to their data and research needs.

Bioenergy.↗

Automated Energy-Dispersive X-ray Spectroscopy Analysis for Multi-Modal Few-Shot Learning

Scanning transmission electron microscopy (STEM) is a powerful tool that allows for the atomic-scale analysis of a materials’ structure, chemistry, and defect domains (Akers et al. 2021). The current generation of microscopes generate vast amounts of data, surpassing the limits of effective manual analysis traditionally performed by domain experts (Spurgeon et al. 2021). While recent strides in machine learning have significantly enhanced the processing of large and intricate datasets acquired through electron microscopy, the prevalent use of proprietary software packages for initial data collection poses a challenge. In many cases, these software packages act as a ‘black box’, constraining user functionality and hindering the output of data in a format that is conducive to seamless integration into machine learning models. This work addresses these challenges by adapting HyperSpy, an open-source Python library, for the analysis and quantification of raw energy dispersive spectroscopy (EDS) data acquired through STEM. The modified HyperSpy code successfully facilitates user-defined segmentation of the data, enabling the integration of atomic %, weight %, and raw EDS spectra for each segmented region into an existing few-shot machine learning model. While initial results reveal discrepancies in quantified atomic and weight percentages when compared to proprietary software, ongoing efforts aim to rectify this issue by refining the fit of the HyperSpy model to the EDS spectra. Overall, this research underscores the potential of open-source tools like HyperSpy to enhance the accessibility of analytical tools, fostering a transparent and user-friendly environment for seamlessly incorporating electron microscopy data into machine learning models.

36 MATERIALS SCIENCE↗

An automated integrated web-based smart tool for open stope design

The Stability Graph is a widely used tool for the design of open stopes in underground mining. Many users of the Stability Graph still apply this design method manually. Although the manual approach has benefits, using multiple graphs and stability number computation charts for each stope surface is time-consuming, even for the experienced mining engineer. Current practice in the use of the method also limits data sharing. This paper presents a StopeSoft web-based tool for open stope stability prediction that is developed on the basis of the Stability Graph method and is available at openstope.com. StopeSoft incorporates flexibility in terms of Stability Graph options and incorporates additional critical factors often overlooked. As a web-based tool, StopeSoft encourages and makes data sharing possible globally, focused on expanding the database and improving the current limitations of the Stability Graph to provide practical, reliable solutions for mining engineers, consultants, and academics. The StopeSoft automated process facilitates the process of open stope stability prediction, saving time and minimizing potential human errors. Statistical treatment of the data accounts for the variability of input parameters to emphasize the probabilistic nature of the Stability Graph method. The probabilistic interpretation of the stability states of stope surfaces eliminates the false feeling of absolute stope performance based on its location on the Stability Graph , as implied by the deterministic approach.

58 GEOSCIENCES↗

Sandbox for Mac Malware v 1.0

This software is an analyzer for automated sandbox analysis of malware on the OS X operating system. It runs inside an OS X virtual machine to collect data about what happens when a given file is opened or run. As of August 2014, there was no sandbox software for Mac OS X malware, as it requires different methods from those used on the Windows OS (which most sandboxes are written for). This software adds OS X analysis capabilities to an existing open-source sandbox, Cuckoo Sandbox (http://cuckoosandbox.org/), which previously only worked for Windows. The analyzer itself can take many different types of files as input: the traditional Mach-O and FAT executables, .app files, zip files, Python scripts, Java archives, and web pages, as well as PDFs and other documents. While the file is running, the analyzer also simulates rudimentary human interaction with clicks and mouse movements in order to bypass the tests some malware use to see if they are being analyzed. The analyzer outputs several different kinds of data: function call traces, network captures, screenshots, and all created and modified files. This work also includes a static analysis Cuckoo module for Mach-O binary files. It extracts file structures, code library imports and exports, and signatures. This data can be used along with the analyzer results to create signatures for malware.

Walkup, Elizabeth↗

Spatiotemporal characterization of ionizing radiation induced DNA damage foci and their relation to chromatin organization

DNA damage sensing proteins have been shown to localize to the sites of DSB within seconds to minutes following ionizing radiation (IR) exposure, resulting in the formation of microscopically visible nuclear domains referred to as radiation-induced foci (RIF). This review characterizes the spatio-temporal properties of RIF at physiological doses, minutes to hours following exposure to ionizing radiation, and it proposes a model describing RIF formation and resolution as a function of radiation quality and nuclear densities. Discussion is limited to RIF formed by three interrelated proteins ATM (Ataxia telangiectasia mutated), 53BP1 (p53 binding protein 1) and ?H2AX (phosphorylated variant histone H2AX). Early post-IR, we propose that RIF mark chromatin reorganization, leading to a local nuclear scaffold rigid enough to keep broken DNA from diffusing away, but open enough to allow the repair machinery. We review data indicating clear kinetic and physical differences between RIF emerging from dense and uncondensed regions of the nucleus. At later time post-IR, we propose that persistent RIF observed days following exposure to ionizing radiation are nuclear ?scars? marking permanent disruption of the chromatin architecture. When DNA damage is resolved, such chromatin modifications should not necessarily lead to growth arrest and it has been shown that persistent RIF can replicate during mitosis. Thus, heritable persistent RIF spanning over tens of Mbp may affect the transcriptome of a large progeny of cells. This opens the door for a non DNA mutation-based mechanism of radiation-induced phenotypes.

59 BASIC BIOLOGICAL SCIENCES↗

Method for discovering relationships in data by dynamic quantum clustering

Data clustering is provided according to a dynamical framework based on quantum mechanical time evolution of states corresponding to data points. To expedite computations, we can approximate the time-dependent Hamiltonian formalism by a truncated calculation within a set of Gaussian wave-functions (coherent states) centered around the original points. This allows for analytic evaluation of the time evolution of all such states, opening up the possibility of exploration of relationships among data-points through observation of varying dynamical-distances among points and convergence of points into clusters. This formalism may be further supplemented by preprocessing, such as dimensional reduction through singular value decomposition and/or feature filtering.

97 MATHEMATICS AND COMPUTING↗

Method for discovering relationships in data by dynamic quantum clustering

Data clustering is provided according to a dynamical framework based on quantum mechanical time evolution of states corresponding to data points. To expedite computations, we can approximate the time-dependent Hamiltonian formalism by a truncated calculation within a set of Gaussian wave-functions (coherent states) centered around the original points. This allows for analytic evaluation of the time evolution of all such states, opening up the possibility of exploration of relationships among data-points through observation of varying dynamical-distances among points and convergence of points into clusters. This formalism may be further supplemented by preprocessing, such as dimensional reduction through singular value decomposition and/or feature filtering.

97 MATHEMATICS AND COMPUTING↗