Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

LADS: Optimizing Data Transfers using Layout-Aware Data Scheduling

While future terabit networks hold the promise of signifi- cantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today s 100 gigabit networks to real- ize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink. Data stor- age infrastructure at both the source and sink and its in- terplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this paper, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network en- vironment, and we present a new bulk data movement framework called LADS for terabit networks. LADS ex- ploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to use zero-copy, OS-bypass hardware when available. It can further im- prove data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared stor- age resource, improving I/O bandwidth, and data transfer rates across the high speed networks.

Data Management↗
DOE OSTI2022

Livewire Data Platform-A Solution for Energy Efficient Mobility Systems (EEMS) Data Sharing

This presentation is an overview of progress made on the Livewire Data Platform since June 2021. Livewire is a publicly accessible platform for sharing energy efficiency and mobility research data funded by DOE's Vehicle Technologies Office. Core services and platform capabilities of Livewire include: Free, secure data storage; Access management that allows data owners to control who sees their data; Data collection and preservation; Quality characterization; Detailed access and download metrics; and Increased visibility of projects and data.

energy efficiency mobility↗
DOE OSTI2024

In Silico Human Mobility Data Science: Leveraging Massive Simulated Mobility Data (Vision Paper)

Human mobility data science using trajectories or check-ins of individuals has many applications. Recently, we have seen a plethora of research efforts that tackle these applications. However, research progress in this field is limited by a lack of large and representative datasets. The largest and most commonly used dataset of individual human trajectories captures fewer than 200 individuals, while datasets of individual human check-ins capture fewer than 100 check-ins per city per day. Thus, it is not clear if findings from the human mobility data science community would generalize to large populations. Since obtaining massive, representative, and individual-level human mobility data is hard to come by due to privacy considerations, the vision of this work is to embrace the use of data generated by large-scale socially realistic microsimulations. Informed by both real data and leveraging social and behavioral theories, massive spatially explicit microsimulations may allow us to simulate entire megacities at the person level. The simulated worlds, which do not capture any identifiable personal information, allow us to perform “in silico” experiments using the simulated world as a sandbox in which we have perfect information and perfect control without jeopardizing the privacy of any actual individual. In silico experiments have become commonplace in other scientific domains such as chemistry and biology, permitting experiments that foster the understanding of concepts without any harm to individuals. This work describes challenges and opportunities for leveraging massive and realistic simulated alternate worlds for in silico human mobility data science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Summary There are many data sources that produce large volumes of data. The Big Data nature requires new distributed processing approaches to extract the valuable information. Real‐time sentiment analysis is one of the most demanding research areas that requires powerful Big Data analytics tools such as Spark. Prior literature survey work has shown that, though there are many conventional sentiment analysis researches, there are only few works realizing sentiment analysis in real time. One major point that affects the quality of real‐time sentiment analysis is the confidence of the generated data. In more clear terms, it is a valuable research question to determine whether the owner that generates sentiment is genuine or not. Since data generated by fake personalities may decrease accuracy of the outcome, a smart/intelligent service that can identify the source of data is one of the key points in the analysis. In this context, we include a fake account detection service to the proposed framework. Both sentiment analysis and fake account detection systems are trained and tested using Naïve Bayes model from Apache Spark's machine learning library. The developed system consists of four integrated software components, ie, (i) machine learning and streaming service for sentiment prediction, (ii) a Twitter streaming service to retrieve tweets, (iii) a Twitter fake account detection service to assess the owner of the retrieved tweet, and (iv) a real‐time reporting and dashboard component to visualize the results of sentiment analysis. The sentiment classification performances of the system for offline and real‐time modes are 86.77% and 80.93%, respectively.

Kılınç, Deniz↗
DOE OSTI2020

Smoky Mountain Data Challenge 2020: An Open Call to Solve Data Problems in the Areas of Neutron Science, Material Science, Urban Modeling and Dynamics, Geophysics, and Biomedical Informatics

The 2020 Smoky Mountains Computational Sciences and Engineering Conference enlists research scientists from across Oak Ridge National Laboratory (ORNL) to be data sponsors and help create data analytics challenges for eminent data sets at the laboratory. This work describes the significance of each of the seven data sets and their as- sociated challenge questions. The challenge questions for each data set were required to cover multiple difficulty levels. An international call for participation was sent to students, and researchers asking them to form teams of up to four people to apply novel data analytics techniques to these data sets.

Parete-Koon, Suzanne↗
DOE OSTI2019

Measuring the Hubble constant and spatial curvature from supernova apparent magnitude, baryon acoustic oscillation, and Hubble parameter data

Cosmic microwave background (CMB) anisotropy (spatial inhomogeneity) data provide the tightest constraints on the Hubble constant, matter density, spatial curvature, and dark energy dynamics. Other data, sensitive to the evolution of only the spatially homogeneous part of the cosmological model, such as Type Ia supernova apparent magnitude, baryon acoustic oscillation distance, and Hubble parameter measurements, can be used in conjunction with the CMB data to more tightly constrain parameters. Recent joint analyses of CMB and such non-CMB data indicate that slightly closed spatial hypersurfaces are favored in nonflat untilted inflation models and that dark energy dynamics cannot be ruled out, and favor a smaller Hubble constant. Here, we show that the constraints that follow from these non-CMB data alone are consistent with those that follow from the CMB data alone and so also consistent with, but weaker than, those that follow from the joint analyses of the CMB and non-CMB data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Data Mining and Visualization of High-Dimensional ICME Data for Additive Manufacturing

Integrated computational materials engineering (ICME) methods combining CALPHAD with process-based simulations can produce rich, high-dimensional data for alloy and process design. In ICME methods for metallurgical applications, the visualization and interpretation of such high-dimensional data has previously been through heat maps represented in 2 or 3 dimensions. While such an approach is ideal when one variable is varied at a time, in the case of high-dimensional data with multiple variables varied simultaneously, as is the case in additive manufacturing, interpreting the trends through two- or three-dimensional heat maps becomes challenging. Here, we propose a strategy of mixed visual data mining and quantitative analysis for high-dimensional metallurgical and process data using high-throughput thermodynamic calculations. Two case studies show the application of the proposed approach. The first case study investigated the effects of feedstock chemistry on the δ ferrite formation in 316L stainless steel powders used for binder jet additive manufacturing. The second case study linked Scheil–Gulliver calculations to a process model for dissimilar joining of aluminum alloys 5356 and 6111 during laser hot-wire additive manufacturing. Both cases contained thousands of calculated data points, showcasing the utility of visual data analysis through parallel coordinate plotting, Pearson correlation coefficient matrices, and scatter matrices compared to traditional process maps. These visualization techniques can be extended to many additive manufacturing problems to capture process–structure–property relationships for additively manufactured components.

36 MATERIALS SCIENCE↗
DOE OSTI2015

Interpreting biomarker data from the COPHES/DEMOCOPHES twin projects: Using external exposure data to understand biomarker differences among countries

In 2011 and 2012, the COPHES/DEMOCOPHES twin projects performed the first ever harmonized human biomonitoring survey in 17 European countries. In more than 1800 mother–child pairs, individual lifestyle data were collected and cadmium, cotinine and certain phthalate metabolites were measured in urine. Total mercury was determined in hair samples. While the main goal of the COPHES/DEMOCOPHES twin projects was to develop and test harmonized protocols and procedures, the goal of the current paper is to investigate whether the observed differences in biomarker values among the countries implementing DEMOCOPHES can be interpreted using information from external databases on environmental quality and lifestyle. In general, 13 countries having implemented DEMOCOPHES provided high-quality data from external sources that were relevant for interpretation purposes. However, some data were not available for reporting or were not in line with predefined specifications. Therefore, only part of the external information could be included in the statistical analyses. Nonetheless, there was a highly significant correlation between national levels of fish consumption and mercury in hair, the strength of antismoking legislation was significantly related to urinary cotinine levels, and we were able to show indications that also urinary cadmium levels were associated with environmental quality and food quality. These results again show the potential of biomonitoring data to provide added value for (the evaluation of) evidence-informed policy making. - Highlights: • External data was collected to interpret HBM data from DEMOCOPHES. • Hg in hair could be related to fish consumption across different countries. • Urinary cotinine was related to strictness of anti-smoking legislation. • Urinary Cd was borderline significantly related to air and food quality. • Lack of comparable data among countries hampered the analysis.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Nuclear Data Sheets for A = 227

The evaluated spectroscopic data are presented for ten known nuclides of mass 227 (Po, At, Rn, Fr, Ra, Ac, Th, Pa, U, Np). For {sup 227}Po, {sup 227}At, {sup 227}Rn, {sup 227}Pa, {sup 227}U and {sup 227}Np nuclei, only the ground-state information is available. Their decay characteristics are mostly unknown. Levels in {sup 227}Fr are known only from the decay of {sup 227}Rn to {sup 227}Fr. This decay scheme at present cannot be normalized to deduce γ intensities per 100 decays due to lack of knowledge about multipolarities of many low-energy transitions. The levels in {sup 227}Ra, {sup 227}Ac and {sup 227}Th are known from several decays and reactions, including particle-transfer data for {sup 227}Ra and {sup 227}Ac. The decay scheme of {sup 227}Ra to {sup 227}Ac was last studied in 1971 using small Ge detectors. Improved γ-ray intensity data need to be obtained with a better γ-detection system. The datasets for {sup 227}Ac have undergone extensive revisions, including detailed data for 231Pa α decay from 1986BaYK report, and single-proton transfer data from 1986MaYU thesis. High-spin (J>13/2 or so) structures are known only for 227Th. Level lifetime data are quite scarce for all the nuclides in this mass chain, thus limiting the knowledge of reduced transition probabilities. Band structures for {sup 227}Fr, {sup 227}Ra, {sup 227}Ac and {sup 227}Th are known in detail, together with evidence of weak octupole deformation and consequent parity-doublet structures. This evaluation was carried out as part of a joint IAEA-ICTP workshop for Nuclear Structure and Decay Data, organized and hosted by the IAEA, Vienna and ICTP, Trieste, March 24–28, 2014. The evaluation work was coordinated by B. Singh (McMaster). This work supersedes previous A=227 evaluation (2001Br31) published by E. Browne which covered literature before May 2001.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2018

ENDF/B-VIII.0: The 8 th Major Release of the Nuclear Reaction Data Library with CIELO-project Cross Sections, New Standards and Thermal Scattering Data

We describe the new ENDF/B-VIII.0 evaluated nuclear reaction data library. ENDF/B-VIII.0 fully incorporates the new IAEA standards, includes improved thermal neutron scattering data and uses new evaluated data from the CIELO project for neutron reactions on 1 H, 16 O, 56 Fe, 235 U, 238 U and 239 Pu described in companion papers in the present issue of Nuclear Data Sheets. The evaluations benefit from recent experimental data obtained in the U.S. and Europe, and improvements in theory and simulation. Notable advances include updated evaluated data for light nuclei, structural materials, actinides, fission energy release, prompt fission neutron and γ-ray spectra, thermal neutron scattering data, and charged-particle reactions. Integral validation testing is shown for a wide range of criticality, reaction rate, and neutron transmission benchmarks. In general, integral validation performance of the library is improved relative to the previous ENDF/B-VII.1 library.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2019

Multi-component background learning automates signal detection for spectroscopic data

Abstract Automated experimentation has yielded data acquisition rates that supersede human processing capabilities. Artificial Intelligence offers new possibilities for automating data interpretation to generate large, high-quality datasets. Background subtraction is a long-standing challenge, particularly in settings where multiple sources of the background signal coexist, and automatic extraction of signals of interest from measured signals accelerates data interpretation. Herein, we present an unsupervised probabilistic learning approach that analyzes large data collections to identify multiple background sources and establish the probability that any given data point contains a signal of interest. The approach is demonstrated on X-ray diffraction and Raman spectroscopy data and is suitable to any type of data where the signal of interest is a positive addition to the background signals. While the model can incorporate prior knowledge, it does not require knowledge of the signals since the shapes of the background signals, the noise levels, and the signal of interest are simultaneously learned via a probabilistic matrix factorization framework. Automated identification of interpretable signals by unsupervised probabilistic learning avoids the injection of human bias and expedites signal extraction in large datasets, a transformative capability with many applications in the physical sciences and beyond.

Ament, Sebastian E.↗
DOE OSTI2019

The CHRS Data Portal, an easily accessible public repository for PERSIANN global satellite precipitation data

The Center for Hydrometeorology and Remote Sensing (CHRS) has created the CHRS Data Portal to facilitate easy access to the three open data licensed satellite-based precipitation datasets generated by our Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) system: PERSIANN, PERSIANN-Cloud Classification System (CCS), and PERSIANN-Climate Data Record (CDR). These datasets have the potential for widespread use by various researchers, professionals including engineers, city planners, and so forth, as well as the community at large. Researchers at CHRS created the CHRS Data Portal with an emphasis on simplicity and the intention of fostering synergistic relationships with scientists and experts from around the world. The following paper presents an outline of the hosted datasets and features available on the CHRS Data Portal, an examination of the necessity of easily accessible public data, a comprehensive overview of the PERSIANN algorithms and datasets, and a walk-through of the procedure to access and obtain the data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Source scaling comparison and validation in Central Italy: data intensive direct S waves versus the sparse data coda envelope methodology

SUMMARY Robustness of source parameter estimates is a fundamental issue in understanding the relationships between small and large events; however, it is difficult to assess how much of the variability of the source parameters can be attributed to the physical source characteristics or to the uncertainties of the methods and data used to estimate the values. In this study, we apply the coda method by Mayeda et al. using the coda calibration tool (CCT), a freely available Java-based code (https://github.com/LLNL/coda-calibration-tool) to obtain a regional calibration for Central Italy for estimating stable source parameters. We demonstrate the power of the coda technique in this region and show that it provides the same robustness in source parameter estimation as a data-driven methodology [generalized inversion technique (GIT)], but with much fewer calibration events and stations. The Central Italy region is ideal for both GIT and coda approaches as it is characterized by high-quality data, including recent well-recorded seismic sequences such as L'Aquila (2009) and Amatrice–Norcia–Visso (2016–2017). This allows us to apply data-driven methods such as GIT and coda-based methods that require few, but high-quality data. The data set for GIT analysis includes ∼5000 earthquakes and more than 600 stations, while for coda analysis we used a small subset of 39 events spanning 3.5 < Mw < 6.33 and 14 well-distributed broad-band stations. For the common calibration events, as well as an additional 247 events (∼1.7 < Mw < ∼5.0) not used in either calibration, we find excellent agreement between GIT-derived and CCT-derived source spectra. This confirms the ability of the coda approach to obtain stable source parameters even with few calibration events and stations. Even reducing the coda calibration data set by 75 per cent, we found no appreciable degradation in performance. This validation of the coda calibration approach over a broad range of event size demonstrates that this procedure, once extended to other regions, represents a powerful tool for future routine applications to homogeneously evaluate robust source parameters on a national scale. Furthermore, the coda calibration procedure can homogenize the Mw estimates for small and large events without the necessity of introducing any conversion scale between narrow-band measures such as local magnitude (ML) and Mw, which has been shown to introduce significant bias.

Morasca, Paola (ORCID:0000000265254867)↗
DOE OSTI2017

The hidden treasure in your data: phasing with unexpected weak anomalous scatterers from routine data sets

Single-wavelength anomalous dispersion (SAD) utilizing anomalous signal from native S atoms, or other atoms with Z ≤ 20, generally requires highly redundant data collected using relatively long-wavelength X-rays. Here, the results from two proteins are presented where the anomalous signal from serendipitously acquired surface-bound Ca atoms with an anomalous data multiplicity of around 10 was utilized to drive de novo structure determination. In both cases, the Ca atoms were acquired from the crystallization solution, and the data-collection strategy was not optimized to exploit the anomalous signal from these scatterers. The X-ray data were collected at 0.98 Å wavelength in one case and at 1.74 Å in the other (the wavelength was optimized for sulfur, but the anomalous signal from calcium was exploited for structure solution). Similarly, using a test case, it is shown that data collected at ~1.0 Å wavelength, where the f '' value for sulfur is 0.28 e, are sufficient for structure determination using intrinsic S atoms from a strongly diffracting crystal. Interestingly, it was also observed that SHELXD was capable of generating a substructure solution from high-exposure data with a completeness of 70% for low-resolution reflections extending to 3.5 Å resolution with relatively low anomalous multiplicity. Considering the fact that many crystallization conditions contain anomalous scatterers such as Cl, Ca, Mn etc. , checking for the presence of fortuitous anomalous signal in data from well diffracting crystals could prove useful in either determining the structure de novo or in accurately assigning surface-bound atoms.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Perdeuteration, crystallization, data collection and comparison of five neutron diffraction data sets of complexes of human galectin-3C

Galectin-3 is an important protein in molecular signalling events involving carbohydrate recognition, and an understanding of the hydrogen-bonding patterns in the carbohydrate-binding site of its C-terminal domain (galectin-3C) is important for the development of new potent inhibitors. The authors are studying these patterns using neutron crystallography. Here, the production of perdeuterated human galectin-3C and successive improvement in crystal size by the development of a crystal-growth protocol involving feeding of the crystallization drops are described. The larger crystals resulted in improved data quality and reduced data-collection times. Furthermore, protocols for complete removal of the lactose that is necessary for the production of large crystals of apo galectin-3C suitable for neutron diffraction are described. Five data sets have been collected at three different neutron sources from galectin-3C crystals of various volumes. It was possible to merge two of these to generate an almost complete neutron data set for the galectin-3C–lactose complex. As a result, these data sets provide insights into the crystal volumes and data-collection times necessary for the same system at sources with different technologies and data-collection strategies, and these insights are applicable to other systems.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Understanding data motion in the modern HPC data center

The utilization and performance of storage, compute, and network resources within HPC data centers have been studied extensively, but much less work has gone toward characterizing how these resources are used in conjunction to solve larger scientific challenges. To address this gap, we present our work in characterizing workloads and workflows at a data-center-wide level by examining all data transfers that occurred between storage, compute, and the external network at the National Energy Research Scientific Computing Center over a three-month period in 2019. Using a simple abstract representation of data transfers, we analyze over 100 million transfer logs from Darshan, HPSS user interfaces, and Globus to quantify the load on data paths between compute, storage, and the wide-area network based on transfer direction, user, transfer tool, source, destination, and time. We show that parallel I/O from user jobs, while undeniably important, is only one of several major I/O workloads that occurs throughout the execution of scientific workflows. We also show that this approach can be used to connect anomalous data traffic to specific users and file access patterns, and we construct time-resolved user transfer traces to demonstrate that one can systematically identify coupled data motion for individual workflows.

Lockwood, GK↗
DOE OSTI2021

D2U: Data Driven User Emulation for the Enhancement of Cyber Testing, Training, and Data Set Generation

Whether testing intrusion detection systems, conducting training exercises, or creating data sets to be used by the broader cybersecurity community, realistic user behavior is a critical component of a cyber range. Existing methods either rely on network level data or replay recorded user actions to approximate real users in a network. Our work is the first to produce generative models trained on actual user data (sequences of application usage) collected on endpoints. Once trained to the user's behavioral data, these models can generate novel sequences of actions %that appear to come from the same distribution as the training data. These sequences of actions are then fed to our custom software via configuration files, which replicate those behaviors on end devices. Notably, our models are platform agnostic and could generate behavior data for any emulation software package. In this paper we present our model generation process, software architecture, and an initial evaluation of the fidelity of our models. Our software is currently deployed in a cyber range to help evaluate the efficacy of defensive cyber technologies. We suggest additional ways that the cyber community as a whole can benefit from more realistic user behavior emulation. The data used to train our model, as well as sample configuration files produced by the model, are available at [redacted].

Oesch, T↗
DOE OSTI2023

Streaming Data from Experimental Facilities to Supercomputers for Real-Time Data Processing

In this paper we demonstrate direct data streaming from instruments and detectors at a large-scale experimental facility to a supercomputer for real-time data processing and feedback. Streaming data to supercomputers introduces the potential for novel scientific applications and workflow models, including the ability to provide real-time feedback from very large datasets during an experiment and the integration of real-time ML training and inference at scale. We discuss a successful demonstration for real-time processing of data from the Advanced Photon Source (APS) on the Polaris supercomputer using an EPICS-based streaming framework. We describe the capabilities of the streaming framework itself, and outline the architecture that allows us to process experimentally derived data on a supercomputer without file-based data transfers. We present throughput measurements that are indicative of system performance capable of sustaining the expected data production rates of the facility, as well as discuss some outstanding challenges and our future directions.

real-time processing↗