Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62

Uniform modelling of the stellar density of thirteen tidal streams within the Galactic halo

ABSTRACT We present the results of fitting a flexible stellar stream density model to a collection of thirteen streams around the Milky Way, using photometric data from DES, DECaLS, and Pan-STARRS. We construct density maps for each stream and characterize their tracks on the sky, width, and distance modulus curves along the length of each stream. We use these measurements to compute lengths and total luminosities of streams and identify substructures. Several streams show prominent substructures, such as stream broadening, gaps, large deviations of stream tracks, and sharp changes in stream densities. Examining the group of streams as a population, as expected we find that streams with globular cluster progenitors are typically narrower than those with dwarf galaxy progenitors, with streams around 100 pc wide showing overlap between the two populations. We also note the average luminosity of globular cluster streams is significantly lower than the typical luminosity of intact globular clusters. The likely explanation is that observed globular cluster streams preferentially come from lower luminosity and lower density clusters. The stream measurements done in a uniform manner presented here will be helpful for more detailed stream studies such as identifying candidate stream members for spectroscopic follow up and stellar stream dynamical modelling.

79 ASTRONOMY AND ASTROPHYSICS↗

Trigger-DAQ and Slow Controls Systems in the Mu2e Experiment

The muon campus program at Fermilab includes the Mu2e experiment that will search for a charged-lepton flavor violating processes where a negative muon converts into an electron in the field of an aluminum nucleus, improving by four orders of magnitude the search sensitivity reached so far. Mu2e's Trigger and Data Acquisition System (TDAQ) uses otsdaq as its solution. Developed at Fermilab, otsdaq uses the artdaq DAQ framework and art analysis framework, under the-hood, for event transfer, filtering, and processing. otsdaq is an online DAQ software suite with a focus on flexibility and scalability, while providing a multi-user, web-based, interface accessible through the Chrome or Firefox web browser. The detector Read Out Controller (ROC), from the tracker and calorimeter, stream out zero-suppressed data continuously to the Data Transfer Controller (DTC). Data is then read over the PCIe bus to a software filter algorithm that selects events which are finally combined with the data flux that comes froma Cosmic Ray Ve to System (CRV). A Detector Control System (DCS) for monitoring, controlling, alarming, and archiving has been developed using the Experimental Physics and Industrial Control System (EPICS) Open Source Platform. The DCS System has also been itegrated into otsdaq. The installation of the TDAQ and the DCS systems in the Mu2e building is planned for 2021-2022, and a prototype has been built at Fermilab's Feynman Computing Center. We report here on the developments and achievements of the integration of Mu2e's DCS system into the online otsdaq software.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The LAMOST survey of background quasars in the vicinity of the Andromeda and Triangulum galaxies. II. Results from the commissioning observations and the pilot surveys

We present new quasars discovered in the vicinity of the Andromeda and Triangulum galaxies with the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, also named the Guoshoujing Telescope, during the 2010 and 2011 observational seasons. Quasar candidates are selected based on the available Sloan Digital Sky Survey, Kitt Peak National Observatory 4 m telescope, Xuyi Schmidt Telescope Photometric Survey optical, and Wide-field Infrared Survey Explorer near-infrared photometric data. We present 509 new quasars discovered in a stripe of ∼135 deg{sup 2} from M31 to M33 along the Giant Stellar Stream in the 2011 pilot survey data sets, and also 17 new quasars discovered in an area of ∼100 deg{sup 2} that covers the central region and the southeastern halo of M31 in the 2010 commissioning data sets. These 526 new quasars have i magnitudes ranging from 15.5 to 20.0, redshifts from 0.1 to 3.2. They represent a significant increase of the number of identified quasars in the vicinity of M31 and M33. There are now 26, 62, and 139 known quasars in this region of the sky with i magnitudes brighter than 17.0, 17.5, and 18.0, respectively, of which 5, 20, and 75 are newly discovered. These bright quasars provide an invaluable collection with which to probe the kinematics and chemistry of the interstellar/intergalactic medium in the Local Group of galaxies. A total of 93 quasars are now known with locations within 2.°5 of M31, of which 73 are newly discovered. Tens of quasars are now known to be located behind the Giant Stellar Stream, and hundreds are behind the extended halo and its associated substructures of M31. The much enlarged sample of known quasars in the vicinity of M31 and M33 can potentially be utilized to construct a perfect astrometric reference frame to measure the minute proper motions (PMs) of M31 and M33, along with the PMs of substructures associated with the Local Group of galaxies. Those PMs are some of the most fundamental properties of the Local Group.

79 ASTRONOMY AND ASTROPHYSICS↗

Quantitative Methods for Comparing Different Polyline Stream Network Models

Two techniques for exploring relative horizontal accuracy of complex linear spatial features are described and sample source code (pseudo code) is presented for this purpose. The first technique, relative sinuosity, is presented as a measure of the complexity or detail of a polyline network in comparison to a reference network. We term the second technique longitudinal root mean squared error (LRMSE) and present it as a means for quantitatively assessing the horizontal variance between two polyline data sets representing digitized (reference) and derived stream and river networks. Both relative sinuosity and LRMSE are shown to be suitable measures of horizontal stream network accuracy for assessing quality and variation in linear features. Both techniques have been used in two recent investigations involving extracting of hydrographic features from LiDAR elevation data. One confirmed that, with the greatly increased resolution of LiDAR data, smaller cell sizes yielded better stream network delineations, based on sinuosity and LRMSE, when using LiDAR-derived DEMs. The other demonstrated a new method of delineating stream channels directly from LiDAR point clouds, without the intermediate step of deriving a DEM, showing that the direct delineation from LiDAR point clouds yielded an excellent and much better match, as indicated by the LRMSE.

58 GEOSCIENCES↗

Synoptic NO3 in Slate River Watershed, Colorado (2022)

The synoptic nitrate (NO3) dataset in the Slate River Watershed, Colorado consists of NO3 data collected at 19 locations three times during the summer of 2022. Stream samples were collected in early summer (early July), mid summer (late August), and late summer (late September). The samples include mainstem, tributary, and point source input water samples. These data were collected to evaluate spatiotemporal variability in stream NO3 during the summer, and evaluate anthropogenic controls on stream NO3 dynamics. This data package contains: (1) a csv of all NO3 samples and (2) a csv of locations for each sampling site. The dataset additionally includes a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata; and a data dictionary (dd.csv) file that contains column/row headers used throughout the files along with a definition, units, and data type. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

EARTH SCIENCE > BIOSPHERE > ECOSYSTEMS > FRESHWATE↗

HERMES: A cyber-energy SAAS platform to provide real-time 2d and 3d interactive visualizations [SWR-23-15]

HERMES is a novel visualization SAAS platform for achieving real-time visualization of large-scale environments involving cyber-energy devices. The platform receives data from emulated environments which include physical hardware and network devices that represent a complex IT / OT system. The platform is capable of streaming, filtering, storing, and visualizing all data within the environment and scaling vertically and horizontally. This capability enables high fidelity visual analysis of events to be performed in real time as well as the collection and storage of historical data for forensic analysis.

Van Natta, Joshua↗

Model Inputs, Outputs, and Scripts associated with: “Combined effects of stream hydrology and land use on basin-scale hyporheic zone denitrification in the Columbia River Basin”

This data package is associated with the publication “Combined effects of stream hydrology and land use on basin‐scale hyporheic zone denitrification in the Columbia River Basin”, published in Water Resource Research (Son et al.2022) available at https://doi.org/10.1029/2021WR031131. This data package includes the key model inputs/outputs of the river corridor model for the Columbia River Basin (CRB) and the model source codes used in the manuscript. The model is a carbon-nitrogen-coupled river corridor model (RCM), and the model is used to quantify hyporheic zone (HZ) denitrification at the NHDPLUS stream reach scales. The RCM used in this study combines empirical substrate models derived from observations and three microbially driven reactions, including two-step denitrification and aerobic respiration, are considered within the HZ. The key input data of the model are exchange flux, residence time, and stream solute (dissolved organic carbon (DOC), dissolved oxygen (DO), and nitrate concentrations). These inputs are constant over time and represent long-term averaged values. This study uses the RCM to explore the spatial patterns of HZ denitrification across reaches with different sizes and land use in the CRB. Our main objective is to use the RCM as a virtual reality model, and the machine-learning models as surrogates that encapsulate the complexities of the physics-based model while identifying the importance of different variables that are not evident in the model conceptualization. We do not include a direct comparison of the modeled HZ denitrification and measurements; however, the RCM can capture the overall spatial patterns of the HZ denitrification because the model inputs and its reaction networks are based on well-established theory and a physical-based model. The combination of the model-based predictions and a machine-learning approach (e.g., random forest) is used to improve our understanding of what variables of the model are associated with spatial patterns of the modeled denitrification across reaches with different sizes and land uses, and to develop a proxy model using measurable variables to reproduce the simulated patterns.This dataset contains five folders: (1) model_inputs, (2) model_outputs, (3) Rscripts, (4) figures, and (5) model_codes. It also contains a readme, file level metadata (FLMD), and data dictionary (dd). Please see the FLMD for a list of all the files contained in this data package and descriptions for each. The model_inputs folder contains the model inputs used to drive the model simulations. The model_outputs folder contains key model output files from the river corridor model. The Rscripts folder contains the Rscripts for pre- and post- processing model results. The figures folder contains the raw figures associated with the manuscript. The model_codes folder includes key model source codes/input files. All files are .jpg, .jpeg, .out, .e, .od, .dat, .sub, .F90, .0, .R, .sbx, .cpg, .sbn, .shx, .shp, .dbf, .prj, .tfw, .tif, .xml, .pdf, or .csv.

54 ENVIRONMENTAL SCIENCES↗

VDiSC: An Open Source Framework for Distributed Smart City Vision and Biometric Surveillance Networks

Recent global growth in the interest of smart cities has led to trillions of dollars of investment toward research and development. These connected cities have the potential to create a symbiosis of technology and society and revolutionize the cost of living, safety, ecological sustainability, and quality of life of societies on a world-wide scale. Some key components of the smart city construct are connected smart grids, self-driving cars, federated learning systems, smart utilities, large-scale public transit, and proactive surveillance systems. While exciting in prospect, these technologies and their subsequent integration cannot be attempted without addressing the potential societal impacts of such a high degree of automation and data sharing. Additionally, the feasibility of coordinating so many disparate tasks will require a fast, extensible, unifying framework. To that end, we propose the Distributed Smart City framework for Vision, or VDiSC. VDiSC serves as a unified biometric API harness that allows for seamless evaluation, deployment, and simple pipeline creation for heterogeneous biometric software. VDiSC additionally provides a fully declarative capability for defining and coordinating custom machine learning and sensor pipelines, allowing the distribution of processes across otherwise incompatible hardware and networks. VDiSC ultimately provides a way to quickly configure, hot-swap, and expand large coordinated or federated systems online without interruptions for maintenance. Because much of the data collected in a smart city contains Personally Identifying Information (PII), VDiSC also provides built-in tools and layers to ensure secure and encrypted streaming, storage, and access of PII data across distributed systems.

Brogan, Joel↗

MetricForensics: A Multi-Level Approach for Mining Volatile Graphs

Advances in data collection and storage capacity have made it increasingly possible to collect highly volatile graph data for analysis. Existing graph analysis techniques are not appropriate for such data, especially in cases where streaming or near-real-time results are required. An example that has drawn significant research interest is the cyber-security domain, where internet communication traces are collected and real-time discovery of events, behaviors, patterns and anomalies is desired. We propose MetricForensics, a scalable framework for analysis of volatile graphs. MetricForensics combines a multi-level “drill down" approach, a collection of user-selected graph metrics and a collection of analysis techniques. At each successive level, more sophisticated metrics are computed and the graph is viewed at a finer temporal resolution. In this way, MetricForensics scales to highly volatile graphs by only allocating resources for computationally expensive analysis when an interesting event is discovered at a coarser resolution first. We test MetricForensics on three real-world graphs: an enterprise IP trace, a trace of legitimate and malicious network traffic from a research institution, and the MIT Reality Mining proximity sensor data. Our largest graph has »3M vertices and »32M edges, spanning 4:5 days. The results demonstrate the scalability and capability of MetricForensics in analyzing volatile graphs; and highlight four novel phenomena in such graphs: elbows, broken correlations, prolonged spikes, and strange stars.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Double-pulse and single-pulse laser-induced breakdown spectroscopy for distinguishing between gaseous and particulate phase analytes

We explore the use of a combination of double-pulse and single-pulse laser-induced breakdown spectroscopy (LIBS) methodologies as a means of differentiating between solid-phase and gaseous-phase analytes (namely, carbon) in an aerosol stream. A range of spectral data was recorded for double-pulse and single-pulse configurations, including both ns and fs prepulse widths, while varying the gas-phase mass percentage of the carbon from about 10% to 90% for various fixed carbon concentrations. The carbon emission response, as measured by the peak-to-continuum ratio, was greater for the double-pulse configuration as compared with the single-pulse response and was also enhanced as the percentage of solid-phase carbon was increased. Using a combination of the double-pulse and single-pulse emission signals, a monotonically increasing response function was found to correlate with the percentage of gas-phase analyte. However, individual data points at the measured gas-phase percentages reveal considerable scatter from the predicted trend. Furthermore, the double-pulse to single-pulse ratio was only pronounced with the ns-ns configuration as compared with the fs-ns scheme. Overall, the LIBS methodology has been demonstrated as a potential means to discriminate between gas-phase and particulate-phase fractions of the same elemental species in an aerosol, although future optimization of the temporal parameters should be explored to improve the precision and accuracy of this approach.

74 ATOMIC AND MOLECULAR PHYSICS↗

Development of an Index of Biotic Integrity for the Sand Hills Ecoregion of the Southeastern United States

Abstract We developed an index of biotic integrity (IBI) for the Sand Hills ecoregion within the upper Coastal Plain of the southeastern United States. This ecoregion exhibits differences in species composition among river basins and possesses a relatively depauperate fish fauna that has been affected by faunal homogenization. We also investigated whether this IBI could be improved by adding benthic macroinvertebrate information. Data collected from 70 wadeable stream sites were used to calculate 45 fish assemblage metrics. Twelve metrics within the categories of species richness, species composition, habitat guild, trophic guild, tolerance level, alien/invasive species, and individual fish condition were retained after screening all metrics for sensitivity to disturbance, redundancy, and ecological content. Heterogeneity of species composition among basins led to a greater reliance on richness, indicator species, trophic guild, and habitat guild metrics rather than on more geographically variable species composition metrics. Characteristic headwater assemblages provided the basis for some metrics, and invader species metrics were included to address faunal homogenization. The Sand Hills IBI (SHIBI) was more accurate ecoregion‐wide than IBIs that were developed for portions of the Sand Hills, correctly classifying about 60% of the disturbed sites compared with less than 30% for the other IBIs. However, the SHIBI was not highly responsive to gradients in site quality, as indicated by an R 2 of 0.30 between SHIBI scores and instream habitat quality scores. This R 2 value was improved to 0.49 by supplementing the SHIBI with a benthic macroinvertebrate index (the Ephemeroptera, Plecoptera, and Trichoptera index). We conclude that ecoregion‐wide IBIs should (1) be based on ecoregion‐wide data that characterize responses to disturbance and natural habitat gradients, (2) maintain a substantial ecological basis by including several metric classes, and (3) include metrics that address faunal homogenization where disturbance has increased the dispersion of nonnative species and native generalist species. In addition, combining information from different assemblages can enhance IBI responsiveness to disturbance and improve performance. Received May 5, 2016; accepted September 19, 2016 Published online December 8, 2016

Paller, Michael H.↗

On low-rank updates to the singular value and Tucker decompositions

The singular value decomposition is widely used in signal processing and data mining. Since the data often arrives in a stream, the problem of updating matrix decompositions under low-rank modification has been widely studied. Brand developed a technique in 2006 that has many advantages. However, the technique does not directly approximate the updated matrix, but rather its previous low-rank approximation added to the new update, which needs justification. Further, the technique is still too slow for large information processing problems. We show that the technique minimizes the change in error per update, so if the error is small initially it remains small. We show that an updating algorithm for large sparse matrices should be sub-linear in the matrix dimension in order to be practical for large problems, and demonstrate a simple modification to the original technique that meets the requirements.

99 GENERAL AND MISCELLANEOUS↗

Experimental determination of radiated internal wave power without pressure field data

We present a method to determine, using only velocity field data, the time-averaged energy flux (J) and total radiated power P for two-dimensional internal gravity waves. Both (J) and P are determined from expressions involving only a scalar function, the stream function ψ. We test the method using data from a direct numerical simulation for tidal flow of a stratified fluid past a knife edge. The results for the radiated internal wave power given by the stream function method agree to within 0.5% with results obtained using pressure and velocity data from the numerical simulation. The results for the radiated power computed from the stream function agree well with power computed from the velocity and pressure if the starting point for the stream function computation is on a solid boundary, but if a boundary point is not available, care must be taken to choose an appropriate starting point. We also test the stream function method by applying it to laboratory data for tidal flow past a knife edge, and the results are found to agree with the direct numerical simulation. The supplementary material includes a Matlab code with a graphical user interface that can be used to compute the energy flux and power from two-dimensional velocity field data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Occupant comfort and behavior: High-resolution data from a 6-month field study of personal comfort systems with 37 real office workers

Personal Comfort Systems (PCS) provide individual occupants local heating and cooling to meet their comfort needs without affecting others in the same space. It saves energy by relaxing ambient temperature requirements for the HVAC system. Aside from these benefits, PCS offers a wealth of data that can describe how individuals interact with heating/cooling devices in their own environment. Recently developed Internet-connected PCS chairs have unlocked this opportunity by generating continuous streams of heating and cooling usage data, along with occupancy status and environmental measurements via embedded sensors. The data allow individuals' comfort and behavior to be learned, and can inform centralized systems to provide ‘just the right’ amount of conditioning to meet occupant needs. In summer 2016, we carried out a study with PCS chairs involving 37 occupants in an office building in California. During the study period, we collected >5 million chair usage data-points and 4500 occupant survey responses, as well as continuous measurements of environmental and HVAC system conditions. The data analysis shows that (1) local temperatures experienced by individual occupants vary quite widely across different parts of the building, even within the same thermal zone; (2) occupants often have different thermal preferences even under the same thermal conditions; (3) PCS control behavior can dynamically describe individuals' thermal preferences; (4) PCS chairs produce much higher comfort satisfaction (96%) than typically achieved in buildings. We conclude that PCS not only provide personalized comfort solutions but also offer individualized feedback that can improve comfort analytics and control decisions in buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Test Data Monitor

The National Security Campus (NSC) collects a large amount of test data used The National Security Campus (NSC) collects a large amount of test data used to accept high value and high rigor product. The data has been used historically to support root cause analysis when anomalies are detected in down-stream processes. The opportunity to use the data for predictive failure analysis however, had never been exploited. The primary goal of the Test Data Monitor (TDM) software is to provide automated capabilities to analyze data in near-real-time and report trends that foreshadow actual product failures. To date, the aerospace industry as a whole is challenged at utilizing collected data to the degree that modern technology allows. As a result of the innovation behind TDM, Honeywell is able to monitor millions of data points through a multitude of SPC algorithms continuously and autonomously so that our personnel resources can more efficiently and accurately direct their attention to suspect processes or features. TDM’s capabilities have been recognized by our U.S. Department of Energy National Nuclear Security Administration (NNSA) sponsor for potential use at other sites within the NNSA. This activity supports multiple initiatives including expectations of the NNSA and broader corporate goals that center around data-based quality controls on production.

Bosas, Joseph↗

Fault tolerance in an inner-outer solver: A GVR-enabled case study

Resilience is a major challenge for large-scale systems. It is particularly important for iterative linear solvers, since they take much of the time of many scientific applications. We show that single bit flip errors in the Flexible GMRES iterative linear solver can lead to high computational overhead or even failure to converge to the right answer. Informed by these results, we design and evaluate several strategies for fault tolerance in both inner and outer solvers appropriate across a range of error rates. We implement them, extending Trilinos’ solver library with the Global View Resilience (GVR) programming model, which provides multi-stream snapshots, multi-version data structures with portable and rich error checking/recovery. Lastly, experimental results validate correct execution with low performance overhead under varied error conditions.

97 MATHEMATICS AND COMPUTING↗

Spectral anisotropy of solar wind turbulence in the inertial range and dissipation range

Wavelet analysis is a tool that can simultaneously analyze spacecraft data in both time and frequency. This enables the magnetic energy spectrum (trace spectrum) to be measured for different angles between the local mean magnetic field B{sub 0} and the direction of the mean flow within a single data set. Examples obtained around solar minimum using Ulysses data at high latitudes and Stereo data for high-speed streams in the ecliptic plane show that the spectral exponent in the inertial range varies nearly monotonically from approximately 1.6 when B{sub 0} is perpendicular to the flow to 2.0 when B{sub 0} is parallel to the flow, roughly consistent with anisotropic theories of incompressible MHD turbulence. This and other new information about the 3D wavector spectrum of solar wind fluctuations in the inertial range and the dissipation range are briefly discussed.

79 ASTRONOMY AND ASTROPHYSICS↗