Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Knowledge Beacons: Web services for data harvesting of distributed biomedical knowledge

The continually expanding distributed global compendium of biomedical knowledge is diffuse, heterogeneous and huge, posing a serious challenge for biomedical researchers in knowledge harvesting: accessing, compiling, integrating and interpreting data, information and knowledge. In order to accelerate research towards effective medical treatments and optimizing health, it is critical that efficient and automated tools for identifying key research concepts and their experimentally discovered interrelationships are developed. As an activity within the feasibility phase of a project called “Translator” (https://ncats.nih.gov/translator) funded by the National Center for Advancing Translational Sciences (NCATS) to develop a biomedical science knowledge management platform, we designed a Representational State Transfer (REST) web services Application Programming Interface (API) specification, which we call a Knowledge Beacon. Knowledge Beacons provide a standardized basic API for the discovery of concepts, their relationships and associated supporting evidence from distributed online repositories of biomedical knowledge. This specification also enforces the annotation of knowledge concepts and statements to the NCATS endorsed the Biolink Model data model and semantic encoding standards (https://biolink.github.io/biolink-model/). Implementation of this API on top of diverse knowledge sources potentially enables their uniform integration behind client software which will facilitate research access and integration of biomedical knowledge.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1997

Examination of Soil Moisture Retrieval Using SIR-C Radar Data and a Distributed Hydrological Model

A major objective of soil moisture-related hydrological-research during NASA's SIR-C/X-SAR mission was to determine and compare soil moisture patterns within humid watersheds using SAR data, ground-based measurements, and hydrologic modeling. Currently available soil moisture-inversion methods using active microwave data are only accurate when applied to bare and slightly vegetated surfaces. Moreover, as the surface dries down, the number of pixels that can provide estimated soil moisture by these radar inversion methods decreases, leading to less accuracy and, confidence in the retrieved soil moisture fields at the watershed scale. The impact of these errors in microwave- derived soil moisture on hydrological modeling of vegetated watersheds has yet to be addressed. In this study a coupled water and energy balance model operating within a topographic framework is used to predict surface soil moisture for both bare and vegetated areas. In the first model run, the hydrological model is initialized using a standard baseflow approach, while in the second model run, soil moisture values derived from SIR-C radar data are used for initialization. The results, which compare favorably with ground measurements, demonstrate the utility of combining radar-derived surface soil moisture information with basin-scale hydrological modeling.

Hsu, A. Y.↗
NASA NTRS1996

Monitoring Data-Structure Evolution in Distributed Message-Passing Programs

Monitoring the evolution of data structures in parallel and distributed programs, is critical for debugging its semantics and performance. However, the current state-of-art in tracking and presenting data-structure information on parallel and distributed environments is cumbersome and does not scale. In this paper we present a methodology that automatically tracks memory bindings (not the actual contents) of static and dynamic data-structures of message-passing C programs, using PVM. With the help of a number of examples we show that in addition to determining the impact of memory allocation overheads on program performance, graphical views can help in debugging the semantics of program execution. Scalable animations of virtual address bindings of source-level data-structures are used for debugging the semantics of parallel programs across all processors. In conjunction with light-weight core-files, this technique can be used to complement traditional debuggers on single processors. Detailed information (such as data-structure contents), on specific nodes, can be determined using traditional debuggers after the data structure evolution leading to the semantic error is observed graphically.

Sarukkai, Sekhar R.↗
NASA NTRS2015

Lidar-Radiometer Inversion Code (LIRIC) for the Retrieval of Vertical Aerosol Properties from Combined Lidar Radiometer Data: Development and Distribution in EARLINET

This paper presents a detailed description of LIRIC (LIdar-Radiometer Inversion Code)algorithm for simultaneous processing of coincident lidar and radiometric (sun photometric) observations for the retrieval of the aerosol concentration vertical profiles. As the lidar radiometric input data we use measurements from European Aerosol Re-search Lidar Network (EARLINET) lidars and collocated sun-photometers of Aerosol Robotic Network (AERONET). The LIRIC data processing provides sequential inversion of the combined lidar and radiometric data by the estimations of column-integrated aerosol parameters from radiometric measurements followed by the retrieval of height-dependent concentrations of fine and coarse aerosols from lidar signals using integrated column characteristics of aerosol layer as a priori constraints. The use of polarized lidar observations allows us to discriminate between spherical and non-spherical particles of the coarse aerosol mode. The LIRIC software package was implemented and tested at a number of EARLINET stations. Inter-comparison of the LIRIC-based aerosol retrievals was performed for the observations by seven EARLNET lidars in Leipzig, Germany on 25 May 2009. We found close agreement between the aerosol parameters derived from different lidars that supports high robustness of the LIRIC algorithm. The sensitivity of the retrieval results to the possible reduction of the available observation data is also discussed.

Chaikovsky, A.↗
NASA NTRS1995

Model-based VQ for image data archival, retrieval and distribution

An ideal image compression technique for image data archival, retrieval and distribution would be one with the asymmetrical computational requirements of Vector Quantization (VQ), but without the complications arising from VQ codebooks. Codebook generation and maintenance are stumbling blocks which have limited the use of VQ as a practical image compression algorithm. Model-based VQ (MVQ), a variant of VQ described here, has the computational properties of VQ but does not require explicit codebooks. The codebooks are internally generated using mean removed error and Human Visual System (HVS) models. The error model assumed is the Laplacian distribution with mean, lambda-computed from a sample of the input image. A Laplacian distribution with mean, lambda, is generated with uniform random number generator. These random numbers are grouped into vectors. These vectors are further conditioned to make them perceptually meaningful by filtering the DCT coefficients from each vector. The DCT coefficients are filtered by multiplying by a weight matrix that is found to be optimal for human perception. The inverse DCT is performed to produce the conditioned vectors for the codebook. The only image dependent parameter used in the generation of codebook is the mean, lambda, that is included in the coded file to repeat the codebook generation process for decoding.

Manohar, Mareboyana↗
NASA NTRS2007

Understanding Local Structure Globally in Earth Science Remote Sensing Data Sets

Empirical probability distributions derived from the data are the signatures of physical processes generating the data. Distributions defined on different space-time windows can be compared and differences or changes can be attributed to physical processes. This presentation discusses on ways to reduce remote sensing data in a way that preserves information, focusing on the rate-distortion theory and using the entropy-constrained vector quantization algorithm.

massive data sets↗
DOE OSTI2022

Feature Extraction for Pipeline Defects Inspection Based Upon Distributed Acoustic Fiber Optic Sensing Data

Fiber-optic distributed acoustic sensing (DAS) is becoming an increasingly important tool for real-time monitoring of energy and civil infrastructure structural health such as pipelines. We present a systematic theoretical study of the potential for DAS to be directly coupled with guided ultrasonic waves typically used in conventional acoustic non-destructive evaluation (NDE) methods for real-time pipeline health monitoring. We are referring to this innovative new NDE technique as ultrasonic guided wave and optical fiber sensor fusion. In the practical application of DAS coupled with guided ultrasonic waves, the structural design of (1) the specific guided waves excited, (2) the physical installation of the acoustic transducers and the fiber optic sensors, and (3) the functional performance specifications (gauge length, sensitivity, Etc.) of fiber optic DAS have an important influence on overall capabilities of the monitoring system. Meanwhile, physics-based analysis of acoustic waves is still a challenge due to the complex nature of the Lamb wave when it propagates, scatters, and disperses in the presence of structural defects. In this work, we simulate carbon steel pipes relevant for oil and gas pipeline applications with diameters of approximately 6-12” and wall thickness of 0.5” as the objects to be monitored. By establishing and implementing these capabilities, we seek to pursue an in-depth study on structural parameter optimization of DAS network, measurement range, and signal processing with an ultimate goal of increasing the sensitivity and efficacy of DAS to defect identification for various modes of corrosion expected in practice. To study the characteristics of scattered acoustic waves and performance of DAS for defect identification, we simulated the response of DAS for multiple pipe structures, defect types, and DAS sensor network configuration using finite element software Ansys, then the properties of signal response are extracted to construct defect-sensitive features. The raw data simulated, and the associated features extracted can ultimately be utilized as annotated training data to benchmark various designs for DAS applications, guided acoustic excitation sources, and learning model parameters to enhance early detection of potentially problematic defects.

Pipeline Defects Inspection, Fiber-optic sensors, ↗
DOE OSTI2024

Out-of-Distribution Detection and Radiological Data Monitoring Using Statistical Process Control

Abstract Machine learning (ML) models often fail with data that deviates from their training distribution. This is a significant concern for ML-enabled devices as data drift may lead to unexpected performance. This work introduces a new framework for out of distribution (OOD) detection and data drift monitoring that combines ML and geometric methods with statistical process control (SPC). We investigated different design choices, including methods for extracting feature representations and drift quantification for OOD detection in individual images and as an approach for input data monitoring. We evaluated the framework for both identifying OOD images and demonstrating the ability to detect shifts in data streams over time. We demonstrated a proof-of-concept via the following tasks: 1) differentiating axial vs. non-axial CT images, 2) differentiating CXR vs. other radiographic imaging modalities, and 3) differentiating adult CXR vs. pediatric CXR. For the identification of individual OOD images, our framework achieved high sensitivity in detecting OOD inputs: 0.980 in CT, 0.984 in CXR, and 0.854 in pediatric CXR. Our framework is also adept at monitoring data streams and identifying the time a drift occurred. In our simulations tracking drift over time, it effectively detected a shift from CXR to non-CXR instantly, a transition from axial to non-axial CT within few days, and a drift from adult to pediatric CXRs within a day—all while maintaining a low false positive rate. Through additional experiments, we demonstrate the framework is modality-agnostic and independent from the underlying model structure, making it highly customizable for specific applications and broadly applicable across different imaging modalities and deployed ML models.

Zamzmi, Ghada↗
DOE OSTI2026

WHONDRS Surface Water Geochemistry and Organic Matter Characterization Data from Streams Distributed across Latin America

This dataset supports a broader study examining global transferability of stream biogeochemistry and was generated in collaboration with the MicroSudAqua (µSudAqua) network (https://microsudaqua.netlify.app/en/). The dataset provides surface water geochemistry (dissolved organic carbon, total dissolved nitrogen, cations) and organic matter characterization (FTICR-MS) from streams in Argentina, Brazil, Chile, and Colombia. Samples were collected across stream orders (1st to 6th order) within five basins. Related data were collected and will be published separately in collaboration with the µSudAqua network. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to this readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) a folder of field photos; (2) a folder of surface water sample data, (3) a folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data; (4) file-level metadata; (5) data dictionary; (6) field metadata; (7) readme; (8) international generic sample number (IGSN) mapping file; and (9) field protocol. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (2) total dissolved nitrogen data and averages; (3) anions and averages; (4) methods codes; (5) FTICR-MS methods; and (15) a subfolder of 9.4 Tesla (9.4T) FTICR-MS data. This folder contains the processed data and three subfolders, one containing the .xml files, one containing the water CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .d, .html, .Rmd, .py, .cal, .json, .jpg, .jpeg, .png, .mov, or .mp4.

Anions↗
NASA NTRSDate not supplied

The Future of Oaks in the Santa Monica Mountains: A Case Study in Using Remote Sensing Data for Species Distributions Models

The Woolsey Fire began on November 8, 2018, and lasted for almost two weeks, during which it burned almost 100,000 acres of valuable landscape and habitat, including a vast area of woodland. The persistence of key woodland species provides aesthetic, monetary, and ecological value to the landscape through carbon sequestration, air temperature moderation, and erosion mitigation, among other ecosystem services. This study investigated the impact of the Woolsey Fire on native woodland species distributions and identified areas suitable for restoration within the Santa Monica Mountains National Recreation Area. The team partnered with the Resource Conservation District of the Santa Monica Mountains; National Park Service, Santa Monica Mountains National Recreation Area; California Department of Parks and Recreation, Los Angeles County Division; County of Los Angeles Fire Department, Prevention Services Bureau, Forestry Division; County of Los Angeles Department of Regional Planning; and the University of Montana. The Earth observations used include data from Landsat 8 Operational Land Imager, NASA ER-2 Jet Airborne Visible InfraRed Imaging Spectrometer, Shuttle Radar Topography Mission, and RapidEye. The team produced maps of burn severity from the Woolsey Fire, its impact on plant species distributions, and habitat suitability projections for 2050 and 2099 to assist partners in prioritizing areas for restoration. A plant community classification was successfully created using Multiple Endmember Spectral Mixture Analysis (MESMA). Overall accuracy was assessed at 90.54% by comparing the classification to validation pixels derived from ground truth information provided by our partners.

Roger Ly↗
NASA NTRS1995

Experimental evaluation of dynamic data allocation strategies in a distributed database with changing workloads

Traditionally, allocation of data in distributed database management systems has been determined by off-line analysis and optimization. This technique works well for static database access patterns, but is often inadequate for frequently changing workloads. In this paper we address how to dynamically reallocate data for partionable distributed databases with changing access patterns. Rather than complicated and expensive optimization algorithms, a simple heuristic is presented and shown, via an implementation study, to improve system throughput by 30 percent in a local area network based system. Based on artificial wide area network delays, we show that dynamic reallocation can improve system throughput by a factor of two and a half for wide area networks. We also show that individual site load must be taken into consideration when reallocating data, and provide a simple policy that incorporates load in the reallocation decision.

Brunstrom, Anna↗
DOE OSTI2021

Estimation of Aerosol Columnar Size Distribution from Spectral Extinction Data in Coastal and Maritime Environment

Aerosol columnar size distributions (SDs) are commonly provided by aerosol inversions based on measurements of both spectral extinction and sky radiance. These inversions developed for a fully clear sky offer few SDs for areas with abundant clouds. Here, we estimate SDs from spectral extinction data alone for cloudy coastal and maritime regions using aerosol refractive index (RI) obtained from chemical composition data. Our estimation involves finding volume and mean radius of lognormally distributed modes of an assumed bimodal size distribution through fitting of the spectral extinction data. We demonstrate that vertically integrated SDs obtained from aircraft measurements over a coastal site have distinct seasonal changes, and these changes are captured reasonably well by the estimated columnar SDs. We also demonstrate that similar seasonal changes occur at a maritime site, and columnar SDs retrieved from the combined extinction and sky radiance measurements are approximated quite well by their extinction only counterparts (correlation exceeds 0.9) during a 7-year period (2013–2019). The level of agreement between the estimated and retrieved SDs depends weakly on wavelength selection within a given spectral interval (roughly 0.4–1 µm). Since the extinction-based estimations can be performed frequently for partly cloudy skies, the number of periods where SDs can be found is greatly increased.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1997

Comparison of OARE Accelerometer Data with Dopant Distribution in Se-Doped GaAs Crystals Grown During USML-1

Experiments were conducted in the crystal growth furnace (CGF) during the first United States Microgravity Laboratory (USML-1), the STS-50 flight of the Space Shuttle Columbia, to determine the segregation behavior of selenium in bulk GaAs in a microgravity environment. After the flight, the selenium-doped GaAs crystals were sectioned, polished, and analyzed to determine the free carrier concentration as a function of position, One of the two crystals initially exhibited an axial concentration profile indicative of diffusion controlled growth, but this profile then changed to that predicted for a complete mixing type growth. An analytical model, proposed by Naumann [R.J. Naumann, J. Crystal Growth 142 (1994) 253], was utilized to predict the maximum allowable microgravity disturbances transverse to the growth direction during the two different translation rates used for each of the experiments. The predicted allowable acceleration levels were 4.86 microgram for the 2.5 micrometers/s furnace translation rate and 38.9 microgram for the 5.0 micrometers/s rate. These predicted values were compared to the Orbital Acceleration Research Experiment (OARE) accelerometer data recorded during the crystal growth periods for these experiments. Based on the analysis of the OARE acceleration data and utilizing the predictions from the analytical model, it is concluded that the change in segregation behavior was not caused by any acceleration events in the microgravity environment.

Moskowitz, Milton E.↗
DOE OSTI2022

An efficient method to propagate model uncertainty when inverting seismic data for time domain seismic moment tensors

SUMMARY We present a computationally efficient method to approximately propagate uncertainty when linearly inverting seismic data for point source, time variable moment tensor components. The method is based on the assumption that the data residual, given by the difference between the observed seismic data and the data predicated by a linear inversion, contains the effects of both data and model uncertainty. Our method uses a distribution of data residuals, added directly to the data, in a pseudo-Monte Carlo scheme. Using the assumption that the data residual is a stochastic process, we use the well-known Karhunen–Loève (KL) theorem to construct a distribution of data residuals, where the required basis functions are constructed using Fourier series. The Fourier series are scaled by a product of a random variable and the real-valued spectral amplitudes of the original data residual’s spectrum. Thus, the Fourier series and spectral amplitudes are eigenfunction-eigenvalue pairs used in the KL-based construction of data residual distribution. Using tests with synthetic data, we show that our method compares closely with a Finite Difference Monte Carlo (FDMC) method that we presented previously. More importantly, the method presented here is computationally several orders of magnitude faster than our previous FDMC method, and requires no a priori assumptions of model and/or data uncertainty.

Poppeliers, Christian (ORCID:0000000159526849)↗
NASA NTRS1991

WetNet: Using SSM/I data interactively for global distribution of rainfall and precipitable water

By completing hardware installation, preparing for comparative studies of SSM/I, radar, and lightning data, it is believed that this will be a powerful combination for evaluating the global distribution of tropical rainfall, and the vertical distribution of latent heating, with strong application to algorithms for use on TRMM, EOS-A, and future GOES spacecraft. Potential data bases will be surveyed, about 5 case studies with surface rainfall, radar, lightning, and sounding data will be identified. SSM/I algorithms will be used to identify convective regions of MCSs. A catalog will be developed of the global profile of heavy tropical rainfall, and how these zones are organized within larger tropical weather systems. Beginning with the first few months of SSM/I data distributed over WetNet, SSM/I radiances will be compared with TOVS radiance (moisture and thermal) and OLR observations. The purpose is to improve understanding of how real world water vapor profiles in the tropical atmosphere are perceived by SSM/I precipitable water algorithm and, at the same time, by the TOVS water vapor channel.

Zipser, Edward J.↗
NASA NTRS1986

Global distribution of SAGE II data products

Techniques employed to map the global distribution of stratospheric aerosols using SAGE II data at four wavelengths (385, 453, 525 and 1020 microns) are described. The methods were devised to integrate the 900 data points scanned by the instrument with account taken of the different times the vertical profiles were obtained. The data have been used to calculate the O3, H2O, NO2 and aerosol optical thicknesses. Sample details of data gathered during April 1985 are discussed, with emphasis on difficulties being encountered in analyzing the differences being observed in the dawn and dusk data.

Rind, D.↗
DOE OSTI2021

Distribution Grid Modeling Using Smart Meter Data

The knowledge of distribution grid models, including topologies and line impedances, is essential for grid monitoring, control and protection. However, such information is often unavailable, incomplete or outdated. The increasing deployment of smart meters (SMs) provides a unique opportunity to tackle this issue. This paper proposes a two-stage framework for distribution grid modeling using SM data. In the first stage, the network topology is identified by reconstructing a weighted Laplacian matrix of distribution networks. In the second stage, a least absolute deviations (LAD) regression model is developed for estimating line impedance of a single branch based on the nonlinear (inverse) power flow model, wherein a conductor library is leveraged to narrow down the solution space. The LAD regression model is originally a mixed-integer nonlinear program whose continuous relaxation is still non-convex. Furthermore, we specially address its convex relaxation and discuss the exactness. The modified regression model is then embedded within a bottom-up sweep algorithm to achieve the identification across the network in a branch-wise manner. Numerical results on the IEEE 13-bus, 37-bus and 69-bus test feeders validate the effectiveness of the proposed methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗