Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “in situ data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS2000

Case Study Analyses of the SUCCESS DC-8 Scanning Lidar Database

Under project SUCCESS (Subsonic Aircraft Contrail and Cloud Effects Special Study) funded by the Atmospheric Effects of Aviation Program, SRI International (SRI) developed an angular scanning backscatter lidar for operation on the NASA DC-8 research aircraft and deployed the scanning lidar during the SUCCESS field campaign. The primary purpose of the lidar was to generate real-time video displays of clouds and contrails above, ahead of, and below the DC-8 as a means to help position the aircraft for optimum cloud and contrail sampling by onboard in situ sensors, and to help extend the geometrical domain of the in situ sampling records. A large, relatively complex lidar database was collected and several data examples were processed to illustrate the value of the lidar data for interpreting the other data records collected during SUCCESS. These data examples were used to develop a journal publication for the special SUCCESS Geophysical Research Letters issue. The data examples justified data analyses of a larger part of the DC-8 lidar database and is the objective of the current study. Efficient processing of the SUCCESS DC-8 scanning lidar database required substantial effort to enhance hardware and software components of the data system that was used for the initial analyses. MATLAB instructions are used to generate altitude and distance color-coded lidar displays corrected for effects introduced by aircraft pitch and forward movement during an angular scan time interval. Onboard in situ sensor atmospheric measurements are propagated to distances ahead of the DC-8 using recorded aircraft velocity so that they can be plotted on the lidar displays for comparison with lidar remotely observed aerosol distributions. Resulting lidar and in situ sensor polar scan displays over extended sampling intervals are integrated into a time series movie format for 36 case studies. Contrails and clouds were detected to ranges of 15 km by the forward-viewing angular scanning lidar and were progressively mapped as the aircraft approached and penetrated them. Near aircraft lidar observations were much better correlated with in situ sensor observations than lidar observations at greater distances ahead of the aircraft. The major cause of this difference was thought to be the about 2 deg. offset of the lidar viewing direction from the flight direction. Contrail spatial distributions were not of the quality obtainable from ground-based lidar observations. This results because contrails tend to become horizontally stratified, vertical distance between angular lidar observations increases with increased distance from the aircraft, and erratic aircraft motions during an angular scan. The most useful lidar observations were made with lidar viewing directions of vertically upward or vertically downward. These provided real-time information on aircraft altitudes to achieve optimum in situ cloud and contrail sampling. At sampling altitudes, the forward viewing angular scanning observations were useful for fine-tuning the aircraft altitude for cloud and contrail penetration. Best information on cloud and contrail properties were obtained from vertically directed lidar observations as the aircraft performed a series of upward and downward penetrations of contrails. This operational mode was especially well suited for lidar and radiometric evaluation of cloud and contrail optical and radiative properties. The vertical viewing lidar detected ice crystals thought to be precipitating from an aircraft contrail and their scavenging by a cirrus cloud layer. The lidar display indicates that the crystals are effective for increasing cirrus cloud density. Vertical angular scanning observations can evaluate the sharp decrease in lidar backscatter for small off-vertical viewing directions that result from horizontally aligned ice crystals and perhaps can provide additional information on crystal shapes. The about 2 deg. offset of the lidar viewing direction from the flight direction is thought to have greatly degraded the forward-viewing angular scanning observations and this mode of operation was not fully evaluated. However, the reasoning for this capability remains valid and the angular scan presentations collected during this program justifies modification of the lidar pod for true forward direction lidar viewing during future cloud and contrail studies.

Uthe, Edward E.↗
NASA NTRS2002

Clouds in the Winter Arctic Tropopause Region

Water vapor in the winter arctic tropopause region is important because, after the tropical tropopause region, the winter arctic tropopause has the coldest temperatures in the tropospheric northern hemisphere. This suggests the potential for cloud formation that can remove water vapor from a part of the atmosphere where radiatively active gases (such as water) exert a disproportionate influence on the earth's radiation budget. Previous work by the same authors has shown that this cloud formation extends into the stratosphere, with 20% of the parcels having ozone values of 300-350 ppbv experiencing ice saturation in any given 10 day period period during the late winter. In fact, temperatures are cold enough that 5-10% of the parcels experience saturation even if the water content is below the prevailing stratospheric value of 5 ppmv. This work describes a case study of clouds observed by aircraft near the winter arctic tropopause during the SAGE Ozone Loss and Validation Experiment (SOLVE). This provided a unique opportunity to examine dehydration processes in this region since in situ water, tracer, cloud particle, and meteorological data were all available simultaneously. During this period, temperatures were cold enough at the tropopause to produce saturation mixing ratios of 3-4 ppmv. Thus, clouds were actually observed within the stratosphere. Back trajectories indicate that the air in these clouds came from lower latitudes and altitudes. The study describes the nature of the clouds, the history of the air, and the possible implications for the upper tropospheric water budget.

Pfister, Leonhard↗
NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly.

Lynnes, Christopher↗
NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly. Reviewed by Mark McInerney ESDIS Deputy Project Manager.

Lynnes, Christopher↗
NASA NTRS2022

Remote Sensing of Aerosol Water Fraction, Dry Size Distribution and Soluble Fraction Using Multi-Angle, Multi-Spectral Polarimetry

A framework to infer volume water fraction, soluble fraction and dry size distributions of fine mode aerosol from multi-angle, multi-spectral polarimetry retrievals of column-averaged ambient aerosol properties is presented. The method is applied to observations of the Research Scanning Polarimeter (RSP) obtained during two NASA aircraft campaigns, namely the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) and the Cloud, Aerosol, and Monsoon Processes-Philippines Experiment (CAMP2Ex). All aerosol retrievals are statistically evaluated using in situ data. Volume water fraction is inferred from the retrieved ambient real part of the refractive index, assuming a dry refractive index of 1.54 and by applying a volume mixing rule to obtain the effective ambient refractive index. The uncertainties in inferred volume water fraction resulting from this simplified model are discussed and estimated to be lower than 0.2 and decreasing with increasing volume water fraction. The daily mean retrieved volume water fractions correlate well with the in situ values with a mean absolute difference of 0.09. Polarimeter-retrieved ambient effective radius for daily data is shown to increase as a function of volume water fraction as expected. Furthermore, the effective variance of the size distributions also increases with increasing effective radius, which we show is consistent with an external mixture of soluble and insoluble aerosol. The relative variations of effective radius and variance over an observation period are then used to estimate the soluble fraction of the aerosol. Daily results of soluble fraction correlate well with in situ observed sulfate mass fraction with a correlation coefficient of 0.79. Subsequently, inferred water and soluble fractions are used to derive dry fine-mode size distributions from their ambient counterparts. While dry effective radii obtained in situ and from RSP show similar ranges, in situ values are generally substantially smaller during the ACTIVATE deployments, which may be due to biases in RSP retrievals or in the in situ observations, or both. Both RSP and in situ observations indicate the dominance of aerosol with low hygroscopicity during the ACTIVATE and CAMP2Ex campaigns. Furthermore, RSP indicates a high degree of external mixing of particles with low and high hygroscopicity. These retrievals of fine mode water volume fraction and soluble fraction may be used for the evaluation of water uptake in atmospheric models. Furthermore, the framework allows to estimate the variation in the concentration of fine-mode aerosol larger than a specific dry radius limit, which can be used as a proxy for the variation in cloud condensation nucleus concentrations. This framework may be applied to multi-angle, multi-spectral satellite data expected to be available in the near future.

Bastiaan Van Diedenhoven↗
DOE OSTI2022

Demonstrating the viability of Lagrangian in situ reduction on supercomputers

Performing exploratory analysis and visualization of large-scale time-varying computational science applications is challenging due to inaccuracies that arise from under-resolved data. In recent years, Lagrangian representations of the vector field computed using in situ processing are being increasingly researched and have emerged as a potential solution to enable exploration. However, prior works have offered limited estimates of the encumbrance on the simulation code as they consider “theoretical” in situ environments. Further, the effectiveness of this approach varies based on the nature of the vector field, benefitting from an in-depth investigation for each application area. With this study, an extended version of Sane et al. (2021), we contribute an evaluation of Lagrangian analysis viability and efficacy for simulation codes executing at scale on a supercomputer. We investigated previously unexplored cosmology and seismology applications as well as conducted a performance benchmarking study by using a hydrodynamics mini-application targeting exascale computing. Here, to inform encumbrance, we integrated in situ infrastructure with simulation codes, and evaluated Lagrangian in situ reduction in representative homogeneous and heterogeneous HPC environments. To inform post hoc accuracy, we conducted a statistical analysis across a range of spatiotemporal configurations as well as a qualitative evaluation. Additionally, our study contributes cost estimates for distributed-memory post hoc reconstruction. In all, we demonstrate viability for each application — data reduction to less than 1% of the total data via Lagrangian representations, while maintaining accurate reconstruction and requiring under 10% of total execution time in over 90% of our experiments.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

The State of Remote Sensing Capabilities of Cascading Hazards over High Mountain Asia

Cascading hazard processes refer to a primary trigger such as heavy rainfall, seismic activity, or snow melt, followed by a chain or web of consequences that can cause subsequent hazards influenced by a complex array of preconditions and vulnerabilities. These interact in multiple ways and can have tremendous impacts on populations proximate to or downstream of these initial triggers. High Mountain Asia (HMA) is extremely vulnerable to cascading hazard processes given the tectonic, geomorphologic, and climatic setting of the region, particularly as it relates to glacial lakes. Given the limitations of in situ surveys in steep and often inaccessible terrain, remote sensing data are a valuable resource for better understanding and quantifying these processes. The present work provides a survey of cascading hazard processes impacting HMA and how these can be characterized using remote sensing sources. We discuss how remote sensing products can be used to address these process chains, citing several examples of cascading hazard scenarios across HMA. This work also provides a perspective on the current gaps and challenges, community needs, and view forward towards improved characterization of evolving hazards and risk across HMA.

Glacial lake outburst floods↗
DOE OSTI2023

Challenges in studying water fluxes within the soil-plant-atmosphere continuum: A tracer-based perspective on pathways to progress

Tracing and quantifying water fluxes in the hydrological cycle is crucial for understanding the current state of ecohydrological systems and their vulnerability to environmental change. Especially the interface between ecosystems and the atmosphere that is strongly mediated by plants is important to meaningfully describe ecohydrological system functioning. Many of the dynamic interactions generated by water fluxes between soil, plant and the atmosphere are not well understood, which is partly due to a lack of interdisciplinary research. This opinion paper reflects the outcome of a discussion among hydrologists, plant ecophysiologists and soil scientists on open questions and new opportunities for collaborative research on the topic "water fluxes in the soil-plant-atmosphere continuum" especially focusing on environmental and artificial tracers. We emphasize the need for a multi-scale experimental approach, where a hypothesis is tested at multiple spatial scales and under diverse environmental conditions to better describe the small-scale processes (i.e., causes) that lead to large-scale patterns of ecosystem functioning (i.e., consequences). Novel in-situ, high-frequency measurement techniques offer the opportunity to sample data at a high spatial and temporal resolution needed to understand the underlying processes. Here we advocate for a combination of long-term natural abundance measurements and event-based approaches. Multiple environmental and artificial tracers, such as stable isotopes, and a suite of experimental and analytical approaches should be combined to complement information gained by different methods. Virtual experiments using process-based models should be used to inform sampling campaigns and field experiments, e.g., to improve experimental designs and to simulate experimental outcomes. On the other hand, experimental data are a pre-requisite to improve our currently incomplete models. Interdisciplinary collaboration will help to overcome research gaps that overlap across different earth system science fields and help to generate a more holistic view of water fluxes between soil, plant and atmosphere in diverse ecosystems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Low Field NMR Relaxometry Characterization of Water Adsorption in Corn Stover Anatomical Fractions

Low magnetic field 2 MHz NMR relaxometry is applied to measure water adsorption processes of intact corn stover anatomical fractions. Comparison with high magnetic field 250 MHz relaxometry of milled corn stover fractions as a function of water activity, i.e., relative humidity, provides insight into the microstructural changes of the biomass and varying water molecular dynamics during adsorption. As a result, the data presented establish a basis for low field portable NMR of biomass in situ in field and processing environments.

09 BIOMASS FUELS↗
DOE OSTI2021

Atmospheric Radiation Measurement (ARM) User Facility 2020 Decadal Vision

The Atmospheric Radiation Measurement (ARM) user facility was established in 1989 by the U.S Department of Energy (DOE) Office of Biological and Environmental Research (BER) to provide an observational basis for studying the Earth’s climate. ARM began collecting observations in 1992 and was designated a user facility in 2003. The facility includes a network of extensively instrumented long-term fixed-location observatories and mobile facilities. The ARM facility also includes an aerial component to augment these ground-based measurements. Because of the diversity of in situ and remotely obtained observations, ARM’s data management infrastructure is designed to collect, process, and deliver data to the research community.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2018

Digital Technologies at NASA for Science and Engineering

While scientific and engineering advancements used to rely primarily on theoretical studies and physical experiments, today digital technology enabled by petaflops-scale supercomputers is an equal, if not a greater, contributor to such achievements. In addition, computational modeling and simulation serves as a predictive tool that is not otherwise available. As a result, the use of high performance computing is integral to NASA's work in all mission areas such as space exploration, aeronautics, and scientific discovery. But traditional supercomputing alone is not sufficient for all of the space agency's needs. The success of many NASA missions depends on solving complex computing challenges, some of which are NP-hard (decision theory) if using classical solution methods. Quantum computing promises an unprecedented ability to solve such intractable problems by harnessing quantum mechanical effects such as tunneling, superposition, and entanglement. Another disruptive digital technology is neuromorphic computing that uses brain-inspired lessons to generate new architectures that are much more energy efficient, and capable of massive parallel processing and learning in-situ. Finally, with large amounts of observational and computational data sets, the opportunities of big data and data analytics can be leveraged to enable deep learning and knowledge discovery - it's all a massive digital transformation. This talk will be an overview how NASA utilizes digital technologies for its science and engineering efforts.

Biswas, Rupak↗
NASA NTRS2023

Fine particulate concentrations over East Asia derived from aerosols measured by the Advanced Himawari Imager using machine learning

Fine particulate matter with a diameter below 2.5 μm (PM 2.5 ) is deleterious to the cardiovascular and respiratory systems. It is often difficult to assess the effects of PM 2.5 on human health over regions with limited ground monitoring sites, especially in East Asia. As an alternative, we estimated near-surface PM 2.5 concentrations by analyzing Advanced Himawari Imager (AHI) Yonsei Aerosol Retrieval (YAER) products. This study incorporates daytime data for East Asia covering the Korean Peninsula, China, Japan, Southeast Asia, and southern Mongolia. We collocated AHI YAER product pixels with meteorological, land-cover, and other ancillary data for the period from March 2018 to February 2019. To estimate PM 2.5 concentrations over wide areas spanning many countries displaying various relationships between aerosol optical depth and PM 2.5 , monthly models were developed by considering both the spatial and temporal characteristics of ground-based PM 2.5 measurements. Random forest machine learning model estimated ground-level mass concentrations of PM 2.5 ; subsequent 10-fold cross validation (CV) yielded a CV R 2 value of 0.81 and a CV root mean squared error (RMSE) of 12.3 μg m -3 . We investigated the spatial pattern of PM 2.5 concentrations over multiple countries and seasonal variation in PM 2.5 concentrations. Diurnal variation of a severe PM 2.5 event in the Korean Peninsula was investigated as a case study. The model captured the extremely heterogeneous spatial distribution of PM 2.5 concentrations peaked around local noon. To measure the capability of the developed model to estimate PM 2.5 concentrations in areas with few in-situ data, its predictive performance was evaluated using a dataset independent of the training process with an R 2 of 0.60 and RMSE of 8.18 μg m −3 . This study demonstrates the potential for satellite-based PM 2.5 estimation for areas with insufficient measuring stations.

Pm2.5↗
DOE OSTI2025

2025 Peregrine in-situ monitoring and training dataset for laser powder bed fusion and binder jet printers

Peregrine, a software tool developed at Oak Ridge National Laboratory (ORNL), was used to collect and analyze in-situ monitoring (ISM) data from a Concept Laser M2 (Colibrium Additive) laser powder bed fusion (L-PBF) printer and an ExOne Innovent (Desktop Metal) binder jet printer. Data for four builds (print jobs) were saved to HDF5 (high performance data) files for release. Additionally, process anomalies were annotated by the authors across 37 image stacks (i.e., print layers) and are also provided as HDF5 files.

36 MATERIALS SCIENCE↗
DOE OSTI2020

A terminology for in situ visualization and analysis systems

The term "in situ processing" has evolved over the last decade to mean both a specific strategy for visualizing and analyzing data and an umbrella term for a processing paradigm. The resulting confusion makes it difficult for visualization and analysis scientists to communicate with each other and with their stakeholders. To address this problem, a group of over 50 experts convened with the goal of standardizing terminology. This paper summarizes their findings and proposes a new terminology for describing in situ systems. An important finding from this group was that in situ systems are best described via multiple, distinct axes: integration type, proximity, access, division of execution, operation controls, and output type. This paper discusses these axes, evaluates existing systems within the axes, and explores how currently used terms relate to the axes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Scalable, In-situ Data Clustering Data Analysis for Extreme Scale Scientific Computing (Final Report)

The objective of this project is to address challenges in the design and development of scalable in-situ data clustering and analytics algorithms and software. Our goal is to develop parallel software consisting of a set of spatio-temporal data clustering and anomaly detection functions, both of which are very important for large-scale analysis and have wide applicability for in-situ runs as well as post-processing analysis. Our design principles for in-situ analysis consider the following: (1) identify parts of the computation can be done close to the data within the nodes, while it is still in memory; (2) extract analysis components can (and should) be performed in remote staging and analysis nodes; (3) develop error-bound approximation methods for applications tolerable for small errors; (4) identify the type of derived distributions and statistics, for spatio-temporal data, that can be kept locally in order to both accelerate computations and meet energy constraints in subsequent iterations and phases; (5) use a self-describing data format so that data can be consistent and understood among local storage (memory and SSDs) and at staging and analysis nodes, thereby providing portability and flexibility; (6) develop service-oriented functions that can schedule in-situ and post-hoc analysis tasks based on the dynamic requirements of applications. Our development focus is to produce the parallel data analysis software/library that will be scalable, reusable, extensible, and generic for applications in different disciplines. The software will be able to run in-situ with the simulations as well as post-hoc analysis. This approach will satisfy many synergistic requirements for data intensive applications executed on data coming from instruments and experiments. In particular, the proposed multilevel approach is directly applicable to perform design tradeoffs for running part of the algorithms near the instruments and the rest on remote (analysis) systems.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

MAMS: High resolution atmospheric/surface properties

The Multispectral Atmospheric Mapping Sensor (MAMS) has been used for a number of investigations over the last 8 years. It has served as the basis for retrieval technique development, for atmospheric process studies, and to retrieve geophysical parameters at the surface (land and ocean). It was used most recently to collect high resolution visible and infrared imagery for the CaPE and STORMFEST experiments during FY91 and FY92. The Wildfire spectrometer (similar to MAMS but with different infrared channels) was also used during STORMFEST. The goals of MAMS during CaPE were: (1) to collect MAMS data to support the overall CaPE science objectives; (2) refine techniques for the retrieval of atmospheric moisture and surface geophysical parameters; (3) map the moisture distributions associated with the sea breeze front over Florida with MAMS ancillary satellite data on a case study basis; (4) identify surface features which may serve as local sources of heat and moisture and influence preferential convective regions over Florida, and (5) process MAMS data and derived relevant geophysical parameters to support other CaPE investigators. For STORMFEST both the MAMS and Wildfire spectrometers were used to collect unique multispectral data to study atmospheric processes. The Wildfire objectives were: (1) to collect high quality Wildfire data in conjunction with other in situ and remote measurements available during the STORMFEST field phase (1 February - 15 March 1992); (2) to develop algorithms to retrieve total ozone content and compare with those from TOMS and HIRS (IR); and (3) along with water vapor imagery, use the ozone data to better understand the 3-dimensional structure and dynamics of jet streaks and frontal systems in a case study investigation. The MAMS objectives were: (1) collect MAMS data to support a GSFC investigation of gravity waves; and (2) process the MAMS data to locate gravity wave features, and produce various moisture products in support of the gravity wave investigation. The research activities over the last year are presented and they focused on analyzing the CaPE data sets and collecting and starting preliminary analysis of the STORMFEST MAMS and Wildfire data.

Jedlovec, Gary J.↗