Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Facilitating better and faster simulations of aerosol-cloud interactions in Earth system models

Focal Area(s): 1. Predictive modeling through the use of AI techniques and AI-derived model components; the use of AI and other tools to design a prediction system comprising a hierarchy of models. 2. Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI. Science Challenge: One major challenge that Earth system models (ESMs) face in providing credible prediction of the Earth system and its water cycle characteristics (e.g., mean state, variability, and extreme events) is to accurately simulate aerosol-cloud interactions (ACI). The physical, chemical, and dynamical processes affecting ACI are extremely complex and they range from nanoscale to planetary scale. In each model development cycle, scientists spend significant efforts investigating model deficiencies and uncertainties associated with aerosols (e.g., emissions, chemical processes, aerosol microphysics, and transport) and clouds (e.g., macrophysics, microphysics, turbulence, and large-scale circulation) in order to develop improved treatments. However, despite decades of active research, ACI is still a major source of uncertainty in climate projections, even though great progress has been made. Specific scientific challenges include: (i) Parameterizations are developed based on limited data; (ii) The complexity of a parameterization required for accurate predictions is not understood; (iii) Incomplete and unknown physics leads to errors in the fully coupled Earth system; and (iv) Complex physics is computationally too expensive to employ in ESMs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Real-time kinematic (RTK) Drone-collected Data and Processed Models of Port Arthur Coastal Neighborhood and Pleasure Island Golf Course, June 2024

The Southeast Texas Urban Integrated field lab’s Co-design team captured aerial photos in the Port Arthur Coastal Neighborhood Community and the Golf Course on Pleasure Island, Texas, in June 2024. Aerial photos taken were through autonomous flight, and models were processed through the DroneDeploy engine. All aerial photos are in .JPG format and contained in zipped files for each area. The processed data package includes 3D models, geospatial data, mappings, and point clouds. Please be aware that DTM, Elevation toolbox, Point Cloud, and Orthomosaic use EPSG: 6588. And 3D Model uses EPSG: 3857.For using these data:- The Adobe Suite gives you great software to open .Tif files.- You can use LASUtility (Windows), ESRI ArcGIS Pro (Windows), or Blaze3D (Windows, Linux) to open a LAS file and view the data it contains.- Open an .OBJ file with a large number of free and commercial applications. Some examples include Microsoft 3D Builder, Apple Preview, Blender, and Autodesk.- You may use ArcGIS, Merkaartor, Blender (with the Google Earth Importer plug-in), Global Mapper, and Marble to open .KML files.- The .tfw world file is a text file used to georeference the GeoTIFF raster images, like the orthomosaic and the DSM. You need suitable software like ArcView to open a .TFW file.This dataset provides researchers with sufficient geometric data and the status quo of the land surface at the locations mentioned above. This dataset will support researchers' decision-making processes under uncertainties.

2D mapping↗
DOE OSTI2021

ExaWorks: Workflows for Exascale

Exascale computers will offer transformative capabilities to combine data-driven and learning-based approaches with traditional simulation applications to accelerate scientific discovery and insight. These software combinations and integrations, however, are difficult to achieve due to challenges of coordination and deployment of heterogeneous software components on diverse and massive platforms. We present the ExaWorks project, which can address many of these challenges: ExaWorks is leading a co-design process to create a workflow Software Development Toolkit (SDK) consisting of a wide range of workflow management tools that can be composed and interoperate through common interfaces. We describe the initial set of tools and interfaces supported by the SDK, efforts to make them easier to apply to complex science challenges, and examples of their application to exemplar cases. Furthermore, we discuss how our project is working with the workflows community, large computing facilities as well as HPC platform vendors to sustainably address the requirements of workflows at the exascale.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Rapid Response Data Science for COVID-19

This report describes the results of a seven day effort to assist subject matter experts address a problem related to COVID-19. In the course of this effort, we analyzed the 29K documents provided as part of the White House's call to action. This involved applying a variety of natural language processing techniques and compression-based analytics in combination with visualization techniques and assessment with subject matter experts to pursue answers to a specific question. In this paper, we will describe the algorithms, the software, the study performed, and availability of the software developed during the effort.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Fast Scanning Probe Microscopy via Machine Learning: Non-Rectangular Scans with Compressed Sensing and Gaussian Process Optimization

Fast scanning probe microscopy enabled via machine learning allows for a broad range of nanoscale, temporally resolved physics to be uncovered. However, such examples for functional imaging are few in number. Here, using piezoresponse force microscopy (PFM) as a model application, a factor of 5.8 reduction in data collection using a combination of sparse spiral scanning with compressive sensing and Gaussian process regression reconstruction is demonstrated. It is found that even extremely sparse spiral scans offer strong reconstructions with less than 6% error for Gaussian process regression reconstructions. Further, the error associated with each reconstructive technique per reconstruction iteration is analyzed, finding the error is similar past ≈15 iterations, while at initial iterations Gaussian process regression outperforms compressive sensing. Finally, this study highlights the capabilities of reconstruction techniques when applied to sparse data, particularly sparse spiral PFM scans, with broad applications in scanning probe and electron microscopies.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Computing rank‐revealing factorizations of matrices stored out‐of‐core

This paper describes efficient algorithms for computing rank-revealing factorizations of matrices that are too large to fit in main memory (RAM), and must instead be stored on slow external memory devices such as disks (out-of-core or out-of-memory). Traditional algorithms for computing rank-revealing factorizations (such as the column pivoted QR factorization and the singular value decomposition) are very communication intensive as they require many vector-vector and matrix-vector operations, which become prohibitively expensive when data is not in RAM. Randomization allows to reformulate new methods so that large contiguous blocks of the matrix are processed in bulk. The paper describes two distinct methods. The first is a blocked version of column pivoted Householder QR, organized as a “left-looking” method to minimize the number of the expensive write operations. The second method results employs a UTV factorization. It is organized as an algorithm-by-blocks to overlap computations and I/O operations. As it incorporates power iterations, it is much better at revealing the numerical rank. Numerical experiments on several computers demonstrate that the new algorithms are almost as fast when processing data stored on slow memory devices as traditional algorithms are for data stored in RAM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Atmospheric Radiation Measurement Climate Research Facility Operations Quarterly Report October 1-December 31, 2016

Individual datastreams from instrumentation at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility fixed and mobile research observatories (sites) are collected and routed to the ARM Data Center (ADC). The Data Management Facility (DMF), a component of the ADC, executes datastream processing in near-real time. Processed data are then delivered approximately daily to the ARM Data Archive, also a component of the ADC, where they are made freely available to the research community. For each instrument, ARM calculates the ratio of the actual number of processed data records received daily at the ARM Data Archive to the expected number of data records. DOE requires national user facilities to report time-based operating data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Quantum Machine Learning Applications in High-Energy Physics

Some of the most significant achievements of the modern era of particle physics, such as the discovery of the Higgs boson, have been made possible by the tremendous effort in building and operating large-scale experiments like the Large Hadron Collider or the Tevatron. In these facilities, the ultimate theory to describe matter at the most fundamental level is constantly probed and verified. These experiments often produce large amounts of data that require storing, processing, and analysis techniques that continually push the limits of traditional information processing schemes. Thus, the High-Energy Physics (HEP) field has benefited from advancements in information processing and the development of algorithms and tools for large datasets. More recently, quantum computing applications have been investigated to understand how the community can benefit from the advantages of quantum information science. Nonetheless, to unleash the full potential of quantum computing, there is a need to understand the quantum behavior and, thus, scale up current algorithms beyond what can be simulated in classical processors. In this work, we explore potential applications of quantum machine learning to data analysis tasks in HEP and how to overcome the limitations of algorithms targeted for Noisy Intermediate-Scale Quantum (NISQ) devices.

Delgado, Andrea↗
DOE OSTI2015

Design of experiments and data analysis challenges in calibration for forensics applications

Forensic science aims to infer characteristics of source terms using measured observables. Our focus is on statistical design of experiments and data analysis challenges arising in nuclear forensics. More specifically, we focus on inferring aspects of experimental conditions (of a process to produce product Pu oxide powder), such as temperature, nitric acid concentration, and Pu concentration, using measured features of the product Pu oxide powder. The measured features, Y, include trace chemical concentrations and particle morphology such as particle size and shape of the produced Pu oxide power particles. Making inferences about the nature of inputs X that were used to create nuclear materials having particular characteristics, Y, is an inverse problem. Therefore, statistical analysis can be used to identify the best set (or sets) of Xs for a new set of observed responses Y. One can fit a model (or models) such as Υ = f(Χ) + error , for each of the responses, based on a calibration experiment and then “invert” to solve for the best set of Xs for a new set of Ys. This perspectives paper uses archived experimental data to consider aspects of data collection and experiment design for the calibration data to maximize the quality of the predicted Ys in the forward models; that is, we assume that well-estimated forward models are effective in the inverse problem. In addition, we consider how to identify a best solution for the inferred X, and evaluate the quality of the result and its robustness to a variety of initial assumptions, and different correlation structures between the responses. In addition, we also briefly review recent advances in metrology issues related to characterizing particle morphology measurements used in the response vector, Y.

36 MATERIALS SCIENCE↗
DOE OSTI2024

CROCUS Air Quality Data at Argonne National Laboratory Prairie Site

The AQT (Vaisala AQT530) instrument provides observations on meteorological conditions, including particulate matter (PM2.5, PM10), gas species concentrations (NO, NO2, O3, CO), and environment temperature and moisture. These measurements are critical for understanding air quality. These measurements are useful for understanding changes in aerosol properties, air quality research, and comparing to model experiments especially in urban environments. These measurements are collected at the Argonne Testbed for Multiscale Observational Science (ATMOS), a prairie field site at Argonne National Laboratory in Lemont, Illinois. Data is available in the netCDF data format, we encourage data users review documentation through Project Pythia to understand how to work with netCDF data https://foundations.projectpythia.org/core/data-formats/netcdf-cf.html. Each file contains one day's worth of data (24 hours, starting at 0000 UTC). The data is aggregated into daily frequency to make it easier to process multiple days, and compress the higher-resolution fields. File naming convention includes the project (CROCUS), location (atmos), data level (raw, a1), date (year, month, day), and hour (0000).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

Dual energy CT: How well can pseudo-monochromatic imaging reduce metal artifacts?

Purpose: Dual Energy CT (DECT) provides so-called monoenergetic images based on a linear combination of the original polychromatic images. At certain patient-specific energy levels, corresponding to certain patient- and slice-dependent linear combination weights, e.g., E = 160 keV corresponds to α = 1.57, a significant reduction of metal artifacts may be observed. The authors aimed at analyzing the method for its artifact reduction capabilities to identify its limitations. The results are compared with raw data-based processing. Methods: Clinical DECT uses a simplified version of monochromatic imaging by linearly combining the low and the high kV images and by assigning an energy to that linear combination. Those pseudo-monochromatic images can be used by radiologists to obtain images with reduced metal artifacts. The authors analyzed the underlying physics and carried out a series expansion of the polychromatic attenuation equations. The resulting nonlinear terms are responsible for the artifacts, but they are not linearly related between the low and the high kV scan: A linear combination of both images cannot eliminate the nonlinearities, it can only reduce their impact. Scattered radiation yields additional noncanceling nonlinearities. This method is compared to raw data-based artifact correction methods. To quantify the artifact reduction potential of pseudo-monochromatic images, they simulated the FORBILD abdomen phantom with metal implants, and they assessed patient data sets of a clinical dual source CT system (100, 140 kV Sn) containing artifacts induced by a highly concentrated contrast agent bolus and by metal. In each case, they manually selected an optimal α and compared it to a raw data-based material decomposition in case of simulation, to raw data-based material decomposition of inconsistent rays in case of the patient data set containing contrast agent, and to the frequency split normalized metal artifact reduction in case of the metal implant. For each case, the contrast-to-noise ratio (CNR) was assessed. Results: In the simulation, the pseudo-monochromatic images yielded acceptable artifact reduction results. However, the CNR in the artifact-reduced images was more than 60% lower than in the original polychromatic images. In contrast, the raw data-based material decomposition did not significantly reduce the CNR in the virtual monochromatic images. Regarding the patient data with beam hardening artifacts and with metal artifacts from small implants the pseudo-monochromatic method was able to reduce the artifacts, again with the downside of a significant CNR reduction. More intense metal artifacts, e.g., as those caused by an artificial hip joint, could not be suppressed. Conclusions: Pseudo-monochromatic imaging is able to reduce beam hardening, scatter, and metal artifacts in some cases but it cannot remove them. In all cases, the CNR is significantly reduced, thereby rendering the method questionable, unless special post processing algorithms are implemented to restore the high CNR from the original images (e.g., by using a frequency split technique). Raw data-based dual energy decomposition methods should be preferred, in particular, because the CNR penalty is almost negligible.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2017

Toward broadband mechanical spectroscopy

Diverse material classes exhibit qualitatively similar behavior when made viscous upon cooling toward the glass transition, suggesting a common theoretical basis. In this paper, we used seven different measurement methods to determine the mechanical relaxation kinetics of a prototype molecular glass former over a temporal range of 13 decades and over a temperature range spanning liquid to glassy states. The data conform to time–temperature superposition for the main (alpha) process and to a scaling relation of schematic mode-coupling theory. Finally, the broadband mechanical measurements demonstrated have fundamental and practical applications in polymer science, geophysics, multifunctional materials, and other areas.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

Speckle contrast diffuse correlation tomography of complex turbid medium flow

Purpose: Developed herein is a three-dimensional (3D) flow contrast imaging system leveraging advancements in the extension of laser speckle contrast imaging theories to deep tissues along with our recently developed finite-element diffuse correlation tomography (DCT) reconstruction scheme. This technique, termed speckle contrast diffuse correlation tomography (scDCT), enables incorporation of complex optical property heterogeneities and sample boundaries. When combined with a reflectance-based design, this system facilitates a rapid segue into flow contrast imaging of larger, in vivo applications such as humans. Methods: A highly sensitive CCD camera was integrated into a reflectance-based optical system. Four long-coherence laser source positions were coupled to an optical switch for sequencing of tomographic data acquisition providing multiple projections through the sample. This system was investigated through incorporation of liquid and solid tissue-like phantoms exhibiting optical properties and flow characteristics typical of human tissues. Computer simulations were also performed for comparisons. A uniquely encountered smear correction algorithm was employed to correct point-source illumination contributions during image capture with the frame-transfer CCD and reflectance setup. Results: Measurements with scDCT on a homogeneous liquid phantom showed that speckle contrast-based deep flow indices were within 12% of those from standard DCT. Inclusion of a solid phantom submerged below the liquid phantom surface allowed for heterogeneity detection and validation. The heterogeneity was identified successfully by reconstructed 3D flow contrast tomography with scDCT. The heterogeneity center and dimensions and averaged relative flow (within 3%) and localization were in agreement with actuality and computer simulations, respectively. Conclusions: A custom cost-effective CCD-based reflectance 3D flow imaging system demonstrated rapid acquisition of dense boundary data and, with further studies, a high potential for translatability to real tissues with arbitrary boundaries. A requisite correction was also found for measurements in the fashion of scDCT to recover accurate speckle contrast of deep tissues.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Data from: "Warming of alpine tundra enhances belowground production and shifts community towards resource acquisition traits"

This archive contains data used to draw conclusions in “Warming of alpine tundra enhances belowground production and shifts community towards resource acquisition traits”, by Yang et al. 2020. Data were collected on Niwot Ridge, in an alpine meadow within the Alpine Treeline Warming Experiment (ATWE) field sites in Colorado, USA. Samples were also processed in the U.S. Geological Survey Forest and Rangeland Ecosystem Science Center, in Boise, Idaho. File formats in this archive include comma-separated values (.csv), portable document format (.pdf), Microsoft Excel (.xlsx), and two types of geospatial files: keyhole markup language (.kml), and ESRI shapefiles (.shp). Leaf scans are .jpg images, and root scans are .tiff/.tif images.The .csv files can be opened using R, Microsoft Excel, or any simple text-editing software such as TextEdit and Notepad. Microsoft Excel files can be opened using Microsoft Excel, and .pdf files can be opened with Adobe Acrobat Reader, Preview, or other compatible programs. Scanned images can be opened using any photo and/or picture viewing software.The .kml file can be opened using Google Earth and Google Maps, and the shapefiles can be opened by any programs compatible with shapefiles, such as the ArcGIS Desktop suite, and QGIS.------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Measures of belowground net primary productivity (BNPP) are required to understand whether aboveground net primary production (ANPP) changes reflect changes in allocation or are indicative of a whole plant NPP response. Plant functional traits provide a key way to scale from the individual plant to the community level, and provide insight into drivers of NPP responses to environmental change. We used infrared heaters to warm an alpine plant community at Niwot Ridge, Colorado, and applied supplemental water to compensate for soil water loss induced by warming. We measured ANPP, BNPP, and leaf and root functional traits across treatments after 5 years of continuous warming. Community-level ANPP and total NPP (ANPP + BNPP) did not respond to heating or watering, but BNPP increased in response to heating. Heating decreased community-level leaf dry matter content and increased total root length, indicating a shift in strategy from resource conservation to acquisition in response to warming.

13C/12C isotope ratio↗
DOE OSTI2024

Visualization Within the Department of Energy: NREL IEEE VIS Application Spotlight

This presentation highlights the role of advanced visualization techniques at the National Renewable Energy Laboratory (NREL) in supporting cutting-edge research across diverse energy domains. From immersive analytics and uncertainty visualization to high-resolution and real-time data analysis, NREL's visualization capabilities enable scientists to explore complex datasets more effectively. These tools are critical for advancing research in materials science, renewable energy technologies, biofuels, electric vehicle infrastructure, energy efficiency - from industrial processes to entire communities - and then bringing these innovations to practice through energy systems integration. NREL's visualization tools drive innovation across renewable energy and grid modernization efforts by providing deeper insights and improving decision-making.

grid modernization↗
DOE OSTI2020

Mechanistic Approach to Analyzing and Improving Unconventional Hydrocarbon Production [Slides]

DOE research is developing the physical basis and tools needed to manage pressure effectively to increase recovery efficiency. By coupling fast, accurate physics with machine learning, DOE is producing science-based platforms any operator can use. DOE’s research portfolio is targeting hydrocarbon transport at multiple scales, with the goal of increasing recovery efficiency. DOE’s research has led to new, fast & accurate platforms for predicting gas production from fractured shales. Using data from the MSEEL-I site to calibrate our physics-based model, we have early results on pressure management. We have shown that both mechanical and chemical processes in the matrix can negatively impact production.

08 HYDROGEN↗
DOE OSTI2021

Interpretable Deep Learning for the Earth System with Fractal Nets

Focal Area 3: Explainable AI Our confidence in the projections made by Earth System Models (ESMs) depends on understanding them to be, in some important respects, faithful representations of the Earth system. Here we present an “explainable Artificial Intelligence (AI)” method that allows us to uncover the dynamical structure of the observed and modeled Earth system, discover hidden links across wide spatiotemporal scales, target model development efforts at poorly-represented dynamics, and optimize observed or modeled data collection to maximize predictive information. Science Challenge: Dynamical system science for the Earth system poses unique challenges given the large degree of internal climate variability. Thus, tools that help us understand how ESMs succeed and fail at representing these dynamics are crucial, particularly in relation to the observed system. Furthermore, the computational and memory constraints on ESM data output motivate in situ analysis of ESM dynamics, including automatic detection of dynamical shifts. Also, of key importance are procedures that leverage ESMs to optimize observational campaigns for improving process representation, reducing structural uncertainty and improving model skill.

54 ENVIRONMENTAL SCIENCES↗