Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data processing and standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
NASA NTRS1987

The Penn State astronomical image processing system

The needs of modern astronomy for image processing set demanding standards in simultaneously requiring fast computation speed, high-quality graphic display, large data storage, and interactive response. An innovative image processing system was designed, integrated, and used; it is based on a supermicro architecture which is tailored specifically for astronomy, which provides a highly cost-effective alternative to the traditional minicomputer installation. The paper describes the design rationale, equipment selection, and software developed to allow other astronomers with similar needs to benefit from the present experience.

Truax, Ryland J.↗
NASA NTRS2014

Complete Decoding and Reporting of Aviation Routine Weather Reports (METARs)

Aviation Routine Weather Report (METAR) provides surface weather information at and around observation stations, including airport terminals. These weather observations are used by pilots for flight planning and by air traffic service providers for managing departure and arrival flights. The METARs are also an important source of weather data for Air Traffic Management (ATM) analysts and researchers at NASA and elsewhere. These researchers use METAR to correlate severe weather events with local or national air traffic actions that restrict air traffic, as one example. A METAR is made up of multiple groups of coded text, each with a specific standard coding format. These groups of coded text are located in two sections of a report: Body and Remarks. The coded text groups in a U.S. METAR are intended to follow the coding standards set by National Oceanic and Atmospheric Administration (NOAA). However, manual data entry and edits made by a human report observer may result in coded text elements that do not follow the standards, especially in the Remarks section. And contrary to the standards, some significant weather observations are noted only in the Remarks section and not in the Body section of the reports. While human readers can infer the intended meaning of non-standard coding of weather conditions, doing so with a computer program is far more challenging. However such programmatic pre-processing is necessary to enable efficient and faster database query when researchers need to perform any significant historical weather analysis. Therefore, to support such analysis, a computer algorithm was developed to identify groups of coded text anywhere in a report and to perform subsequent decoding in software. The algorithm considers common deviations from the standards and data entry mistakes made by observers. The implemented software code was tested to decode 12 million reports and the decoding process was able to completely interpret 99.93 of the reports. This document presents the deviations from the standards and the decoding algorithm. Storing all decoded data in a database allows users to quickly query a large amount of data and to perform data mining on the data. Users can specify complex query criteria not only on date or airport but also on weather condition. This document also describes the design of a database schema for storing the decoded data, and a Data Warehouse web application that allows users to perform reporting and analysis on the decoded data. Finally, this document presents a case study correlating dust storms reported in METARs from the Phoenix International airport with Ground Stops issued by Air Route Traffic Control Centers (ATCSCC). Blowing widespread dust is one of the weather conditions when dust storm occurs. By querying the database, 294 METARs were found to report blowing widespread dust at the Phoenix airport and 41 of them reported such condition only in the Remarks section of the reports. When METAR is a data source for an ATM research, it is important to include weather conditions not only from the Body section but also from the Remarks section of METARs.

METAR Decoder/Parser↗
NASA NTRS2020

Perspectives on Data Reproducibility and Replicability in Paleoclimate and Climate Science

This paper summarizes the current state of reproducibility and replicability in the fields of climate and paleoclimate science, including brief histories of their development and applications in climate science, new and recent approaches towards improvement of reproducibility and replicability, and challenges. Recommendations for addressing those challenges include: development of searchable, auto-updated, interlinked, multi-archive public paleoclimate repositories for raw and processed digital datasets; cross-center standardized code base cases, improved data storage techniques, and a focus on replicability for climate simulation storage and access; and support of the development and community awareness of findable, accessible, interoperable and reusable (FAIR) principles by funding agencies and publishers. This paper is largely based on the May 2018 presentations of a panel of researchers to the Committee on Reproducibility and Replicability in Science, part of the National Academies of Science, Engineering, and Medicine. The commentary and recommendations made here are in alignment with those of its Consensus Study Report on Reproducibility and Replicability in Science (2019).

data repositories↗
NASA NTRS2011

Ares I-X Best Estimated Trajectory and Comparison with Pre-Flight Predictions

The Ares I-X trajectory reconstruction produced best estimated trajectories of the flight test vehicle ascent through stage separation, and of the first and upper stage entries after separation. The trajectory reconstruction process combines on-board, ground-based, and atmospheric measurements to produce the trajectory estimates. The Ares I-X vehicle had a number of on-board and ground based sensors that were available, including inertial measurement units, radar, air- data, and weather balloons. However, due to problems with calibrations and/or data, not all of the sensor data were used. The trajectory estimate was generated using an Iterative Extended Kalman Filter algorithm, which is an industry standard processing algorithm for filtering and estimation applications. This paper describes the methodology and results of the trajectory reconstruction process, including flight data preprocessing and input uncertainties, trajectory estimation algorithms, output transformations, and comparisons with preflight predictions.

Karlgaard, Christopher D.↗
NASA NTRS2011

Ares I-X Best Estimated Trajectory Analysis and Results

The Ares I-X trajectory reconstruction produced best estimated trajectories of the flight test vehicle ascent through stage separation, and of the first and upper stage entries after separation. The trajectory reconstruction process combines on-board, ground-based, and atmospheric measurements to produce the trajectory estimates. The Ares I-X vehicle had a number of on-board and ground based sensors that were available, including inertial measurement units, radar, air-data, and weather balloons. However, due to problems with calibrations and/or data, not all of the sensor data were used. The trajectory estimate was generated using an Iterative Extended Kalman Filter algorithm, which is an industry standard processing algorithm for filtering and estimation applications. This paper describes the methodology and results of the trajectory reconstruction process, including flight data preprocessing and input uncertainties, trajectory estimation algorithms, output transformations, and comparisons with preflight predictions.

Karlgaard, Christopher D.↗
NASA NTRS1988

IUE's legacy for the future: The final archive and goals for its implementation

Requirements for the IUE archive, and how the signal/noise (S/N) ratio in photometrically corrected images can be enhanced considerably by cross-correlating the fixed pattern in a data image with that in a suitable flat-field image are described. From these cross-correlations it is feasible to derive an accurate geometrical correction to apply to the data image before applying the intensity transfer functions. The standard IUE processing software does not generate a sufficiently accurate geometric correction so that typical spatial errors of 1 to 2 pixels conspire with the large fixed pattern in raw images to produce significant misregistration noise. Tests on flat-field images demonstrate that an explicit geometric correction procedure can avoid most of the misregistration noise and can thereby improve the S/N ratio of IUE data by factors of 1.5 to 2.4.

Linsky, J. L.↗
DOE OSTI2023

A Processing and Analytics System for Microscopy Data Workflows: The Pycroscopy Ecosystem of Packages

Major advancements in fields as diverse as biology and quantum computing have relied on a multitude of microscopy techniques. Despite the considerable proliferation of these instruments, significant bottlenecks remain in terms of processing, analysis, storage, and retrieval of the acquired datasets. Aside from lack of file standards, individual domain-specific analysis packages are often disjoint from the underlying datasets, and thus keeping track of analysis and processing steps remains tedious for the end-user, hampering reproducibility. Here, in this study, the pycroscopy ecosystem of packages is introduced, an open-source python-based ecosystem underpinned by a common data model. The data model, termed the N-dimensional spectral imaging data format, is realized in pycroscopy's sidpy package. This package is built on top of dask arrays, thus leveraging dask array attributes, but expanding them to accelerate microscopy relevant analysis and visualization. Several examples of the use of the pycroscopy ecosystem to create workflows for data ingestion and analysis of scanning transmission electron microscopy (STEM) and scanning probe microscopy data are shown. Adoption of such standardized routines will be critical to usher in the next generation of autonomous instruments where processing, computation, and meta-data storage will be critical to overall experimental operations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Multispectral satellite observations for arid land studies

Multispectral satellite data when properly calibrated and standardized can be used synergistically for a quantitative analysis of processes and surface characteristics, and for quantifying land surface change. Relationships among multispectral satellite data (visible reflectance, surface temperature and polarization difference of microwave emission at 37 GHz frequency) have been used to develop hypotheses concerning the relative sensitivity of these data to varied land surface characteristics, which needs to be verified by field observations. Radiative transfer models have also been developed to understand these multispectral data. Interannual variations of visible reflectance and polarization difference for the period 1982-1986 over the Sahel and the Sudan zones of Africa show a lagged response with respect to the rainfall deficit during recovery from drought, which needs to be understood in terms of biophysical parameters.

Choudhury, Bhaskar J.↗
NASA NTRS1984

Utilization of spaceborne SAR data for mapping

Recent developments in automated processing of digital SEASAT SAR imagery have made feasible the generation of large-scale high-resolution maps. Standard preprocessing of raw data into digital images results in geometrically distorted imagery. Computer algorithms have been developed for unsupervised pixel location, geometric rectification, and mosaicking of multiple-image frames without ground control points. These algorithms utilize knowledge of the spacecraft trajectory data, the imaging geometry, and the coherent properties of the sensor to generate the required processing parameters. This paper discusses the advantages as well as the inherent limitations of this technique, analyzes the associated errors, and presents results using SEASAT SAR imagery. Also discussed are the results of the recent shuttle imaging radar (SIR-A) experiment as well as a follow-on experiment (SIR-B) planned for 1984.

Curlander, J. C.↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 4: Utah FORGE Well 78B-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 3: Utah FORGE Well 56-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
NASA NTRS2018

GMI-IPS: Processing & Visualization Software Used in ATom DC-8 Aircraft Studies

NASA's Atmospheric Tomography Mission (ATom) deployed in each of the four seasons during 2016-2018, the DC-8 aircraft in order to establish global-scale datasets intended to improve the representation of chemically reactive gases in global atmospheric chemistry models (ACMs). The Global Modeling Initiative (GMI) executed simulations for each ATom flight using the GMI Chemistry Transport Model (GMI-CTM) to provide species concentrations of chemical gases along the DC-8 flight transects. To solve the problem of translating the GMI-CTM simulation data to the unique spatial resolutions of each ATom flight, the GMI ICARTT Processing Software (GMI-IPS) was developed.The GMI-IPS is written in Python and provides data processing, flight extraction, and visualization support for aircraft research projects using ICARTT format, which is a standard format for airborne instrument data. Additionally, the GMI-IPS interpolates global gridded model data from Hierarchical Data Format (HDF) to ICARTT compatible flight transects. Software classes for instruments and collections provided by the ATom DC-8 aircraft such as MER10, MMS, etc. are derived from a common base class. Other functionality provided by the GMI-IPS are: deriving missing flight entries along a transect, reading ICARTT entries from file, and providing Python data structures for storing flight and model information, and more.The GMI-IPS is GIT source controlled, has approximately 30,000 lines of code, and supports parallelization across data collections. It delivered GMI-CTM data for more than forty distinct DC-8 aircraft flights that took place under ATom. The output ICARTT files adhere to format standard V1.1, and pass the scan utility provided by NASA LaRC Airborne Science Data for Atmospheric Composition. This presentation will include a software and methods overview, and results from ATom, including assessments using the GMI-CTM showing how well observations from ATom flight transects represent a broader region.

Damon, M. R.↗
DOE OSTI2024

Analysis of Rig Parameter Data Using Drilling Process Modeling Constraints, Volume 5: Utah FORGE Well 16B(78)-32

Drill rig parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. While insightful into the drilling process, these measurements are of reduced value without a standard to aid in data evaluation and decision making. In the main body of this work (Volume 1), a method is demonstrated whereby rock reduction model constraints are used to interpret drilling response parameters; the method could be applied in real-time to improve decision-making in the field and to further discern technology performance during post-drilling evaluations. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) in computational algorithms. The method presented has applicability to development of advanced analytics on future geothermal wells using real-time electronic data recording for improved performance and reduced drilling costs. A drilling cost model is also used to show the tradeoff between rate of penetration and bit life and the influence on interval drilling costs. Details of the bit specifications and performance are cataloged in an independent volume, documented under separate cover, for each of the four wells, and include Volume 2: Utah FORGE 16A(78)-32; Volume 3: Utah FORGE 56-32; Volume 4: Utah FORGE 78B-32 and Volume 5: Utah FORGE 16B(78)-32.

15 GEOTHERMAL ENERGY↗
NASA NTRS1974

An overview of the National Space Science data Center Standard Information Retrieval System (SIRS)

A general overview is given of the National Space Science Data Center (NSSDC) Standard Information Retrieval System. A description, in general terms, the information system that contains the data files and the software system that processes and manipulates the files maintained at the Data Center. Emphasis is placed on providing users with an overview of the capabilities and uses of the NSSDC Standard Information Retrieval System (SIRS). Examples given are taken from the files at the Data Center. Detailed information about NSSDC data files is documented in a set of File Users Guides, with one user's guide prepared for each file processed by SIRS. Detailed information about SIRS is presented in the SIRS Users Guide.

Shapiro, A.↗
DOE OSTI2025

Search for charged lepton flavor violating 𝑍 and 𝑍′ boson decays in proton-proton collisions at $\sqrt{s}$ = 13 TeV

A search for flavor violating decays of the 𝑍 boson to charged leptons is performed using data from proton-proton collisions at $\sqrt{s}$ =13 TeV collected with the CMS detector at the LHC, corresponding to an integrated luminosity of 138 fb −1 . Each of the decays 𝑍 → 𝑒⁢𝜇, 𝑍 → 𝑒⁢𝜏, and 𝑍 → 𝜇⁢𝜏 is considered. The data are consistent with the backgrounds expected from standard model processes. For the 𝑍 → 𝑒⁢𝜇 channel the observed (expected) 95% confidence level upper limit on the branching fraction is 1.9⁢(2.0) × 10 −7 , which is the most stringent direct limit to date on this process; the corresponding limits for the 𝑍 → 𝑒⁢𝜏 and 𝑍 → 𝜇⁢𝜏 channels are 13.8⁢(11.4) × 10 −6 and 12.0⁢(5.3) × 10 −6 , respectively. Additionally, the 𝑒⁢𝜇 final state is used to search for lepton flavor violating decays of 𝑍′ resonances in the mass range from 110 to 500 GeV. No significant excess is observed above the predicted background levels.

Hayrapetyan, A. [Yerevan Physics Institute]↗
DOE OSTI2025

ILAMBv2.7 benchmarking results comparing E3SMv2.1 land-atmosphere coupled (BGCv2LNDATM) and stand alone land (ELM) simulations with CMIP6 emission driven historical simulations

This dataset contains land model benchmarking results for the Energy Exascale Earth System Model version 2.1 (E3SMv2.1), including outputs from both coupled biogeochemistry simulations and stand-alone land model simulations. These results are compared against several emission-driven historical simulations from the Coupled Model Intercomparison Project Phase 6 (CMIP6). Benchmarking was conducted using the International Land Model Benchmarking (ILAMB) package, version 2.7 (ILAMBv2.7). CMIP6 model outputs were sourced from the Earth System Grid Federation (ESGF), while the E3SMv2.1 results were derived from raw model outputs. These outputs underwent processing steps such as time serialization, conservative regridding, and data standardization to ensure comparability. For spatial interpolation, the Earth System Modeling Framework (ESMF) tool, ESMF_RegridWeightGen, was employed to generate regridding weights, enabling the transformation of E3SM’s native cubed-sphere grid to a regular latitude-longitude grid.

Feng, Sha [PNNL]↗
DOE OSTI2025

GLBRC Soil Yearlong Incubation 13C-SIP-Lipidomics

Data package for Lipids represent a dynamic, yet stable pool of microbially-derived soil carbon This data is published under a CC0 license. The authors encourage data reuse and request attribution by referencing the below citations for the data packages and associated manuscript. Please cite as: Rempfert KR, Bell SL, Kasanke CP, Kyle JE, Hofmockel KS. 2025. GLBRC Soil Yearlong Incubation 13C-SIP-Lipidomics. [Data Set] PNNL DataHub. doi: Rempfert KR, Bell SL, Kasanke CP, Kyle JE, Hofmockel KS. 2025. MSV000097435: GLBRC soil yearlong incubation 13C-SIP-Lipidomics [Data Set] MassIVE. doi:10.25345/C57659T3K Rempfert KR, Bell SL, Kasanke CP, Kyle JE, Hofmockel KS. 2025. Lipids represent a dynamic, yet stable pool of microbially-derived soil carbon. In Prep This data package consists of compound-specific 13C SIP-lipidomics data from a yearlong tracer incubation experiment designed to investigate microbial lipid persistence in switchgrass bioenergy crop soils. In order to explore how lipid structure may modulate the persistence of C in soil lipids, we leveraged soils from two sites (Michigan - sandy texture, Wisconsin - silty texture) operated by the U.S. Department of Energy-funded Great Lakes Bioenergy Research Center (GLBRC). These sites had comparable climates, identical management practices, but contrasting soil textures, allowing us to assess the variability of lipid accrual or degradation in soils as well as provide insight regarding the degree to which edaphic properties may regulate the retention of soil lipids. Untargeted lipidomics analyses were performed to identify 13C-labeled lipids in the soil microbiome after long-term incubation. Soils were supplemented with 100 micrograms glucose per gram dry soil (99 atom % 13C or natural abundance for paired control) and incubated; samples were collected two months and one year after glucose addition. Lipid extracts (MPLEx) were analyzed by LC-MS/MS and identified using LIQUID. Calculation of isotopic enrichment of lipids was performed by targeted approach using TarMet to quantify lipid isotopologues and IsoCorrectoR to correct for natural abundance isotopes. Contents: Data package contents reported here are the first version and contain downstream analysis files for the raw LC-MS mass spectrometry files (.mzXML) deposited at the MassIVE database repository under accession MSV000097435 (80 experimental runs; 5.85 GB) | MassIVE DOI: 10.25345/C57659T3K. Support files include the additional data download 'Read Me' file containing data descriptor information. Reported data download contents are structured for compliance with project data sharing guidelines, community standards initiatives, and sponsor stakeholder policies supporting FAIR data principles. Data processing software, analysis tools, and data workflows are listed below corresponding to the host repository long-term location. Available Data Downloads (0.3 GB): "GLBRC soil yearlong incubation 13C-SIP-Lipidomics_readme.txt" - 'Read Me' data package content file (txt) "GLBRC_DataPackage_analysis files" - Data processing files (Rmd) and saved intermediate data processing outputs (rds, csv, xlsx) "GLBRC_13C_lipidomics_dataset.xlsx" - processed data in tabular format (xlsx) Linked Software: LIQUID LC-MS Analysis Software | 10.5281/zenodo.6459462 Lipid Mini-On Software Tools | 10.5281/zenodo.1492803 pmartR Omics Statistical Software | 10.5281/zenodo.6108667 xcms (v4.3.3) TarMet (v1.1.1) IsoCorrectoR (1.24.0) Funding Acknowledgments: This research was supported by an Early Career Research Program award funded by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research (OBER) Genomic Science program under FWP 68292, FWP 07880 and EMSL Exploratory Research Project 51095. A portion of this work was performed in the William R. Wiley Environmental Molecular Sciences Laboratory, a national scientific user facility sponsored by OBER and located at Pacific Northwest National Laboratory (PNNL). PNNL is a multi-program national laboratory operated by Battelle for the DOE under Contract DE-AC05-76RLO1830.

Rempfert, Kaitlin R [Pacific Northwest National La↗
NASA NTRS1972

NERVA mass properties computer methodology

The computer codes are presented that were used for the weight, center of gravity, and mass moment of inertia calculations and documentation for the NERVA. The two programs, E15301 and 12001B, are the basic tools used for weights work on the NERVA program. Used in conjunction with one another, they allow for rapid weights estimates for new concepts or changes while allowing for orderly documentation, which is a necessity in any effective weights effort. Program E15301 requires that each component be resolved into a collection of standard shapes. Separate subroutines then process geometric and material density data for each entry. The master program the performs all calculations necessary to combine these results into a total weight, center of gravity, and moment of inertia for the component. Program 12001B accepts previously calculated weight and center of gravity data for individual components. The program sums weights and calculates center of gravities and moments of inertia for systems of up to four levels of assemblies. The output identifies the engine parts in a format usable directly in a formal report.

Hall, I. K.↗