Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Automatic Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
NASA NTRS2013

PROcess Based Diagnostics PROBE

Many of the aspects of the climate system that are of the greatest interest (e.g., the sensitivity of the system to external forcings) are emergent properties that arise via the complex interplay between disparate processes. This is also true for climate models most diagnostics are not a function of an isolated portion of source code, but rather are affected by multiple components and procedures. Thus any model-observation mismatch is hard to attribute to any specific piece of code or imperfection in a specific model assumption. An alternative approach is to identify diagnostics that are more closely tied to specific processes -- implying that if a mismatch is found, it should be much easier to identify and address specific algorithmic choices that will improve the simulation. However, this approach requires looking at model output and observational data in a more sophisticated way than the more traditional production of monthly or annual mean quantities. The data must instead be filtered in time and space for examples of the specific process being targeted.We are developing a data analysis environment called PROcess-Based Explorer (PROBE) that seeks to enable efficient and systematic computation of process-based diagnostics on very large sets of data. In this environment, investigators can define arbitrarily complex filters and then seamlessly perform computations in parallel on the filtered output from their model. The same analysis can be performed on additional related data sets (e.g., reanalyses) thereby enabling routine comparisons between model and observational data. PROBE also incorporates workflow technology to automatically update computed diagnostics for subsequent executions of a model. In this presentation, we will discuss the design and current status of PROBE as well as share results from some preliminary use cases.

PROBE↗
NASA NTRS2011

Automated X-ray and Optical Analysis of the Virtual Observatory and Grid Computing

We are developing a system to combine the Web Enabled Source Identification with X-Matching (WESIX) web service, which emphasizes source detection on optical images,with the XAssist program that automates the analysis of X-ray data. XAssist is continuously processing archival X-ray data in several pipelines. We have established a workflow in which FITS images and/or (in the case of X ray data) an X-ray field can be input to WESIX. Intelligent services return available data (if requested fields have been processed) or submit job requests to a queue to be performed asynchronously. These services will be available via web services (for non-interactive use by Virtual Observatory portals and applications) and through web applications (written in the Django web application framework). We are adding web services for specific XAssist functionality such as determining .the exposure and limiting flux for a given position on the sky and extracting spectra and images for a given region. We are improving the queuing system in XAssist to allow for "watch lists" to be specified by users, and when X-ray fields in a user's watch list become publicly available they will be automatically added to the queue. XAssist is being expanded to be used as a survey planning 1001 when coupled with simulation software, including functionality for NuStar, eRosita, IXO, and the Wide Field Xray Telescope (WFXT), as part of an end to end simulation/analysis system. We are also investigating the possibility of a dedicated iPhone/iPad app for querying pipeline data, requesting processing, and administrative job control.

Ptak, A.↗
NASA NTRS2001

Runway Incursion Prevention System ADS-B and DGPS Data Link Analysis Dallas-Fort Worth International Airport

A Runway Incursion Prevention System (RIPS) was tested at the Dallas - Ft. Worth International Airport in October 2000. The system integrated airborne and ground components to provide both pilots and controllers with enhanced situational awareness, supplemental guidance cues, a real-time display of traffic information, and warning of runway incursions in order to prevent runway incidents while also improving operational capability. Rockwell Collins provided and supported a prototype Automatic Dependent Surveillance - Broadcast (ADS-B) system using 1090 MHz and a prototype Differential GPS (DGPS) system onboard the NASA Boeing 757 research aircraft. This report describes the Rockwell Collins contributions to the RIPS flight test, summarizes the development process, and analyzes both ADS-B and DGPS data collected during the flight test. In addition, results are report on interoperability tests conducted between the NASA Advanced General Aviation Transport Experiments (AGATE) ADS-B flight test system and the NASA Boeing 757 ADS-B system.

Timmerman, J.↗
DOE OSTI2020

Removing imaging systematics from galaxy clustering measurements with Obiwan: application to the SDSS-IV extended Baryon Oscillation Spectroscopic Survey emission-line galaxy sample

This article presents the application of a new tool, Obiwan, which uses image simulations to determine the selection function of a galaxy redshift survey and calculate three-dimensional (3D) clustering statistics. Obiwan relies on a forward model of the process by which images of the night sky are transformed into a 3D large-scale structure catalogue, and offers several advantages over more traditional map-based techniques – such as operating on individual exposures and adopting a maximum likelihood approach. The photometric pipeline automatically detects and models galaxies and then generates a catalogue of such galaxies with detailed information for each one of them, including their location, redshift, and so on. Systematic biases in the imaging data are therefore imparted into the catalogues and must be accounted for in any scientific analysis of their information content. Obiwan simulates this process for samples selected from the Legacy Surveys imaging data. This imaging data will be used to select target samples for the next-generation Dark Energy Spectroscopic Instrument (DESI) experiment. Here, we apply Obiwan to a portion of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey emission-line galaxies (ELGs). Systematic biases in the data are clearly identified and removed. We compare the 3D clustering results to those obtained by the map-based approach applied to the complete eBOSS Data Release 16 (DR16) sample. We find the results are consistent, thereby validating the eBOSS DR16 ELG catalogues, which is used to obtain cosmological results.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1988

Automatic mathematical modeling for real time simulation system

A methodology for automatic mathematical modeling and generating simulation models is described. The models will be verified by running in a test environment using standard profiles with the results compared against known results. The major objective is to create a user friendly environment for engineers to design, maintain, and verify their model and also automatically convert the mathematical model into conventional code for conventional computation. A demonstration program was designed for modeling the Space Shuttle Main Engine Simulation. It is written in LISP and MACSYMA and runs on a Symbolic 3670 Lisp Machine. The program provides a very friendly and well organized environment for engineers to build a knowledge base for base equations and general information. It contains an initial set of component process elements for the Space Shuttle Main Engine Simulation and a questionnaire that allows the engineer to answer a set of questions to specify a particular model. The system is then able to automatically generate the model and FORTRAN code. The future goal which is under construction is to download the FORTRAN code to VAX/VMS system for conventional computation. The SSME mathematical model will be verified in a test environment and the solution compared with the real data profile. The use of artificial intelligence techniques has shown that the process of the simulation modeling can be simplified.

Wang, Caroline↗
NASA NTRS2020

Dynamic IT Security Database and Analytics for Launch Control Systems Software

During the Summer 2020 session, I worked with intern Destani S. Van Arsdalen of EGS Software. Together, we co-created a tool to aid the dynamic investigation, updated over time,of the security compliance of LCS COTS and open source software. We originally planned touse spreadsheet software for management and analysis, but through this exploratoryproject, chose to use Python and JSON after receiving feedback on our project’s current anddesired capabilities at that time.At first, the project was solely designed to help on-board new COTS software, based on aquestionnaire that could be filled out for each software package. This, combined with usingthe spreadsheet application’s web-query capabilities to fetch information from the NVD,allowed presentation and analytics cells to automatically populate as elements of themanually-filled questionnaire changed. While this system was promising, we decided tochange technologies for a few reasons. In the spreadsheet, single cells could not hold complexdata like arrays and objects. The automatic population of cells and dynamic updates made itdifficult to manage and add new features. And finally, it had limited extensibility sinceadding new software required significant understanding of how both the spreadsheet wasconstructed, and the more obscure, proprietary scripting languages packaged with it.The pivot to a standard computer science database language of JSON, aided by thescripting capabilities of Python, greatly helped to improve the project’s functionality. First,and most importantly, the script’s import and analysis of database data is easilyreproducible. Additional data analysis can be modularly added without requiringmodification of the script and is capable of routine scheduling. The revised process can besplit into three parts. First, the conversion of LCS asset and software documentation into theJSON hierarchical database format. Second, the merging of this database with the NVD,forming a new data structure, using CPEs of the CVE object as a linking element betweenthem. And third, the automatically performed analytics and analysis of the combined data,in a modular and extensible format, to produce better informed business decisions. The outputted graphs, for example, are automatically generated by the Python script inconnection with the combined database. This allows updated graphs and any analytics to be re-rendered automatically following updates to the LCS’s initial asset documentation. Afinal report can then be programmatically and easily constructed from these sources to allow fully reproducible metrics for heavily evidenced risk management decisions.

it↗
DOE OSTI2021

A meta-learning based distribution system load forecasting model selection framework

This paper presents a meta-learning based, automatic distribution system load forecasting model selection framework. Furthermore, the framework includes the following processes: feature extraction, candidate model preparation and labeling, offline training, and online model recommendation. Using load forecasting needs and data characteristics as input features, multiple metalearners are used to rank the candidate load forecast models based on their forecasting accuracy. Then, a scoring-voting mechanism is proposed to weights recommendations from each meta-leaner and make the final recommendations. Heterogeneous load forecasting tasks with different temporal and technical requirements at different load aggregation levels are set up to train, validate, and test the performance of the proposed framework. Simulation results demonstrate that the performance of the meta-learning based approach is satisfactory in both seen and unseen forecasting tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

SU-F-J-116: Clinical Experience-Based Verification and Improvement of a 4DCT Program

Purpose: To demonstrate the role of continuous improvement fulfilled by the Medical Physicist in clinical 4DCT and CBCT scanning. Methods: Lung (SABR and Standard) patients’ 4D respiratory motion and image data were reviewed over a 3, 6 and 12 month period following commissioning testing. By identifying trends of clinically relevant parameters and respiratory motions, variables were tested with a programmable motion phantom and assessed. Patient traces were imported to a motion phantom and 4DCT and CBCT imaging were performed. Cos6 surrogate and sup-inf motion was also programmed into the phantom to simulate the long exhale of patients for image contrast tests. Results: Patient surrogate motion amplitudes were 9.9+5.2mm (3–35) at 18+6bpm (6–30). Expiration/Inspiration time ratios of 1.4+0.5second (0.6–2.9) showed image contrast effects evident in the AveCT and 3DCBCT images. Small differences were found for patients with multiple 4DCT data sets. Patient motion assessments were simulated and verified with the phantom within 2mm. Initial image reviews to check for reconstructed artefacts and data loss identified a small number of patients with irregularities in the automatic placement of inspiration and expiration points. Conclusion: The Physicist’s involvement in the continuous improvements of a clinically commissioned technique, processes and workflows continues beyond the commissioning stage of a project. Our experience with our clinical 4DCT program shows that Physics presence is required at the clinical 4DCT scan to assist with technical aspects of the scan and also for clinical image quality assessment prior to voluming. The results of this work enabled the sharing of information from the Medical Physics group with the Radiation Oncologists and Radiation Therapists. This results in an improved awareness of clinical patient respiration variables and how they may affect 4D simulation images and also may also affect the treatment verification images.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Image-domain DAS 3D VSP elastic transmission tomography

SUMMARY Full-wavefield elastic imaging of active-source seismic data acquired by downhole receivers commonly offers higher-resolution subsurface images in the vicinity of a borehole compared to conventional surface seismic data sets, which can lack higher-frequency wavefield components due to longer travel paths and increased attenuation. An increasingly used approach for downhole acquisition is vertical seismic profiling (VSP), which has become more attractive when coupled with distributed acoustic sensing (DAS) using optical fibres installed in wells. The main difficulty for generating high-quality images with full-wavefield imaging tools for DAS VSP data, though, is the need for an accurate velocity model. To build plausible velocity models using active-source DAS VSP data, we adopt a 3-D image-domain elastic transmission tomography technique, originally developed for surface-recorded passive (microseismic) data, by exchanging the source and receiver positions (i.e. reciprocity) to mimic a passive-seismic surface monitoring scenario. The inversion approach exploits various images for each source constructed through time-reverse imaging (TRI) of downgoing P- and S-wave first-arrival waveforms. The TRI process uses the kinetic term of the (extended) PS energy imaging condition that exhibits sufficient sensitivity to velocity model errors. The method automatically updates the P- and S-wave velocity models to optimize image focusing via adjoint-state inversion. We illustrate the efficacy of the adopted elastic inversion technique using an active-source DAS 3-D VSP field data set acquired in the North Slope of Alaska. The numerical experiments demonstrate that the inverted elastic velocity models can be further used in full-wavefield acoustic/elastic imaging algorithms to obtain accurate subsurface images.

Oren, Can (ORCID:000000019534265X)↗
NASA NTRS1994

Assessment of the SFC database for analysis and modeling

SFC is one of the four clusters that make up the Integrated Work Control System (IWCS), which will integrate the shuttle processing databases at Kennedy Space Center (KSC). The IWCS framework will enable communication among the four clusters and add new data collection protocols. The Shop Floor Control (SFC) module has been operational for two and a half years; however, at this stage, automatic links to the other 3 modules have not been implemented yet, except for a partial link to IOS (CASPR). SFC revolves around a DB/2 database with PFORMS acting as the database management system (DBMS). PFORMS is an off-the-shelf DB/2 application that provides a set of data entry screens and query forms. The main dynamic entity in the SFC and IOS database is a task; thus, the physical storage location and update privileges are driven by the status of the WAD. As we explored the SFC values, we realized that there was much to do before actually engaging in continuous analysis of the SFC data. Half way into this effort, it was realized that full scale analysis would have to be a future third phase of this effort. So, we concentrated on getting to know the contents of the database, and in establishing an initial set of tools to start the continuous analysis process. Specifically, we set out to: (1) provide specific procedures for statistical models, so as to enhance the TP-OAO office analysis and modeling capabilities; (2) design a data exchange interface; (3) prototype the interface to provide inputs to SCRAM; and (4) design a modeling database. These objectives were set with the expectation that, if met, they would provide former TP-OAO engineers with tools that would help them demonstrate the importance of process-based analyses. The latter, in return, will help them obtain the cooperation of various organizations in charting out their individual processes.

Centeno, Martha A.↗
NASA NTRS2001

New Data Source for Studying and Modelling the Topside Ionosphere

The existing uncertainties about density profiles in the topside ionosphere, i.e., in the height regime from hmF2 to approx. 2000 km, requires the search for new data sources. Millions of ionograms had been recorded by the ISIS and Alouette satellites in the sixties and seventies, that never were analyzed in terms of electron density profiles. In recent years an effort started to digitize the analog recordings to prepare the ionograms for computerized analysis. This paper shows how the digital ionograms are processed and the electron density profiles (from satellite orbit altitude, 1400 km for ISIS-2, down to the F peak) are calculated. The most difficult part of the task is the automatic scaling of the echo traces in the ISIS ionograms. Unlike the ionograms from modern ionosondes, the ISIS ionograms do not identify the wave polarization of the different echo traces, so physical logic must be applied to identify the ordinary ()) and extraordinary (X) traces, and this is not always successful. Characteristic resonance features seen in the topside ionograms occur at the gyro and plasma frequencies. An elaborate scheme was developed to identify these resonance frequencies in order to determine the local plasma and gyrofrequencies. This information helps in the identification of the O and X traces, and it provides the starting density of the electron density profile. The inversion of the echo traces into electron density profiles uses the same modified Chebyshev polynomial fitting technique that is successfully applied in the ground-based Digisonde network. The automatic topside ionogram scaler with true height algorithm TOPIST is successfully scaling approx. 70% of the ionograms. An 'editing process' is available to manually scale the more difficult ionograms. The home page for the ISIS project is at http://nssdc.gsfc.nasa.gov/space/isis/isis-status.html. It provides access to as of January 2001, 3000,000 digitized ISIS ionogram data and to related software. A search page lets users select data location, time, and a host of other search criteria. The automated processing of the ISIS ionograms will begin later this year and the electron density profiles will be made available from the project home page. The ISIS data restoration efforts are supported through NASA's Applied Systems and Information Research Program.

Huang, Xue-Qin↗
NASA NTRS2019

An Automatic Light Rain Detection Algorithm on NASA MPLNET Lidar Observations in the Frame of WMO GALION Project

The water cycle strongly influences life on Earth. In particular, the precipitation modifies the atmospheric column thermodynamics through the process of evaporation and serves as a proxy for latent heat modulation. For this reason, a correct precipitation parameterization (especially low-intensity precipitation) at global scale, bedsides improving our understanding of the hydrological cycle, it is crucial to reduce the associated uncertainty of the global climate models to correctly forecast future scenarios, i.e. to apply fast mitigation strategies. In this study we developed an algorithm to automatically detect precipitation from lidar measurements obtained by the National and Aeronautics Space Administration (NASA) Micropulse lidar network (MPLNET) permanent observational site in Goddard. The algorithm, once full operational, will deliver in Near Real Time (latency 1.5h) a new rain mask product that will be publicly available on MPLNET website as part of the new Version 3 Level 1.5 data. The methodology, based on an image processing technique, can detect only light precipitation events (defined by intensity and duration) as the morphological filters used through the detection process are applied on the lidar volume depolarization ratio range corrected composite images, i.e. heavy rain events are unusable as the lidar signal is completely extinguished after few meters in the precipitation or no signal detected because of the water accumulated on the receiver optics. Results from the algorithm, besides filling a gap in precipitation and virga detection by radars, are of particular interest for the scientific community because will help to better understand long-term aerosol-cloud interactions and aerosol atmospheric removal (scavenging effect) by rain as multi-year database being available for several MPLNET permanent observational sites across the globe. Moreover, we developed the automatic algorithm at Universitat Politecnica de Catalunya (UPC) Barcelona, the unique permanent observation station member of MPLNET and the European Aerosol Lidar Network (EARLINET) In the future the algorithm can be then easily applied to any other lidar and/or ceilometer network infrastructure in the frame of World Meteorological Organization (WMO) Global Aerosol Watch (GAW) aerosol lidar observation network (GALION)

Simone Lolli↗
DOE OSTI2025

Event Log / Raw Data

The WFIP3 event log is a curated record spanning 578 days of meteorological phenomena and field observations that complements the campaign’s high-frequency measurements. The log combines manually documented daily weather discussions with automatically derived indicators of key atmospheric processes, providing standardized, publicly available context to support model evaluation, forecast verification, and case-study selection for offshore boundary-layer research.

17 WIND ENERGY↗
NASA NTRS1985

Processing of Image Data by Integrated Circuits

Sensors combined with logic and memory circuitry. Cross-correlation of two inputs accomplished by transversal filter. Position of image taken to point where image and template data yield maximum value correlation function. Circuit used for controlling robots, medical-image analysis, automatic vehicle guidance, and precise pointing of scientific cameras.

Armstrong, R. W.↗
DOE OSTI2022

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities

Buildings account for 35% of the global final energy demand. Efficiency improvements and advanced control strategies have a significant impact in the reduction of energy costs and CO2 emissions. Building energy simulation is widely used to help planners, contractors, and building owners analyse diverse options regarding the planning and management of energy consumption in buildings. Furthermore, recent advances in data processing and computing have led to the development of sophisticated machine learning algorithms that can learn from large datasets, e.g., sensor data from buildings, and use them to develop building-specific adaptive and automatic energy controllers. Control algorithms, such as deep reinforcement learning can tune themselves, are model-free, and economical to implement. In this paper, we introduce an integrated simulation environment that combines CitySim, a fast building energy simulator, and TensorFlow, a platform for efficient implementation of advanced machine learning algorithms. The integration is achieved via Keras - an API for TensorFlow - and a set of text and csv files for data transfer between the applications. This new environment will allow researchers to investigate novel learning control algorithms, and demonstrate their robustness and potential for diverse applications in the built environment. We present two case studies for energy savings and demand response, respectively.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

NeuNorm

NeuNorm is a scipp-based Python library for neutron imaging normalization and time-of-flight (TOF) data processing at Oak Ridge National Laboratory imaging facilities (MARS at HFIR and VENUS at SNS). NeuNorm 2.0 is a complete, scipp-based rewrite of the original NeuNorm normalization library, adding HDF5 output, automatic uncertainty propagation, and TOF/event-mode processing.

Zhang, Chen [Oak Ridge National Laboratory (ORNL),↗
NASA NTRS1978

DSN system performance test software

The system performance test software is currently being modified to include additional capabilities and enhancements. Additional software programs are currently being developed for the Command Store and Forward System and the Automatic Total Recall System. The test executive is the main program. It controls the input and output of the individual test programs by routing data blocks and operator directives to those programs. It also processes data block dump requests from the operator.

Martin, M.↗