Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2001

Improvements in AVHRR Daytime Cloud Detection Over the ARM NSA Site

Clouds play an important role in the radiation budget over Arctic and Antarctic. Because of limited surface observing capabilities, it is necessary to detect clouds over large areas using satellite imagery. At low and mid-latitudes, satellite-observed visible (VIS; 0.65 micrometers) and infrared (IR; 11 micrometers) radiance data are used to derive cloud fraction, temperature, and optical depth. However, the extreme variability in the VIS surface albedo makes the detection of clouds from satellite a difficult process in polar regions. The IR data often show that the surface is nearly the same temperature or even colder than clouds, further complicating cloud detection. Also, the boundary layer can have large areas of haze, thin fog, or diamond dust that are not seen in standard satellite imagery. Other spectral radiances measured by satellite imagers provide additional information that can be used to more accurately discriminate clouds from snow and ice. Most techniques currently use a fixed reflectance or temperature threshold to decide between clouds and clear snow. Using a subjective approach, Minnis et al. (2001) found that the clear snow radiance signatures vary as a function of viewing and illumination conditions as well as snow condition. To routinely process satellite imagery over polar regions with an automated algorithm, it is necessary to account for this angular variability and the change in the background reflectance as snow melts, vegetation grows over land, and melt ponds form on pack ice. This paper documents the initial satellite-based cloud product over the Atmospheric Radiation Measurement (ARM) North Slope of Alaska (NSA) site at Barrow for use by the modeling community. Cloud amount and height are determined subjectively using an adaptation of the methodology of Minnis et al. (2001) and the radiation fields arc determined following the methods of Doelling et al. (2001) as applied to data taken during the Surface Heat and Energy Budget of the Arctic (SHEBA). The procedures and data produced in this empirically based analysis will also facilitate the development of the automated algorithm for future processing of satellite data over the ARM NSA domain. Results are presented for May, June, and July 1998. ARM surface data are use to partially validate the results taken directly over the ARM site.

Chakrapani, V.↗
NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗
DOE OSTI2024

Clearance Tracker

The manual process of managing security clearance was cumbersome and inefficient, involving the emailing of a fillable PDF from person to person. There was no visibility into the progress of a clearance submission. Clearance Tracker provides an automated workflow with data validation and notifications to manage the clearance process. The automated workflow routes notifications to individuals when their input is required. All participants can track the progress of clearance requests.

Short, RickyD↗
NASA NTRS2019

Automated Fiber Placement Manufactured Composites for Science Applications

What automated composite laminate manufacturing isWhy automation is of interest in science applicationsHow composite automation is being considered for science instrument applicationsAnd, about test data showing high stiffness materials processed with automation results in reduced material strength while stiffness and coefficient of thermal expansion are mostly unaffected.

Segal, Ken↗
DOE OSTI2024

Oscilloscope Data Push Program

This paper details the development of a Python program designed to automate the data acquisition and conversion for an oscilloscope for the purposes of a one-off/temporary data acquisition system for users that readily need data, and do not have the option of obtaining a Data Acquisition (DAQ) solution. Creating DAQ systems for analyzing a system requires expensive electronics and a dedicated team of engineers for support. Traditionally, manual data collection and processing are time consuming and prone to error. By automating these processes, the cost, efficiency and accuracy of data handling are improved upon. This project involves the creation of a program that interacts with the oscilloscope. During this interaction, there are various functions being performed such as the acquisition of waveform data via floating points, generating plots with the acquired wave points, and storing of floating points in a CSV file format for future reference and plotting purposes. While the initial aim of the project included continuous logging to a cloud database, this was deferred due to time constraints. The results portrayed an almost-instant rate of data collection with a buffer time, showcasing the potential for further integration and real-time data processing.

Osei-Tutu, Jason↗
NASA NTRS1990

Adaptive pattern recognition by mini-max neural networks as a part of an intelligent processor

In this decade and progressing into 21st Century, NASA will have missions including Space Station and the Earth related Planet Sciences. To support these missions, a high degree of sophistication in machine automation and an increasing amount of data processing throughput rate are necessary. Meeting these challenges requires intelligent machines, designed to support the necessary automations in a remote space and hazardous environment. There are two approaches to designing these intelligent machines. One of these is the knowledge-based expert system approach, namely AI. The other is a non-rule approach based on parallel and distributed computing for adaptive fault-tolerances, namely Neural or Natural Intelligence (NI). The union of AI and NI is the solution to the problem stated above. The NI segment of this unit extracts features automatically by applying Cauchy simulated annealing to a mini-max cost energy function. The feature discovered by NI can then be passed to the AI system for future processing, and vice versa. This passing increases reliability, for AI can follow the NI formulated algorithm exactly, and can provide the context knowledge base as the constraints of neurocomputing. The mini-max cost function that solves the unknown feature can furthermore give us a top-down architectural design of neural networks by means of Taylor series expansion of the cost function. A typical mini-max cost function consists of the sample variance of each class in the numerator, and separation of the center of each class in the denominator. Thus, when the total cost energy is minimized, the conflicting goals of intraclass clustering and interclass segregation are achieved simultaneously.

Szu, Harold H.↗
DOE OSTI2023

Data transfer for STAR grid jobs

The Solenoidal Tracker at RHIC (STAR) is a multipurpose experiment at the Relativistic Heavy Ion Collider (RHIC) with the primary goal to study the formation and properties of the quark-gluon plasma. STAR is an international collaboration of member institutions and laboratories from around the world. Yearly data-taking period produces PBytes of raw data collected by the experiment. STAR primarily uses its dedicated facility at BNL to process this data, but has routinely leveraged distributed systems, both high throughput (HTC) and high performance (HPC) computing clusters, to significantly augment the processing capacity available to the experiment. The ability to automate the efficient transfer of large data sets on reliable, scalable, and secure infrastructure is critical for any large-scale distributed processing campaign. For more than a decade, STAR computing has relied upon GridFTP with its x509-based authentication to build such data transfer systems and integrate them into its larger production workflow. The end of support by the community for both GridFTP and the x509 standard requires STAR to investigate other approaches to meet its distributed processing needs. In this study we investigate two multi-purpose data distribution systems, Globus.org and XRootD, as alternatives to GridFTP. We compare both their performance and the ease by which each service is integrated into the type of secure and automated data transfer systems STAR has previously built using GridFTP. The presented approach and study may be applicable to other distributed data processing use cases beyond STAR.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Unified architecture for data-driven metadata tagging of building automation systems

This article presents a Unified Architecture (UA) for automated point tagging of Building Automation System (BAS) data, based on a combination of data-driven approaches. Advanced energy analytics applications—including fault detection and diagnostics and supervisory control—have emerged as a significant opportunity for improving the performance of our built environment. Effective application of these analytics depends on harnessing structured data from the various building control and monitoring systems, but typical BAS implementations do not employ any standardized metadata schema. While standards such as Project Haystack and Brick Schema have been developed to address this issue, the process of structuring the data, i.e., tagging the points to apply a standard metadata schema, has, to date, been a manual process. This process is typically costly, labor-intensive, and error-prone. In this work we address this gap by proposing a UA that automates the process of point tagging by leveraging the data accessible through connection to the BAS, including time-series data and the raw point names. The UA intertwines supervised classification and unsupervised clustering techniques from machine learning and leverages both their deterministic and probabilistic outputs to inform the point tagging process. Furthermore, we extend the UA to embed additional input and output data-processing modules that are designed to address the challenges associated with the real-time deployment of this automation solution. We test the UA on two datasets for real-life buildings: (i) commercial retail buildings and (ii) office buildings from the National Renewable Energy Laboratory (NREL) campus. We report the proposed methodology correctly applied 85–90% and 70–75% of the tags in each of these test scenarios, respectively for two significantly different building types used for testing UA's fully-functional prototype. The proposed UA, therefore, offers promising approach for automatically tagging BAS data as it reaches close to 90% accuracy. Further building upon this framework to algorithmically identify the equipment type and their relationships is an apt future research direction to pursue.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented data standards and automated data pipelines for the following data types: 1) drilling data, 2) geospatial datasets, and 3) DAS data. An additional data pipeline is proposed for stimulation data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how we can improve this process.

cloud-optimized↗
DOE OSTI2023

An Update on the Geothermal Data Repository's Data Standards and Pipelines: Geospatial Data and Distributed Acoustic Sensing Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has implemented data standards and automated data pipelines for the following data types: 1) drilling data, 2) geospatial datasets, and 3) DAS data. An additional data pipeline is proposed for stimulation data. These data standards and pipelines are intended to improve the real-world applicability of geothermal machine learning outputs through improving the quality of data. More specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, allowing more time to be spent on actual research. By automating this process, the burden of standardization is taken off of the user, overall increasing the availability of standardized data. This paper provides an update on the GDR's transition toward data standardization through automated data pipelines and calls for feedback from the community on how we can improve this process.

cloud-optimized↗
DOE OSTI2022

Koopman Model Predictive Control for Eco-Driving of Automated Vehicles

In this paper, we develop a data-driven process for building a model predictive control (MPC) for eco-driving of automated vehicles. The process involves performing system identification in which the non-linear vehicle dynamics model is approximated by the Koopman operator, a linear predictor of higher state-dimension, in a data-driven framework. This approach allows us to formulate the eco-driving problem in a constrained quadratic program that leads to a computationally fast MPC. The MPC is then implemented as a closed-loop control of an electric vehicle in numerical simulations for demonstration.

autonomous vehicle↗
DOE OSTI2020

Vulnerabilities in Artificial Intelligence and Machine Learning Applications and Data

Artificial intelligence (AI) applications driven by machine learning (ML) are transformational technologies within the international nuclear security regime. Advancements realized by AI—faster and improved data insights, more efficient and automated processes, reductions in human error—enable nuclear security applications such as behavior analysis for insider threat mitigation, source tracking of stolen nuclear material, and facial recognition software for physical protection. In addition to the advantages, however, there are also inherent vulnerabilities and threats associated with its use and risk mitigations must be built into any AI/ML-enabled systems. This work provides a background on AI and ML and different data types used in the field, including open-source intelligence information (OSINT) that is discoverable by AI tools and application data that are used by AI tools for decision-making and automation. Current and potential AI applications and vulnerabilities related to their use within the nuclear security regime are also discussed.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2022

Correction and calibration of atmospheric impact observations in GOES GLM data

The Earth's atmosphere is impacted daily by both meteoroids and artificial objects. Calibrated observations of the emitted light at sufficiently high sampling rates can enable or improve the estimation of impactor attributes such as size, cohesion, trajectory, and composition, but are difficult to obtain owing to the unpredictability, brevity, and high dynamic (brightness) range of impacts. Ground-based camera systems have successfully monitored small regions of the atmosphere at video frame rates and with limited radiometric capabilities, but most impacts occur over the 70% of the Earth's surface covered by water and are therefore missed by these networks. The Geostationary Lightning Mapper (GLM) instruments aboard Geostationary Operational Environmental Satellites 16 and 17 provide near-hemispherical coverage at 500 frames per second. These data have been shown to contain the signatures of many independently confirmed impacts, often from both viewing angles simultaneously, and constitute an observational resource that is currently unparalleled in the public domain. NASA's Asteroid Threat Assessment Project has implemented an automated impact detection pipeline that processes data from GLM daily. Given a detected impact, the GLM data contain a wealth of information for use in quantitative follow-up analyses. However, impact events differ from lightning in ways that violate key assumptions built into GLM's design. The result is that GLM's onboard processing introduces errors into pixel observations of impact events and the calibrated energies near the periphery of the detector may be substantially overestimated. We present methods for mitigating these and other issues to produce a data product more suitable for impact analyses than the existing GLM lightning product.

58 GEOSCIENCES↗
NASA NTRS1971

Implementation of radiation shielding calculation methods. Volume 1: Synopsis of methods and summary of results

The work performed in the following areas is summarized: (1) Analysis of Realistic nuclear-propelled vehicle was analyzed using the Marshall Space Flight Center computer code package. This code package includes one and two dimensional discrete ordinate transport, point kernel, and single scatter techniques, as well as cross section preparation and data processing codes, (2) Techniques were developed to improve the automated data transfer in the coupled computation method of the computer code package and improve the utilization of this code package on the Univac-1108 computer system. (3) The MSFC master data libraries were updated.

Capo, M. A.↗
NASA NTRS1984

Designing systems to satisfy their users - The coming changes in aviation weather and the development of a central weather processor

Attention is given to the development history of the Central Weather Processor (CWP) program of the Federal Aviation Administration. The CWP will interface with high speed digital communications links, accept data and information products from new sources, generate data processing products, and provide meteorologists with the capability to automate data retrieval and dissemination. The CWP's users are operational (air traffic controllers, meteorologists and pilots), institutional (logistics, maintenance, testing and evaluation personnel), and administrative.

Bush, M. W.↗
NASA NTRS1991

A unified approach for composite cost reporting and prediction in the ACT program

The Structures Technology Program Office (STPO) at NASA Langley Research Center has held two workshops with representatives from the commercial airframe companies to establish a plan for development of a standard cost reporting format and a cost prediction tool for conceptual and preliminary designers. This paper reviews the findings of the workshop representatives with a plan for implementation of their recommendations. The recommendations of the cost tracking and reporting committee will be implemented by reinstituting the collection of composite part fabrication data in a format similar to the DoD/NASA Structural Composites Fabrication Guide. The process of data collection will be automated by taking advantage of current technology with user friendly computer interfaces and electronic data transmission. Development of a conceptual and preliminary designers' cost prediction model will be initiated. The model will provide a technically sound method for evaluating the relative cost of different composite structural designs, fabrication processes, and assembly methods that can be compared to equivalent metallic parts or assemblies. The feasibility of developing cost prediction software in a modular form for interfacing with state of the art preliminary design tools and computer aided design (CAD) programs is assessed.

Freeman, W. Tom↗
DOE OSTI2021

SNEWPY: A Data Pipeline from Supernova Simulations to Neutrino Signals

Current neutrino detectors will observe hundreds to thousands of neutrinos from a Galactic supernovae, and future detectors will increase this yield by an order of magnitude or more. With such a data set comes the potential for a huge increase in our understanding of the explosions of massive stars, nuclear physics under extreme conditions, and the properties of the neutrino. However, there is currently a large gap between supernova simulations and the corresponding signals in neutrino detectors, which will make any comparison between theory and observation very difficult. SNEWPY is an open-source software package which bridges this gap. The SNEWPY code can interface with supernova simulation data to generate from the model either a time series of neutrino spectral fluences at Earth, or the total time-integrated spectral fluence. Data from several hundred simulations of core-collapse, thermonuclear, and pair-instability supernovae is included in the package. This output may then be used by an event generator such as sntools or an event rate calculator such as SNOwGLoBES. Additional routines in the SNEWPY package automate the processing of the generated data through the SNOwGLoBES software and collate its output into the observable channels of each detector. In this paper we describe the contents of the package, the physics behind SNEWPY, the organization of the code, and provide examples of how to make use of its capabilities.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗