Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Probabilistic Modeling of Commercial Building Occupancy Patterns Using Location-Based Map Data: Preprint

Considering occupancy patterns is crucial to simulate buildings' energy use. Current energy models use inputs that simplify the actual diversity in occupancy into static occupancy patterns and are not able to represent the numerous variations in occupancy patterns between buildings and across different locations. Recently, inferring occupancy schedules from metered electricity consumption data was used to model occupancy in commercial buildings. However, the translation from metered data to occupancy schedules requires many assumptions that might not capture the reality, and the process is hindered by the availability of data from advanced metering infrastructure. With the development of information technologies, occupancy modeling should not be limited to traditional approaches. The prevalence of social networks and location services with real-time user feedback provides publicly accessible data via Maps Application Programming Interfaces (APIs) such as Google Maps, SafeGraph, Mapbox, Foursquare, etc. This paper presents an automated framework for modeling parametric occupancy patterns using such APIs to calibrate commercial district buildings' energy models. This process includes three main steps: data extraction and processing, parametric schedules generation, and schedules integration. We demonstrated this framework in districts where we used maps API to generate more accurate behavioral patterns for operations and electric vehicle charging events. We used these patterns to determine differences in energy use across key sociodemographic and spatial parameters. The presented method has the potential for worldwide applications. Users can utilize this framework to extract data for selected locations of interest to create more realistic behavioral patterns for commercial facilities across different districts.

building energy modeling↗
DOE OSTI2021

The CanBikeCO Mini Pilot: Preliminary Results and Lessons Learned

In fall 2020, the Colorado Energy Office, as part of the State of Colorado’s “Can Do Colorado” initiative, initiated a project aimed at encouraging energy-efficient transportation during the COVID-19 pandemic. The initial mini-pilot provided e-bikes to 13 low-income households under an individual ownership model. This report assesses the impact of providing this additional mobility option on the travel behavior of participants. It also outlines the lessons learned from deploying a continuous monitoring platform to track the travel behavior. These lessons will influence the evaluation component for the full pilot, which will cover multiple geographic regions, starting in summer 2021, and run for 2 years. The continuous data collection was enabled by a customized version of the open-source e-mission platform, called CanBikeCO, configured with a behavioral gamification feature. The Colorado Energy Office used this system to collect a unique data set consisting of 3 months of partially automated travel diaries, combining sensed and surveyed data and linked with demographic information, from 12 participants. The data collection process worked well overall: users generally liked the app, appreciated the game, and did not complain about battery life. The long tracking period introduced behavioral challenges in user engagement, which we plan to address using repeated patterns and automated status checks for the full pilot. The analysis results, based on the subset of trips with user-reported labels (68%), indicate that the e-bike was the dominant commute mode share (31%), in sharp contrast to the census bicycle commute mode share (<1%). E-bike trips primarily replaced single-occupancy vehicle (SOV) trips (28%), followed closely by walking (24%) and regular bike (20%). The nonmotorized mode replacement corresponds to lower travel time and increased productivity enabled by the program. The emissions impact analysis of the program, computed using trip-level energy intensity factors, indicates savings of 1,367 lbs. of CO 2 . Although the results are strongly positive, the narrow demographic profile of study participants, their limited mobility alternatives, and nonuniform labeling indicate caution in broader interpretation. These preliminary results do suggest that such programs, supported by real-time education and support from program managers, can simultaneously meet equity and sustainability goals. The planned full pilot, addressing the data collection challenges and broadening the geographic scope, will provide additional insights into the generality of this approach.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

The CanBikeCO Mini Pilot: Procedure and Preliminary Results

In fall 2020, the Colorado Energy Office, as part of the State of Colorado's "Can Do Colorado" initiative, initiated a project aimed at encouraging energy-efficient transportation during the COVID-19 pandemic. The initial mini-pilot provided e-bikes to 13 low-income households under an individual ownership model. This report assesses the impact of providing this additional mobility option on the travel behavior of participants. It also outlines the lessons learned from deploying a continuous monitoring platform to track the travel behavior. These lessons will influence the evaluation component for the full pilot, which will cover multiple geographic regions, start in summer 2021, and run for 2 years. The continuous data collection was enabled by a customized version of the open-source e-mission platform, called CanBikeCO, configured with a behavioral gamification feature. The Colorado Energy Office used this system to collect a unique data set consisting of 3 months of partially automated travel diaries, combining sensed and surveyed data and linked with demographic information, from 12 participants. The data collection process worked well overall: users generally liked the app, appreciated the game, and did not complain about battery life. The long tracking period introduced behavioral challenges in user engagement, which we plan to address using repeated patterns and automated status checks for the full pilot. The analysis results, based on the subset of trips with user-reported labels (68%), indicate that the e-bike was the dominant commute mode share (31%), in sharp contrast to the census bicycle commute mode share (<1%). E-bike trips primarily replaced single-occupancy vehicle (SOV) trips (28%), followed closely by walking (24%) and regular bike (20%). The non-motorized mode replacement corresponds to lower travel time and increased productivity enabled by the program. The emissions impact analysis of the program, computed using trip-level energy intensity factors, indicates savings of 1,367 lbs. of CO2. Although the results are strongly positive, the narrow demographic profile of study participants, their limited mobility alternatives, and nonuniform labeling indicate caution in broader interpretation. These preliminary results do suggest that such programs, supported by real-time education and support from program managers, can simultaneously meet equity and sustainability goals. The planned full pilot, addressing the data collection challenges and broadening the geographic scope, will provide additional insights into the generality of this approach.

ADVANCED PROPULSION SYSTEMS↗
NASA NTRS2000

Automation of the CFD Process on Distributed Computing Systems

A script system was developed to automate and streamline portions of the CFD process. The system was designed to facilitate the use of CFD flow solvers on supercomputer and workstation platforms within a parametric design event. Integrating solver pre- and postprocessing phases, the fully automated ADTT script system marshalled the required input data, submitted the jobs to available computational resources, and processed the resulting output data. A number of codes were incorporated into the script system, which itself was part of a larger integrated design environment software package. The IDE and scripts were used in a design event involving a wind tunnel test. This experience highlighted the need for efficient data and resource management in all parts of the CFD process. To facilitate the use of CFD methods to perform parametric design studies, the script system was developed using UNIX shell and Perl languages. The goal of the work was to minimize the user interaction required to generate the data necessary to fill a parametric design space. The scripts wrote out the required input files for the user-specified flow solver, transferred all necessary input files to the computational resource, submitted and tracked the jobs using the resource queuing structure, and retrieved and post-processed the resulting dataset. For computational resources that did not run queueing software, the script system established its own simple first-in-first-out queueing structure to manage the workload. A variety of flow solvers were incorporated in the script system, including INS2D, PMARC, TIGER and GASP. Adapting the script system to a new flow solver was made easier through the use of object-oriented programming methods. The script system was incorporated into an ADTT integrated design environment and evaluated as part of a wind tunnel experiment. The system successfully generated the data required to fill the desired parametric design space. This stressed the computational resources required to compute and store the information. The scripts were continually modified to improve the utilization of the computational resources and reduce the likelihood of data loss due to failures. An ad-hoc file server was created to manage the large amount of data being generated as part of the design event. Files were stored and retrieved as needed to create new jobs and analyze the results. Additional information is contained in the original.

Tejnil, Ed↗
NASA NTRS1993

Design and specification of a centralized manufacturing data management and scheduling system

As was revealed in a previous study, the Materials and Processes Laboratory's Productivity Enhancement Complex (PEC) has a number of automated production areas/cells that are not effectively integrated, limiting the ability of users to readily share data. The recent decision to utilize the PEC for the fabrication of flight hardware has focused new attention on the problem and brought to light the need for an integrated data management and scheduling system. This report addresses this need by developing preliminary designs specifications for a centralized manufacturing data management and scheduling system for managing flight hardware fabrication in the PEC. This prototype system will be developed under the auspices of the Integrated Engineering Environment (IEE) Oversight team and the IEE Committee. At their recommendation the system specifications were based on the fabrication requirements of the AXAF-S Optical Bench.

Farrington, Phillip A.↗
NASA NTRS2016

Achieving Fast Operational Intelligence in NASA's Deep Space Network Through Complex Event Processing

NASA’s Deep Space Network (DSN) is a complex, global project, in which the expertise of human operators remain crucial for its successful operation. To find ways to save costs in operations and to improve its services, a number of modernization efforts are underway in the DSN. One such effort is a research and technology development task at the Jet Propulsion Laboratory that is investigating the use of complex event processing (CEP) for intelligent assessment of situations, trend analysis, and advanced automation. The technology leverages the significant business intelligence (BI) and data science advancements made in the enterprise industries over the last several years. The open source big data processing engine Apache SparkTM and the high-throughput, distributed messaging system Apache Kafka form the core of the DSN Complex Event Processing (DCEP) framework. This paper discusses the system engineering perspective of why achieving efficient, lower-cost operations in the DSN is a challenging problem, how the DCEP system handles the use cases that help realize intelligent operations, and how this solution fits into the overall model of the planned DSN Follow-the- Sun Operations (FtSO).

Choi, Joshua S.↗
DOE OSTI2020

A systematic feature extraction and selection framework for data-driven whole-building automated fault detection and diagnostics in commercial buildings

In data-driven automated fault detection and diagnostics (AFDD) modeling for building energy systems, feature engineering is a critical process of extracting information from high-dimensional and noisy sensor measurement and turning it into informative and representative inputs or features for data-driven modeling. However, few studies specifically discuss the feature engineering, especially the interactions between feature extraction and feature selection in whole-building AFDD. We developed a systematic feature extraction and selection framework for whole-building AFDD. In this framework, features are aggressively extracted from raw sensor data using statistical feature extraction techniques with various window sizes and statistics. With many features extracted, a hybrid feature selection algorithm that combines the filter and wrapper method then selects the best feature set. The framework considers diversity in the duration of fault behavior among fault types in whole-building AFDD, thus achieving high model generalization. We implemented our developed framework in a virtual testbed calibrated with measured data from Oak Ridge National Laboratory's Flexible Research Platform designed to mimic the operation of a typical small commercial building. The AFDD model is trained by the simulation data generated from the virtual testbed. The results show that (1) the developed framework improves the generalization of the AFDD model by 10.7% compared with literature-reported feature extraction and selection methods and (2) features with diverse window sizes and statistics are selected, providing insight into physical systems beyond the current understanding of buildings and faults and improving the detection and diagnostics of multiple fault types.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2017

Model-Based Thermal System Design Optimization for the James Webb Space Telescope

Spacecraft thermal model validation is normally performed by comparing model predictions with thermal test data and reducing their discrepancies to meet the mission requirements. Based on thermal engineering expertise, the model input parameters are adjusted to tune the model output response to the test data. The end result is not guaranteed to be the best solution in terms of reduced discrepancy and the process requires months to complete. A model-based methodology was developed to perform the validation process in a fully automated fashion and provide mathematical bases to the search for the optimal parameter set that minimizes the discrepancies between model and data. The methodology was successfully applied to several thermal subsystems of the James Webb Space Telescope (JWST). Global or quasiglobal optimal solutions were found and the total execution time of the model validation process was reduced to about two weeks. The model sensitivities to the parameters, which are required to solve the optimization problem, can be calculated automatically before the test begins and provide a library for sensitivity studies. This methodology represents a crucial commodity when testing complex, large-scale systems under time and budget constraints. Here, results for the JWST Core thermal system will be presented in detail.

Cataldo, Giuseppe↗
DOE OSTI2020

Emerging materials intelligence ecosystems propelled by machine learning

We report that the age of cognitive computing and artificial intelligence (AI) is just dawning. Inspired by its successes and promises, several AI ecosystems are blossoming, many of them within the domain of materials science and engineering. These materials intelligence ecosystems are being shaped by several independent developments. Machine learning (ML) algorithms and extant materials data are utilized to create surrogate models of materials properties and performance predictions. Materials data repositories, which fuel such surrogate model development, are mushrooming. Automated data and knowledge capture from the literature (to populate data repositories) using natural language processing approaches is being explored. The design of materials that meet target property requirements and of synthesis steps to create target materials appear to be within reach, either by closed-loop active-learning strategies or by inverting the prediction pipeline using advanced generative algorithms. AI and ML concepts are also transforming the computational and physical laboratory infrastructural landscapes used to create materials data in the first place. Surrogate models that can outstrip physics-based simulations (on which they are trained) by several orders of magnitude in speed while preserving accuracy are being actively developed. Automation, autonomy and guided high-throughput techniques are imparting enormous efficiencies and eliminating redundancies in materials synthesis and characterization. The integration of the various parts of the burgeoning ML landscape may lead to materials-savvy digital assistants and to a human-machine partnership that could enable dramatic efficiencies, accelerated discoveries and increased productivity. Here, we review these emergent materials intelligence ecosystems and discuss the imminent challenges and opportunities. The materials research landscape is being transformed by the infusion of approaches based on machine learning. This Review discusses the emerging materials intelligence ecosystems and the potential of human-machine partnerships for fast and efficient virtual materials screening, development and discovery.

36 MATERIALS SCIENCE↗
NASA NTRS1984

Practical considerations in aeroelastic design

The structural design process for large transport aircraft is described. Critical loads must be determined from a large number of load cases within the flight maneuver envelope. The structural design is also constrained by considerations of producibility, reliability, maintainability, durability, and damage tolerance, as well as impact dynamics and multiple constraints due to flutter and aeroelasticity. Aircraft aeroelastic design considerations in three distinct areas of product development (preliminary design, advanced design, and detailed design) are presented and contrasted. The present state of the art is challenged to solve the practical difficulties associated with design, analysis, and redesign within cost and schedule constraints. The current practice consists of largely independent engineering disciplines operating with unorganized data interfaces. The need is then demonstrated for a well-planned computerized aeroelastic structural design optimization system operating with a common interdisciplinary data base. This system must incorporate automated interfaces between modular programs. In each phase of the design process, a common finite-element model for static and dynamic optimization is required to reduce errors due to modeling discrepancies. As the design proceeds from the simple models in preliminary design to the more complex models in advanced and detailed design, a means of retrieving design data from the previous models must be established.

Rommel, B. A.↗
NASA NTRS1996

Technology Directions for the 21st Century, volume 1

For several decades, semiconductor device density and performance have been doubling about every 18 months (Moore's Law). With present photolithography techniques, this rate can continue for only about another 10 years. Continued improvement will need to rely on newer technologies. Transition from the current micron range for transistor size to the nanometer range will permit Moore's Law to operate well beyond 10 years. The technologies that will enable this extension include: single-electron transistors; quantum well devices; spin transistors; and nanotechnology and molecular engineering. Continuation of Moore's Law will rely on huge capital investments for manufacture as well as on new technologies. Much will depend on the fortunes of Intel, the premier chip manufacturer, which, in turn, depend on the development of mass-market applications and volume sales for chips of higher and higher density. The technology drivers are seen by different forecasters to include video/multimedia applications, digital signal processing, and business automation. Moore's Law will affect NASA in the areas of communications and space technology by reducing size and power requirements for data processing and data fusion functions to be performed onboard spacecraft. In addition, NASA will have the opportunity to be a pioneering contributor to nanotechnology research without incurring huge expenses.

Crimi, Giles F.↗
DOE OSTI2026

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1994

Automated inspection of turbine blades: Challenges and opportunities

Current inspection methods for complex shapes and contours exemplified by aircraft engine turbine blades are expensive, time-consuming and labor intensive. The logistics support of new manufacturing paradigms such as integrated product-process development (IPPD) for current and future engine technology development necessitates high speed, automated inspection of forged and cast jet engine blades, combined with a capability of retaining and retrieving metrology data for process improvements upstream (designer-level) and downstream (end-user facilities) at commercial and military installations. The paper presents the opportunities emerging from a feasibility study conducted using 3-D holographic laser radar in blade inspection. Requisite developments in computing technologies for systems integration of blade inspection in production are also discussed.

Mehta, Manish↗
NASA NTRS2018

The NANOGrav 11-year Data Set: High-Precision Timing of 45 Millisecond Pulsars

We present high-precision timing data over time spans of up to 11 years for 45 millisecond pulsars observed as part of the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) project, aimed at detecting and characterizing low-frequency gravitational waves. The pulsars were observed with the Arecibo Observatory and/or the Green Bank Telescope at frequencies ranging from 327 MHz to 2.3 GHz. Most pulsars were observed with approximately monthly cadence, and six high-timing-precision pulsars were observed weekly. All were observed at widely separated frequencies at each observing epoch in order to fit for time-variable dispersion delays. We describe our methods for data processing, time-of-arrival (TOA) calculation, and the implementation of a new, automated method for removing outlier TOAs. We fit a timing model for each pulsar that includes spin, astrometric, and (for binary pulsars) orbital parameters; time-variable dispersion delays; and parameters that quantify pulse-profile evolution with frequency. The timing solutions provide three new parallax measurements, two new Shapiro delay measurements, and two new measurements of significant orbital-period variations. We fit models that characterize sources of noise for each pulsar. We find that 11 pulsars show significant red noise, with generally smaller spectral indices than typically measured for non-recycled pulsars, possibly suggesting a different origin. A companion paper uses these data to constrain the strength of the gravitational-wave background

Arzoumanian, Zaven↗
NASA NTRS2016

DeepSAT: A Deep Learning Approach to Tree-Cover Delineation in 1-m NAIP Imagery for the Continental United States

High resolution tree cover classification maps are needed to increase the accuracy of current land ecosystem and climate model outputs. Limited studies are in place that demonstrates the state-of-the-art in deriving very high resolution (VHR) tree cover products. In addition, most methods heavily rely on commercial softwares that are difficult to scale given the region of study (e.g. continents to globe). Complexities in present approaches relate to (a) scalability of the algorithm, (b) large image data processing (compute and memory intensive), (c) computational cost, (d) massively parallel architecture, and (e) machine learning automation. In addition, VHR satellite datasets are of the order of terabytes and features extracted from these datasets are of the order of petabytes. In our present study, we have acquired the National Agriculture Imagery Program (NAIP) dataset for the Continental United States at a spatial resolution of 1-m. This data comes as image tiles (a total of quarter million image scenes with ~60 million pixels) and has a total size of ~65 terabytes for a single acquisition. Features extracted from the entire dataset would amount to ~8-10 petabytes. In our proposed approach, we have implemented a novel semi-automated machine learning algorithm rooted on the principles of "deep learning" to delineate the percentage of tree cover. Using the NASA Earth Exchange (NEX) initiative, we have developed an end-to-end architecture by integrating a segmentation module based on Statistical Region Merging, a classification algorithm using Deep Belief Network and a structured prediction algorithm using Conditional Random Fields to integrate the results from the segmentation and classification modules to create per-pixel class labels. The training process is scaled up using the power of GPUs and the prediction is scaled to quarter million NAIP tiles spanning the whole of Continental United States using the NEX HPC supercomputing cluster. An initial pilot over the state of California spanning a total of 11,095 NAIP tiles covering a total geographical area of 163,696 sq. miles has produced true positive rates of around 88 percent for fragmented forests and 74 percent for urban tree cover areas, with false positive rates lower than 2 percent for both landscapes.

Imagery↗
NASA NTRS2004

Data processing pipeline with transaction-oriented data sharing

This paper makes three contributions to the area of modern science data processing systems. First, the paper describes the science data processing pipeline, developed at the Multi-mission Image Processing Lab of JPL, for transforming raw space data into high quality image data and automating the distribution of data using a high-performance file transaction service. File Exchange Interface is the file transaction service developed MIPL. Second, it presents the FEI component architecture in the are of file transaction management, security,a nd file integrity verfication. Finally, the paper presents the federated model for the FEI service to demonstrate how to create a pool of file trasaction services to support load balancing and service fallover, and simplify service management.

science data processing↗
NASA NTRS2004

A Simple, Scalable, Script-based Science Processor

The production of Earth Science data from orbiting spacecraft is an activity that takes place 24 hours a day, 7 days a week. At the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC), this results in as many as 16,000 program executions each day, far too many to be run by human operators. In fact, when the Moderate Resolution Imaging Spectroradiometer (MODIS) was launched aboard the Terra spacecraft in 1999, the automated commercial system for running science processing was able to manage no more than 4,000 executions per day. Consequently, the GES DAAC developed a lightweight system based on the popular Per1 scripting language, named the Simple, Scalable, Script-based Science Processor (S4P). S4P automates science processing, allowing operators to focus on the rare problems occurring from anomalies in data or algorithms. S4P has been reused in several systems ranging from routine processing of MODIS data to data mining and is publicly available from NASA.

Lynnes, Christopher↗
NASA NTRS2001

NASA IVHM Technology Experiment for X-vehicles (NITEX)

The purpose of the NASA IVHM Technology Experiment for X-vehicles (NITEX) is to advance the development of selected IVHM technologies in a flight environment and to demonstrate the potential for reusable launch vehicle ground processing savings. The technologies to be developed and demonstrated include system-level and detailed diagnostics for real-time fault detection and isolation, prognostics for fault prediction, automated maintenance planning based on diagnostic and prognostic results, and a microelectronics hardware platform. Complete flight The Evolution of Flexible Insulation as IVHM consists of advanced sensors, distributed data acquisition, data processing that includes model-based diagnostics, prognostics and vehicle autonomy for control or suggested action, and advanced data storage. Complete ground IVHM consists of evolved control room architectures, advanced applications including automated maintenance planning and automated ground support equipment. This experiment will advance the development of a subset of complete IVHM.

Sandra, Hayden↗