Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Sequential experiment design”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1979

Numbers of center points appropriate to blocked response surface experiments

Tables are given for the numbers of center points to be used with blocked sequential designs of composite response surface experiments as used in empirical optimum seeking. The tables also give the star point radii for exact orthogonal blocking. The center point options vary from a lower limit of one to an upper limit equal to the numbers proposed by Box and Hunter for approximate rotatability and uniform variance, and exact orthogonal blocking. Some operating characteristics of the proposed options are described.

Holms, A. G.↗
NASA NTRS2017

Efficient Testing Combining Design of Experiment and Learn-to-Fly Strategies

Rapid modeling and efficient testing methods are important in a number of aerospace applications. In this study efficient testing strategies were evaluated in a wind tunnel test environment and combined to suggest a promising approach for both ground-based and flight-based experiments. Benefits of using Design of Experiment techniques, well established in scientific, military, and manufacturing applications are evaluated in combination with newly developing methods for global nonlinear modeling. The nonlinear modeling methods, referred to as Learn-to-Fly methods, utilize fuzzy logic and multivariate orthogonal function techniques that have been successfully demonstrated in flight test. The blended approach presented has a focus on experiment design and identifies a sequential testing process with clearly defined completion metrics that produce increased testing efficiency.

Murphy, Patrick C.↗
DOE OSTI2024

Practical and Optimal Sequential Bayesian Experimental Design for Complex Systems Incorporating Human Experimenter Preferences (Final Scientific/Technical Report)

Experiments are indispensable for developing models of complex systems. Carefully designed experiments can provide substantial savings for these expensive data-acquisition opportunities. However, designs based on heuristics are often suboptimal for systems with multiphysics, nonlinear dynamics, and uncertain and noisy environments. Optimal experimental design, while leveraging predictive models, seeks to systematically quantify and maximize the value of experiments. In this project, we focused on the design of multiple experiments, where current approaches are largely suboptimal: batch-design does not adapt to new data acquired during the experiment campaign (no feedback), and greedy/myopic design ignores future dynamics and consequences (no lookahead). We developed the mathematical framework and computational methods for sequential optimal experimental design (sOED) for complex systems. We enabled tractable model-based sOED in a rigorous manner through novel algorithms based on reinforcement learning, and investigated the effects of human experimenters on the design process. Our methods are fully Bayesian, able to quantify and update uncertainty in a principled manner. The traits aimed by our approach—mathematical rigor and optimality, human effects and uncertainty quantification, computational practicality—are crucial for elevating the standards of artificial intelligence (AI) to support decision-making in scientific domains, and contribute toward trust and realistic adoption of AI in experimental design practice.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2013

Validating an Air Traffic Management Concept of Operation Using Statistical Modeling

Validating a concept of operation for a complex, safety-critical system (like the National Airspace System) is challenging because of the high dimensionality of the controllable parameters and the infinite number of states of the system. In this paper, we use statistical modeling techniques to explore the behavior of a conflict detection and resolution algorithm designed for the terminal airspace. These techniques predict the robustness of the system simulation to both nominal and off-nominal behaviors within the overall airspace. They also can be used to evaluate the output of the simulation against recorded airspace data. Additionally, the techniques carry with them a mathematical value of the worth of each prediction-a statistical uncertainty for any robustness estimate. Uncertainty Quantification (UQ) is the process of quantitative characterization and ultimately a reduction of uncertainties in complex systems. UQ is important for understanding the influence of uncertainties on the behavior of a system and therefore is valuable for design, analysis, and verification and validation. In this paper, we apply advanced statistical modeling methodologies and techniques on an advanced air traffic management system, namely the Terminal Tactical Separation Assured Flight Environment (T-TSAFE). We show initial results for a parameter analysis and safety boundary (envelope) detection in the high-dimensional parameter space. For our boundary analysis, we developed a new sequential approach based upon the design of computer experiments, allowing us to incorporate knowledge from domain experts into our modeling and to determine the most likely boundary shapes and its parameters. We carried out the analysis on system parameters and describe an initial approach that will allow us to include time-series inputs, such as the radar track data, into the analysis

Statistical emulation↗
NASA NTRS1990

The FIFE information system

A description is given of the FIFE (First ISLSCP Field Experiment) information system, which was developed to serve FIFE investigators as a tool for designing the experiment and for organizing and manipulating the complex data set. Fulfilling these functions on an experiment-driven timeline led to abandoning the classical sequential development paradigm of software engineering in favor of a more responsive and broadly based approach. The design, development, and operation of the information system supporting the experiment had to be flexible and under direct day-to-day control of scientist/users. Because of the organization around scientific requirements, the system was able to incorporate diverse data types in a systematic way as they became available, to add scientific rigor by identifying data gaps at an early stage, and to provide real-time quality assurance. These factors are important for designing and building future databases and long-term information systems to support interdisciplinary scientific research.

Strebel, Donald E.↗
NASA NTRS1962

Sequential Phenomena in Psychophysical Judgments: A Theoretical Analysis

This paper deals with an analysis of psychophysical detection experiments designed to assess the limit of a human observer's level of sensitivity. A mathematical theory or the detection process is introduced that, in contrast to previous theories, provides an analysis of the sequential effects observed in psychophysical data. Two variations of the detection task are considered: information feedback and no-information feedback. In the feedback situation the subject is given information concerning the correctness of his responses, whereas in the no-feedback situation he is not. Data from a visual detection experiment with no-information feedback, and from an auditory detection experiment with information feedback are analyzed in terms of the theory. Finally, some general results are derived concerning the relationship between performance in the feedback situation and the no-feedback situation.

Atkinson, R. C.↗
NASA NTRS2018

Space-Filling Designs for Multi-Layer Nested Factors

This articles considers computer experiments where levels for continuous factors are selected in sequential order with the level selected for one factor directly a ecting the range of possible levels for the nested factor, and so on for a nite number of factors. In addition, we assume the nested relationships between the factors have no closed form solution. In this paper, we propose an approach for constructing a multi-layer nested factor design, or multi-NFD for short. This space- lling design approach takes advan- tage of the maximin criterion and can be analyzed using a standard Gaussian process model. While the multi-NFD approach can be adapted for future computer experi- ments involving factor relationships of this type, we present results from a particular aerospace computer simulation study.

Gryder, Ryan W.↗
NASA NTRS2015

Statistical Engineering in Air Traffic Management Research

NASA is working to develop an integrated set of advanced technologies to enable efficient arrival operations in high-density terminal airspace for the Next Generation Air Transportation System. This integrated arrival solution is being validated and verified in laboratories and transitioned to a field prototype for an operational demonstration at a major U.S. airport. Within NASA, this is a collaborative effort between Ames and Langley Research Centers involving a multi-year iterative experimentation process. Designing and analyzing a series of sequential batch computer simulations and human-in-the-loop experiments across multiple facilities and simulation environments involves a number of statistical challenges. Experiments conducted in separate laboratories typically have different limitations and constraints, and can take different approaches with respect to the fundamental principles of statistical design of experiments. This often makes it difficult to compare results from multiple experiments and incorporate findings into the next experiment in the series. A statistical engineering approach is being employed within this project to support risk-informed decision making and maximize the knowledge gained within the available resources. This presentation describes a statistical engineering case study from NASA, highlights statistical challenges, and discusses areas where existing statistical methodology is adapted and extended.

Wilson, Sara R.↗
NASA NTRS1992

The Critical Point Facility (CPF)

The Critical Point Facility (CPF) is an ESA multiuser facility designed for microgravity research onboard Spacelab. It has been conceived and built to offer investigators opportunities to conduct research on critical point phenomena in microgravity. This facility provides the high precision and stability temperature standards required in this field of research. It has been primarily designed for the purpose of optical investigations of transparent fluids. During a Spacelab mission, the CPF automatically processes several thermostats sequentially, each thermostat corresponding to an experiment. The CPF is now integrated in Spacelab at Kennedy Space Center, in preparation for the International Microgravity Lab. mission. The CPF was designed to submit transparent fluids to an adequate, user defined thermal scenario, and to monitor their behavior by using thermal and optical means. Because they are strongly affected by gravity, a good understanding of critical phenomena in fluids can only be gained in low gravity conditions. Fluids at the critical point become compressed under their own weight. The role played by gravity in the formation of interfaces between distinct phases is not clearly understood.

Source record↗
NASA NTRS1974

Automated microbial metabolism laboratory

The design and rationale of an advanced labeled release experiment based on single addition of soil and multiple sequential additions of media into each of four test chambers are outlined. The feasibility for multiple addition tests was established and various details of the methodology were studied. The four chamber battery of tests include: (1) determination of the effect of various atmospheric gases and selection of that gas which produces an optimum response; (2) determination of the effect of incubation temperature and selection of the optimum temperature for performing Martian biochemical tests; (3) sterile soil is dosed with a battery of C-14 labeled substrates and subjected to experimental temperature range; and (4) determination of the possible inhibitory effects of water on Martian organisms is performed initially by dosing with 0.01 ml and 0.5 ml of medium, respectively. A series of specifically labeled substrates are then added to obtain patterns in metabolic 14CO2 (C-14)O2 evolution.

Source record↗
NASA NTRS1993

Efficient partitioning and assignment on programs for multiprocessor execution

The general problem studied is that of segmenting or partitioning programs for distribution across a multiprocessor system. Efficient partitioning and the assignment of program elements are of great importance since the time consumed in this overhead activity may easily dominate the computation, effectively eliminating any gains made by the use of the parallelism. In this study, the partitioning of sequentially structured programs (written in FORTRAN) is evaluated. Heuristics, developed for similar applications are examined. Finally, a model for queueing networks with finite queues is developed which may be used to analyze multiprocessor system architectures with a shared memory approach to the problem of partitioning. The properties of sequentially written programs form obstacles to large scale (at the procedure or subroutine level) parallelization. Data dependencies of even the minutest nature, reflecting the sequential development of the program, severely limit parallelism. The design of heuristic algorithms is tied to the experience gained in the parallel splitting. Parallelism obtained through the physical separation of data has seen some success, especially at the data element level. Data parallelism on a grander scale requires models that accurately reflect the effects of blocking caused by finite queues. A model for the approximation of the performance of finite queueing networks is developed. This model makes use of the decomposition approach combined with the efficiency of product form solutions.

Standley, Hilda M.↗
DOE OSTI2024

Optimal experimental design: Formulations and computations

Questions of ‘how best to acquire data’ are essential to modelling and prediction in the natural and social sciences, engineering applications, and beyond. Optimal experimental design (OED) formalizes these questions and creates computational methods to answer them. This article presents a systematic survey of modern OED, from its foundations in classical design theory to current research involving OED for complex models. We begin by reviewing criteria used to formulate an OED problem and thus to encode the goal of performing an experiment. We emphasize the flexibility of the Bayesian and decision-theoretic approach, which encompasses information-based criteria that are well-suited to nonlinear and non-Gaussian statistical models. We then discuss methods for estimating or bounding the values of these design criteria; this endeavour can be quite challenging due to strong nonlinearities, high parameter dimension, large per-sample costs, or settings where the model is implicit. A complementary set of computational issues involves optimization methods used to find a design; we discuss such methods in the discrete (combinatorial) setting of observation selection and in settings where an exact design can be continuously parametrized. Finally we present emerging methods for sequential OED that build non-myopic design policies, rather than explicit designs; these methods naturally adapt to the outcomes of past experiments in proposing new experiments, while seeking coordination among all experiments to be performed. Throughout, we highlight important open questions and challenges.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Machine learning for automated experimentation in scanning transmission electron microscopy

Abstract Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centric experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows, as well as the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Payload concepts for investigations of electrostatic dust motion on the lunar surface

Significant experimental and computational investigations have explored the feasibility of electrostatically-motivated dust motion on the lunar surface. The motion of lunar dust influences our understanding of the evolution of the surface and may also present a hazard to future exploration vehicles and astronauts. The possibility of a sustained exploration presence on the lunar surface opens the door to long-term experiments on the lunar surface, akin to the science facilities on the International Space Station. We have identified four measurements/observations that would significantly advance our understanding of dust-plasma interactions on the lunar surface. In this context, we provide conceptual designs for payloads to obtain these observations: a Langmuir probe, dust deposit witness plate, regolith charge measurement instrument, and cameras to look for evidence of horizon glow. These payloads could deploy independently and sequentially, or together as a suite. The proposed payloads would provide key observations that would inform future modeling efforts and direct future in situ experiments to understand the dust-plasma environment, both for planetary science and spacecraft design applications.

42 ENGINEERING↗
DOE OSTI2022

Influence of sequential stimulation practices on geochemical alteration of shale

Water-based hydraulic fracturing fluids (HFFs) can chemically interact with formation shale, resulting in altered porosity and permeability of the host rock. Experimental investigations of spatial and temporal shale-HFF interactions are helpful in interpreting chemical compositions of the injectate, as well as predicting alteration of hydraulic properties in the reservoir due to mineral dissolution and precipitation. Most bench-top experiments designed to study shale-HFF chemical interactions, either using batch reactors or flow-through setups, are carried out assuming that the acid spearhead has already become mixed with neutral HFFs. During operations, however, HFFs are typically injected according to a sequenced pumping schedule, starting with a concentrated acid spearhead, followed by multiple additions of near-neutral pH HFFs containing chemical amendments and proppant. In this study, we use geochemical modeling to consider whether this pre-mixed experimental protocol provides results directly comparable to a sequential discrete fluid-shale interaction protocol. Our results show that for the batch system, the transient evolution in major ion concentrations is faster with the sequential procedure. After 2 h of reaction time, the two protocols converge to the same aqueous concentrations. In a flow-through geometry, the pre-mixed model predicts extensive chemical alteration close to the injection point but negligible alteration downstream. In contrast, the sequential model predicts mineral reactions over hundreds of meters along the flow path. The extent of shale alteration in the sequential model at a given location depends on shale mineralogy and where the acid spearhead resides during the shut-in period. The predictive model developed in this study can help experimentalists to design bench-top tests and operators to better translate the results of laboratory experiments into practical applications.

Li, Qingyun↗
NASA NTRS2001

Space Station Human Factors: Designing a Human-Robot Interface

The experiments described in this paper are part of a larger joint MIT/NASA research effort and focus on the development of a methodology for designing and evaluating integrated interfaces for highly dexterous and multifunctional telerobot. Specifically, a telerobotic workstation is being designed for an Extravehicular Activity (EVA) anthropomorphic space station telerobot called Robonaut. Previous researchers have designed telerobotic workstations based upon performance of discrete subsets of tasks (for example, peg-in-hole, tracking, etc.) without regard for transitions that operators go through between tasks performed sequentially in the context of larger integrated tasks. The experiments presented here took an integrated approach to describing teleoperator performance and assessed how subjects operating a full-immersion telerobot perform during fine position and gross position tasks. In addition, a Robonaut simulation was also developed as part of this research effort, and experimentally tested against Robonaut itself to determine its utility. Results show that subject performance of teleoperated tasks using both Robonaut and the simulation are virtually identical, with no significant difference between the two. These results indicate that the simulation can be utilized as both a Robonaut training tool, and as a powerful design platform for telepresence displays and aids.

Rochlis, Jennifer L.↗
DOE OSTI2025

Bayesian D‐Optimal Designs for Gaussian Process Surrogate Models

Computer experiments often employ space-filling strategies to create surrogate models with strong predictive performance. The impact of model parameter estimation for Gaussian process surrogates, however, is often overlooked. Obtaining a better initial estimate of the covariance lengthscale parameter, θ, can greatly improve the resulting Gaussian process fit through more effective sequential acquisitions during active learning. In this work, we propose a novel initial design maximizing the Bayesian D-optimality criterion of the Gaussian process lengthscale parameter. Previously published results have shown the emphasis on lengthscale estimation to be promising, but relied on an empirically driven design creation process. Our Bayesian D-optimal designs are rooted in information theory and lead to more informative sequential acquisitions by improving lengthscale estimation. In many cases, these gains eventually result in better surrogates than those seeded with space-filling initial designs. Furthermore, Bayesian D-optimal designs can be tailored to either isotropic or anisotropic covariance structures, and the Bayesian framework enables the inclusion of prior knowledge in the design process, offering greater flexibility and adaptability. Through several simulation studies, we demonstrate the advantages of Bayesian D-optimal designs in terms of both lengthscale estimation accuracy and predictive performance during active learning.

Bayesian experimental design↗