Automated and Autonomous Experiments in Electron and Scanning Probe Microscopy
Not Available
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
SEARCH · Engineering Papers
Results for “automated and autonomous experiments”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Deep learning for electron and scanning probe microscopy: From materials design to atomic fabrication
Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment in imaging. Here, we discuss recent progress in application of machine learning methods in scanning transmission electron microscopy and scanning probe microscopy, from applications such as data compression and exploratory data analysis to physics learning to atomic fabrication.
36 MATERIALS SCIENCE↗
Active oversight and quality control in standard Bayesian optimization for autonomous experiments
The fusion of experimental automation and machine learning has catalyzed a new era in materials research, prominently featuring Gaussian Process (GP) Bayesian Optimization (BO) driven autonomous experiments. Here we introduce a Dual-GP approach that enhances traditional GPBO by adding a secondary surrogate model to dynamically constrain the experimental space based on real-time assessments of the raw experimental data. This Dual-GP approach enhances the optimization efficiency of traditional GPBO by isolating more promising space for BO sampling and more valuable experimental data for primary GP training. We also incorporate a flexible, human-in-the-loop intervention method in the Dual-GP workflow to adjust for unanticipated results. We demonstrate the effectiveness of the Dual-GP model with synthetic model data and implement this approach in autonomous pulsed laser deposition experimental data. This Dual-GP approach has broad applicability in diverse GPBO-driven experimental settings, providing a more adaptable and precise framework for refining autonomous experimentation for more efficient optimization.
36 MATERIALS SCIENCE↗
Improving Self-Driving Labs: Quantifying System-Level Experiment Repeatability and Broadening Instrument-Level Compatibility
Modular Autonomous Research System (MARS) is a self-driving laboratory (SDL) which performs wet-lab science with peptide-lanthanide combinations in an automated and, ultimately, an autonomous manner to aid in soil analysis for domestic lithium mining. Autonomous experimentation involves automated experimentation, experiment planning, and active learning. MARS consists of a 6-axis robotic arm (UR5e) on a linear rail, pipette robots (Opentrons 2), and microplate readers. These components transport, operate on, and collect data with chemical solutions in standard labware. For effective autonomy, MARS must perform system-level labware operations repeatably, plan experiments autonomously, and be portable between research-domains. Repeatability is evaluated by labware placement precision, such that future operations can properly locate labware, as well as the elapsed time, so that low variance mean estimates of experiment duration can inform high-level researcher decision making. Autonomous experiment planning is the next step to decouple experimentation from human management; however, there is a conflict between the ideal system-level experiment goals and the constraints imposed by instruments’ limitations. Sub-domain portability is a long-term goal to extend MARS’ research beyond the chemistry of peptide-lanthanide binding to other sub-domains without having to invest significant overhead to system retrofitting. To address these goals, we manually trained the robotic arm labware placement and modelled statistical failurerate and uncertainty Additionally, we benchmarked the duration and variance of each experiment sub-operation as a heuristic for research decision making. Next, we use a parameterized geometric program (PGP) approach to design experiments that optimize system-level objectives and satisfy instrument-level constraints. Lastly, we proposed a Python framework to maximize MARS’ extensibility to other scientific sub-domains through a JSON-based experiment specification.
36 MATERIALS SCIENCE↗
Automated scanning probe microscopy of combinatorial ferroelectric libraries: Gaussian-process-guided exploration and noise-aware experiment planning
Combinatorial materials libraries provide an efficient route for mapping composition–property relationships, but their broader impact depends on rapid, quantitative, and functionally relevant characterization. Scanning Probe Microscopy (SPM), including piezoresponse force microscopy (PFM), offers significant potential for quantitative, functionally relevant combi-library readouts. Here, we implement a fully automated SPM workflow for ferroelectric combinatorial libraries and benchmark Gaussian-process-based Bayesian optimization strategies for autonomous experiment planning. The workflow integrates automated probe motion, contact optimization, imaging, and dual amplitude resonance tracking-PFM spectroscopy, and uses scalarized spectroscopic observables to guide subsequent measurements. Stage motion, probe engagement, in-contact tuning, imaging, spectroscopy, and the choice of the next measurement location all proceed without human input. We demonstrate the approach on Sm-doped BiFeO 3 and Zn x Mg 1−x O libraries. By comparing vanilla Bayesian optimization with a measured-noise variant, we show that explicit treatment of local reproducibility can improve modeling of composition-dependent response when the measured variance is physically meaningful, but can also reduce robustness when variability is dominated by outliers or topographic artifacts. Furthermore, these results establish automated SPM as a bridge between combinatorial synthesis and quantitative functional characterization.
Liu, Yu [University of Tennessee, Knoxville, TN (U↗
Disentangling Electronic Transport and Hysteresis at Individual Grain Boundaries in Hybrid Perovskites via Automated Scanning Probe Microscopy
Underlying the rapidly increasing photovoltaic efficiency and stability of metal halide perovskites (MHPs) is the advancement in the understanding of the microstructure of polycrystalline MHP thin film. Over the past decade, intense efforts have been aimed at understanding the effect of microstructures on MHP properties, including chemical heterogeneity, strain disorder, phase impurity, etc. It has been found that grain and grain boundary (GB) are tightly related to lots of microscale and nanoscale behavior in MHP thin films. Atomic force microscopy (AFM) is widely used to observe grain and boundary structures in topography and subsequently to study the correlative surface potential and conductivity of these structures. For now, most AFM measurements have been performed in imaging mode to study the static behavior; in contrast, AFM spectroscopy mode allows us to investigate the dynamic behavior of materials, e.g., conductivity under sweeping voltage. However, a major limitation of AFM spectroscopy measurements is that they require manual operation by human operators, and as such only limited data can be obtained, hindering systematic investigations of these microstructures. Here, we designed a workflow combining the conductive AFM measurement with a machine learning (ML) algorithm to systematically investigate grain boundaries in MHPs. The trained ML model can extract GBs locations from the topography image, and the workflow drives the AFM probe to each GB location to perform a current–voltage (IV) curve automatically. Then, we are able to have IV curves at all GB locations, allowing us to systematically understand the property of GBs. Using this method, we discovered that the GB junction points are less conductive, potentially more photoactive, and can play critical roles in MHP stability, while most previous works only focused on the difference between GB and grains.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Toward Unified Autonomous Scattering Experiments: A Cross-Facility Case Study at ALS and PETRA III
Autonomous experiments rely on the integration of control, data acquisition, analysis, and decision-making frameworks. While such systems have been demonstrated at individual facilities, adapting them to additional instruments remains challenging due to differences in local infrastructure. We present a modular workflow that connects existing open-source tools for data access (Tiled), workflow orchestration (Prefect), analysis and visualization (pyFAI, Plotly Dash), and Gaussian-process-based adaptive sampling (gpCAM) into a unified framework for autonomous scattering experiments. The same configuration operates across two synchrotron beamlines (ALS 7.3.3 and PETRA III P03) with only minimal facility-specific adjustments, as shown in proof-of-concept demonstrations. This validates that a consistent design emphasizing modularity and shared interfaces can ease deployment across diverse experimental environments. The resulting framework provides a flexible foundation for extending autonomous control and analysis capabilities beyond a single beamline or instrument.
47 OTHER INSTRUMENTATION↗
Cyber Framework for Steering and Measurements Collection Over Instrument-Computing Ecosystems
We propose a framework to develop cyber solutions to support the remote steering of science instruments and measurements collection over instrument-computing ecosystems. It is based on provisioning separate data and control connections at the network level, and developing software modules consisting of Python wrappers for instrument commands and Pyro server-client codes that make them available across the ecosystem network. We demonstrate automated measurement transfers and remote steering operations in a microscopy use case for materials research over an ecosystem of Nion microscopes and computing platforms connected over site networks. The proposed framework is currently under further refinement and being adopted to science workflows with automated remote experiments steering for autonomous chemistry laboratories and smart energy grid simulations.
Al Najjar, Anees↗
AEcroscopy: A Software–Hardware Framework Empowering Microscopy Toward Automated and Autonomous Experimentation
Microscopy has been pivotal in improving the understanding of structure-function relationships at the nanoscale and is by now ubiquitous in most characterization labs. However, traditional microscopy operations are still limited largely by a human-centric click-and-go paradigm utilizing vendor-provided software, which limits the scope, utility, efficiency, effectiveness, and at times reproducibility of microscopy experiments. Here, in this work, a coupled software–hardware platform is developed that consists of a software package termed AEcroscopy (short for Automated Experiments in Microscopy), along with a field-programmable-gate-array device with LabView-built customized acquisition scripts, which overcome these limitations and provide the necessary abstractions toward full automation of microscopy platforms. The platform works across multiple vendor devices on scanning probe microscopes and electron microscopes. It enables customized scan trajectories, processing functions that can be triggered locally or remotely on processing servers, user-defined excitation waveforms, standardization of data models, and completely seamless operation through simple Python commands to enable a plethora of microscopy experiments to be performed in a reproducible, automated manner. This platform can be readily coupled with existing machine-learning libraries and simulations, to provide automated decision-making and active theory-experiment optimization to turn microscopes from characterization tools to instruments capable of autonomous model refinement and physics discovery.
47 OTHER INSTRUMENTATION↗
Bayesian Conavigation: Dynamic Designing of the Material Digital Twins via Active Learning
Scientific advancement is universally based on the dynamic interplay between theoretical insights, modeling, and experimental discoveries. However, this feedback loop is often slow, including delayed community interactions and the gradual integration of experimental data into theoretical frameworks. This challenge is particularly exacerbated in domains dealing with high-dimensional object spaces, such as molecules and complex microstructures. Hence, the integration of theory within automated and autonomous experimental setups, or theory in the loop-automated experiment, is emerging as a crucial objective for accelerating scientific research. The critical aspect is to use not only theory but also on-the-fly theory updates during the experiment. Furthermore, we introduce a method for integrating theory into the loop through Bayesian conavigation of theoretical model space and experimentation. Our approach leverages the concurrent development of surrogate models for both simulation and experimental domains at the rates determined by latencies and costs of experiments and computation, alongside the adjustment of control parameters within theoretical models to minimize epistemic uncertainty over the experimental object spaces. This methodology facilitates the creation of digital twins of material structures, encompassing both the surrogate model of behavior that includes the correlative part and the theoretical model itself. While being demonstrated here within the context of functional responses in ferroelectric materials, our approach holds promise for broader applications, such as the exploration of optical properties in nanoclusters, microstructure-dependent properties in complex materials, and properties of molecular systems.
Microscopy↗
Towards Autonomous Experiments by Connecting High Performance Microscopy with High Performance Computing
The digitization of controls, data, and analysis in microscopy is bringing the idea of autonomous microscopes closer to reality than ever before. Automated transmission electron microscopy (TEM) is already fairly routine for some experiments the only require simple repetitive tasks such as imaging biological macromolecules for single particle cryoEM [1], tilt series for electron tomography [2], and movies for crystallography [3]. The vast majority of TEM experiments are conducted completely by human operators who choose the regions of interest, optimize experimental parameters, and make decisions about data quality visually during an experiment. The field is still a long way from having completely autonomous TEMs that can adapt to sample difficulties and tune experimental parameters based on data quality and desired experimental outcomes. Part of the issue is the lack of capability for feeding information learned from on-line, live data analysis back into the on-going experiment [4]. Furthermore, this presentation will discuss current capabilities for large scale data reduction and analysis using high performance computing (i.e. supercomputing) and progress towards developing a true feed-back loop that places data analysis and theory in the experimental loop.
97 MATHEMATICS AND COMPUTING↗
Beyond Optimization: Exploring Novelty Discovery in Autonomous Experiments
Autonomous experiments (AEs) are transforming how scientific research is conducted by integrating artificial intelligence with automated experimental platforms. Current AEs primarily focus on the optimization of a predefined target; while accelerating this goal, such an approach limits the discovery of unexpected or unknown physical phenomena. Here, we introduce a novel framework, INS 2 ANE (Integrated Novelty Score−Strategic Autonomous Non-Smooth Exploration), to enhance the discovery of novel phenomena in autonomous microscopy experimentation. Our method integrates two key components: (1) a novelty scoring system that evaluates the uniqueness of experimental results and (2) a strategic sampling mechanism that promotes exploration of under-sampled regions even if they appear less promising by conventional criteria. We validate this approach on a preacquired data set with a known ground truth comprising of image−spectral pairs. We further implement the process on autonomous scanning probe microscopy experiments. INS 2 ANE significantly increases the diversity of explored phenomena in comparison to conventional optimization routines, enhancing the likelihood of discovering previously unobserved phenomena. These results demonstrate the potential for autonomous microscopy experiments to enhance the scientific discovery by navigating complex experimental spaces to uncover novel phenomena.
Materials↗
Autonomous x-ray scattering
Autonomous experimentation (AE) is an emerging paradigm that seeks to automate the entire workflow of an experiment, including—crucially—the decision-making step. Beyond mere automation and efficiency, AE aims to liberate scientists to tackle more challenging and complex problems. We describe our recent progress in the application of this concept at synchrotron x-ray scattering beamlines. We automate the measurement instrument, data analysis, and decision-making, and couple them into an autonomous loop. We exploit Gaussian process modeling to compute a surrogate model and associated uncertainty for the experimental problem, and define an objective function exploiting these. We provide example applications of AE to x-ray scattering, including imaging of samples, exploration of physical spaces through combinatorial methods, and coupling to in situ processing platforms These uses demonstrate how autonomous x-ray scattering can enhance efficiency, and discover new materials.
77 NANOSCIENCE AND NANOTECHNOLOGY↗
Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning
Recent progress in machine learning methods and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs) have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. Here, the discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods and can be particularly impactful for destructive or irreversible measurements.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Enabling Modular Autonomous Feedback‐Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration
Abstract Materials acceleration platforms (MAPs) operate on the paradigm of integrating combinatorial synthesis, high‐throughput characterization, automatic analysis, and machine learning. Within a MAP, one or multiple autonomous feedback loops may aim to optimize materials for certain functional properties or to generate new insights. The scope of a given experiment campaign is defined by the range of experiment and analysis actions that are integrated into the experiment framework. Herein, the authors present a method for integrating many actions within a hierarchical experimental laboratory automation and orchestration (HELAO) framework. They demonstrate the capability of orchestrating distributed research instruments that can incorporate data from experiments, simulations, and databases. HELAO interfaces laboratory hardware and software distributed across several computers and operating systems for executing experiments, data analysis, provenance tracking, and autonomous planning. Parallelization is an effective approach for accelerating knowledge generation provided that multiple instruments can be effectively coordinated, which the authors demonstrate with parallel electrochemistry experiments orchestrated by HELAO. Efficient implementation of autonomous research strategies requires device sharing, asynchronous multithreading, and full integration of data management in experimental orchestration, which to the best of the authors’ knowledge, is demonstrated for the first time herein.
36 MATERIALS SCIENCE↗
Autonomous Experiments in Scanning Probe Microscopy and Spectroscopy: Choosing Where to Explore Polarization Dynamics in Ferroelectrics
Polarization dynamics in ferroelectric materials are explored via automated experiment in piezoresponse force microscopy/spectroscopy (PFM/S). A Bayesian optimization (BO) framework for imaging is developed, and its performance for a variety of acquisition and pathfinding functions is explored using previously acquired data. The optimized algorithm is then deployed on an operational scanning probe microscope (SPM) for finding areas of large electromechanical response in a thin film of PbTiO 3 , with results showing that, with just 20% of the area sampled, most high-response clusters were captured. Furthermore, this approach can allow performing more complex spectroscopies in SPM that were previously not possible due to time constraints and sample stability. Improvements to the framework to enable the incorporation of more prior information and improve efficiency further are modeled and discussed.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Exploring Benchmarks for Self-Driving Labs using Color Matching
Self Driving Labs (SDLs) that combine automation of experimental procedures with autonomous decision making are gaining popularity as a means of increasing the throughput of scientific workflows. The task of identifying quantities of supplied colored pigments that match a target color, the color matching problem, provides a simple and flexible SDL test case, as it requires experiment proposal, sample creation, and sample analysis, three common components in autonomous discovery applications. We present a robotic solution to the color matching problem that allows for fully autonomous execution of a color matching protocol. Our solution leverages the WEI science factory platform to enable portability across different robotic hardware, the use of alternative optimization methods for continuous refinement, and automated publication of results for experiment tracking and post-hoc analysis.
Artificial intelligence↗
Automated tracking of prefabricated components for areal-time evaluator to optimize and automateinstallation
Trade associations for prefabricated construction estimate that about 50% of prefabricated wall projects have alignment problems that lead to defects and rework. Additionally, component installation times average between 30 and 60 minutes per component. To address these issues, a real-time evaluator (RTE) system was introduced to decrease cost and automate prefabricated component installation by reducing the installation time, decreasing rework, and enhancing energy performance through higher installation quality. The RTE uses commonly available hardware and software to perform autonomous tracking to measure the real-time location and orientation of components as they are crane-lifted and installed. The hardware, software, and algorithms that allow the autonomous tracking of components are detailed. An algorithm to automate the initial search for a component with three attached retroreflectors is proposed. Algorithms to automate the measurement of component position and orientation are also proposed. Simple lab-scale proof-of-concept experiments were conducted to assess the algorithms for automation of component searching, measurement of real-time movement, and measurement of component orientation. With additional development, the system can be used as a tool to generate the commands for autonomous crane operation or single-task construction robots.
Hayes, Nolan↗