SEARCH · Engineering Papers

Results for “misclassification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2015

SU-E-J-241: Wavelet-Based Temporal Feature Extraction From DCE-MRI to Identify Sub-Volumes of Low Blood Volume in Head-And-Neck Cancer

Purpose: A previous study showed that large sub-volumes of tumor with low blood volume (BV) (poorly perfused) in head-and-neck (HN) cancers are significantly associated with local-regional failure (LRF) after chemoradiation therapy, and could be targeted with intensified radiation doses. This study aimed to develop an automated and scalable model to extract voxel-wise contrast-enhanced temporal features of dynamic contrastenhanced (DCE) MRI in HN cancers for predicting LRF. Methods: Our model development consists of training and testing stages. The training stage includes preprocessing of individual-voxel DCE curves from tumors for intensity normalization and temporal alignment, temporal feature extraction from the curves, feature selection, and training classifiers. For feature extraction, multiresolution Haar discrete wavelet transformation is applied to each DCE curve to capture temporal contrast-enhanced features. The wavelet coefficients as feature vectors are selected. Support vector machine classifiers are trained to classify tumor voxels having either low or high BV, for which a BV threshold of 7.6% is previously established and used as ground truth. The model is tested by a new dataset. The voxel-wise DCE curves for training and testing were from 14 and 8 patients, respectively. A posterior probability map of the low BV class was created to examine the tumor sub-volume classification. Voxel-wise classification accuracy was computed to evaluate performance of the model. Results: Average classification accuracies were 87.2% for training (10-fold crossvalidation) and 82.5% for testing. The lowest and highest accuracies (patient-wise) were 68.7% and 96.4%, respectively. Posterior probability maps of the low BV class showed the sub-volumes extracted by our model similar to ones defined by the BV maps with most misclassifications occurred near the sub-volume boundaries. Conclusion: This model could be valuable to support adaptive clinical trials with further validation. The framework could be extendable and scalable to extract temporal contrastenhanced features of DCE-MRI in other tumors. We would like to acknowledge NIH for funding support: UO1 CA183848.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2015

SU-E-T-261: Development of An Automated System to Detect Patient Identification and Positioning Errors Prior to Radiotherapy Treatment

Purpose: To develop a system that can automatically detect patient identification and positioning errors using 3D computed tomography (CT) setup images and kilovoltage CT (kVCT) planning images. Methods: Planning kVCT images were collected for head-and-neck (H&N), pelvis, and spine treatments with corresponding 3D cone-beam CT (CBCT) and megavoltage CT (MVCT) setup images from TrueBeam and TomoTherapy units, respectively. Patient identification errors were simulated by registering setup and planning images from different patients. Positioning errors were simulated by misaligning the setup image by 1cm to 5cm in the six anatomical directions for H&N and pelvis patients. Misalignments for spine treatments were simulated by registering the setup image to adjacent vertebral bodies on the planning kVCT. A body contour of the setup image was used as an initial mask for image comparison. Images were pre-processed by image filtering and air voxel thresholding, and image pairs were assessed using commonly-used image similarity metrics as well as custom -designed metrics. A linear discriminant analysis classifier was trained and tested on the datasets, and misclassification error (MCE), sensitivity, and specificity estimates were generated using 10-fold cross validation. Results: Our workflow produced MCE estimates of 0.7%, 1.7%, and 0% for H&N, pelvis, and spine TomoTherapy images, respectively. Sensitivities and specificities ranged from 98.0% to 100%. MCEs of 3.5%, 2.3%, and 2.1% were obtained for TrueBeam images of the above sites, respectively, with sensitivity and specificity estimates between 96.2% and 98.4%. MCEs for 1cm H&N/pelvis misalignments were 1.3/5.1% and 9.1/8.6% for TomoTherapy and TrueBeam images, respectively. 2cm MCE estimates were 0.4%/1.6% and 3.1/3.2%, respectively. Vertebral misalignment MCEs were 4.8% and 4.9% for TomoTherapy and TrueBeam images, respectively. Conclusion: Patient identification and gross misalignment errors can be robustly and automatically detected using 3D setup images of two imaging modalities across three commonly-treated anatomical sites.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2015

MO-F-CAMPUS-J-05: Toward MRI-Only Radiotherapy: Novel Tissue Segmentation and Pseudo-CT Generation Techniques Based On T1 MRI Sequences

Purpose: To develop and validate a 4 class tissue segmentation approach (air cavities, background, bone and soft-tissue) on T1 -weighted brain MRI and to create a pseudo-CT for MRI-only radiation therapy verification. Methods: Contrast-enhanced T1-weighted fast-spin-echo sequences (TR = 756ms, TE= 7.152ms), acquired on a 1.5T GE MRI-Simulator, are used.MRIs are firstly pre-processed to correct for non uniformity using the non parametric, non uniformity intensity normalization algorithm. Subsequently, a logarithmic inverse scaling log(1/image) is applied, prior to segmentation, to better differentiate bone and air from soft-tissues. Finally, the following method is enrolled to classify intensities into air cavities, background, bone and soft-tissue:Thresholded region growing with seed points in image corners is applied to get a mask of Air+Bone+Background. The background is, afterward, separated by the scan-line filling algorithm. The air mask is extracted by morphological opening followed by a post-processing based on knowledge about air regions geometry. The remaining rough bone pre-segmentation is refined by applying 3D geodesic active contours; bone segmentation evolves by the sum of internal forces from contour geometry and external force derived from image gradient magnitude.Pseudo-CT is obtained by assigning −1000HU to air and background voxels, performing linear mapping of soft-tissue MR intensities in [-400HU, 200HU] and inverse linear mapping of bone MR intensities in [200HU, 1000HU]. Results: Three brain patients having registered MRI and CT are used for validation. CT intensities classification into 4 classes is performed by thresholding. Dice and misclassification errors are quantified. Correct classifications for soft-tissue, bone, and air are respectively 89.67%, 77.8%, and 64.5%. Dice indices are acceptable for bone (0.74) and soft-tissue (0.91) but low for air regions (0.48). Pseudo-CT produces DRRs with acceptable clinical visual agreement to CT-based DRR. Conclusion: The proposed approach makes it possible to use T1-weighted MRI to generate accurate pseudo-CT from 4-class segmentation.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

SU-F-J-169: A Feasibility Study of Using MRI Alone in Abdominal Radiotherapy

Purpose: To demonstrate the feasibility of a MRI alone workflow to support treatment planning and image guidance for abdominal radiotherapy. Methods: Abdominal MR images (in-phase/out-phase/fat/water) were acquired for a patient with breath-hold using a Dixon pulse sequence. Air masks were created on in-phase images using intensity thresholding and morphological processing methods in order to separate air from bone. Pseudo-CT and DRRs were generated using a published method. To investigate the effect of heterogeneity corrections on dose calculations using pseudo-CT, three different plans (3-field 3D, 5-field IMRT and 2-arc VMAT) were performed to mimic pancreatic treatments (1.8Gy/fraction over 28 fractions). Results: The DRRs created from pseudo-CT were of comparable quality as those created from CT. Comparing dose calculations with and without heterogeneity corrections between the 3 different plans, the biggest dosimetric differences were seen in the VMAT plan where modulation must occur across air-tissue interfaces such as those of the stomach and bowel. The DVHs for the VMAT plan showed ∼84cc difference at V50Gy in the small bowel. In terms of pseudo-CT quality, some small volumes of air in the bowel and stomach were misclassified as bone. The VMAT plan was re-optimized on pseudo-CT with 0 HU in the misclassified areas. The V50Gy in the small bowel differed by ∼90cc between the new VMAT plan with and without heterogeneity corrections. Conclusion: We found that the use of MRI alone is feasible for abdominal treatment planning and image guidance. A difference between calculations with and without heterogeneity corrections was found that is most pronounced for VMAT where the traversal of air-tissue interfaces is unavoidable. Future work will be performed to minimize misclassification between bone and air.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

SU-F-T-115: Uncertainty in the Esophagus Dose in Retrospective Epidemiological Study of Breast Cancer Radiotherapy Patients

Purpose: Epidemiological studies of second cancer risks in breast cancer radiotherapy patients often use generic patient anatomy to reconstruct normal tissue doses when CT images of patients are not available. To evaluate the uncertainty involved in the dosimetry approach, we evaluated the esophagus dose in five sample patients by simulating breast cancer treatments. Methods: We obtained the diagnostic CT images of five anonymized adult female patients in different Body Mass Index (BMI) categories (16– 36kg/m2) from National Institutes of Health Clinical Center. We contoured the esophagus on the CT images and imported them into a Treatment Planning System (TPS) to create treatment plans and calculate esophagus doses. Esophagus dose was calculated once again via experimentally-validated Monte Carlo (MC) transport code, XVMC under the same geometries. We compared the esophagus doses from TPS and the MC method. We also investigated the degree of variation in the esophagus dose across the five patients and also the relationship between the patient characteristics and the esophagus doses. Results: Eclipse TPS using Analytical Anisotropic Algorithm (AAA) significantly underestimates the esophagus dose in breast cancer radiotherapy compared to MC. In the worst case, the esophagus dose from AAA was only 40% of the MC dose. The Coefficient of Variation across the patients was 48%. We found that the maximum esophagus dose was up to 2.7 times greater than the minimum. We finally observed linear relationship (Dose = 0.0218 × BMI – 0.1, R2=0.54) between patient’s BMI and the esophagus doses. Conclusion: We quantified the degree of uncertainty in the esophagus dose in five sample breast radiotherapy patients. The results of the study underscore the importance of individualized dose reconstruction for the study cohort to avoid misclassification in the risk analysis of second cancer. We are currently extending the number of patients up to 30.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2017

Simulated rRNA/DNA Ratios Show Potential To Misclassify Active Populations as Dormant

ABSTRACT The use of rRNA/DNA ratios derived from surveys of rRNA sequences in RNA and DNA extracts is an appealing but poorly validated approach to infer the activity status of environmental microbes. To improve the interpretation of rRNA/DNA ratios, we performed simulations to investigate the effects of community structure, rRNA amplification, and sampling depth on the accuracy of rRNA/DNA ratios in classifying bacterial populations as “active” or “dormant.” Community structure was an insignificant factor. In contrast, the extent of rRNA amplification that occurs as cells transition from dormant to growing had a significant effect ( P < 0.0001) on classification accuracy, with misclassification errors ranging from 16 to 28%, depending on the rRNA amplification model. The error rate increased to 47% when communities included a mixture of rRNA amplification models, but most of the inflated error was false negatives (i.e., active populations misclassified as dormant). Sampling depth also affected error rates ( P < 0.001). Inadequate sampling depth produced various artifacts that are characteristic of rRNA/DNA ratios generated from real communities. These data show important constraints on the use of rRNA/DNA ratios to infer activity status. Whereas classification of populations as active based on rRNA/DNA ratios appears generally valid, classification of populations as dormant is potentially far less accurate. IMPORTANCE The rRNA/DNA ratio approach is appealing because it extracts an extra layer of information from high-throughput DNA sequencing data, offering a means to determine not only the seedbank of taxa present in communities but also the subset of taxa that are metabolically active. This study provides crucial insights into the use of rRNA/DNA ratios to infer the activity status of microbial taxa in complex communities. Our study shows that the approach may not be as robust as previously supposed, particularly in complex communities composed of populations employing different growth strategies, and identifies factors that inflate the erroneous classification of active populations as dormant.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2016

Methodological Guidelines for Accurate Detection of Viruses in Wild Plant Species

Here ecological understanding of disease risk, emergence, and dynamics and of the efficacy of control strategies relies heavily on efficient tools for microorganism identification and characterization. Misdetection, such as the misclassification of infected hosts as healthy, can strongly bias estimates of disease prevalence and lead to inaccurate conclusions. In natural plant ecosystems, interest in assessing microbial dynamics is increasing exponentially, but guidelines for detection of microorganisms in wild plants remain limited, particularly so for plant viruses. To address this gap, we explored issues and solutions associated with virus detection by serological and molecular methods in noncrop plant species as applied to the globally important Barley yellow dwarf virus PAV ( Luteoviridae ), which infects wild native plants as well as crops. With enzyme-linked immunosorbent assays (ELISA), we demonstrate how virus detection in a perennial wild plant species may be much greater in stems than in leaves, although leaves are most commonly sampled, and may also vary among tillers within an individual, thereby highlighting the importance of designing effective sampling strategies. With reverse transcription-PCR (RT-PCR), we demonstrate how inhibitors in tissues of perennial wild hosts can suppress virus detection but can be overcome with methods and products that improve isolation and amplification of nucleic acids. These examples demonstrate the paramount importance of testing and validating survey designs and virus detection methods for noncrop plant communities to ensure accurate ecological surveys and reliable assumptions about virus dynamics in wild hosts.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Robust Containment Queries over Collections of Rational Parametric Curves via Generalized Winding Numbers

Point containment queries for regions bound by watertight geometric surfaces, i.e., closed and without self-intersections, can be evaluated straightforwardly with a number of well-studied algorithms. When this assumption on domain geometry is not met, such methods are either unusable, or prone to misclassifications that can lead to cascading errors in downstream applications. More robust point classification schemes based on generalized winding numbers have been proposed, as they are indifferent to these imperfections. However, existing algorithms are limited to point clouds and collections of linear elements. We extend this methodology to encompass more general curved shapes with an algorithm that evaluates the winding number scalar field over unstructured collections of rational parametric curves. In particular, we evaluate the winding number for each curve independently, making the derived containment query robust to how the curves are arranged. We ensure geometric fidelity in our queries by treating each curve as equivalent to an adaptively constructed polyline that provably has the same generalized winding number at the point of interest. Our algorithm is numerically stable for points that are arbitrarily close to the model, and explicitly treats points that are coincident with curves. We demonstrate the improvements in computational performance granted by this method over conventional techniques as well as the robustness induced by its application.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Robust Containment Queries over Collections of Trimmed NURBS Surfaces via Generalized Winding Numbers

Here, we propose a containment query that is robust to the watertightness of regions bound by trimmed NURBS surfaces, as this property is difficult to guarantee for in-the-wild CAD models. Containment is determined through the generalized winding number (GWN), a mathematical construction that is indifferent to the arrangement of surfaces in the shape. Applying contemporary techniques for the 3D GWN to trimmed NURBS surfaces requires some form of geometric discretization, introducing computational inefficiency to the algorithm and even risking containment misclassifications near the surface. In contrast, our proposed method leverages properties of the 3D solid angle to solve the relevant surface integral using a boundary formulation with rapidly converging adaptive quadrature. Batches of queries are further accelerated by memoizing (i.e., caching and reusing) quadrature node positions and tangents as they are evaluated. We demonstrate that our GWN method is robust to complex trimming geometry in a CAD model, and is accurate up to arbitrary precision at arbitrary distances from the surface. The derived containment query is therefore robust to model non-watertightness while respecting all curved features of the input shape.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Development of a Random-Forest Cloud-Regime Classification Model Based on Surface Radiation and Cloud Products

Various methods have been developed to characterize cloud type, otherwise referred to as cloud regime. These include manual sky observations, combining radiative and cloud vertical properties observed from satellite, surface-based remote sensing, and digital processing of sky imagers. While each method has inherent advantages and disadvantages, none of these cloud-typing methods actually includes measurements of surface shortwave or longwave radiative fluxes. Here, a method that relies upon detailed, surface-based radiation and cloud measurements and derived data products to train a random-forest machine-learning cloud classification model is introduced. Measurements from five years of data from the ARM Southern Great Plains site were compiled to train and independently evaluate the model classification performance. A cloud-type accuracy of approximately 80% using the random-forest classifier reveals that the model is well suited to predict climatological cloud properties. Furthermore, an analysis of the cloud-type misclassifications is performed. While physical cloud types may be misreported, the shortwave radiative signatures are similar between misclassified cloud types. From this, we assert that the cloud-regime model has the capacity to successfully differentiate clouds with comparable cloud–radiative interactions. Therefore, we conclude that the model can provide useful cloud-property information for fundamental cloud studies, inform renewable energy studies, and be a tool for numerical model evaluation and parameterization improvement, among many other applications.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

CAT: computer aided triage improving upon the Bayes risk through ε-refusal triage rules

Manual extraction of information from electronic pathology (epath) reports to populate the Surveillance, Epidemiology, and End Result (SEER) database is labor intensive. Systematizing the data extraction automatically using machine-learning (ML) and natural language processing (NLP) is desirable to reduce the human labor required to populate the SEER database and to improve the timeliness of the data. This enables scaling up registry efficiency and collection of new data elements. To ensure the integrity, quality, and continuity of the SEER data, the misclassification error of ML and NPL algorithms needs to be negligible. Current algorithms fail to achieve the precision of human experts who can bring additional information in their assessments. Differences in registry format and the desire to develop a common information extraction platform further complicate the ML/NLP tasks. The purpose of our study is to develop triage rules to partially automate registry workflow to improve the precision of the auto-extracted information. This paper presents a mathematical framework to improve the precision of a classifier beyond that of the Bayes classifier by selectively classifying item that are most likely to be correct. This results in a triage rule that only classifies a subset of the item. We characterize the optimal triage rule and demonstrate its usefulness in the problem of classifying cancer site from electronic pathology reports to achieve a desired precision. From the mathematical formalism, we propose a heuristic estimate for triage rule based on post-processing the soft-max output from standard machine learning algorithms. We show, in test cases, that the triage rule significantly improve the classification accuracy.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2023

Testing and Design of Discriminants for Local Seismic Events Recorded during the Redmond Salt Mine Monitoring Experiment

The Redmond Salt Mine (RSM) Monitoring Experiment in Utah was designed to record seismoacoustic data at distances less than 50 km for algorithm testing and development. During the experiment from October 2017 to July 2019, six broadband seismic stations were operating at a time, with three of them having fixed locations for the duration, whereas the three other stations were moved to different locations every one-and-half to two-and-half months. RSM operations consist of nighttime underground blasting several times per week. The RSM is located in proximity to a belt of active seismicity, allowing direct comparison of natural and anthropogenic sources. Using the recorded data set, we built 1373 events with local magnitude (M L ) of -2.4 and lower to 3.3. For 75 blasts (RMEs) from the Redmond Salt Mine and 206 tectonic earthquakes (EQs), both M L and the coda duration magnitude (MC) are well constrained. We used these events to test and design discriminants that separate the RMEs from the EQs and are effective at local distances. The discriminants consist of M L -M C , low-frequency Sg to high-frequency Sg, Pg/Sg phase-amplitude ratios, and Rg/Sg spectral amplitude ratios, as well as different combinations of two or more of these classifiers. The areas under the receiver operating characteristic curves (AUCs) of 0.92–1.0 for M L -M C , low-frequency Sg to high-frequency Sg, and Rg/Sg indicate that these discriminants are very effective. Conversely, the AUC of only 0.57 for Pg/Sg suggests that this discriminant is only slightly better than a random classifier. Among the effective classifiers, Rg/Sg, shows the lowest likelihood of misclassification (4.3%) for the populations. In conclusion, results of joint discriminant analyses suggest that even the arguably ineffective single classifier, like Pg/Sg in this case, can provide some value when used in combination with others.

58 GEOSCIENCES↗

DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗

DOE OSTI2021

Evaluating Deception Detection Model Robustness To Linguistic Variation

With the increasing use of automated, machine learning-driven tools and the downstream impact that algorithmic judgements can have, it is critical to develop models that are robust to evolving or manipulated inputs. Evaluating the reliability of multimodal models across linguistic variations to understand model susceptibility to intentional linguistic adversarial attacks as well as natural linguistic variations is essential in this pursuit. We present extensive analysis of model robustness and susceptibility to linguistic variations in the setting of deceptive news detection, a difficult classification task that is an increasingly important problem to solve with the impact of misinformation spread online. We evaluate the effectiveness of incorporating adversarial defense strategies and measure model susceptibility to state-of-the-art adversarial attacks using two types of linguistic attacks — character and word perturbations. We consider two multiclass prediction tasks — a 3-way classification of tweets as trustworthy, propaganda, or disinformation; and a 4-way classification as clickbait, hoax, satire, or conspiracy — and compare the performance of three embeddings that have been state-of-the-art for several NLP tasks — GloVe, ELMo, and BERT — to highlight consistent trends in susceptibility, high confidence misclassifications, and high impact failures. We find that character or mixed ensemble models are the most effective defense mechanisms and that character perturbations are a more effective attack than word perturbations for deception classification.

adversarial evaluation↗

DOE OSTI2021

Eject, crash, or survive: Using machine learning to predict orbital instability of exoplanetary systems

Astronomers throughout history, including titans like Kepler and Newton, have tackled planetary dynamics and orbital instability. Despite strides taken in research, understanding the evolution of planetary orbits remains an intricate, computationally expensive, and analytically unsolved problem. I apply machine learning classification methods to numerical simulations of planetary systems in order to predict the long-term fate of the planet - whether the planet remains in a stable orbit or not. My method uses the first 41.1 years (≤ 500 orbits) of data from a planet’s simulation to calculate 17 dynamically-motivated metrics; I trained my classifier on these features to predict a planet’s stability after 107 years. At 84.33%, my classifier was comparable in accuracy to pre-existing literature, despite using significantly less computational power than most other methods. In my research, I found that the standard deviation of eccentricity, mass ratios for neighboring planets, and semi-major axis ratio with the outer planet neighbor to be the most predictive features of instability. I propose reasons for the importance of these features, their role in planetary dynamics, as well as possible explanations for why some planets were misclassified. By understanding the important metrics of instability and reasons for misclassification, we can begin to understand more about system architectures, orbital motion and dynamics, and the formation and evolution of the exoplanetary systems. This is applicable in our own Solar System, and with exoplanet discovery missions such as TESS, this research becomes especially relevant in understanding the new exoplanetary systems we discover.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Eject, crash, or survive: Using machine learning to predict orbital instability of exoplanetary systems

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

A Machine Learns to Predict the Stability of Highly Diverse Multi-planetary Systems

Machine learning has proven to be an invaluable tool for characterizing the stability of planets in simplified planetary systems. In this work, we investigate the performance of a machine learning classifier on tightlypacked systems containing a rich diversity of planets, from Earths to Jupiters. Using information derived from short numerical simulations about a planet’s early orbital evolution and its relationship with the most massive planets in the system, we train a random forest classifier to predict instability with a > 88 percent accuracy. Our classifier relies on relative planet masses and the standard deviation of eccentricity for much of its predictive power. Most misclassified planets lie along a multi-dimensional boundary between stable and unstable planets, indicating that their early orbital evolution is ambiguous. The major reason for misclassification in this work is timescale: because our classifier uses information from only the first 137 years of simulation data, it is blind to late time interactions that cause or prevent instability. Machine learning methods like those utilized in this work provide powerful tools to complement numerical simulations across a wide range of planetary architectures.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Autonomous System Subversion Tactics: Prototypes and Recommended Countermeasures

One of the fielding requirements for Advanced and Small Modular Reactors (AR/SMR) is the ability to support remote and autonomous operations. Autonomous Control Systems (ACS) are found on platforms such as Autonomous Space Vehicles, Cruise Missiles, and advanced driver-assistance systems. Each of these ACS implementations depends upon a set of decision support subsystems responsible for supporting Autonomous Mission Managers (names vary based upon field and author preferences). These Autonomous Mission Managers receive inputs from system sensors (e.g., LIDAR collection from an automobile travelling down a street; transients from a nuclear reactor), and perform a set of classifications (e.g., Red Traffic Light; Small Pedestrian at 10m; Load Rejection; Single Coolant Pump Trip), and then use these classifications in combination with recommendation algorithms to achieve platform goals (e.g., Stop the Vehicle at the Traffic Light, Avoid the Small Pedestrian; Trip the Reactor to prevent a Safety Event). The design, implementation, and fielding of an ACS capability will alter the cyber-attack surface such that existing risk management plans will need to be updated to include how to protect and defend against data-science and decision-support-system attack classes. These attack classes would include protection of the design and training environments where algorithm selection and testing and training data would be obvious attack vectors. These attack classes would also require an informed set of detection and response procedures to identify anomalous behaviors and document best practices for anomaly assessment and vulnerability mitigation and remediation. Last year we published a Cyber Threat Assessment Methodology for Autonomous and Remote Operations for AR/SMRs along with a companion publication on Cyber Attack and Defense Use Cases. The focus of the methodology was on describing and enumerating ACS processes, components, and functions such that security engineers could: evaluate subversion options against the target; identify threat actor attributes and capabilities derived from each subversion option; and identify security controls and response countermeasures. The Use Cases document offered detailed methodology examples including an assessment of a Military Base SMR, an Autonomous System Decision Loop, and implementation of AR/SMR Machine Learning algorithms. Our proposal at the end of last year was to focus on implementation of subversion prototypes related to the last Use Case area: AR/SMR Machine Learning (ML) Algorithms. We included six attack scenarios in our Use Cases paper: a Poisoning Attack against ML functions implemented using an FPGA; a Trojaning Attack against ML classifiers exploiting the excitability of Nuclear Engineers; a Backdooring Attack against ML Training environments to ensure persistence of an attack vector; a False Positive Evasion Attack against multi-factor Access Control Systems using clever inputs; an Inference Attack against ML models by an Insider with access to the Operational environment; and an Adversarial Reprogramming Attack against a Material Access Control Video Surveillance System. At the beginning of this year these six attack scenarios were provided to our research teams at Georgia Tech and Idaho State University and each team successfully implemented a subversion attack against a ML implementation to include transient misclassifications. While this is a notable outcome from this type of research, this paper offers the reader insight into not only how to structure and execute these types of attacks, but into the thought process behind how the researcher investigated the problem space, performed initial algorithm implementation, and the trial-and-error behind arriving at the successful subversion prototypes. We include in this paper a set of associated Scenarios on how these subversion prototypes could be implemented and an initial set of guidance for AR/SMR architects, Nuclear Regulators, and Cyber Defenders to implement awareness and defense capabilities into their current operational portfolios.

42 ENGINEERING↗