Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “false negative”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Simulated rRNA/DNA Ratios Show Potential To Misclassify Active Populations as Dormant

ABSTRACT The use of rRNA/DNA ratios derived from surveys of rRNA sequences in RNA and DNA extracts is an appealing but poorly validated approach to infer the activity status of environmental microbes. To improve the interpretation of rRNA/DNA ratios, we performed simulations to investigate the effects of community structure, rRNA amplification, and sampling depth on the accuracy of rRNA/DNA ratios in classifying bacterial populations as “active” or “dormant.” Community structure was an insignificant factor. In contrast, the extent of rRNA amplification that occurs as cells transition from dormant to growing had a significant effect ( P < 0.0001) on classification accuracy, with misclassification errors ranging from 16 to 28%, depending on the rRNA amplification model. The error rate increased to 47% when communities included a mixture of rRNA amplification models, but most of the inflated error was false negatives (i.e., active populations misclassified as dormant). Sampling depth also affected error rates ( P < 0.001). Inadequate sampling depth produced various artifacts that are characteristic of rRNA/DNA ratios generated from real communities. These data show important constraints on the use of rRNA/DNA ratios to infer activity status. Whereas classification of populations as active based on rRNA/DNA ratios appears generally valid, classification of populations as dormant is potentially far less accurate. IMPORTANCE The rRNA/DNA ratio approach is appealing because it extracts an extra layer of information from high-throughput DNA sequencing data, offering a means to determine not only the seedbank of taxa present in communities but also the subset of taxa that are metabolically active. This study provides crucial insights into the use of rRNA/DNA ratios to infer the activity status of microbial taxa in complex communities. Our study shows that the approach may not be as robust as previously supposed, particularly in complex communities composed of populations employing different growth strategies, and identifies factors that inflate the erroneous classification of active populations as dormant.

59 BASIC BIOLOGICAL SCIENCES↗

Managing the Performance/Error Tradeoff of Floating-point Intensive Applications

Modern embedded systems are becoming more reliant on real-valued arithmetic as they employ mathematically complex vision algorithms and sensor signal processing. Double-precision floating point is the most commonly used precision in computer vision algorithm implementations. A single-precision floating point can provide a performance boost due to less memory transfers, less cache occupancy, and relatively faster mathematical operations on some architectures. However, adopting it can result in loss of accuracy. Identifying which parts of the program can run in single-precision floating point with low impact on error is a manual and tedious process. In this paper, we propose an automatic approach to identify parts of the program that have a low impact on error using shadow-value analysis. Our approach provides the user with a performance/error tradeoff, using which the user can decide how much accuracy can be sacrificed in return for performance improvement. We illustrate the impact of the approach using a well known implementation of Apriltag detection used in robotics vision. In conclusion, we demonstrate that an average 1.3x speedup can be achieved with no impact on tag detection, and a 1.7x speedup with only 4% false negatives.

97 MATHEMATICS AND COMPUTING↗

Designing Size Consistent Statistics for Accurate Anomaly Detection in Dynamic Networks

An important task in network analysis is the detection of anomalous events in a network time series. These events could merely be times of interest in the network timeline or they could be examples of malicious activity or network malfunction. Hypothesis testing using network statistics to summarize the behavior of the network provides a robust framework for the anomaly detection decision process. Unfortunately, choosing network statistics that are dependent on confounding factors like the total number of nodes or edges can lead to incorrect conclusions (e.g., false positives and false negatives). In this article, we describe the challenges that face anomaly detection in dynamic network streams regarding confounding factors. We also provide two solutions to avoiding error due to confounding factors: the first is a randomization testing method that controls for confounding factors, and the second is a set of size-consistent network statistics that avoid confounding due to the most common factors, edge count and node count.

97 MATHEMATICS AND COMPUTING↗

Ensuring statistical reproducibility of ocean model simulations in the age of hybrid computing

Novel high performance computing systems that feature hybrid architectures require large scale code refactoring to unravel underlying exploitable parallelism. Such redesign can often be accompanied with machine-precision changes as the order of computation cannot always be maintained. For chaotic systems like climate models, these round-off level differences can grow rapidly. Systematic errors may also manifest initially as machine-precision differences. Isolating genuine round off level differences from such errors remains a challenge. Here, we apply two-sample equality of distribution tests to evaluate statistical reproducibility of the ocean model component of US Department of Energy's Energy Exascale Earth System Model (E3SM). A 2-year control simulation ensemble is compared to a modified ensemble as a test case - after a known non-bit-for-bit change in a model component is introduced - to evaluate the null hypothesis that the two ensembles are statistically indistinguishable. To quantify the false negative rates of these tests, we conduct a formal power analysis using a targeted suite of short simulation ensembles. The ensemble suite contains several perturbed ensembles, each with a progressively different climate than the baseline ensemble - obtained by perturbing the magnitude of a single model tuning parameter, the Gent and McWilliams κ, in a controlled manner. The null hypothesis is evaluated for each of perturbed ensembles using these tests. The power analysis informs on the detection limits of the tests for given ensemble size allowing model developers to evaluate the impact of an introduced non-bit-for-bit change to the model.

Mahajan, Salil↗

PECAN2

PECAN2, Pose Classification with 3D Atomic Network 2, presents an innovative approach to pose classification in computational modeling, particularly in the context of molecular docking. Traditional methods like docking, which rely on physics-based calculations, are prone to inaccuracies in predicting binding poses during experimental testing. While machine learning (ML) approaches, such as ML-driven pose classification, have been introduced to address these issues, they often depend on the availability of crystal structure data. In contrast, PECAN2 introduces a novel pose classification method that operates independently of crystal structures. Instead, it utilizes a 3D atomic neural network with Point Cloud Network (PCN) to establish correlations between docking scores and experimental data. This approach marks a departure from previous studies that heavily relied on crystal structures for labeling. The key innovation lies in the ability of PECAN2 to enhance the performance of molecular docking by filtering out false positives and false negatives through the correlation between docking scores and experimental data.

Shim, Heesung↗

Bloom filter de-identifier

A bloom filter is a binary data structure that is designed to be memory efficient and able to determine whether an element is present in a set. The bloom filter is a probabilistic data structure. It can tell you whether an element, or set of elements, is likely to exist within a set. It can also tell you whether an element, or set of elements, does not exist within a set. So false positives are probable, but false negatives are not. Our novel approach utilizes the bloom filter to determine how many bigrams (two letter sequence) each value has in common. The resulting metric approximates the relative similarity of two text values.

Bleeker, Amelia↗

Random forest models accurately classify synthetic opioids using high-dimensionality mass spectrometry datasets

Detection of novel threat agents presents several challenges, a principle one being the development of untargeted methods to screen an increasing number of threat chemicals whose exact structures are unknown. With the use of Machine Learning (ML) tools, we can guide the development of analytical methods for broad-spectrum detection of unbounded threat chemical families in complex mixtures. Toward this goal, we used nominal mass and high-resolution mass spectrometry data for hundreds of synthetic opioids and non-opioid compounds. We tested two ML techniques, logistic regression and random forest, to develop models towards a practical, implementable method for opioid detection. We found that of these tested ML methods, random forest models resulted in the highest validation accuracy (95+%) for both nominal mass and high-resolution classification of opioids versus non-opioids, with low false positive and false negative rates. The RF models were then used to successfully predict the classification of 10 compounds—five opioids and five non-opioids not part of the training and validation analysis. This application of ML is a critical step towards the development of field-deployable nominal mass spectrometers with ML-driven analyses for classification of emergent threats.

Arasteh, Kourosh [Lawrence Livermore National Labo↗

An ensemble model of QSAR tools for regulatory risk assessment

Quantitative structure activity relationships (QSARs) are theoretical models that relate a quantitative measure of chemical structure to a physical property or a biological effect. QSAR predictions can be used for chemical risk assessment for protection of human and environmental health, which makes them interesting to regulators, especially in the absence of experimental data. For compatibility with regulatory use, QSAR models should be transparent, reproducible and optimized to minimize the number of false negatives. In silico QSAR tools are gaining wide acceptance as a faster alternative to otherwise time-consuming clinical and animal testing methods. However, different QSAR tools often make conflicting predictions for a given chemical and may also vary in their predictive performance across different chemical datasets. In a regulatory context, conflicting predictions raise interpretation, validation and adequacy concerns. To address these concerns, ensemble learning techniques in the machine learning paradigm can be used to integrate predictions from multiple tools. By leveraging various underlying QSAR algorithms and training datasets, the resulting consensus prediction should yield better overall predictive ability. We present a novel ensemble QSAR model using Bayesian classification. The model allows for varying a cut-off parameter that allows for a selection in the desirable trade-off between model sensitivity and specificity. The predictive performance of the ensemble model is compared with four in silico tools (Toxtree, Lazar, OECD Toolbox, and Danish QSAR) to predict carcinogenicity for a dataset of air toxins (332 chemicals) and a subset of the gold carcinogenic potency database (480 chemicals). Leave-one-out cross validation results show that the ensemble model achieves the best trade-off between sensitivity and specificity (accuracy: 83.8 % and 80.4 %, and balanced accuracy: 80.6 % and 80.8 %) and highest inter-rater agreement [kappa (κ): 0.63 and 0.62] for both the datasets. The ROC curves demonstrate the utility of the cut-off feature in the predictive ability of the ensemble model. In conclusion, this feature provides an additional control to the regulators in grading a chemical based on the severity of the toxic endpoint under study.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Two-level structural sparsity regularization for identifying lattices and defects in noisy images

Here, this paper presents a regularized regression model with a two-level structural sparsity penalty applied to locate individual atoms in a noisy scanning transmission electron microscopy image (STEM). In crystals, the locations of atoms is symmetric, condensed into a few lattice groups. Therefore, by identifying the underlying lattice in a given image, individual atoms can be accurately located. We propose to formulate the identification of the lattice groups as a sparse group selection problem. Furthermore, real atomic scale images contain defects and vacancies, so atomic identification based solely on a lattice group may result in false positives and false negatives. To minimize error, model includes an individual sparsity regularization in addition to the group sparsity for a within-group selection, which results in a regression model with a two-level sparsity regularization. We propose a modification of the group orthogonal matching pursuit (gOMP) algorithm with a thresholding step to solve the atom finding problem. The convergence and statistical analyses of the proposed algorithm are presented. The proposed algorithm is also evaluated through numerical experiments with simulated images. The applicability of the algorithm on determination of atom structures and identification of imaging distortions and atomic defects was demonstrated using three real STEM images. In conclusion, we believe this is an important step toward automatic phase identification and assignment with the advent of genomic databases for materials.

36 MATERIALS SCIENCE↗

Interlaboratory Evaluation of in Vitro Cytotoxicity and Inflammatory Responses to Engineered Nanomaterials: The NIEHS Nano GO Consortium

Background: Differences in interlaboratory research protocols contribute to the conflicting data in the literature regarding engineered nanomaterial (ENM) bioactivity. Objectives: Grantees of a National Institute of Health Sciences (NIEHS)-funded consortium program performed two phases of in vitro testing with selected ENMs in an effort to identify and minimize sources of variability. Methods: Consortium program participants (CPPs) conducted ENM bioactivity evaluations on zinc oxide (ZnO), three forms of titanium dioxide (TiO2), and three forms of multiwalled carbon nanotubes (MWCNTs). In addition, CPPs performed bioassays using three mammalian cell lines (BEAS-2B, RLE-6TN, and THP-1) selected in order to cover two different species (rat and human), two different lung epithelial cells (alveolar type II and bronchial epithelial cells), and two different cell types (epithelial cells and macrophages). CPPs also measured cytotoxicity in all cell types while measuring inflammasome activation [interleukin-1β (IL-1β) release] using only THP-1 cells. Results: The overall in vitro toxicity profiles of ENM were as follows: ZnO was cytotoxic to all cell types at ≥ 50 μ g/mL, but did not induce IL-1β. TiO2 was not cytotoxic except for the nanobelt form, which was cytotoxic and induced significant IL-1β production in THP-1 cells. MWCNTs did not produce cytotoxicity, but stimulated lower levels of IL-1β production in THP-1 cells, with the original MWCNT producing the most IL-1β. Conclusions: The results provide justification for the inclusion of mechanism-linked bioactivity assays along with traditional cytotoxicity assays for in vitro screening. In addition, the results suggest that conducting studies with multiple relevant cell types to avoid false-negative outcomes is critical for accurate evaluation of ENM bioactivity.

cell viability↗

DETECTING FIRE WITH MACHINE LEARNING-ENABLED VISUAL MONITORING FOR NUCLEAR POWER PLANT ENVIRONMENTS

Nuclear power plants are experiencing significant cost challenges to remain competitive with other energy-generation utilities. Unlike other industries, the cost of operation and maintenance activities is mostly attributed to workforce costs. To mitigate this, nuclear power plant stakeholders are increasingly interested in the development and deployment of machine learning methods to potentially automate or augment manually intensive tasks to reduce costs, especially for monitoring activities. One monitoring function that is visually demanding and that can occur frequently to meet the requirements of a fire protection program is visually monitoring an area for fire occurrence. Currently, fire watch activities consist of a worker physically stationed at a given location with the sole responsibility of observing a given area to ensure a fire is detected and mitigated promptly. This effort focused on the development and evaluation of a suitable deep convolutional neural network to classify individual video frames at a sub-second frequency for the occurrence of “fire” and “no fire” in varying industrial environments similar to nuclear power plants. It is believed that a trained neural network model could be integrated with existing facility video surveillance camera feeds to generate alerts when fire inferences occur in individual frames captured at sub-second temporal resolutions. Extensive effort was dedicated to identifying and curating suitable imagery training data representing varying environments and scene settings with and without flame features to maximize generalization in nuclear power plant environments. The data collection effort resulted in the aggregation of a large, labeled image library exceeding 12,000 images to support model training for diverse industrial environments. A deep neural network model incorporating parallel multi-scale capabilities was developed and trained to support accurate image-based detection of flame incidents of varying sizes and spectral feature properties within heterogeneous scenes. Analysis results show that the trained model can achieve high inference accuracy despite heterogeneous scene environments and components. Testing accuracy exceeded 95.0 percent with very low false positive and false negative inferences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Laser-induced breakdown spectroscopy for the classification of unknown powders

Laser-induced breakdown spectroscopy (LIBS) was used to discern between two biological agent surrogates (Bacillus atrophaeus and ovalbumin) and potential interferent compounds (mold spores, humic acid, house dust, and Arizona road dust). Multiple linear regression and neural network analysis models were constructed by using B. atrophaeus and ovalbumin spectra, and limits of detection were calculated. Classification of the agent surrogates' LIBS spectra was attempted by using a neural network model. False negative rates of 0% were observed for B. atrophaeus (100 colony forming units) spore spectra with the neural network model used for classification.

60 APPLIED LIFE SCIENCES↗

Genometa - A Fast and Accurate Classifier for Short Metagenomic Shotgun Reads

Metagenomic studies use high-throughput sequence data to investigate microbial communities in situ. However, considerable challenges remain in the analysis of these data, particularly with regard to speed and reliable analysis of microbial species as opposed to higher level taxa such as phyla. We here present Genometa, a computationally undemanding graphical user interface program that enables identification of bacterial species and gene content from datasets generated by inexpensive high-throughput short read sequencing technologies. Our approach was first verified on two simulated metagenomic short read datasets, detecting 100% and 94% of the bacterial species included with few false positives or false negatives. Subsequent comparative benchmarking analysis against three popular metagenomic algorithms on an Illumina human gut dataset revealed Genometa to attribute the most reads to bacteria at species level (i.e. including all strains of that species) and demonstrate similar or better accuracy than the other programs. Lastly, speed was demonstrated to be many times that of BLAST due to the use of modern short read aligners. Our method is highly accurate if bacteria in the sample are represented by genomes in the reference sequence but cannot find species absent from the reference. This method is one of the most user-friendly and resource efficient approaches and is thus feasible for rapidly analysing millions of short reads on a personal computer.

59 BASIC BIOLOGICAL SCIENCES↗

Uncertainty quantification in ToxCast high throughput screening

High throughput screening (HTS) projects like the U.S. Environmental Protection Agency’s ToxCast program are required to address the large and rapidly increasing number of chemicals for which we have little to no toxicity measurements. Concentration-response parameters such as potency and efficacy are extracted from HTS data using nonlinear regression, and models and analyses built from these parameters are used to predict in vivo and in vitro toxicity of thousands of chemicals. How these predictions are impacted by uncertainties that stem from parameter estimation and propagated through the models and analyses has not been well explored. While data size and complexity makes uncertainty quantification computationally expensive for HTS datasets, continued advancements in computational resources have allowed these computational challenges to be met. This study uses nonparametric bootstrap resampling to calculate uncertainties in concentration-response parameters from a variety of HTS assays. Using the ToxCast estrogen receptor model for bioactivity as a case study, we highlight how these uncertainties can be propagated through models to quantify the uncertainty in model outputs. Uncertainty quantification in model outputs is used to identify potential false positives and false negatives and to determine the distribution of model values around semi-arbitrary activity cutoffs, increasing confidence in model predictions. At the individual chemical-assay level, curves with high variability are flagged for manual inspection or retesting, focusing subject-matter-expert time on results that need further input. This work improves the confidence of predictions made using HTS data, increasing the ability to use this data in risk assessment.

59 BASIC BIOLOGICAL SCIENCES↗

An effective “three-in-one” screening assay for testing drug and nanoparticle toxicity in human endothelial cells

Evaluating nanoparticle (NP) toxicity in human cell systems is a fundamental requirement for future NP biomedical applications. In this study, we have designed a screening assay for assessing different types of cell death induced by NPs in human umbilical vein endothelial cell (HUVEC) culture. This assay consists of WST-8, LDH and Hoechst 33342 staining, all performed in one well, which enables an evaluation of cell viability, necrosis and apoptosis, respectively, in the same cell sample. The 96-well format and automated processing of fluorescent images enhances the assay rapidity and reproducibility. After testing the assay functionality with agents that induced different types of cell death, we investigated the endothelial toxicity of superparamagnetic iron oxide nanoparticles (SPIONs, 8 nm), silica nanoparticles (SiNPs, 7–14 nm) and carboxylated multiwall carbon nanotubes (CNTCOOHs, 60 nm). Our results indicated that all the tested NP types induced decreases in cell viability after 24 hours at a concentration of 100 μg/ml. SPIONs caused the lowest toxicity in HUVECs. By contrast, SiNPs induced pronounced necrosis and apoptosis. A time course experiment showed the gradual toxic effect of all the tested NPs. CNTCOOHs inhibited tetrazolium derivatives at 100 μg/ml, causing false negative results from the WST-8 and LDH assay. In summary, our data demonstrate that the presented “three-in-one” screening assay is capable of evaluating NP toxicity effectively and reliably. Due to its simultaneous utilization of two different methods to assess cell viability, this assay is also capable of revealing, if NPs interfere with tetrazolium salts.

59 BASIC BIOLOGICAL SCIENCES↗

Enhanced detection of prion infectivity from blood by preanalytical enrichment with peptoid-conjugated beads

Prions cause transmissible infectious diseases in humans and animals and have been found to be transmissible by blood transfusion even in the presymptomatic stage. However, the concentration of prions in body fluids such as blood and urine is extremely low; therefore, direct diagnostic tests on such specimens often yield false-negative results. Quantitative preanalytical prion enrichment may significantly improve the sensitivity of prion assays by concentrating trace amounts of prions from large volumes of body fluids. Here, we show that beads conjugated to positively charged peptoids not only captured PrP aggregates from plasma of prion-infected hamsters, but also adsorbed prion infectivity in both the symptomatic and preclinical stages of the disease. Bead absorbed prion infectivity efficiently transmitted disease to transgenic indicator mice. We found that the readout of the peptoid-based misfolded protein assay (MPA) correlates closely with prion infectivity in vivo, thereby validating the MPA as a simple, quantitative, and sensitive surrogate indicator of the presence of prions. The reliable and sensitive detection of prions in plasma will enable a wide variety of applications in basic prion research and diagnostics.

59 BASIC BIOLOGICAL SCIENCES↗

Magnetic microparticle-based multimer detection system for the detection of prion oligomers in sheep

Transmissible spongiform encephalopathies (TSEs) are zoonotic fatal neurodegenerative diseases in animals and humans. TSEs are commonly known as bovine spongiform encephalopathy in cattle, scrapie in sheep and goats, chronic wasting disease in cervids, and Creutzfeldt–Jakob disease in humans. The putative transmissible agents are infectious prion proteins (PrPSc), which are formed by the conversion of the normal prion protein on the glycoprotein cell surface in the presence of other PrPSc. Reports of the transmission of TSEs through blood raised considerable concern about the safety of blood and blood products. To address this issue, many laboratories attempted to develop a sensitive and accurate blood diagnostic test to detect PrPSc. Previously, we reported that, compared to normal controls, the multimer detection system (MDS) was more efficient in detecting PrPSc in infected hamster brain homogenate, mouse plasma spiked with purified PrPSc from scrapie mouse brain, and scrapie-infected hamster plasmas. MDS differentiates prion multimers from the cellular monomer through the multimeric expression of epitopes on prion multimers, in contrast to the monomeric form. In this study, MDS detected PrPSc in plasma samples from scrapie-infected sheep expressing clinical symptoms, demonstrating 100% sensitivity and specificity in these samples. Plasma samples from asymptomatic lambs at the preclinical stage (8-month-old naturally infected offspring of scrapie-infected parents expressing a highly susceptible genotype) tested positive with 50% sensitivity and 100% specificity. In the first of two coded analyses using clinical scrapie-infected sheep and normal healthy samples, MDS successfully identified all but one of the clinical samples with 92% sensitivity and 100% specificity. Similar results were obtained in the second coded analysis using preclinical samples. MDS again successfully identified all but one of the samples with 87% sensitivity and 100% specificity. The false-negative sample was subjected to a protease pretreatment. In conclusion, MDS could accurately detect scrapie in plasma samples at both preclinical and clinical stages. From these studies, we conclude that MDS could be a promising tool for the early diagnosis of TSEs from blood samples.

59 BASIC BIOLOGICAL SCIENCES↗

Final Report for CRADA Agreement , AL-C-2006-01 with Microsens Biotechnologies: Detection of the Abnormal Prion Protein in Blood by Improving the Extraction of this Protein

Several conditions were examined to optimize the extraction protocol using Seprion beads for the abnormal prion protein. Different combinations of water, hexafluro-2-propanol and formic acid were used. The results of these extraction protocols showed that the magnetic beads coated with Seprion reagents were subject to degradation, themselves, when the extraction conditions that would solubilize the abnormal prion protein were used. These compounds caused interference in the immunoassay for the abnormal prion protein and rendered these protocols incompatible with the assay systems. In an attempt to overcome this problem, another approach was then used. The coated beads were used as an integral part of the assay platform. After washing away denaturing agents, the beads with the 'captured' abnormal prion were incubated directly in the immunoassay, followed by analysis by the capillary electrophoresis. When a capillary electrophoresis electro-kinetic separation was attempted, the beads disturbed the analysis making it impossible to interpret. A pressure separation method was then developed for capillary electrophoresis analysis. When 20 samples, 5 of which were positive were analyzed, the assay identified 4 of the 5 positives and had no false positives. When a larger number of samples were analyzed the results were not as good - there were false positives and false negatives. It was then observed that the amount of beads that were loaded was dependent upon how long the beads were allowed to settle before loading them into the capillary. This resulted in unacceptable variations in the results and explained that when large numbers of samples were evaluated the results were not consistent. Because the technical difficulties with using the Seprion beads could not be overcome at this time, another approach is underway that is outside of the scope of this CRADA. No further agreements have been developed. Because the results were not favorable, no manuscripts were written nor intellectual property developed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗