SEARCH · Engineering Papers

Results for “label quality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Improve Learning from Crowds via Generative Augmentation

Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high-quality augmentation using Generative Adversarial Networks: 1) the generated annotations should follow the distribution of authentic ones, which is measured by a discriminator; 2) the generated annotations should have high mutual information with the ground-truth labels, which is measured by an auxiliary network. Extensive experiments and comparisons against an array of state-of-the-art learning from crowds methods on three real-world datasets proved the effectiveness of our data augmentation framework. It shows the potential of our algorithm for low-budget crowdsourcing in general.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2023

pmartR 2.0 : A Quality Control, Visualization, and Statistics Pipeline for Multiple Omics Datatypes

The pmartR (https://github.com/pmartR/pmartR) package was designed for the quality control (QC) and analysis of mass spectrometry (MS) data, tailored to specific characteristics of proteomic (isobaric or labelled), metabolomic, and lipidomic datasets. Since its initial release, the tool has been expanded to address the needs of its growing userbase and now includes QC and statistics for nuclear magnetic resonance (NMR) metabolomic data, and leverages the DESeq2, edgeR, and limma-voom R packages for some transcriptomic data analyses. These improvements have made progress towards a unified omics processing pipeline for ease of reporting and streamlined statistical purposes. The package’s statistics and visualization capabilities have also been expanded by adding support for paired data and by integrating pmartR with the trelliscopejs R package for the quick creation of trellis displays (https://github.com/hafen/trelliscopejs). Here, we present relevant examples of each of these enhancements to pmartR and highlight how each new feature benefits the omics community.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

Clinical Natural Language Processing for Radiation Oncology: A Review and Practical Primer

Natural language processing (NLP), which aims to convert human language into expressions that can be analyzed by computers, is one of the most rapidly developing and widely used technologies in the field of artificial intelligence. Natural language processing algorithms convert unstructured free text data into structured data that can be extracted and analyzed at scale. In medicine, this unlocking of the rich, expressive data within clinical free text in electronic medical records will help untap the full potential of big data for research and clinical purposes. Recent major NLP algorithmic advances have significantly improved the performance of these algorithms, leading to a surge in academic and industry interest in developing tools to automate information extraction and phenotyping from clinical texts. Thus, these technologies are poised to transform medical research and alter clinical practices in the future. Radiation oncology stands to benefit from NLP algorithms if they are appropriately developed and deployed, as they may enable advances such as automated inclusion of radiation therapy details into cancer registries, discovery of novel insights about cancer care, and improved patient data curation and presentation at the point of care. However, challenges remain before the full value of NLP is realized, such as the plethora of jargon specific to radiation oncology, nonstandard nomenclature, a lack of publicly available labeled data for model development, and interoperability limitations between radiation oncology data silos. Successful development and implementation of high quality and high value NLP models for radiation oncology will require close collaboration between computer scientists and the radiation oncology community. Here, we present a primer on artificial intelligence algorithms in general and NLP algorithms in particular; provide guidance on how to assess the performance of such algorithms; review prior research on NLP algorithms for oncology; and describe future avenues for NLP in radiation oncology research and clinics.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2022

Cooling Egypt: Cost and benefits of room air conditioner efficiency improvement in Egypt

Egypt is one of the largest countries in the Middle East and North Africa region with a population of a total of 106 million (as of July 2022). Increasing urbanization, population growth, and increasing temperatures will likely continue to drive demand for electricity from air conditioning, with corresponding negative impacts on the peak load, air quality, and greenhouse gas emissions from refrigerant use and energy consumption. Since 2003, Egypt has adopted a standards and labeling program for various appliances, and has improved its room air conditioner (RAC) energy efficiency requirements in 2016 and 2017. However, its market is still dominated by inefficient fixed-speed drive (FSD) RACs due to a price-sensitive market. Seasonal metrics such as Cooling Season Performance Factor (CSPF) measure the energy efficiency of FSD and variable-speed drive (VSD) RACs on the same scale. Leading markets, most notably China, have already adopted a new MEPS in 2020 and 2022, which align largely with the United for Efficiency (U4E) Model Regulations Guidelines. India has also adopted a CSPF metric. Both India and China are now experiencing (1) increased demand and consumption of higher efficiency VSD RACs, as well as (2) a corresponding drop in prices from the economies of scale of production of these higher efficiency RACs. Because China is a leading manufacturer in the global air conditioner (AC) market, its adoption of the model U4E regulations have effectively raised the energy efficiency of the ACs in the global market. To achieve the benefits of taking a similar approach to Egypt’s market, we offer the following recommendations: 1. Adopt a common metric to measure the efficiency of both FSD and VSD RACs 2. Raise the stringency of MEPS to the level of the U4E Model Regulation Guidelines 3. Restructure the labeling program from the “A++” labeling scheme to a simpler A through E or a star-rating program 4. Strengthen the compliance regime governing the Egyptian AC MEPS 5. Assist small and medium AC manufacturers in Egypt to transition manufacturing lines to produce low-cost, high-efficiency VSD RACs. Adopting these recommendations would reduce Egypt’s electricity consumption to 11 TWh in 2030 and 89 TWh in 2050—with corresponding carbon dioxide (CO2) emissions reductions and avoided generation capacity requirements compared to the baseline.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2025

Deploying Adversarial Attacks in Super-Resolution Models

Reliable super-resolution methods are crucial for applications like remote sensing, grid resilience and disaster impact analysis, and standoff biometrics. These methods infuse additional high-frequency information into reconstructions, allowing for better contextualization and image intelligence. However, super-resolution models can also introduce hallucinations or other unseen vulnerabilities that could be exploited by an adversary. This is further compounded by the prominence of deep learning in these models, as models are often blindly applied on out-of-distribution images. In this work, we implement adversarial attacks in common open-source super-resolution models and examine their impact on reconstructions and downstream classification tasks. We find that an adversarially trained super-resolution model can produce high-quality reconstructions that degrade downstream classifications. Moreover, these attacks do not require access to low-resolution imagery or class labels at inference time. These results demonstrate the vulnerability of super-resolution methods to malicious actors and motivates the development of a detector for super-resolution adversarial attacks. Further exploration of adversarial attacks in this domain is required to ensure trustworthiness and robustness of super-resolution models for national security applications.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

NASA NTRS1977

Remote sensing data processing: Two years ago, today, and two years from today

Beginning with a survey of the state-of-the-art of processing remotely sensed data in early 1975, significant developments between that time and the present are chronicled, and technologies for early 1979 are projected. Current technical issues discussed include: training selection and labeling; classification and mensuration; use of satellite indicators to supplement predictions; small scale field structures; physical factors; ancillary data; geometric quality; and the cost of processing.

Holmes, Q. A.↗

DOE OSTI2020

Direction-optimizing Label Propagation and its Application for Community Detection

Label Propagation is a machine learning algorithm typically used for classification. It has also been found to be an effective method for detecting communities in networks. It has two attractive features as a community detection method: it has nearly linear runtime and it requires no \textit{a priori} community information. We propose a new Direction Optimizing Label Propagation Algorithm (DOLPA) that relies on the use of {\em frontiers} and alternates between label {\em push} and label {\em pull} operations to enhance the performance of LPA. Specifically, DOLPA has parameters for tuning the processing order of vertices in a graph. This reduces the number of edges visited and improves the quality of solution. We apply DOLPA to community detection and present the design and implementation of the algorithm as well as its shared-memory parallelization using OpenMP. Empirically, we evaluate our algorithm using synthetic graphs as well as real-world networks. Compared with the state-of-art \textit{Parallel Label Propagation} algorithm, we achieve at least two times the F-Score while reducing the runtime by 50\% for synthetic graphs with overlapping communities. We also compare DOLPA against the state-of-art parallel implementation of the Louvain method using the same graphs and show that DOLPA achieves about three times the F-Score at 10\% the runtime. On real-world graphs, we get a speedup of up to $10\times$ using 64 threads.

Liu, X↗

DOE OSTI2024

PMU Data Quality and Sensor Health Monitoring

Phasor Measurement Units (PMUs) play a critical role in the evolution of the electric power industry by providing high-precision, real-time monitoring of essential power system metrics. However, effectively detecting abnormalities and critical events from PMU data is a complex task, complicated by intricate temporal patterns, a scarcity of labeled data for training algo- rithms, and constraints on online computational power. In this study, we apply TranAD, an innovative algorithm that combines transformer architectures with the refinement of adversarial learning, to both synthetic and real-world PMU datasets for developing a data quality and sensor online health monitoring platform for utilities. Our findings reveal that TranAD not only provides efficient detection and localization but also enhances the detail with which abnormalities are detected, marking a a significant step forward in the field of clean data acquisition processes for power system monitoring

deep neural network, machine learning (ML)↗

NASA NTRS1979

Accuracy assessment in the Large Area Crop Inventory Experiment

The Accuracy Assessment System (AAS) of the Large Area Crop Inventory Experiment (LACIE) was responsible for determining the accuracy and reliability of LACIE estimates of wheat production, area, and yield, made at regular intervals throughout the crop season, and for investigating the various LACIE error sources, quantifying these errors, and relating them to their causes. Some results of using the AAS during the three years of LACIE are reviewed. As the program culminated, AAS was able not only to meet the goal of obtaining accurate statistical estimates of sampling and classification accuracy, but also the goal of evaluating component labeling errors. Furthermore, the ground-truth data processing matured from collecting data for one crop (small grains) to collecting, quality-checking, and archiving data for all crops in a LACIE small segment.

Houston, A. G.↗

DOE OSTI2025

A self-supervised robotic system for autonomous contact-based spatial mapping of semiconductor properties

Integrating robotically driven contact-based material characterization techniques into self-driving laboratories can enhance measurement quality, reliability, and throughput. While deep learning models support robust autonomy, current methods lack reliable pixel-precision positioning and require extensive labeled data. To overcome these challenges, we propose an approach for building self-supervised autonomy into contact-based robotic systems that teach the robot to follow domain expert measurement principles at high throughputs. We demonstrate the performance of this approach by autonomously driving a 4-DOF robotic probe for 24 hours to characterize semiconductor photoconductivity at 3025 uniquely predicted poses across a gradient of drop-casted perovskite film compositions, achieving throughputs of more than 125 measurements per hour. Spatially mapping photoconductivity onto each drop-casted film reveals compositional trends and regions of inhomogeneity, valuable for identifying manufacturing defects. With this self-supervised neural network–driven robotic system, we enable high-precision and reliable automation of contact-based characterization techniques at high throughputs, thereby allowing measurement of previously inaccessible yet important semiconductor properties for self-driving laboratories.

Science & Technology - Other Topics↗

DOE OSTI2026

Total Dissolved Nitrogen and Ammonia Data for the East River Watershed, Colorado (2015-2025)

This data package contains mean values for total dissolved nitrogen (TDN) and ammonia concentrations for water samples taken from the East River Watershed in Colorado. The East River is part of the Watershed Function Scientific Focus Area (WFSFA) located in the Upper Colorado River Basin, United States. TDN was analyzed using a Shimadzu Total Nitrogen Module (TNM-1) combined with the TOC-VCSH analyzer (Shimadzu Corporation, Japan). TNM-1 is a non-specific measurement of total nitrogen (TN). All nitrogen species in samples are combusted to nitrogen monoxide and nitrogen dioxide, then reacted with ozone to form an excited state of nitrogen dioxide. Upon returning to ground state, light energy is emitted. Then, TDN is measured using a chemiluminescence detector. Ammonia was determined using a Lachat's QuikChem 8500 Series 2 Flow Injection Analysis System (LACHAT Instruments, QuckChem 8500 series 2, Automated Ion Analyzer, Loveland, Colorado). When ammonia in water samples is heated (60 degrees C) with salicylate and hypochlorite in an alkaline phosphate buffer, an emerald green color is produced which is proportional to the ammonia concentration. The color is intensified by the addition of nitroprusside. Ethylenediaminetetraacetic acid (EDTA) is added to the buffer to prevent the interference of metal ions (Ca, Mg, and Fe etc.). Ammonia-N is then determined by LACHAT flow injection and a colorimetric assay at an absorbance wavelength 660 nm. (Reference: LACHAT Instruments: QuickChem Method 90-107-06-3-A, Determination of Ammonia by Flow Injection Analysis (High Throughput, Salicylate Method/DCIC) (Multi Matrix method). Written by Lynn Egan (Application group), February 08, 2011.) All files are labeled by location and variable, and data reported are the mean values upon replicate measurements. All samples were analyzed under a rigorous quality assurance and quality control (QA/QC) process as detailed in the methods. This data package contains (1) a zip file (tdn_ammonia_data_2015-2025.zip) containing a total of 299 files: 298 data files of ammonia and TDN data from across the Lawrence Berkeley National Laboratory (LBNL) Watershed Function Scientific Focus Area (SFA) which is reported in .csv files per location and a locations.csv (1 file) with latitude and longitude for each location; (2) a file-level metadata (v7_20260901_flmd.csv) file that lists each file contained in the dataset with associated metadata; (3) a data dictionary (v7_20260901_dd.csv) file that contains terms/column_headers used throughout the files along with a definition, units, and data type; (4) PDF and docx files for the determination of Method Detection Limits (MDLs) for TDN data, which has been updated in 2026-08; and (5) PDF and docx files for the detemination of Method Detection Limits (MDLs) for Ammonia and the Interferences by LACHAT Flow Injection Analysis. Missing values within the anion data files are noted as either "-9999" or "0.0" for not detectable (N.D.) data. There are a total of 105 locations containing TDN and Ammonia-N data. Update 2020-10-07: Updated the data files to remove times from the timestamps, so that only dates remain. The data values have not changed. Update 2021-04-11: Added Determination of Method Detection Limits (MDLs) for DIC, NPOC and TDN Analyses and Determination of Method Detection Limit for Ammonia and the Interferences by LACHAT Flow Injection Analysis documents, which can be accessed as PDFs or with Microsoft Word.Update on 6/10/2022: versioned updates to this dataset was made along with these changes: (1) updated total dissolved nitrogen and ammonia data for all locations up to 2021-12-31, (2) removal of units from column headers in datafiles, (3) added row underneath headers to contain units of variables, (4) restructure of units to comply with CSV reporting format requirements, (5) added -9999 for empty numerical cells, and (6) the addition of the file-level metadata (flmd.csv) and data dictionary (dd.csv) were added to comply with the File-Level Metadata Reporting Format. Update on 2022-09-09: Updates were made to reporting format specific files (file-level metadata and data dictionary) to correct swapped file names, add additional details on metadata descriptions on both files, add a header_row column to enable parsing, and add version number and date to file names (v2_20220909_flmd.csv and v2_20220909_dd.csv). Update on 2022-12-20: Updates were made to both the data files and reporting format specific files. Units were listed incorrectly, but have been fixed to reflect correct units (ug/L). File level metadata (flmd) and data dictionary (dd) files were updated to reflect the updated versions of these files. Available data was added up until 2022-06-01. Update on 2023-08-08: Updates were made to both the data files and reporting format specific files. New available anion data was added, up until 2023-01-05. The file level metadata and data dictionary files were updated to reflect the additional data added. Update on 2024-03-11: Updates were made to both the data files and reporting format specific files. New available anion data was added, up until 2023-10-27. Further, revisions to the data files were made to remove incorrect data points (from 1970 and 2001). The reporting format specific files were updated to reflect the additional data added. Revised versions of the PDF and docx files for determination of MDLs for TDN were added to replace previous versions. Update on 2025-05-15: Updates were made to both the data files and reporting format specific files. New available TDN and Ammonia-N data was added, up until the end of WY2024 (September 30, 2024). International Generic Sample Numbers (IGSNs), when registered, were added to the data files. The reporting format specific files were updated to reflect the additional data added. Update on 2026-09-01: Updates were made to both the data files and reporting format specific files. New available TDN and Ammonia-N data was added, up until the end of WY2025 (September 30, 2025). Updated versions, as of 2026-08-10, of the PDF and docx files for determination of MDLs for TDN data were added to this dataset.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2005

Adaptable Constrained Genetic Programming: Extensions and Applications

An evolutionary algorithm applies evolution-based principles to problem solving. To solve a problem, the user defines the space of potential solutions, the representation space. Sample solutions are encoded in a chromosome-like structure. The algorithm maintains a population of such samples, which undergo simulated evolution by means of mutation, crossover, and survival of the fittest principles. Genetic Programming (GP) uses tree-like chromosomes, providing very rich representation suitable for many problems of interest. GP has been successfully applied to a number of practical problems such as learning Boolean functions and designing hardware circuits. To apply GP to a problem, the user needs to define the actual representation space, by defining the atomic functions and terminals labeling the actual trees. The sufficiency principle requires that the label set be sufficient to build the desired solution trees. The closure principle allows the labels to mix in any arity-consistent manner. To satisfy both principles, the user is often forced to provide a large label set, with ad hoc interpretations or penalties to deal with undesired local contexts. This unfortunately enlarges the actual representation space, and thus usually slows down the search. In the past few years, three different methodologies have been proposed to allow the user to alleviate the closure principle by providing means to define, and to process, constraints on mixing the labels in the trees. Last summer we proposed a new methodology to further alleviate the problem by discovering local heuristics for building quality solution trees. A pilot system was implemented last summer and tested throughout the year. This summer we have implemented a new revision, and produced a User's Manual so that the pilot system can be made available to other practitioners and researchers. We have also designed, and partly implemented, a larger system capable of dealing with much more powerful heuristics.

Janikow, Cezary Z.↗

DOE OSTI2025

Replace Human Intelligence with Fast and Smart Geometric Reasoning and Graph Neural Network to Accelerate Next Gen ModSim Workflows

We present an agent-guided approach to CAD geometry decomposition that automates hex/hybrid meshing with graph neural networks (GNNs) to accelerate next-generation ModSim workflows. Our end-to-end pipeline (i) reduces 3D boundary-representation (B-Rep) models to a 2D chordal axis skeleton (CAT) and then to a 1D bipartite graph of surface and curve nodes, (ii) assigns per node labels as Cubit® WebCut actions, (iii) trains a multi-action GNN under supervised learning, and (iv) predicts five surface-node and three curve-node actions on out-of-distribution test geometries. Each graph node carries geometric, topological, and meshing attributes drawn from the B-Rep “skin” and CAT “skeleton,” with two-way mappings across 3D↔2D↔1D representations to maintain traceability back to 3D CAD. The supervised learning model exhibits stable convergence of the binary cross-entropy loss and achieves 98.7% accuracy on unseen lattice models. To operationalize decision-making, we rank predicted commands by geometric significance and prototyped the agent-guided workflow through the Cubit® Meshing PowerTool GUI. As a stretch goal, we explore reinforcement learning (RL) to reduce or remove label requirements and to learn policies for action sequences that maximize total reward (e.g., size of hex-meshable regions and resulting hex mesh quality). When all-hex meshing is not feasible, the agent assists in producing hybrid meshes—prioritizing hex in critical regions and transitioning to tetrahedral elements (tets) elsewhere—maintaining fidelity while ensuring robustness. The overarching objective is to replace manual, heuristics-based decomposition with data-driven, reproducible automation, cutting meshing turnaround time by orders of magnitude. We anticipate direct impact on simulation workflows through intelligent, scalable decomposition of complex CAD models into hex-meshable subdomains.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Closing the Loop between In Situ Stress Complexity and EGS Fracture Complexity

42 ENGINEERING↗

DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Semisupervised Learning for Seismic Monitoring Applications

The impressive performance that deep neural networks demonstrate on a range of seismic monitoring tasks depends largely on the availability of event catalogs that have been manually curated over many years or decades. However, the quality, duration, and availability of seismic event catalogs vary significantly across the range of monitoring operations, regions, and objectives. Semisupervised learning (SSL) enables learning from both labeled and unlabeled data and provides a framework to leverage the abundance of unreviewed seismic data for training deep neural networks on a variety of target tasks. We apply two SSL algorithms (mean-teacher and virtual adversarial training) as well as a novel hybrid technique (exponential average adversarial training) to seismic event classification to examine how unlabeled data with SSL can enhance model performance. In general, we find that SSL can perform as well as supervised learning with fewer labels. We also observe in some scenarios that almost half of the benefits of SSL are the result of the meaningful regularization enforced through SSL techniques and may not be attributable to unlabeled data directly. Lastly, the benefits from unlabeled data scale with the difficulty of the predictive task when we evaluate the use of unlabeled data to characterize sources in new geographic regions. Finally, in geographic areas where supervised model performance is low, SSL significantly increases the accuracy of source-type classification using unlabeled data.

58 GEOSCIENCES↗

NASA NTRS1985

An operational ASDAR system

The story of the Aircraft to Satellite Data Relay (ASDAR) program began when airline meteorologists realized that B-747's and other commercial jets provided cockpit displays of digital values for outside air temperature and winds. Later, when a few B-747's were used to carry portable air quality monitoring equipment for the Global Air Sampling Program (GASP), scientists at NASA-Lewis explored ways in which these digital values could be used to label data collected during the GASP flights. Digital values of GASP analyses were recorded along with digital values of location and altitude, time, winds, and temperature, obtained by microprocessors from within the host aircraft's avionics. These data suggested a way in which manually recorded in-flight meteorological reports could be replaced by an automatic system, which could record winds and air temperatures as often as desired. NASA's prototype ASDAR showed that automated data relay by meteorological geostationary satellites could be accomplished from an aircraft. Testing of the instruments and analyses of its data are examined.

Sparkman, James K., Jr.↗

DOE OSTI2021

Patient-Reported Outcomes in Pediatric Cancer Registration Trials: A US Food and Drug Administration Perspective

Pediatric patient-reported outcome (PRO) data can help inform the US Food and Drug Administration’s (FDA’s) benefit-risk assessment of cancer therapeutics by quantifying symptom and functional outcomes from the patient’s perspective. This study assessed use of PROs in commercial pediatric oncology trials submitted to the FDA for regulatory review. FDA databases were searched to identify pediatric oncology product applications approved between 1997 and 2020. Sponsor-submitted documents were reviewed to determine whether PRO data were collected, which instruments were used, and the quality of collected data (ie, sample size, completion rates, and use of fit-for-purpose instruments). The role of PROs in each trial (endpoint hierarchy) was also recorded in addition to whether any PRO endpoints were included in product labeling. We reviewed 17 pediatric oncology applications, 4 of which included PRO data: denosumab, tisagenlecleucel, larotrectinib, and selumetinib. In these 4 instances, PROs served as exploratory endpoints and were not incorporated in product labeling. Trials that collected PRO data were phase II or phase I/II single-arm studies with sample sizes of 28 to 88 patients. Symptomatic adverse events (AEs) were characterized using clinician-reported Common Terminology Criteria for Adverse Events (CTCAE) without additional patient self-report. PROs were infrequently used in pediatric cancer registration trials. When PROs were used, PRO data were limited by lack of a clear research objective and corresponding prospective statistical analysis plan. Contemporary PRO symptom libraries, such as the National Cancer Institute’s Pediatric PRO-CTCAE, may provide an opportunity to better evaluate the occurrence and impact of symptomatic AEs, from the patient’s perspective, in pediatric oncology trials.

Oncology↗