Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “extractants”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Xylan-cellulose core structure of oat water-extractable β-glucan macromolecule: Insight into interactions and organization of the cell wall complex

Water-extractable β-glucan with high molar mass (HM) determines health benefits of oat food. Oat β-glucan was extracted by a standardized in vitro digestion method and co-existing water-extractable polysaccharide (WEP) fraction and its HM-arabinoxylan (HM-AX) subfraction were isolated to identify their highly acid-resistant subunit and investigate molecular interactions between constituent polymers. The WEP and HM-AX samples consisted of arabinoxylans (AXs) (74 and 76 %, respectively), however, cellulose constituted the secondary component (6.6 and 12.8 %, respectively). Multi-detection HPSEC along with specific enzymatic hydrolysis of AXs revealed the presence of the HM-xylan domain (16 and 34 %, respectively) built of numerous single- and multi-component populations with random coil and rod-like conformations, which were embedded in a xylan matrix with spherical conformation and controlled the macromolecular shape. Unlike single-component populations, the multi-component ones were resistant to hydrolytic action of AX-hydrolyzing enzymes and represented the subunits that anchor matrix polysaccharides onto cellulose surface. These results indicate that water-extractable β-glucan macromolecule comprises as integral element a cellulose core with two linking populations, HM-xylan and low molar mass glucomannan, which are surrounded by a feruloylated AX-arabinan-arabinogalactan composite and next laminated by β-glucan matrix. Here, the stiff cellulose-xylan backbone is the basis of HM β-glucan organization, controlled by its cellulose-like segments.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Elucidating the speciation of extracted lanthanides by diglycolamides

Many studies over the past few decades have been devoted to addressing the application of diglycolamides (DGAs) for hydrometallurgical-based, f-element separations. Work to date has shown the molecular structure of a DGA derivative can have a significant impact on intra-lanthanide partitioning patterns. More recent studies have pushed towards probing the structure function relationship of the lanthanide-DGA complex to enable the design of more efficient lanthanide separation systems. Spectroscopic techniques, such as UV–Visible, Fourier Transformed Infrared Spectroscopy (FT-IR), Nuclear Magnetic Resonance (NMR), and Extended X-ray Absorption Fine Structure (EXAFS), provide information regarding the inner-sphere coordination of a given lanthanide-DGA complex. Scattering techniques, such as Dynamic Light Scattering (DLS), Small-Angle X-ray Scattering (SAXS), and Small-Angle Neutron Scattering (SANS), address nanoscale structures including aggregate sizes and morphology. Here, this review assesses the current state-of-knowledge regarding lanthanide-DGA hydrometallurgical (i.e., solvent extraction systems) interrogated using spectroscopic and scattering techniques to characterize the extracted Ln 3+ DGA species. Of particular interest to this review is the impact of varied diluents, inclusion and variation of phase modifiers, and DGA derivatization on system characteristics. While there has been extensive literature on the application of DGAs for f-element separations, the literature lacks a collective assessment of the speciation of Ln 3+ in the organic phase. This review provides new insights into the field of DGA separations, explicitly with an application to intra-lanthanide separations. Specifically, this review illustrates the importance of both the co-extraction anion (Cl - , NO 3 - , or ClO 4 - ) as it pertains to both the aggregate size and Ln 3+ distribution coefficient. It is evident the ability of the anion to disrupt the hydrogen bonding network limits both aggregate size and distribution coefficients according to the Hoffmeister series. This suggests the importance of the large, softer anions with a low charge-to-surface area ratio on encouraging hydrogen bond interactions. In addition, the co-extracted cation (H + vs Na + ) is important for mitigating transfer of Ln 3+ from the aqueous to the organic phase through extensive hydrogen bonding networks. These networks are responsible for forming supramolecular aggregates where a change in morphology is observed with increasing concentrations of H + and/or Ln 3+ in the organic phase.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Extracting a mixing parameter from 2D radiographic imaging of variable-density turbulent flow

We extract a suitably averaged fluctuating density from the two-dimensional radiographic image of a flow. The X-ray attenuation is given by the Beer–Lambert law which exponentially damps the incident beam intensity by a factor proportional to the density, opacity and thickness of the target. By making reasonable assumptions for the mean density, opacity and effective thickness of the target flow, we estimate the density fluctuation contribution to the attenuation. The extracted density fluctuations averaged across the thickness of the flow in the direction of the beam may be used to form the density–specific-volume correlation $b$. In a statistical description of variable-density turbulence, $b$ quantifies the degree of mixedness. The ability to extract a measure of mixedness from experimental data would be a powerful tool that could be used in the validation of mix models. The scheme proposed is tested for DNS data computed for variable density buoyancy-driven mixing. We quantify the deficits in the extracted value of $b$ due to target thickness, Atwood number and modeled signal noise. This analysis justifies using the proposed scheme to infer the mix parameter from thin targets at moderate to low Atwood numbers. Furthermore, to illustrate how the scheme might be used in a practical problem, we demonstrate its application to a radiographic image of counter-shear flow obtained from experiments at the National Ignition Facility.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Extracting Material Property Measurements from Scientific Literature with Limited Annotations

Extracting material property data from scientific text is pivotal for advancing data-driven research in chemistry and materials science; however, the extensive annotation effort required to produce training data for named entity recognition (NER) models for this task often makes it a barrier to extracting specialized data sets. Here, in this work, we present a comparative study of the conventional, supervised NER methodology to alternative few-shot learning architectures and large language model (LLM)-based approaches that mitigate the need to label large training data sets. We find that the best-performing LLM (GPT-4o) not only excels in directly extracting relevant material properties based on limited examples but also enhances supervised learning through data augmentation. We supplement our findings with error and data quality assessments to provide a nuanced understanding of factors that impact property measurement extraction.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Advancing Chemical Separations: Unraveling the Structure and Dynamics of Phase Splitting in Liquid–Liquid Extraction

Liquid-liquid extraction (LLE), the go-to process for a variety of chemical separations, is limited by spontaneous organic phase splitting upon sufficient solute loading, called third phase formation. In this study we explore the applicability of critical phenomena theory to gain insight into this deleterious phase behavior with the goal of improving separations efficiency and minimizing waste. Here, a series of samples representative of rare earth purification were constructed to include each of one light and one heavy lanthanide (cerium and lutetium) paired with one of two common malonamide extractants (DMDOHEMA and DMDBTDMA). The resulting postextraction organic phases are chemically complex and often form rich hierarchical structures whose statics and dynamics near the critical point were probed herein with small-angle X-ray scattering and high-speed X-ray photon correlation spectroscopy. Despite their different extraction behaviors, all samples show remarkably similar critical behavior with exponents well described by classical critical point theory consistent with the 3D Ising model, where the critical behavior is characterized by fluctuations with a single diverging length scale. This unexpected result indicates a significant reduction in relevant chemical parameters at the critical point, indicating that the underlying behavior of phase transitions in LLE rely on far fewer variables than are generally assumed. The obtained scalar order parameter is attributed to the extractant fraction of the extractant/diluent mixture, revealing that other solution components and their respective concentrations simply shift the critical temperature but do not affect the nature of the critical fluctuations. These findings point to an opportunity to drastically simplify studies of liquid-liquid phase separation and phase diagram development in general while providing insights into LLE process improvement.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Mesostructuring in Liquid–Liquid Extraction Organic Phases Originating from Critical Points

Organic phase structure plays an important role in solute extraction energetics and phase behavior of liquid-liquid extraction (LLE) systems. For a binary extractant (amphiphile)/solvent mixture of relevance to LLE, we find that the organic phase mesostructuring is consistent with extractant concentration fluctuations as the compositional isotherm traverses the Widom line above its liquid-liquid critical point. This reveals a different mechanism for the well-documented heterogeneities in LLE organic phases that are typically attributed to micellization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Asymmetric pathways for lithium extraction and recovery based on the two-phase equilibrium of layered oxides

Electrochemical intercalation offers a promising platform for Li + extraction. However, only limited types of electrode materials have been investigated. The challenge to broaden and tailor materials for electrochemical intercalation-based Li + extraction lies in the lack of understanding of material’s response upon co-intercalation of multiple ions, therefore, paired process design to enable reversible Li + extraction and recovery. Here, we showcase the design of asymmetric ion pathways for Li + extraction and recovery for host material with complex Li + and Na + interaction using layered cobalt oxide as a model material. The two-phase equilibrium of Na 0.48 CoO 2 and Li 0.94 CoO 2 governs Li + selectivity when a high depth of intercalation is achieved (low vacancy level). We show that the relative rate between ion exchange and intercalation is critical to determine the ion pathways. The relationship can be quantitatively compared using the average pseudo ion exchange rate (C pseudoIX ) and the intercalation rate (C inter ). The ion pathways at the three regimes with C pseudoIX > C inter , C pseudoIX ~ C inter , and C pseudoIX < C inter are constructed. By selecting the optimized ion pathway and particle size, we demonstrate 9.7×10 4 Li + selectivity with 99% purity Li + recovery from an initial 1:1000 Li: Na molar ratio solution using 115 mAh/g specific capacity.

electrochemistry↗
DOE OSTI2020

Deep learning on the 2-dimensional Ising model to extract the crossover region with a variational autoencoder

The 2-dimensional Ising model on a square lattice is investigated with a variational autoencoder in the non-vanishing field case for the purpose of extracting the crossover region between the ferromagnetic and paramagnetic phases. The encoded latent variable space is found to provide suitable metrics for tracking the order and disorder in the Ising configurations that extends to the extraction of a crossover region in a way that is consistent with expectations. The extracted results achieve an exceptional prediction for the critical point as well as agreement with previously published results on the configurational magnetizations of the model. The performance of this method provides encouragement for the use of machine learning to extract meaningful structural information from complex physical systems where little a priori data is available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Identifying COVID-19 cases and extracting patient reported symptoms from Reddit using natural language processing

We used social media data from “covid19positive” subreddit, from 03/2020 to 03/2022 to identify COVID-19 cases and extract their reported symptoms automatically using natural language processing (NLP). We trained a Bidirectional Encoder Representations from Transformers classification model with chunking to identify COVID-19 cases; also, we developed a novel QuadArm model, which incorporates Question-answering, dual-corpus expansion, Adaptive rotation clustering, and mapping, to extract symptoms. Our classification model achieved a 91.2% accuracy for the early period (03/2020-05/2020) and was applied to the Delta (07/2021–09/2021) and Omicron (12/2021–03/2022) periods for case identification. We identified 310, 8794, and 12,094 COVID-positive authors in the three periods, respectively. The top five common symptoms extracted in the early period were coughing (57%), fever (55%), loss of sense of smell (41%), headache (40%), and sore throat (40%). During the Delta period, these symptoms remained as the top five symptoms with percent authors reporting symptoms reduced to half or fewer than the early period. During the Omicron period, loss of sense of smell was reported less while sore throat was reported more. Our study demonstrated that NLP can be used to identify COVID-19 cases accurately and extracted symptoms efficiently.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Radiation-Induced Effects on the Extraction Properties of Hexa-n-octylnitrilo-triacetamide (HONTA) Complexes of Americium and Europium

The candidate An(III)/Ln(III) separation ligand hexa-n-octylnitrilo-triacetamide (HONTA) was irradiated under envisioned SELECT (Solvent Extraction from Liquid waste using Extractants of CHON-type for Transmutation) process conditions (n-dodecane/0.1 M HNO 3 ) using a solvent test loop in conjunction with cobalt-60 gamma irradiation. The extent of HONTA radiolysis and complimentary degradation product formation was quantified by HPLC-ESI-MS/MS. Further, the impact of HONTA radiolysis on process performance was evaluated by measuring the change in 243 Am and 154 Eu distribution ratios as a function of absorbed gamma dose. HONTA was found to decay exponentially with increasing dose, affording a dose coefficient of d = (4.48 ± 0.19) × 10 –3 kGy –1 . Multiple degradation products were detected by HPLC-ESI-MS/MS with dioctylamine being the dominant quantifiable species. Both 243 Am and 154 Eu distribution ratios exhibited an induction period of ~70 kGy for extraction (0.1 M HNO 3 ) and back-extraction (4.0 M HNO 3 ) conditions, after which both values decreased with absorbed dose. The decrease in distribution ratios was attributed to a combination of the destruction of HONTA and ingrowth of dioctylamine, which is capable of interfering in metal ion complexation. Furthermore, the loss of HONTA with absorbed gamma dose was predominantly attributed to its reaction with the n-dodecane radical cation (R˙ + ). These R˙ + reaction kinetics were measured for HONTA and its 241 Am and 154 Eu complexes using picosecond pulsed electron radiolysis techniques. All three second-order rate coefficients (k) were essentially diffusion limited in n-dodecane indicating a significant reaction pathway: k(HONTA + R˙ + ) = (7.6 ± 0.8) × 10 9 M –1 s –1 , k(Am(HONTA) 2 + R˙ + ) = (7.1 ± 0.7) × 10 10 M –1 s –1 , and k(Eu(HONTA) 2 + R˙ + ) = (9.5 ± 0.5) × 10 10 M –1 s –1 . HONTA-metal ion complexation afforded an order-of-magnitude increase in rate coefficient. Additional nanosecond time-resolved measurements showed that both direct and indirect HONTA radiolysis yielded the short-lived (< 100 ns) HONTA radical cation and a second long-lived (µs) species identified as the HONTA triplet excited state. The latter was confirmed by a series of oxygen quenching picosecond pulsed electron measurements, affording a quenching rate coefficient of k( 3 [HONTA]* + O 2 ) = 2.2 × 10 8 M –1 s –1 . Overall, both the HONTA radical cation and triplet excited state are important precursors to the suite of measured HONTA degradation products.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2025

Ultrafast pre-solvated dodecane hole capture and subsequent damage of used nuclear fuel extraction ligands DEHBA, DEH i BA, HONTA, CMPO, HEH[EHP] and TBP

Here, two classes of used nuclear fuel (UNF) extraction ligands, amide (DEHBA, DEH i BA, HONTA) and organophosphorus (CMPO, HEH[EHP], TBP), were selected to study radiation induced damage at picosecond to nanosecond timescale using electron pulse radiolysis in n-dodecane (DD) and supported by quantum chemical calculations. Spectra after radiolysis of 200 mM extraction ligands were recorded in DD/0.3 M DCM. Absorption peaks at 365, 365, 400 and 387 nm in case of DEHBA, DEH i BA, HONTA and CMPO respectively are assigned to triplet excited states. Additional absorption peaks at 420, 460 and 600 nm of DEHBA, DEH i BA and HONTA respectively were identified as due to ligand radical cations. A concentration dependent absorption peak at 600 nm in the case of CMPO was observed and assigned due to a combination of CMPO˙ + , (CMPO) 2 ˙ + and possibly a radical degradation product of CMPO. Weak absorption peaks at 650 and 550 nm in case of HEH[EHP] and TBP were observed and tentatively assigned to their radical cations. A two-component DD˙ + decay in the presence of ligands was observed due to different ligand oxidation mechanisms: ultrafast capture of pre-solvated DD holes and diffusive capture of solvated DD holes. At high extraction ligand concentrations (>100 mM), the majority of DD holes were captured via the ultrafast pre-solvated pathway in <10 ps with C 37 values of 389, 401, 270, 374, 458 and 340 mM for DEHBA, DEHiBA, HONTA, CMPO, HEH[EHP] and TBP respectively. Following ultrafast capture, the remainder of DD holes became solvated and were captured with k = (2.32 ± 0.13), (1.78 ± 0.12), (1.38 ± 0.2), (0.98 ± 0.081), (1.09 ± 0.08) and (1.77 ± 0.046) × 10 10 for DEHBA, DEH i BA, HONTA, CMPO, HEH[EHP] and TBP respectively. Subsequent hole transfer from the extraction ligands˙ + to the low IP solute tri-p-tolylamine (TTA) showed only 4–16% hole transfer, most likely indicating ligand˙ + degradation in 0.9–4.6 ns.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2021

Light extraction in tandem organic light emitting diodes

Since the invention of organic light emitting diodes (OLEDs), great research efforts have been dedicated to improving their efficiency and lifetime. For high-brightness applications, tandem OLED structures have advantages because of the lower current densities required to achieve high brightness. With the successful development of highly efficient charge generation layers, high brightness tandem OLEDs are used in displays and lighting. However, the major challenge for tandem OLEDs is the low light extraction efficiency, because about 50% of the light is trapped inside the device as waveguide modes. In this Perspective, we first review the recent works done on light extraction, analyze different waveguide mode extraction structures, and then identify the key factors determining the extraction efficiencies in tandem OLEDs.

42 ENGINEERING↗
DOE OSTI2023

DEHP− extractant binding to trivalent lanthanide Er3+: Fast binding accompanied by concerted angular motions of hydration water

Solvent extraction of trivalent rare earth metal ions by organophosphorus extractants proceeds via binding of phosphoric acid headgroups to the metal ion. Water molecules in the tightly bound first hydration shell of the metal ions must be displaced by oxygen atoms from phosphoric acid headgroups. Here, we use classical molecular dynamics simulations to explore the event in which a fully hydrated Er3+ binds to its first phosphoric acid headgroup. Approach of the headgroup into the region between the first and second hydration shells leads to a fast ejection of a water molecule that is accompanied by reordering of the hydration water molecules, including discretization of their angular positions and collective rotation about the metal ion. The water molecule ejected from the first shell is located diametrically opposite from the binding oxygen. Headgroup binding places a headgroup oxygen closer to Er3+ than its first hydration shell and creates a loosely bound water that subsequently exchanges between the first shell and its environment. This second exchange of water also occurs at discrete angular positions. This geometrical aspect of binding may be of relevance to understanding the binding and transport of ion–extractant complexes that are expected to occur at the organic–aqueous liquid–liquid interface used in solvent extraction processes.

Chemistry↗
DOE OSTI2026

Spectral deconvolution without the deconvolution: Extracting temperature from x-ray Thomson scattering spectra without the source-and-instrument function

X-ray Thomson scattering (XRTS) probes the dynamic structure factor of the system, but the measured spectrum is broadened by the combined source-and-instrument function (SIF) of the setup. In order to extract properties such as temperature from an XRTS spectrum, the broadening by the SIF needs to be removed. Recent work [Dornheim et al. Nat. Commun. 13 , 7911 (2022)] has suggested that the SIF may be deconvolved using the two-sided Laplace transform. However, the extracted information can depend strongly on the shape of the input SIF, and the SIF is in practice challenging to measure accurately. Here, we propose an alternative approach: we demonstrate that considering ratios of Laplace-transformed XRTS spectra collected at different scattering angles is equivalent to performing the deconvolution, but without the need for explicit knowledge of the SIF. From these ratios, it is possible to directly extract the temperature from the scattering spectra, when the system is in thermal equilibrium. We find the method to be generally robust to spectral noise and physical differences between the spectrometers, and we explore situations in which the method breaks down. Furthermore, the fact that consistent temperatures can be extracted for systems in thermal equilibrium indicates that non-equilibrium effects could be identified by inconsistent temperatures of a few eV between the ratios of three or more scattering angles.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Conceptual Design for a Blanket Tritium Extraction Test Stand

Future fusion reactors must be able to breed the tritium they will consume. Several breeding and tritium extraction technologies are under investigation internationally. PbLi is of particular interest as a breeder material. A new forced convection PbLi loop, the Tritium Extraction eXperiment (TEX), is being designed and constructed to investigate tritium extraction from PbLi. Specifically, TEX will serve to investigate the extraction efficiency of various vacuum permeator membranes and configurations. The major components of the loop include a moving magnet pump, reverse permeator, furnace test section, analysis chamber, supply tank, and plenum. As significant hazards are present in such experiments, safety is an integral focus of the experimental design.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

An integrated manifold learning approach for high-dimensional data feature extractions and its applications to online process monitoring of additive manufacturing

As an effective dimension reduction and feature extraction technique, manifold learning has been successfully applied to high-dimensional data analysis. With the rapid development of sensor technology, a large amount of high-dimensional data such as image streams can be easily available. Thus, a promising application of manifold learning is in the field of sensor signal analysis, particular for the applications of online process monitoring and control using high-dimensional data. The objective of this study is to develop a manifold learning-based feature extraction method for process monitoring of Additive Manufacturing (AM) using online sensor data. Due to the non-parametric nature of most existing manifold learning methods, their performance in terms of computational efficiency, as well as noise resistance has yet to be improved. To address this issue, this study proposes an integrated manifold learning approach termed multi-kernel metric learning embedded isometric feature mapping (MKML-ISOMAP) for dimension reduction and feature extraction of online high-dimensional sensor data such as images. Based on the extracted features with the utilization of supervised classification and regression methods, an online process monitoring methodology for AM is implemented to identify the actual process quality status. Finally, in the numerical simulation and real-world case studies, the proposed method demonstrates excellent performance in both prediction accuracy and computational efficiency.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES): a method for populating knowledge bases using zero-shot learning

Abstract Motivation Creating knowledge bases and ontologies is a time consuming task that relies on manual curation. AI/NLP approaches can assist expert curators in populating these knowledge bases, but current approaches rely on extensive training data, and are not able to populate arbitrarily complex nested knowledge schemas. Results Here we present Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES), a Knowledge Extraction approach that relies on the ability of Large Language Models (LLMs) to perform zero-shot learning and general-purpose query answering from flexible prompts and return information conforming to a specified schema. Given a detailed, user-defined knowledge schema and an input text, SPIRES recursively performs prompt interrogation against an LLM to obtain a set of responses matching the provided schema. SPIRES uses existing ontologies and vocabularies to provide identifiers for matched elements. We present examples of applying SPIRES in different domains, including extraction of food recipes, multi-species cellular signaling pathways, disease treatments, multi-step drug mechanisms, and chemical to disease relationships. Current SPIRES accuracy is comparable to the mid-range of existing Relation Extraction methods, but greatly surpasses an LLM’s native capability of grounding entities with unique identifiers. SPIRES has the advantage of easy customization, flexibility, and, crucially, the ability to perform new tasks in the absence of any new training data. This method supports a general strategy of leveraging the language interpreting capabilities of LLMs to assemble knowledge bases, assisting manual knowledge curation and acquisition while supporting validation with publicly-available databases and ontologies external to the LLM. Availability and implementation SPIRES is available as part of the open source OntoGPT package: https://github.com/monarch-initiative/ontogpt.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

SeqMask: Behavior Extraction Over Cyber Threat Intelligence Via Multi-Instance Learning

Abstract Identification and extraction of Tactics, Techniques and Procedures (TTPs) for Cyber Threat Intelligence (CTI) restore the full picture of cyber attacks and guide the analysts to assess the system risk. Existing frameworks can hardly provide uniform and complete processing mechanisms for TTPs information extraction without adequate knowledge background. A multi-instance learning approach named SeqMask is proposed in this paper as a solution. SeqMask extracts behavior keywords from CTI evaluated by the semantic impact, and predicts TTPs labels by conditional probabilities. Still, the framework has two mechanisms to determine the validity of keywords. One using expert experience verification. The other verifies the distortion of the classification effect by blocking existing keywords. In the experiments, SeqMask reached 86.07% and 73.99% in F1 scores for TTPs classifications. For the top 20% of keywords, the expert approval rating is 92.20%, where the average repetition of keywords whose scores between 100% and 90% is 60.02%. Particularly, when the top 65% of the keywords were blocked, the F1 decreased to about 50%; when removing the top 50%, the F1 was under 31%. Further, we also validate the possibility of extracting TTPs from full-size CTI and malware whose F1 are improved by 2.16% and 0.81%.

Ge, Wenhan↗