Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “relational databases”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Exploration and Analysis of Legacy Data at the Hanford Site Using PHOENIX - 19044

Radioactive contaminants pose risks to human health and the environment. These risks are influenced by factors such as disposal/release and transport mechanisms, location in the environment, potential exposure receptors, and the presence of comingled contaminants. Understanding the behavior, transport, and distribution of radionuclides in the environment is essential to facilitating risk-informed decision-making related to remedial design, action, and monitoring. These decisions are based on significant amounts of complex data that must be accessed, integrated, and analyzed to effectively evaluate data and formulate appropriate conclusions and recommendations. Often, data volume, complexity, and legacy access issues pose severe challenges for analysts, regulators, and stakeholders interested in evaluating scientific data to support decision-making. To address these challenges at the U.S. Department of Energy Hanford Site, Pacific Northwest National Laboratory (PNNL) has developed PHOENIX (PNNL-Hanford Online Environmental Information exchange) to facilitate risk-informed decision-making by streamlining investigative and analytical processes through the integration and visualization of previously isolated datasets created under distinct regulatory or operational regimes, each serving a specific purpose. With the large volume of data dispersed across many Hanford environmental databases, finding the correct sources of relevant data, accessing them, and relating them in a meaningful way can prove challenging and time consuming. By analyzing Hanford databases, performing substantial optimization using modern data processing methods and developing extensive queries on the refactored data, PHOENIX enables users to access, integrate, display, visualize, and retrieve data that would have been historically unfeasible to examine. The data and information accessible through PHOENIX help inform remediation-related assessments and evaluations, predict and quantify the risks that radioactive and chemical contaminants pose to human health and the environment, and communicate this information to decision makers responsible for risk-based remediation actions. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2012

CALIBRATING 100 YEARS OF POLAR FACULAE MEASUREMENTS: IMPLICATIONS FOR THE EVOLUTION OF THE HELIOSPHERIC MAGNETIC FIELD

Although the Sun's polar magnetic fields are thought to provide important clues for understanding the 11 year sunspot cycle, including the observed variations of its amplitude and period, the current database of high-quality polar field measurements spans relatively few sunspot cycles. In this paper, we address this deficiency by consolidating Mount Wilson Observatory polar faculae data from four data reduction campaigns, validating it through a comparison with facular data counted automatically from Michelson Doppler Imager (MDI) intensitygrams, and calibrating it against polar field measurements taken by the Wilcox Solar Observatory and average polar field and total polar flux calculated using MDI line-of-sight magnetograms. Our results show that the consolidated polar facular measurements are in excellent agreement with both polar field and polar flux estimates, making them an ideal proxy to study the evolution of the polar magnetic field. Additionally, we combine this database with sunspot area measurements to study the role of the polar magnetic flux in the evolution of the heliospheric magnetic field (HMF). We find that there is a strong correlation between HMF and polar flux at solar minimum and that, taken together, polar flux and sunspot area are better at explaining the evolution of the HMF during the last century than sunspot area alone.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2018

Daily Temperature and Precipitation Data for 223 Former-USSR Stations (NDP-040)

The stations in this dataset are considered by RIHMI to comprise one of the best networks suitable for temperature and precipitation monitoring over the the former-USSR. Factors involved in choosing these 223 stations included length or record, amount of missing data, and achieving reasonably good geographic coverage. There are indeed many more stations with daily data over this part of the world, and hundreds more station records are available through NOAA's Global Historical Climatology Network - Daily (GHCND) database. The 223 stations comprising this database are included in GHCND, but different data processing, updating, and quality assurance methods/checks mean that the agreement between records will vary depending on the station. The relative quality and accuracy of the common station records in the two databases also cannot be easily assessed. As of this writing, most of the common stations contained in the GHCND have more recent records, but not necessarily records starting as early as the records available here. This database contains four variables: daily mean, minimum, and maximum temperature, and daily total precipitation (liquid equivalent). Temperature were taken three times a day from 1881-1935, four times a day from 1936-65, and eight times a day since 1966. Daily mean temperature is defined as the average of all observations for each calendar day. Daily maximum/minimum temperatures are derived from maximum/minimum thermometer measurements. See the measurement description file for further details. Daily precipitation totals are also available (to the nearest tenth of a millimeter) for each station. Throughout the record, daily precipitation is defined as the total amount of precipitation recorded during a 24-h period, snowfall being converted to a liquid total by melting the snow in the gauge. From 1936 on, rain gauges were checked several times each day; the cumulative total of all observations during a calendar day was presumably used as the daily total. Again, see the measurement description file for further details. For access to the data files, click this link to the CDIAC data transition website: http://cdiac.ess-dive.lbl.gov/ndps/ndp040.html

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1995

A relational metric, its application to domain analysis, and an example analysis and model of a remote sensing domain

An objective and quantitative method has been developed for deriving models of complex and specialized spheres of activity (domains) from domain-generated verbal data. The method was developed for analysis of interview transcripts, incident reports, and other text documents whose original source is people who are knowledgeable about, and participate in, the domain in question. To test the method, it is applied here to a report describing a remote sensing project within the scope of the Earth Observing System (EOS). The method has the potential to improve the designs of domain-related computer systems and software by quickly providing developers with explicit and objective models of the domain in a form which is useful for design. Results of the analysis include a network model of the domain, and an object-oriented relational analysis report which describes the nodes and relationships in the network model. Other products include a database of relationships in the domain, and an interactive concordance. The analysis method utilizes a newly developed relational metric, a proximity-weighted frequency of co-occurrence. The metric is applied to relations between the most frequently occurring terms (words or multiword entities) in the domain text, and the terms found within the contexts of these terms. Contextual scope is selectable. Because of the discriminating power of the metric, data reduction from the association matrix to the network is simple. In addition to their value for design. the models produced by the method are also useful for understanding the domains themselves. They can, for example, be interpreted as models of presence in the domain.

Mcgreevy, Michael W.↗
DOE OSTI2014

Annotation and sequence diversity of transposable elements in common bean (Phaseolus vulgaris)

Common bean (Phaseolus vulgaris) is an important legume crop grown and consumed worldwide.With the availability of the common bean genome sequence, the next challenge is to annotate the genome and characterize functional DNA elements. Transposable elements (TEs) are the most abundant component of plant genomes and can dramatically affect genome evolution and genetic variation. Thus, it is pivotal to identify TEs in the common bean genome. In this study, we performed a genome-wide transposon annotation in common bean using a combination of homology and sequence structure-based methods.We developed a 2.12-Mb transposon database which includes 791 representative transposon sequences and is available upon request or from www.phytozome.org. Of note, nearly all transposons in the database are previously unrecognized TEs. More than 5,000 transposon-related expressed sequence tags (ESTs) were detected which indicates that some transposons may be transcriptionally active. Two Ty1-copia retrotransposon families were found to encode the envelope-like protein which has rarely been identified in plant genomes. Also, we identified an extra open reading frame (ORF) termed ORF2 from 15 Ty3-gypsy families that was located between the ORF encoding the retrotransposase and the 3 LTR. The ORF2 was in opposite transcriptional orientation to retrotransposase. Sequence homology searches and phylogenetic analysis suggested that the ORF2 may have an ancient origin, but its function is not clear. These transposon data provide a useful resource for understanding the genome organization and evolution and may be used to identify active TEs for developing transposon-tagging system in common bean and other related genomes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

A Data Processing Pipeline To Extract A Knowledge Graph From Sec Documents For Socio-technical Analysis Of Critical Infrastructure Influence

The code is written in Python and consists of the following pipeline that is implemented in Apache Airflow. This pipeline intends to understand the companies that are directly or indirectly involved with a type of critical infrastructure system at some point in that system's lifecycle. The pipeline takes a configuration file that specifies a list of initial companies to consider, a geographic region of interest (disk) expressed as a latitude/longitude point and distance, and a set of SEC form types from which to extract entities and relations. There are three main components to this pipeline as currently implemented: Social Network Extraction, Critical Infrastructure Network Extraction, and Inference and Fusion. First, Social Network Extraction, implemented as the `organizations_sec` component of the workflow graph queries the SEC EDGAR webservice using the list of initial companies from the configuration file. Given this, it extracts metadata that documents the number of each type of form for the given set of companies and their location. This forms metadata represents a catalog of data sources for the extracted social network knowledge graph. The pipeline then downloads these forms from the website and saves them in a build directory for further processing. These documents are then parsed for entities and relations. Second, the Critical Network Extraction component extracts entities and relations for a critical infrastructure sector. Currently, we focus on Electric Vehicle charging stations and this information is available via the Department of Energy (DOE) database on fueling stations maintained by NREL. Third, the Inference and Fusion component relates the social network graph to the critical infrastructure graph in order to understand the impact of a company within a geographic region. Relations include ownership of the EV Charging Station asset as well as maintenance/ownership of the EV payment networks. The fused network can be represented in many ways and currently we emit a knowledge graph.

Weaver, GabrielA.↗
DOE OSTI2020

A database of battery materials auto-generated using ChemDataExtractor

A database of battery materials is presented which comprises a total of 292,313 data records, with 214,617 unique chemical-property data relations between 17,354 unique chemicals and up to five material properties: capacity, voltage, conductivity, Coulombic efficiency and energy. 117,403 data are multivariate on a property where it is the dependent variable in part of a data series. The database was auto-generated by mining text from 229,061 academic papers using the chemistry-aware natural language processing toolkit, ChemDataExtractor version 1.5, which was modified for the specific domain of batteries. The collected data can be used as a representative overview of battery material information that is contained within text of scientific papers. Public availability of these data will also enable battery materials design and prediction via data-science methods. To the best of our knowledge, this is the first auto-generated database of battery materials extracted from a relatively large number of scientific papers. We also provide a Graphical User Interface (GUI) to aid the use of this database.

25 ENERGY STORAGE↗
DOE OSTI2018

Association between media coverage and prevalence of idiopathic environmental intolerance attributed to electromagnetic field in Taiwan

Highlights: • The prevalence rate of IEI-EMF in Taiwan declined form 13.3% in 2007 to 4.6% in 2012. • Residents in Taiwan had increasing exposure to EMF related to mobile phone use during the same period of time. • Media coverage of EMF related issues had the similar trend with the decreased IEI-EMF prevalence rate. • Most media coverage conveyed information on the negative health impacts of EMF. • Risk communications seem to be more effective than precautionary measures in coping with the public fears about EMF. Idiopathic environmental intolerance attributed to electromagnetic fields (IEI-EMF) describes symptoms sufferers attribute to exposure to electromagnetic fields (EMF). In Taiwan, the prevalence rate of IEI-EMF was 13.3% in 2007, but a survey using the same method found the rate declined to 4.6% in 2012. Because media reports may encourage readers to attribute their symptoms to EMF, the change might be related to media coverage. We searched articles indexed in the largest newspaper database in Taiwan to evaluate the association between media coverage and the prevalence of IEI-EMF. We also assessed the effects of other potential affecting factors. The number of newspaper articles related to EMF and IEI-EMF increased from 2005 to 2007 and then has been decreasing until 2012, which is compatible with the change in the prevalence of IEI-EMF. However, from 2007 to 2012, the other potential affecting factors such as density of mobile phone base stations, number of mobile phone users, total mobile phone calling time, and number of text messages sent through mobile phones all increased in Taiwan. This finding indicated a positive association between media coverage and the prevalence of IEI-EMF in Taiwan, which might also be true in other countries.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
NASA NTRS2017

Medical Optimization Network for Space Telemedicine Resources

INTRODUCTION: Long-duration missions beyond low Earth orbit introduce new constraints to the space medical system. Beyond the traditional limitations in mass, power, and volume, consideration must be given to other factors such as the inability to evacuate to Earth, communication delays, and limitations in clinical skillsets. As NASA develops the medical system for an exploration mission, it must have an ability to evaluate the trade space of what resources will be most important. The Medical Optimization Network for Space Telemedicine Resources (MONSTR) was developed over the past year for this reason, and is now a system for managing data pertaining to medical resources and their relative importance when addressing medical conditions. METHODS: The MONSTR web application with a Microsoft SQL database backend was developed and made accessible to Tableau v9.3 for analysis and visualization. The database was initially populated with a list of medical conditions of concern for an exploration mission taken from the Integrated Medical Model (IMM), a probabilistic model designed to quantify in-flight medical risk. A team of physicians working within the Exploration Medical Capability Element of NASA's Human Research Program compiled a list diagnostic and treatment medical resources required to address best- and worst-case scenarios of each medical condition using a terrestrial standard of care and entered this data into the system. This list included both tangible resources (e.g. medical equipment, medications) and intangible resources (e.g. clinical skills required to perform a procedure). The physician team then assigned criticality values to each instance of a resource, representing the importance of that resource to diagnosing or treating its associated condition(s). Medical condition probabilities of occurrence during a Mars mission were pulled from the IMM and imported into the MONSTR database for use within a resource criticality-weighting algorithm. DISCUSSION: The MONSTR tool is a novel approach to assess the relative value of individual resources needed for the diagnosis and treatment of medical conditions. Future work will add resources for prevention and long term care of these conditions. Once data collection is complete, MONSTR will provide the operational and research communities at NASA with information to support informed decisions regarding areas of research investment, future crew training, and medical supplies manifested as part of any exploration medical system.

Rubin, D.↗
NASA NTRS1992

COINS: A composites information database system

An automated data abstraction form (ADAF) was developed to collect information on advanced fabrication processes and their related costs. The information will be collected for all components being fabricated as part of the ACT program and include in a COmposites INformation System (COINS) database. The aim of the COINS development effort is to provide future airframe preliminary design and fabrication teams with a tool through which production cost can become a deterministic variable in the design optimization process. The effort was initiated by the Structures Technology Program Office (STPO) of the NASA LaRC to implement the recommendations of a working group comprised of representatives from the commercial airframe companies. The principal working group recommendation was to re-institute collection of composite part fabrication data in a format similar to the DOD/NASA Structural Composites Fabrication Guide. The fabrication information collection form was automated with current user friendly computer technology. This work in progress paper describes the new automated form and features that make the form easy to use by an aircraft structural design-manufacturing team.

Siddiqi, Shahid↗
DOE OSTI2014

Evaluation of Application Space Expansion for the Sensor Fish

The Pacific Northwest National Laboratory has developed an instrument known as the sensor fish that can be released into downstream passage routes at hydropower facilities to collect data on the physical conditions that a fish might be exposed to during passage through a turbine. The US Department of Energy Wind and Water Power Program sees value in expanding the sensor fish application space beyond large Kaplan turbines in the northwest United States to evaluate conditions to which a greater variety of fish species are exposed. Development of fish-friendly turbines requires an understanding of both physical passage conditions and biological responses to those conditions. Expanding the use of sensor fish into other application spaces will add to the knowledge base of physical passage conditions and could also enhance the use of sensor fish as a site-specific tool in mitigating potential impacts to fish populations from hydropower. The Oak Ridge National Laboratory (ORNL) National Hydropower Assessment Program (NHAAP) database contains hydropower facility characteristics that, along with national fish distribution data, were used to evaluate potential interactions between fish species and project characteristics related to downstream passage issues. ORNL developed rankings for the turbine types in the NHAAP database in terms of their potential to impact fish through injury or mortality during downstream turbine passage. National-scale fish distributions for 31 key migratory species were spatially intersected with hydropower plant locations to identify facilities where turbines with a high threat to fish injury or mortality overlap with the potential range of a sensitive fish species. A dataset was produced that identifies hydropower facilities where deployment of the sensor fish technology might be beneficial in addressing issues related to downstream fish passage. The dataset can be queried to target specific geographic regions, fish species, license expiration dates, generation capacity levels, ownership characteristics, turbine characteristics, or any combination of these metrics.

13 HYDRO ENERGY↗
NASA NTRS2000

The NASA ASTP Combined-Cycle Propulsion Database Project

The National Aeronautics and Space Administration (NASA) communicated its long-term R&D goals for aeronautics and space transportation technologies in its 1997-98 annual progress report (Reference 1). Under "Pillar 3, Goal 9" a 25-year-horizon set of objectives has been stated for the Generation 3 Reusable Launch Vehicle ("Gen 3 RLV") class of space transportation systems. An initiative referred to as "Spaceliner 100" is being conducted to identify technology roadmaps in support of these objectives. Responsibility for running "Spaceliner 100" technology development and demonstration activities have been assigned to NASA's agency-wide Advanced Space Transportation Program (ASTP) office located at the Marshall Space Flight Center. A key technology area in which advances will be required in order to meet these objectives is propulsion. In 1996, in order to expand their focus beyond "allrocket" propulsion systems and technologies (see Appendix A for further discussion), ASTP initiated technology development and demonstration work on combined-cycle airbreathing/rocket propulsion systems (ARTT Contracts NAS8-40890 through 40894). Combined-cycle propulsion (CCP) activities (see Appendix B for definitions) have been pursued in the U.S. for over four decades, resulting in a large documented knowledge base on this subject (see Reference 2). In the fall of 1999 the Combined-Cycle Propulsion Database (CCPD) project was established with the primary purpose of collecting and consolidating CCP related technical information in support of the ASTP's ongoing technology development and demonstration program. Science Applications International Corporation (SAIC) was selected to perform the initial development of the Database under its existing support contract with MSFC (Contract NAS8-99060) because of the company's unique combination of capabilities in database development, information technology (IT) and CCP knowledge. The CCPD is summarized in the descriptive 2-page flyer appended to this paper as Appendix C. The purpose of this paper is to provide the reader with an understanding of the objectives of the CCPD and relate the progress that has been made toward meeting those objectives.

Hyde, Eric H.↗
DOE OSTI2014

System and method employing a self-organizing map load feature database to identify electric load types of different electric loads

A method identifies electric load types of a plurality of different electric loads. The method includes providing a self-organizing map load feature database of a plurality of different electric load types and a plurality of neurons, each of the load types corresponding to a number of the neurons; employing a weight vector for each of the neurons; sensing a voltage signal and a current signal for each of the loads; determining a load feature vector including at least four different load features from the sensed voltage signal and the sensed current signal for a corresponding one of the loads; and identifying by a processor one of the load types by relating the load feature vector to the neurons of the database by identifying the weight vector of one of the neurons corresponding to the one of the load types that is a minimal distance to the load feature vector.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Age-related telomere attrition causes aberrant gene expression in sub-telomeric regions

Telomere attrition has been proposed as a biomarker and causal factor in aging. In addition to causing cellular senescence and apoptosis, telomere shortening has been found to affect gene expression in subtelomeric regions. Here, we analyzed the distribution of age-related differentially expressed genes from the GTEx RNA sequencing database of 54 tissue types from 979 human subjects and found significantly more upregulated than downregulated genes in subtelomeric regions as compared to the genome-wide average. Our data demonstrate spatial relationships between telomeres and gene expression in aging.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Estimation of trabecular bone parameters in children from multisequence MRI using texture-based regression

Purpose: This paper presents a statistical approach for the prediction of trabecular bone parameters from low-resolution multisequence magnetic resonance imaging (MRI) in children, thus addressing the limitations of high-resolution modalities such as HR-pQCT, including the significant exposure of young patients to radiation and the limited applicability of such modalities to peripheral bones in vivo. Methods: A statistical predictive model is constructed from a database of MRI and HR-pQCT datasets, to relate the low-resolution MRI appearance in the cancellous bone to the trabecular parameters extracted from the high-resolution images. The description of the MRI appearance is achieved between subjects by using a collection of feature descriptors, which describe the texture properties inside the cancellous bone, and which are invariant to the geometry and size of the trabecular areas. The predictive model is built by fitting to the training data a nonlinear partial least square regression between the input MRI features and the output trabecular parameters. Results: Detailed validation based on a sample of 96 datasets shows correlations >0.7 between the trabecular parameters predicted from low-resolution multisequence MRI based on the proposed statistical model and the values extracted from high-resolution HRp-QCT. Conclusions: The obtained results indicate the promise of the proposed predictive technique for the estimation of trabecular parameters in children from multisequence MRI, thus reducing the need for high-resolution radiation-based scans for a fragile population that is under development and growth.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Bay Area Regional Energy: Network Integrated Commercial Retrofits (BRICR) Project. Final Report

The BRICR project applied large-scale building energy modeling concepts with the aim of reducing the cost of energy efficiency targeting, design, and project development, and measurement of energy savings for energy efficiency programs implemented by local governments that serve small and medium commercial buildings (SMB). The project leveraged the services and resources of existing local government energy programs serving disadvantaged and hard-to-reach SMB customers. In contrast to programs run by utilities, local government programs generally do not have direct access to energy billing records for an entire class of customers in a geographic area, which prior research demonstrated useful for large-scale building energy model baseline development and calibration. , However, local governments are rich in public records that offer important clues about physical attributes and uses that, along with behavior, determine energy use. Relying only on public records, BRICR demonstrated development of credible baseline energy models for 3,792 office, retail, and hotel buildings. Publicly disclosed annual energy use data from a local energy benchmarking program and anonymized data from the Building Performance Database, the nation’s largest dataset about energy-related characteristics of buildings, were utilized to validate and calibrate energy models via an innovative method comparing distributions of energy intensity by fuel type for portfolios of buildings of similar size, vintage, and use. Portfolio calibration does not provide certainty that an energy model fits an individual building; the method is useful when billing data is not accessible – a common situation for researchers, energy service providers and ESCOs, local governments, and any party other than a utility. A software component was developed, the BRICR gem, which automates simulation when relevant data is added or edited by the user to a file saved in the standardized BuildingSync XML schema for energy audit data. The component was demonstrated as a simplified means to generate a mass of energy models corresponding to public records containing basic attributes such as building scale, location, use, year built, and aspect ratio in combination with building energy code prototype data corresponding to use and vintage. The component was also demonstrated as a simplified means to automate energy simulation when attributes are revised; the intention was to enable iterative improvement of the baseline model and energy savings estimates for common energy conservation measures as users revise relevant attributes based on their observations. In the context of institutional change and uncertainty for the participating local government energy programs, 13 whole building retrofits were completed. Impacts were measured by applying the CalTRACK2.0 methods to standardize measurement of normalized metered energy consumption. The GRIDMeter methods of stratified sampling and individual load shape analysis were applied to adjust for impacts of the effect of COVID-19 on retrofitted buildings in the context of all local buildings of similar size and use. Excluding impacts of the pandemic, retrofitted buildings demonstrated between 1.6% and 25.1% reduction in energy use. The project contributed use cases and feedback that helped inform evolution of the software tools and data formats that were combined for the first time in the BRICR project.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Machine Learning and Data Science to Advance Laboratory Earthquake Prediction and Illuminate the Mechanics of Precursors to Failure

Earthquakes represent one of our greatest natural hazards and in recent years human induced seismicity is adding to the threat. Even a modest improvement in the ability to forecast devastating large earthquakes or smaller shallow events associated with fluid injection could save thousands of lives and billions of dollars. Current efforts to forecast earthquakes are limited by knowledge of earthquake physics and hampered by a lack of reliable lab or field observations. However, recent work has provided a critical opportunity for advancement. We have found: 1) clear and consistent precursors prior to earthquake-like failure in the laboratory and 2) that lab earthquakes can be predicted using machine learning (ML). These works show that stick-slip failure events –the lab equivalent of earthquakes– are preceded by a cascade of micro-failure events that radiate elastic energy in a manner that foretells catastrophic failure. Remarkably, ML predicts the fault zone stress state, the failure time and in some cases the magnitude of lab earthquakes. In addition, the observations include clear precursors to failure in the form of changes in fault zone properties prior to lab earthquakes. Precursors have been observed in previous laboratory studies but their origin is poorly understood and their possible connection to ML based earthquake prediction is unknown. The work conducted under our project has dramatically expanded these efforts. We have developed an integrated data science approach to illuminate the physics of earthquake precursors and lab earthquake prediction. Our work has accelerated the development of ML, artificial intelligence (AI), and related data science approaches by providing massive data sets that are tightly connected to critical scientific problems and by bringing together leading subject matter experts and data scientists. Earthquake physics involves phenomena that are far from equilibrium. Our work has leveraged data science methods to illuminate these phenomena and investigate how they relate to earthquake prediction. In addition to a large database with many types of labeled events that is available to everyone, our work has advanced the fundamental understanding of seismic forecasting, earthquake physics, and fault rheology

58 GEOSCIENCES↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗