Leveraging local LLM for log analysis: Integrating OLLAMA into electronic logging systems
Explore the source record for details and available documents.
43 PARTICLE ACCELERATORS↗
SEARCH · Engineering Papers
Results for “log analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Machine Learning for Well Log Analysis in Uranium Mining
This project explores the use of Artificial Intelligence (AI) and Machine Learning (ML) techniques to automate well log analysis for uranium mining. Geophysical log data—spontaneous potential, resistivity, and gamma ray—were used to classify lithology, correlate well logs and identify roll front zonation patterns, which are critical for locating uranium ore bodies. Supervised ML algorithms such as eXtreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost), and Random Forest were trained to classify lithology with high accuracy. Gradient Boosting Machines (GBM), XGBoost, Random Forest, and Neural Networks were also used for role front zone identification. Moreover, a Fast Dynamic Time Warping (FastDTW) algorithm was employed for well log correlation. Additionally, sample lag was addressed using dynamic programming. Results demonstrate the potential of AI and ML to streamline well log analysis and enhance uranium exploration workflows.
11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
Requirements-Driven Log Analysis Extended Abstract
Imagine that you are tasked to help a project improve their testing effort. In a realistic scenario it will quickly become clear, that having an impact is diffcult. First of all, it will likely be a challenge to suggest an alternative approach which is significantly more automated and/or more effective than current practice. The reality is that an average software system has a complex input/output behavior. An automated testing approach will have to auto-generate test cases, each being a pair (i; o) consisting of a test input i and an oracle o. The test input i has to be somewhat meaningful, and the oracle o can be very complicated to compute. Second, even in case where some testing technology has been developed that might improve current practice, it is then likely difficult to completely change the current behavior of the testing team unless the technique is obviously superior and does everything already done by existing technology. So is there an easier way to incorporate formal methods-based approaches than the full edged test revolution? Fortunately the answer is affirmative. A relatively simple approach is to benefit from possibly already existing logging infrastructure, which after all is part of most systems put in production. A log is a sequence of events, generated by special log recording statements, most often manually inserted in the code by the programmers. An event can be considered as a data record: a mapping from field names to values. We can analyze such a log using formal methods, for example checking it against a formal specification. This separates running the system for analyzing its behavior. It is not meant as an alternative to testing since it does not address the important in- put generation problem. However, it offers a solution which testing teams might accept since it has low impact on the existing process. A single person might be assigned to perform such log analysis, compared to the entire testing team changing behavior.
log analysis↗
Database-Agnostic Log Analysis and Monitoring Framework
Prior to my internship, I was informed that a previous intern had built a tool to analyse MongoDB logs and look for invalid access attempts, which served as a great reference point for my project. I was initially tasked with expanding on her prototype and filling in the gaps such as integrating it with the main monitoring tool the lab uses. Eventually, the scope grew, expanding to support other databases and a growing collection of tools. I organized the framework around an observer pattern, meaning one point in the program sending updates to the rest of the framework. Every time a log was read and parsed, it was sent to be processed by the tools, using the type of event as a means to determine which tools should get a chance to act on the log. This decouples the tools from the log reader, making future updates and additions much easier. The framework processes MongoDB logs at ~135,000 entries per second and PostgreSQL logs at ~170,500 entries per second, accurately detecting anomalies such as slow queries and connections from unknown addresses. This framework serves to fill gaps in database monitoring tools currently implemented at the lab, such as tracking failed authentication for PostgreSQL and MongoDB which had very minimal or none before this framework. National labs such as Fermilab hold sensitive data and valuable computing resources, making them attractive targets. Monitoring intrusion attempts on databases is made much easier by this comprehensive monitoring suite.
Clark, Dylan [Unlisted, US, IL; Fermilab]↗
User Evaluation of the NASA Technical Report Server Recommendation Service
We present the user evaluation of two recommendation server methodologies implemented for the NASA Technical Report Server (NTRS). One methodology for generating recommendations uses log analysis to identify co-retrieval events on full-text documents. For comparison, we used the Vector Space Model (VSM) as the second methodology. We calculated cosine similarities and used the top 10 most similar documents (based on metadata) as 'recommendations'. We then ran an experiment with NASA Langley Research Center (LaRC) staff members to gather their feedback on which method produced the most 'quality' recommendations. We found that in most cases VSM outperformed log analysis of co-retrievals. However, analyzing the data revealed the evaluations may have been structurally biased in favor of the VSM generated recommendations. We explore some possible methods for combining log analysis and VSM generated recommendations and suggest areas of future work.
Nelson, Michael L.↗
User Evaluation of the NASA Technical Report Server Recommendation Service
We present the user evaluation of two recommendation server methodologies implemented for the NASA Technical Report Server (NTRS). One methodology for generating recommendations uses log analysis to identify co-retrieval events on full-text documents. For comparison, we used the Vector Space Model (VSM) as the second methodology. We calculated cosine similarities and used the top 10 most similar documents (based on metadata) as recommendations . We then ran an experiment with NASA Langley Research Center (LaRC) staff members to gather their feedback on which method produced the most quality recommendations. We found that in most cases VSM outperformed log analysis of co-retrievals. However, analyzing the data revealed the evaluations may have been structurally biased in favor of the VSM generated recommendations. We explore some possible methods for combining log analysis and VSM generated recommendations and suggest areas of future work.
Nelson, Michael L.↗
Improving Production in the Emerging Paradox Oil Play (Final Technical Report)
This report documents the results of the DOE-funded project Improving Production in the Emerging Paradox Oil Play, focused on integrating geological characterization, petrophysical analysis, reservoir simulation, and field development planning to improve hydrocarbon recovery from the Paradox Basin. Project activities included core and log analysis, assessment of reservoir heterogeneity and compartmentalization, evaluation of completion and stimulation strategies, and development of an uncertainty-aware field development framework. Findings highlight the importance of selective completions, geology-driven well placement, and targeted stimulation in vertically heterogeneous carbonate reservoirs.
02 PETROLEUM↗
What Can We Learn from One Billion Ground System Log Messages?
Shortage of log-based data in a ground system they have traditionally been the under achievers in a satellite ground system. This is due to several factors: Once log messages scroll out of view on the TTC event console window they are soon forgotten. Application and system log files are scattered across directories within a system, across a multitude of servers, and across one or more databases making access cumbersome. Typical tools to perform log file content searching are generally crude and typically only employed as part of trouble-shooting exercises.As we move towards satellite constellations and fleets and add even more status information, the number of messages keeps growing. One mission now estimates that they could generate 3,000,000 messages per day 1 billion per year - for the life of their mission. What to do with those 1 billion messages? That is the challenge. With the recent technological advances in the management of large data sets, text-based processing, and data analytics, there are now capabilities that we can provide to the ground system engineers and satellite operators to address what we postulate are missed opportunities. Advanced real-time log analysis can allow us to be less reactionary in favor of being more proactive. Analytics goals include the ability to: Identify root cause of unexpected events, failures or error conditions enabled by correlating disparate data. Detect security breaches attempts before they are successful. Help admins ensure IT resources continue running optimally. Identify trends and patterns that may indicate impending failures or error conditions for valuable assets before they happen. Compare satellites in a fleet or constellation in terms of number of alarms, number of command sent to them, etc.. Answer questions like "Are the operations support needs increasing over the past year?" or "Have we seen this combination of alarm conditions before?" But really, once the tools are readily available the users will start realizing what can be done with their new powers. In this presentation we will show the results of analyzing millions of actual mission operations log messages, how the results can be displayed to the user, and how new products now available as open source can be applied to the challenges of large scale time-tagged text-based mission operations messages. Flight operations team members believe that this is a powerful new option for how they assess overall system and space asset health. Technical descriptions of the design, tools, and storage will be provided. One billion messages? Bring'em on!
Orsborne, Sharon↗
A Scala DSL for RETE-Based Runtime Verification
Runtime verification (RV) consists in part of checking execution traces against formalized specifications. Several systems have emerged, most of which support specification notations based on state machines, regular expressions, temporal logic, or grammars. The field of Artificial Intelligence (AI) has for an even longer period of time studied rule-based production systems, which at a closer look appear to be relevant for RV, although seemingly focused on slightly different application domains, such as for example business processes and expert systems. The core algorithm in many of these systems is the Rete algorithm. We have implemented a Rete-based runtime verification system, named LogFire (originally intended for offline log analysis but also applicable to online analysis), as an internal DSL in the Scala programming language, using Scala's support for defining DSLs. This combination appears attractive from a practical point of view. Our contribution is in part conceptual in arguing that such rule-based frameworks originating from AI may be suited for RV.
matching problem↗
Informing Plant Asset Reliability and Availability Through AI-Driven Analysis of Operator Logs
The availability and reliability of nuclear power plant (NPP) structures, systems, and components (SSCs) are critical parameters for NPP safety. Tracking these parameters is necessary but costly and labor-intensive, requiring the collection and evaluation of SSC event data such as shutdowns, startups, and failures. To show how these events are needed for the parameters an example is given: one measure of reliability is based on the number of equipment failure events and the number of run hours (i.e., the time from a startup event to a shutdown event). Here, this work investigates using artificial intelligence (AI) to mine NPP operator log entry texts for SSC event data. Four AI approaches were explored for identifying these events, including natural language processing (NLP) methods, generative AI, generative AI combined with NLP, and topic modeling. A key challenge addressed with all four approaches is the brevity of operator log entries. Among these four a neural network–based NLP method was shown to be the most promising for this application, achieving F1 scores of 86.0% for shutdowns, 92.2% for startups, and 80.4% for failures on a subject-matter-expert-curated dataset from NPP operator logs, compared to a baseline of 66.6% for a random classifier. This shows that NLP methods can perform better than generative AI. Additionally, the NLP methods combined with generative AI were shown to perform better than generative AI alone. Generative AI was most successful at providing the background information for the NLP methods to use. This work demonstrates the potential to use AI to automate parameter collection from NPP operator log entries and other records.
97 - MATHEMATICS AND COMPUTING↗
Accuracy for detection of simulated lesions: comparison of fluid-attenuated inversion-recovery, proton density--weighted, and T2-weighted synthetic brain MR imaging
OBJECTIVE: The objective of our study was to determine the effects of MR sequence (fluid-attenuated inversion-recovery [FLAIR], proton density--weighted, and T2-weighted) and of lesion location on sensitivity and specificity of lesion detection. MATERIALS AND METHODS: We generated FLAIR, proton density-weighted, and T2-weighted brain images with 3-mm lesions using published parameters for acute multiple sclerosis plaques. Each image contained from zero to five lesions that were distributed among cortical-subcortical, periventricular, and deep white matter regions; on either side; and anterior or posterior in position. We presented images of 540 lesions, distributed among 2592 image regions, to six neuroradiologists. We constructed a contingency table for image regions with lesions and another for image regions without lesions (normal). Each table included the following: the reviewer's number (1--6); the MR sequence; the side, position, and region of the lesion; and the reviewer's response (lesion present or absent [normal]). We performed chi-square and log-linear analyses. RESULTS: The FLAIR sequence yielded the highest true-positive rates (p < 0.001) and the highest true-negative rates (p < 0.001). Regions also differed in reviewers' true-positive rates (p < 0.001) and true-negative rates (p = 0.002). The true-positive rate model generated by log-linear analysis contained an additional sequence-location interaction. The true-negative rate model generated by log-linear analysis confirmed these associations, but no higher order interactions were added. CONCLUSION: We developed software with which we can generate brain images of a wide range of pulse sequences and that allows us to specify the location, size, shape, and intrinsic characteristics of simulated lesions. We found that the use of FLAIR sequences increases detection accuracy for cortical-subcortical and periventricular lesions over that associated with proton density- and T2-weighted sequences.
NASA Discipline Neuroscience↗
Treatment of singularities in cracked bodies
Three-dimensional finite-element analyses of middle-crack tension (M-T) and bend specimens subjected to mode I loadings were performed to study the stress singularity along the crack front. The specimen was modeled using 20-node isoparametric elements. The displacements and stresses from the analysis were used to estimate the power of singularities using a log-log regression analysis along the crack front. The analyses showed that finite-sized cracked bodies have two singular stress fields of the form rho = C sub o (theta, z) r to the -1/2 power + D sub o (theta, phi) R to the lambda rho power. The first term is the cylindrical singularity with the power -1/2 and is dominant over the middle 96 pct (for Poisson's ratio = 0.3) of the crack front and becomes nearly zero at the free surface. The second singularity is a vertex singularity with the vertex point located at the intersection of the crack front and the free surface. The second term is dominant at the free surface and becomes nearly zero away from the the boundary layer. The thickness of the boundary layer depends on Poisson's ratio of the material and is independent of the specimen type. The thickness of the boundary layer varied from 0 pct to about 5 pct of the total specimen thickness as Poisson's ratio varied from 0.0 to 0.45. Because there are two singular stress fields near the free surface, the strain energy release rate (G) is an appropriate parameter to measure the severity of the crack.
Shivakumar, K. N.↗
Treatment of singularities in a middle-crack tension specimen
A three-dimensional finite-element analysis of a middle-crack tension specimen subjected to mode I loading was performed to study the stress singularity along the crack front. The specimen was modeled using 20-node isoparametric elements with collapsed nonsingular elements at the crack front. The displacements and stresses from the analysis were used to estimate the power of singularities, by a log-log regression analysis, along the crack front. Analyses showed that finite-sized cracked bodies have two singular stress fields. Because of two singular stress fields near the free surface and the classical square root singularity elsewhere, the strain energy release rate appears to be an appropriate parameter all along the crack front.
Shivakumar, K. N.↗
Treatment of singularities in cracked bodies
Three-dimensional finite-element analyses of middle-crack tension (M-T) and bend specimens subjected to mode I loadings were performed to study the stress singularity along the crack front. The specimen was modeled using 20-node isoparametric elements. The displacements and stresses from the analysis were used to estimate the power of singularities using a log-log regression analysis along the crack front. The analyses showed that finite-sized cracked bodies have two singular stress fields of the form rho = C sub o (theta, z) r to the -1/2 power + D sub o (theta, phi) R to the lambda rho power. The first term is the cylindrical singularity with the power -1/2 and is dominant over the middle 96 pct (for Poisson's ratio = 0.3) of the crack front and becomes nearly zero at the free surface. The second singularity is a vertex singularity with the vertex point located at the intersection of the crack front and the free surface. The second term is dominant at the free surface and becomes nearly zero away from the boundary layer. The thickness of the boundary layer depends on Poisson's ratio of the material and is independent of the specimen type. The thickness of the boundary layer varied from 0 pct to about 5 pct of the total specimen thickness as Poisson's ratio varied from 0.0 to 0.45. Because there are two singular stress fields near the free surface, the strain energy release rate (G) is an appropriate parameter to measure the severity of the crack.
Shivakumar, K. N.↗
LLUDA (Low-cost Logging for Ubiquitous Decarbonization and Analysis) [SWR-24-131]
This repository features an Arduino-based GPS logger, called LLUDA, which uses the Adafruit Ultimate GPS FeatherWing with the Adafruit Feather M0 Adalogger, designed to capture and log GPS data in CSV format onto an SD card (embedded inside the Feather M0 Adalogger). The FeatherWing is powered through the vehicle's auxiliary power (from cigarette lighter) socket via a micro USB cable. The data it collects can be used to analyze vehicle operations and support vehicle modeling applications, helping to assess the potential for electrification.
Fakhimi, Setayesh [National Renewable Energy Labor↗
TraceContract
TraceContract is an API (Application Programming Interface) for trace analysis. A trace is a sequence of events, and can, for example, be generated by a running program, instrumented appropriately to generate events. An event can be any data object. An example of a trace is a log file containing events that a programmer has found important to record during a program execution. Trace - Contract takes as input such a trace together with a specification formulated using the API and reports on any violations of the specification, potentially calling code (reactions) to be executed when violations are detected. The software is developed as an internal DSL (Domain Specific Language) in the Scala programming language. Scala is a relatively new programming language that is specifically convenient for defining such internal DSLs due to a number of language characteristics. This includes Scala s elegant combination of object-oriented and functional programming, a succinct notation, and an advanced type system. The DSL offers a combination of data-parameterized state machines and temporal logic, which is novel. As an extension of Scala, it is a very expressive and convenient log file analysis framework.
Kavelund, Klaus↗
Application of a dedicated micro-processor for automatic data acquisition of load strain and acoustic-emission data
This paper describes an automatic system for acquisition, logging and analysis of stress, strain and acoustic emission data. The complete system is a combination of several standard items of equipment and a newly developed intelligent data logger which has the capability to send data directly to a large scale computer for reduction and plotting. The data logging and all interfacing with peripheral equipment are accomplished using digital electronics. The heart of the system is a four-bit microprocessor and digital cassette tape unit. The system, which is controlled by instructions from a cathode ray tube terminal, has the capability to transmit data from the cassette unit to a large computer via a time sharing option. Typical results are presented for acoustic emission count rate and total count curves superimposed on stress-strain diagrams for boron/epoxy advanced composites.
Wilson, G. H.↗
Preliminary geological investigation of AIS data at Mary Kathleen, Queensland, Australia
The Airborne Imaging Spectrometer (AIS) was flown over granitic, volcanic, and calc-silicate terrain around the Mary Kathleen Uranium Mine in Queensland, in a test of its mineralocial mapping capabilities. An analysis strategy and restoration and enhancement techniques were developed to process the 128 band AIS data. A preliminary analysis of one of three AIS flight lines shows that the data contains considerable spectral variation but that it is also contaminated by second-order leakage of radiation from the near-infrared region. This makes the recognition of expected spectral absorption shapes very difficult. The effect appears worst in terrains containing considerable vegetation. Techniques that try to predict this supplementary radiation coupled with the log residual analytical technique show that expected mineral absorption spectra can be derived. The techniques suggest that with additional refinement correction procedures, the Australian AIS data may be revised. Application of the log residual analysis method has proved very successful on the cuprite, Nevada data set, and for highlighting the alunite, linite, and SiOH mineralogy.
Huntington, J. F.↗