Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ORACLES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

TU-AB-202-10: How Effective Are Current Atlas Selection Methods for Atlas-Based Auto-Contouring in Radiotherapy Planning?

Purpose: To investigate the effectiveness of atlas selection methods for improving atlas-based auto-contouring in radiotherapy planning. Methods: 275 H&N clinically delineated cases were employed as an atlas database from which atlases would be selected. A further 40 previously contoured cases were used as test patients against which atlas selection could be performed and evaluated. 26 variations of selection methods proposed in the literature and used in commercial systems were investigated. Atlas selection methods comprised either global or local image similarity measures, computed after rigid or deformable registration, combined with direct atlas search or with an intermediate template image. Workflow Box (Mirada-Medical, Oxford, UK) was used for all auto-contouring. Results on brain, brainstem, parotids and spinal cord were compared to random selection, a fixed set of 10 “good” atlases, and optimal selection by an “oracle” with knowledge of the ground truth. The Dice score and the average ranking with respect to the “oracle” were employed to assess the performance of the top 10 atlases selected by each method. Results: The fixed set of “good” atlases outperformed all of the atlas-patient image similarity-based selection methods (mean Dice 0.715 c.f. 0.603 to 0.677). In general, methods based on exhaustive comparison of local similarity measures showed better average Dice scores (0.658 to 0.677) compared to the use of either template image (0.655 to 0.672) or global similarity measures (0.603 to 0.666). The performance of image-based selection methods was found to be only slightly better than a random (0.645). Dice scores given relate to the left parotid, but similar results patterns were observed for all organs. Conclusion: Intuitively, atlas selection based on the patient CT is expected to improve auto-contouring performance. However, it was found that published approaches performed marginally better than random and use of a fixed set of representative atlases showed favourable performance. This research was funded via InnovateUK Grant 600277 as part of Eurostars Grant E!9297. DP,BS,MG,TK are employees of Mirada Medical Ltd.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Biomass-burning smoke's properties and its interactions with marine stratocumulus clouds in WRF-CAM5 and southeastern Atlantic field campaigns

Abstract. A large part of the uncertainty in climate projections comes from uncertain aerosol properties and aerosol–cloud interactions as well as the difficulty in remotely sensing them. The southeastern Atlantic functions as a natural laboratory to study biomass-burning smoke and to constrain this uncertainty. We address these gaps by comparing the Weather Research and Forecasting with Chemistry Community Atmosphere Model (WRF-CAM5) to the multi-campaign observations ORACLES (ObseRvations of Aerosols above CLouds and their intEractionS), CLARIFY (CLoud–Aerosol–Radiation Interaction and Forcing), and LASIC (Layered Atlantic Smoke Interactions with Clouds) in the southeastern Atlantic in August 2017 to evaluate a large range of the model's aerosol chemical properties, size distributions, processes, and transport, as well as aerosol–cloud interactions. Overall, while WRF-CAM5 is able to represent smoke properties and transport, some key discrepancies highlight the need for further analysis. Observations of smoke composition show an overall decrease in aerosol mean diameter as smoke ages over 4–12 d, while the model lacks this trend. A decrease in the mass ratio of organic aerosol (OA) to black carbon (BC), OA:BC, and the OA mass to carbon monoxide (CO) mixing ratio, OA:CO, suggests that the model is missing processes that selectively remove OA from the particle phase, such as photolysis and heterogeneous aerosol chemistry. A large (factor of ∼2.5) enhancement in sulfate from the free troposphere (FT) to the boundary layer (BL) in observations is not present in the model, pointing to the importance of properly representing secondary sulfate aerosol formation from marine dimethyl sulfide and gaseous SO2 smoke emissions. The model shows a persistent overprediction of aerosols in the marine boundary layer (MBL), especially for clean conditions, which multiple pieces of evidence link to weaker aerosol removal in the modeled MBL than reality. This evidence includes several model features, such as not representing observed shifts towards smaller aerosol diameters, inaccurate concentration ratios of carbon monoxide and black carbon, underprediction of heavy rain events, and little evidence of persistent biases in modeled entrainment. The average below-cloud aerosol activation fraction (NCLD/NAER) remains relatively constant in WRF-CAM5 between field campaigns (∼0.65), while it decreases substantially in observations from ORACLES (∼0.78) to CLARIFY (∼0.5), which could be due to the model misrepresentation of clean aerosol conditions. WRF-CAM5 also overshoots an observed upper limit on liquid cloud droplet concentration around NCLD= 400–500 cm−3 and overpredicts the spread in NCLD. This could be related to the model often drastically overestimating the strength of boundary layer vertical turbulence by up to a factor of 10. We expect these results to motivate similar evaluations of other modeling systems and promote model development to reduce critical uncertainties in climate simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

An attribution of the low single-scattering albedo of biomass burning aerosol over the southeastern Atlantic

Abstract. Aerosol over the remote southeastern Atlantic is some of the most sunlight-absorbing aerosol on the planet: the in situ free-tropospheric single-scattering albedo at the 530 nm wavelength (SSA530 nm) ranges from 0.83 to 0.89 within ORACLES (ObseRvations of Aerosols above CLouds and their intEractionS) aircraft flights from late August–September. Here we seek to explain the low SSA. The SSA depends strongly on the black carbon (BC) number fraction, which ranges from 0.15 to 0.4. Low organic aerosol (OA)-to-BC mass ratios of 8–14 and modified combustion efficiency values >0.975 point indirectly to the dry, flame-efficient combustion of primarily grass fuels, with back trajectories ending in the miombo woodlands of Angola. The youngest aerosol, aged 4–5 d since emission, occupied the top half of a 5 km thick plume sampled directly west of Angola with a vertically consistent BC:ΔCO (carbon monoxide) ratio, indicating a homogenization of the source emissions. The younger aerosol, transported more quickly off of the continent by stronger winds, overlaid older, slower-moving aerosol with a larger mean particle size and fraction of BC-containing particles. This is consistent with ongoing gas condensation and the coagulation of smaller non-BC particles upon the BC-containing particles. The particle volumes and OA:BC mass ratios of the older aerosol were smaller, attributed primarily to evaporation following fragmentation, instead of dilution or thermodynamics. The CLARIFY (CLoud–Aerosol–Radiation Interaction and Forcing: Year 2017) aircraft campaign sampled aerosols that had traveled further to reach the more remote Ascension Island. CLARIFY reported higher BC number fractions, lower OA:BC mass ratios, and lower SSA yet larger mass absorption coefficients compared to this study's. Values from one ORACLES 2017 flight, held midway to Ascension Island, are intermediate, confirming the long-range changes. Overall the data are most consistent with continuing oxidation through fragmentation releasing aerosols that subsequently enter the gas phase, reducing the OA mass, rather than evaporation through dilution or thermodynamics. The data support the following best fit: SSA530nm=0.801+0055⋅(OA:BC) (r=0.84). The fires of southern Africa emit approximately one-third of the world's carbon; the emitted aerosols are distinct from other regional smoke emissions, and their composition needs to be represented appropriately to realistically depict regional aerosol radiative effects.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Aerosol hygroscopicity over the southeast Atlantic Ocean during the biomass burning season – Part 1: From the perspective of scattering enhancement​​​​​​​

Aerosol hygroscopicity plays a vital role in aerosol radiative forcing. One key parameter describing hygroscopicity is the scattering enhancement factor, f(RH), defined as the ratio of the scattering coefficient at humidified relative humidity (RH) to its dry value. Here, we utilize the f(80 %) from ORACLES (ObseRvations of Aerosols above CLouds and their intEractionS) 2016 and 2018 airborne measurements to investigate the hygroscopicity of aerosols, its vertical distribution, its relationship with chemical composition, and its sensitivity to organic aerosol (OA) hygroscopicity over the southeast Atlantic (SEA) Ocean during the biomass burning (BB) season. We found that aerosol hygroscopicity remains steady above 2 km, with a mean f(80 %) of 1.40 ± 0.17. Below 2 km, aerosol hygroscopicity increases with decreasing altitude, with a mean f(80 %) of 1.51 ± 0.22, consistent with higher values of BB aerosol hygroscopicity found in the literature. The hygroscopicity parameter of OA (κ OA ) is retrieved from the Mie model with a mean value of 0.11 ± 0.08, which is in the middle to upper range compared to the literature. Higher OA hygroscopicity is related to aerosols that are more aged, oxidized, and present at lower altitudes. The enhanced biomass burning aerosol (BBA) hygroscopicity at lower altitudes is mainly due to a lower OA fraction, increased sulfate fraction, and greater κ OA at lower altitudes. We propose a parameterization that quantifies f(RH) with chemical composition and κOA based on Mie simulation of internally mixed OA–(NH 4 ) 2 SO 4 –BC mixtures. The good agreement between the predictions and the ORACLES measurements implies that the aerosols in the SEA during the BB season can be largely represented by the OA–(NH 4 ) 2 SO 4 –BC internal mixture with respect to the f(RH) prediction. The sensitivity of f(RH) to κ OA indicates that applying a constant κ OA is only suitable when the OA fraction is low and κ OA shows limited variation. However, in situations deviating these two criteria, κ OA can notably impact scattering coefficients and aerosol radiative effect; therefore, accounting for κ OA variability is recommended.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Quantifying the impacts of marine aerosols over the southeast Atlantic Ocean using a chemical transport model: implications for aerosol–cloud interactions

The southeast Atlantic region, characterized by persistent stratocumulus clouds, has one of the highest uncertainties in aerosol radiative forcing and significant variability across climate models. In this study, we analyze the seasonally varying role of marine aerosol sources and identify key uncertainties in aerosol composition at cloud-relevant altitudes over the southeast Atlantic using the GEOS-Chem chemical transport model. We evaluate simulated aerosol optical depth (AOD) and speciated aerosol concentrations against those collected from ground observations and aircraft campaigns such as LASIC, ORACLES, and CLARIFY, conducted during 2017. The model consistently underestimates AOD relative to AERONET, particularly at remote locations like Ascension Island. However, when compared with aerosol mass concentrations from aircraft campaigns during the biomass burning period, it performs adequately at cloud-relevant altitudes, with a normalized mean bias (NMB) between -3.5 % (CLARIFY) and -7.5 % (ORACLES). At these altitudes, in the model, organic aerosols (63 %) dominate during the biomass burning period, while sulfate (41 %) prevails during austral summer, when dimethylsulfide (DMS) emissions peak in the model. Our findings indicate that marine sulfate can account for up to 69 % of total sulfate during the high-DMS period. Sensitivity analyses indicate that refining DMS emissions and oxidation chemistry may increase sulfate aerosol produced from marine sources, highlighting that there remains large uncertainty as to the role of DMS emissions in the marine boundary layer. Additionally, we find marine primary organic aerosol emissions may substantially increase total organic aerosol concentrations, particularly during austral summer. This study underscores the imperative need to refine marine emissions and their chemical transformations, as aerosols from marine sources are a major component of total aerosols at cloud-relevant altitudes and may impact uncertainties in aerosol radiative forcing over the southeast Atlantic.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1982

Optimal Regulator Algorithms for the Control of Linear Systems

Control-theory design package, called Optimal Regulator Algorithms for Control of Linear Systems (ORACLS), aids in design of controllers and optimal filters for systems modeled by linear, time invariant differential and difference equations. ORACLS is particularly attractive rigorous tool for dealing with multi-input and multi-output dynamic systems in both continuous and discrete forms.

Armstrong, E. S.↗
NASA NTRS1981

Data base management system analysis and performance testing with respect to NASA requirements

Several candidate Data Base Management Systems (DBM's) that could support the NASA End-to-End Data System's Integrated Data Base Management System (IDBMS) Project, later rescoped and renamed the Packet Management System (PMS) were evaluated. The candidate DBMS systems which had to run on the Digital Equipment Corporation VAX 11/780 computer system were ORACLE, SEED and RIM. Oracle and RIM are both based on the relational data base model while SEED employs a CODASYL network approach. A single data base application which managed stratospheric temperature profiles was studied. The primary reasons for using this application were an insufficient volume of available PMS-like data, a mandate to use actual rather than simulated data, and the abundance of available temperature profile data.

Martin, E. A.↗
NASA NTRS1986

ADAMS: AIRLAB data management system user's guide

The AIRLAB Data Management System (ADAMS) is an online environment that supports research at NASA's AIRLAB. ADAMS provides an easy to use interactive interface that eases the task of documenting and managing information about experiments and improves communication among project members. Data managed by ADAMS includes information about experiments, data sets produced, software and hardware available in AIRLAB as well as that used in a particular experiment, and an on-line engineer's notebook. The User's Guide provides an overview of the ADAMS system as well as details of the operations available within ADAMS. A tutorial section takes the user step-by-step through a typical ADAMS session. ADAMS runs under the VAX/VMS operating system and uses the ORACLE database management system and DEC/FMS (the Forms Management System). ADAMS can be run from any VAX connected via DECnet to the ORACLE host VAX. The ADAMS system is designed for simplicity, so interactions within the underlying data management system and communications network are hidden from the user.

Conrad, C. L.↗
NASA NTRS1986

Database interfaces on NASA's heterogeneous distributed database system

The purpose of the ORACLE interface is to enable the DAVID program to submit queries and transactions to databases running under the ORACLE DBMS. The interface package is made up of several modules. The progress of these modules is described below. The two approaches used in implementing the interface are also discussed. Detailed discussion of the design of the templates is shown and concluding remarks are presented.

Huang, S. H. S.↗
NASA NTRS1990

An SQL query generator for CLIPS

As expert systems become more widely used, their access to large amounts of external information becomes increasingly important. This information exists in several forms such as statistical, tabular data, knowledge gained by experts and large databases of information maintained by companies. Because many expert systems, including CLIPS, do not provide access to this external information, much of the usefulness of expert systems is left untapped. The scope of this paper is to describe a database extension for the CLIPS expert system shell. The current industry standard database language is SQL. Due to SQL standardization, large amounts of information stored on various computers, potentially at different locations, will be more easily accessible. Expert systems should be able to directly access these existing databases rather than requiring information to be re-entered into the expert system environment. The ORACLE relational database management system (RDBMS) was used to provide a database connection within the CLIPS environment. To facilitate relational database access a query generation system was developed as a CLIPS user function. The queries are entered in a CLlPS-like syntax and are passed to the query generator, which constructs and submits for execution, an SQL query to the ORACLE RDBMS. The query results are asserted as CLIPS facts. The query generator was developed primarily for use within the ICADS project (Intelligent Computer Aided Design System) currently being developed by the CAD Research Unit in the California Polytechnic State University (Cal Poly). In ICADS, there are several parallel or distributed expert systems accessing a common knowledge base of facts. Expert system has a narrow domain of interest and therefore needs only certain portions of the information. The query generator provides a common method of accessing this information and allows the expert system to specify what data is needed without specifying how to retrieve it.

Snyder, James↗
NASA NTRS2000

The Network Configuration of an Object Relational Database Management System

The networking and implementation of the Oracle Database Management System (ODBMS) requires developers to have knowledge of the UNIX operating system as well as all the features of the Oracle Server. The server is an object relational database management system (DBMS). By using distributed processing, processes are split up between the database server and client application programs. The DBMS handles all the responsibilities of the server. The workstations running the database application concentrate on the interpretation and display of data.

Diaz, Philip↗
NASA NTRS2000

A Novel High Efficient Laser Transmitter Design for a Space-borne Ozone Differential Lidar (DIAL)

Development of a UV laser transmitter capable of operating from a space platform is a critical step in enabling global earth observations of aerosols and ozone at resolutions greater than current passive instrument capabilities. Tropospheric chemistry is well recognized as the next frontier for global atmospheric measurement. NASA Langley Research Center (LaRC) and the Canadian Space Agency (CSA) have jointly studied the requirements for a satellite based, global ozone monitoring instrument. The study, called Ozone Research using Advanced Cooperative Lidar Experiment (ORACLE) has defined the Differential Absorption Lidar (DIAL) instrument performance, weight and power, and configuration requirements for a space based measurement. In order to achieve the measurement resolution and acceptable signal-to-noise from lidar returns, 500mJ/pulse (5 Watts average power) is required at both 305-308nm and 315-320nm wavelengths. These are consecutive pulses, in a 10 Hz, double-pulsed format. The two wavelengths are used as the on- and off-lines for the ozone DIAL measurement. NASA Langley is currently developing technology for a UV laser transmitter capable of meeting the ORACLE requirements. Experimental efforts to date have shown that the UV generation scheme is viable, and that energies greater than l00mJ/pulse are possible. In this paper, we will briefly discuss the down select process for the proposed laser design, the study effort to date and the laser system design, including both primary and alternate approaches. We will describe UV laser technology that minimizes the total number of optical components (for enhanced reliability) as well as the number of UV coated optics required to transmit the light from the laser (for enhanced optical damage resistance). While the goal is to develop a laser that will produce 500 mJ of energy, we will describe an optional design that will produce output energies between 100 - 200 mJ/unit and techniques for combining multiple laser modules in order to transmit a minimum of 500mJ of UV energy in each pulse of the on- and off-line pulse pairs. This modular laser approach provides redundancy and significantly reduces development time, risk and cost when compared to the development of a single, 500mJ double-pulsed laser subsystem. Finally, we will summarize the laser development effort to date, including results that include the highest known UV energy ( 130 mJ at 320nm) ever produced by a solid-state laser operating in this wavelength region.

Edwards, W. C.↗
NASA NTRS1984

Control Systems

Boeing Commercial Airplane Company's Flight Control Department engineers relied on Langley developed software package known as ORACLS to develop an advanced control synthesis package for both continuous and discrete control system. Package was used by Boeing for computerized analysis of new system designs. Resulting applications include a multiple input/output control system for the terrain-following navigation equipment of the Air Forces B-1 Bomber, and another for controlling in flight changes of wing camber on an experimental airplane. ORACLS is one of 1,300 computer programs available from COSMIC.

Source record↗
NASA NTRS2009

Service Management Database for DSN Equipment

This data- and event-driven persistent storage system leverages the use of commercial software provided by Oracle for portability, ease of maintenance, scalability, and ease of integration with embedded, client-server, and multi-tiered applications. In this role, the Service Management Database (SMDB) is a key component of the overall end-to-end process involved in the scheduling, preparation, and configuration of the Deep Space Network (DSN) equipment needed to perform the various telecommunication services the DSN provides to its customers worldwide. SMDB makes efficient use of triggers, stored procedures, queuing functions, e-mail capabilities, data management, and Java integration features provided by the Oracle relational database management system. SMDB uses a third normal form schema design that allows for simple data maintenance procedures and thin layers of integration with client applications. The software provides an integrated event logging system with ability to publish events to a JMS messaging system for synchronous and asynchronous delivery to subscribed applications. It provides a structured classification of events and application-level messages stored in database tables that are accessible by monitoring applications for real-time monitoring or for troubleshooting and analysis over historical archives.

Zendejas, Silvino↗
NASA NTRS2005

NASA Interactive Forms Type Interface - NIFTI

A flexible database query, update, modify, and delete tool was developed that provides an easy interface to Oracle forms. This tool - the NASA interactive forms type interface, or NIFTI - features on-the- fly forms creation, forms sharing among users, the capability to query the database from user-entered criteria on forms, traversal of query results, an ability to generate tab-delimited reports, viewing and downloading of reports to the user s workstation, and a hypertext-based help system. NIFTI is a very powerful ad hoc query tool that was developed using C++, X-Windows by a Motif application framework. A unique tool, NIFTI s capabilities appear in no other known commercial-off-the- shelf (COTS) tool, because NIFTI, which can be launched from the user s desktop, is a simple yet very powerful tool with a highly intuitive, easy-to-use graphical user interface (GUI) that will expedite the creation of database query/update forms. NIFTI, therefore, can be used in NASA s International Space Station (ISS) as well as within government and industry - indeed by all users of the widely disseminated Oracle base. And it will provide significant cost savings in the areas of user training and scalability while advancing the art over current COTS browsers. No COTS browser performs all the functions NIFTI does, and NIFTI is easier to use. NIFTI s cost savings are very significant considering the very large database with which it is used and the large user community with varying data requirements it will support. Its ease of use means that personnel unfamiliar with databases (e.g., managers, supervisors, clerks, and others) can develop their own personal reports. For NASA, a tool such as NIFTI was needed to query, update, modify, and make deletions within the ISS vehicle master database (VMDB), a repository of engineering data that includes an indentured parts list and associated resource data (power, thermal, volume, weight, and the like). Since the VMDB is used both as a collection point for data and as a common repository for engineering, integration, and operations teams, a tool such as NIFTI had to be designed that could expedite the creation of database query/update forms which could then be shared among users.

Jain, Bobby↗
NASA NTRS2011

Automated Test Case Generation for an Autopilot Requirement Prototype

Designing safety-critical automation with robust human interaction is a difficult task that is susceptible to a number of known Human-Automation Interaction (HAI) vulnerabilities. It is therefore essential to develop automated tools that provide support both in the design and rapid evaluation of such automation. The Automation Design and Evaluation Prototyping Toolset (ADEPT) enables the rapid development of an executable specification for automation behavior and user interaction. ADEPT supports a number of analysis capabilities, thus enabling the detection of HAI vulnerabilities early in the design process, when modifications are less costly. In this paper, we advocate the introduction of a new capability to model-based prototyping tools such as ADEPT. The new capability is based on symbolic execution that allows us to automatically generate quality test suites based on the system design. Symbolic execution is used to generate both user input and test oracles user input drives the testing of the system implementation, and test oracles ensure that the system behaves as designed. We present early results in the context of a component in the Autopilot system modeled in ADEPT, and discuss the challenges of test case generation in the HAI domain.

Giannakopoulou, Dimitra↗
NASA NTRS2014

Towards Test Driven Development for Computational Science with pFUnit

Developers working in Computational Science & Engineering (CSE)/High Performance Computing (HPC) must contend with constant change due to advances in computing technology and science. Test Driven Development (TDD) is a methodology that mitigates software development risks due to change at the cost of adding comprehensive and continuous testing to the development process. Testing frameworks tailored for CSE/HPC, like pFUnit, can lower the barriers to such testing, yet CSE software faces unique constraints foreign to the broader software engineering community. Effective testing of numerical software requires a comprehensive suite of oracles, i.e., use cases with known answers, as well as robust estimates for the unavoidable numerical errors associated with implementation with finite-precision arithmetic. At first glance these concerns often seem exceedingly challenging or even insurmountable for real-world scientific applications. However, we argue that this common perception is incorrect and driven by (1) a conflation between model validation and software verification and (2) the general tendency in the scientific community to develop relatively coarse-grained, large procedures that compound numerous algorithmic steps.We believe TDD can be applied routinely to numerical software if developers pursue fine-grained implementations that permit testing, neatly side-stepping concerns about needing nontrivial oracles as well as the accumulation of errors. We present an example of a successful, complex legacy CSE/HPC code whose development process shares some aspects with TDD, which we contrast with current and potential capabilities. A mix of our proposed methodology and framework support should enable everyday use of TDD by CSE-expert developers.

pFUnit↗
NASA NTRS2012

Requirements-Driven Log Analysis Extended Abstract

Imagine that you are tasked to help a project improve their testing effort. In a realistic scenario it will quickly become clear, that having an impact is diffcult. First of all, it will likely be a challenge to suggest an alternative approach which is significantly more automated and/or more effective than current practice. The reality is that an average software system has a complex input/output behavior. An automated testing approach will have to auto-generate test cases, each being a pair (i; o) consisting of a test input i and an oracle o. The test input i has to be somewhat meaningful, and the oracle o can be very complicated to compute. Second, even in case where some testing technology has been developed that might improve current practice, it is then likely difficult to completely change the current behavior of the testing team unless the technique is obviously superior and does everything already done by existing technology. So is there an easier way to incorporate formal methods-based approaches than the full edged test revolution? Fortunately the answer is affirmative. A relatively simple approach is to benefit from possibly already existing logging infrastructure, which after all is part of most systems put in production. A log is a sequence of events, generated by special log recording statements, most often manually inserted in the code by the programmers. An event can be considered as a data record: a mapping from field names to values. We can analyze such a log using formal methods, for example checking it against a formal specification. This separates running the system for analyzing its behavior. It is not meant as an alternative to testing since it does not address the important in- put generation problem. However, it offers a solution which testing teams might accept since it has low impact on the existing process. A single person might be assigned to perform such log analysis, compared to the entire testing team changing behavior.

log analysis↗