Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software effort estimations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Examining Reuse in LaSRS++-Based Projects

NASA Langley Research Center (LaRC) developed the Langley Standard Real-Time Simulation in C++ (LaSRS++) to consolidate all software development for its simulation facilities under one common framework. A common framework promised a decrease in the total development effort for a new simulation by encouraging software reuse. To judge the success of LaSRS++ in this regard, reuse metrics were extracted from 11 aircraft models. Three methods that employ static analysis of the code were used to identify the reusable components. For the method that provides the best estimate, reuse levels fall between 66% and 95% indicating a high degree of reuse. Additional metrics provide insight into the extent of the foundation that LaSRS++ provides to new simulation projects. When creating variants of an aircraft, LaRC developers use object-oriented design to manage the aircraft as a reusable resource. Variants modify the aircraft for a research project or embody an alternate configuration of the aircraft. The variants inherit from the aircraft model. The variants use polymorphism to extend or redefine aircraft behaviors to meet the research requirements or to match the alternate configuration. Reuse level metrics were extracted from 10 variants. Reuse levels of aircraft by variants were 60% - 99%.

Madden, Michael M.↗

Working and Learning with Knowledge in the Lobes of a Humanoid's Mind

Humanoid class robots must have sufficient dexterity to assist people and work in an environment designed for human comfort and productivity. This dexterity, in particular the ability to use tools, requires a cognitive understanding of self and the world that exceeds contemporary robotics. Our hypothesis is that the sense-think-act paradigm that has proven so successful for autonomous robots is missing one or more key elements that will be needed for humanoids to meet their full potential as autonomous human assistants. This key ingredient is knowledge. The presented work includes experiments conducted on the Robonaut system, a NASA and the Defense Advanced research Projects Agency (DARPA) joint project, and includes collaborative efforts with a DARPA Mobile Autonomous Robot Software technical program team of researchers at NASA, MIT, USC, NRL, UMass and Vanderbilt. The paper reports on results in the areas of human-robot interaction (human tracking, gesture recognition, natural language, supervised control), perception (stereo vision, object identification, object pose estimation), autonomous grasping (tactile sensing, grasp reflex, grasp stability) and learning (human instruction, task level sequences, and sensorimotor association).

Ambrose, Robert↗

Serendipitous Geodesy from Bennu's Short-Lived Moonlets

The Origins, Spectral Interpretation, Resource Identification, and Security-Regolith Explorer (OSIRIS-REx; or OREx) spacecraft arrived at its target, near-Earth asteroid (101955) Bennu, on December 3, 2018. The OSIRIS-REx spacecraft has since collected a wealth of scientific information in order to select a suitable site for sampling. Shortly after insertion into orbit on December 31, 2018, particles were identified in starfield images taken by the navigation camera (NavCam 1). Several groups within the OSlRlS-REx team analyzed the particle data in an effort to better understand this newfound activity of Bennu and to investigate the potential sensitivity of the particles to Bennu's geophysical parameters. A number of particles were identified through automatic and manual methods in multiple images, which could be turned into short sequences of optical tracking observations. Here, we discuss the precision orbit determination (OD) effort focused on these particles at NASA GSFC, which involved members of the Independent Navigation Team (INT) in particular. The particle data are combined with other OSIRIS-REx tracking data (radiometric from OSN and optical landmark data) using the NASA GSFC GEODYN orbit determination and geodetic parameter estimation software. We present the results of our study, particularly those pertaining to the gravity field of Bennu. We describe the force modeling improvements made to GEODYN specifically for this work, e.g., with a raytracing-based modeling of solar radiation pressure. The short-lived, low-flying moonlets enable us to determine a gravity field model up to a relatively high degree and order: at least degree 6 without constraints, and up to degree 10 when applying Kaula-like regularization. We can backward- and forward-integrate the trajectory of these particles to the ejection and landing sites on Bennu. We assess the recovered field by its impact on the OSIRIS-REx trajectory reconstruction and prediction quality in the various mission phases (e.g., Orbital A, Detailed Survey, and Orbital B).

Mazarico, Erwan↗

In-Situ Resource Utilization Modeling of a Lunar Water Processing System

A key element of achieving a sustained surface presence, such as defined in NASA’s Artemis plan, is In-Situ Resource Utilization (ISRU). ISRU is the practice of using local resources to provide mission consumables that reduce system launch mass requirements, and regenerate resources (chiefly, water and oxygen) for propulsion and life support supporting both Lunar and Martian missions. ISRU systems require multiple complex processes, such as excavation, chemical reactors, and electrolysis subsystems that must operate in harmony to optimize the overall system process from beginning to end. The Mission Analysis and Integration Tool (MAIT) was previously developed with MATLAB in FY22 to connect individual subsystem models into a customized, flexible framework for the purpose of technology downselect, optimization, and end-to-end process planning. Beginning in FY24, MAIT was leveraged and evolved using MATLAB/Simulink due to its ability to communicate with a vast number of other programming languages and makes up the backbone of data flow between inputs and outputs to the subsystem models. MAIT initially evaluated a suite of ISRU-related technologies, including the water processing Lunar Auger Dryer for ISRU (LADI) system with integrated upstream excavation and downstream electrolysis subsystems. With individual models consolidated, the MAIT tool generated over 60,000 cases during its parametric sweeps; these system iterations produced valuable insight into the optimal LADI geometry for minimizing energy demands, estimating carbothermal reactor and radiator mass relationships, and calculated the power dynamics of the electrolysis unit. Advanced efforts with advanced models will include examining multiple production targets to demonstrate the ability to scale ISRU technologies supporting the Space Technology Mission Directorate’s (STMD) commercialization strategy, and increase the MAIT software capability to handle a wide array of ISRU system models beyond the Lunar environment, e.g. production of propellant for a Martian lander.

Avery Carlson↗

Quantification of Uncertainty in Extreme Scale Computations (QUEST)

QUEST was a SciDAC Institute comprising Sandia National Laboratories, Los Alamos National Laboratory, the University of Southern California, the Massachusetts Institute of Technology, the University of Texas at Austin, and Duke University. The mission of QUEST is to: (1) develop a broad class of uncertainty quantification (UQ) methods/tools, and (2) provide UQ expertise and software to other SciDAC projects, thereby enabling/guiding their UQ activities. The USC effort centered on the development of reduced models and efficient algorithms for implementing various components of the UQ pipeline. USC personnel were responsible for the development of adaptive bases, adaptive quadrature, and reduced models to be used in estimation and inference.

97 MATHEMATICS AND COMPUTING↗

In-Situ Resource Utilization Modeling of a Lunar Water Processing System

A key element of achieving a sustained surface presence, such as defined in NASA’s Artemis plan, is In-Situ Resource Utilization (ISRU). ISRU is the practice of using local resources to provide mission consumables that reduce system launch mass requirements, and regenerate resources (chiefly, water and oxygen) for propulsion and life support supporting both Lunar and Martian missions. ISRU systems require multiple complex processes, such as excavation, chemical reactors, and electrolysis subsystems that must operate in harmony to optimize the overall system process from beginning to end. The Mission Analysis and Integration Tool (MAIT) was previously developed with MATLAB in FY22 to connect individual subsystem models into a customized, flexible framework for the purpose of technology downselect, optimization, and end-to-end process planning. Beginning in FY24, MAIT was updated and became the capital program in the Systems Engineering and Integration (SE&I) ISRU Modeling and Analysis (SIMA) project. Prior work was leveraged and evolved using MATLAB/Simulink due to its ability to communicate with a vast number of other programming languages and makes up the backbone of data flow between inputs and outputs to the subsystem models. MAIT initially evaluated a suite of ISRU-related technologies, including the water processing Lunar Auger Dryer for ISRU (LADI) system with integrated upstream excavation and downstream electrolysis subsystems. With individual models consolidated, the MAIT tool generated over 5,000 cases during its first round of parametric sweeps on the water processing architecture at multiple production targets; the system analysis produced valuable insight into the optimal LADI geometry that minimized energy demands, estimated effects to cold trap size and radiator requirements, and calculated the power dynamics of the electrolysis unit and liquid oxygen storage volume. Additional efforts are being made to demonstrate the ability to scale ISRU technologies supporting the Space Technology Mission Directorate’s (STMD) commercialization strategy and increase the MAIT software capability. Work is ongoing to handle a wide array of ISRU system models beyond the Lunar environment, e.g. production of propellant for a Martian lander.

Avery Carlson↗

In-Situ Resource Utilization Modeling of a Lunar Water Processing System

A key element of achieving a sustained surface presence, such as defined in NASA’s Artemis plan, is In-Situ Resource Utilization (ISRU). ISRU is the practice of using local resources to provide mission consumables that reduce system launch mass requirements, and regenerate resources (chiefly, water and oxygen) for propulsion and life support supporting both Lunar and Martian missions. ISRU systems require multiple complex processes, such as excavation, chemical reactors, and electrolysis subsystems that must operate in harmony to optimize the overall system process from beginning to end. The Mission Analysis and Integration Tool (MAIT) was previously developed with MATLAB in FY22 to connect individual subsystem models into a customized, flexible framework for the purpose of technology downselect, optimization, and end-to-end process planning. Beginning in FY24, MAIT was updated and became the capital program in the Systems Engineering and Integration (SE&I) ISRU Modeling and Analysis (SIMA) project. Prior work was leveraged and evolved using MATLAB/Simulink due to its ability to communicate with a vast number of other programming languages and makes up the backbone of data flow between inputs and outputs to the subsystem models. MAIT initially evaluated a suite of ISRU-related technologies, including the water processing Lunar Auger Dryer for ISRU (LADI) system with integrated upstream excavation and downstream electrolysis subsystems. With individual models consolidated, the MAIT tool generated over 5,000 cases during its first round of parametric sweeps on the water processing architecture at multiple production targets; the system analysis produced valuable insight into the optimal LADI geometry that minimized energy demands, estimated effects to cold trap size and radiator requirements, and calculated the power dynamics of the electrolysis unit and liquid oxygen storage volume. Additional efforts are being made to demonstrate the ability to scale ISRU technologies supporting the Space Technology Mission Directorate’s (STMD) commercialization strategy and increase the MAIT software capability. Work is ongoing to handle a wide array of ISRU system models beyond the Lunar environment, e.g. production of propellant for a Martian lander.

Avery Carlson↗

Contaminated Soil Volume Estimation at the Maywood Site - 12292

As part of the ongoing remediation process at the Maywood Formerly Utilized Sites Remedial Action Program properties, Argonne National Laboratory assisted the U.S. Army Corps of Engineers (USACE) New York District in revising contaminated soil volume estimates for the remaining areas of the Stepan/Sears properties that require soil remediation. As part of the volume estimation process, an initial conceptual site model (ICSM) was prepared for the entire site that captured existing information (with the exception of soil sampling results) pertinent to the possible location of surface and subsurface contamination above cleanup requirements. This ICSM was based on historical anecdotal information, aerial photographs, and the logs from several hundred soil cores that identified the depth of fill material and the depth to bedrock under the site. Specialized geostatistical software developed by Argonne was used to update the ICSM with historical sampling results and down-hole gamma survey information for hundreds of soil core locations; both sampling results and down-hole gamma data were coded to identify whether the results indicated the presence of contamination above site cleanup requirements. Significant effort was invested in developing complete electronic data sets for the site by incorporating data contained in various scanned documents, maps, etc. The updating process yielded both a best guess estimate of contamination volumes and upper and lower bounds on the volume estimate that reflected the estimate's uncertainty. The site-wide contaminated volume estimate (with associated uncertainty) was adjusted to reflect areas where remediation was complete; the result was a revised estimate of the remaining soil volumes requiring remediation that the USACE could use for planning. Other environmental projects may benefit from this process for estimating the volume of contaminated soil. A comparison of sample and DHG results for various stations with the site ICSM provides one measure of the quality of site understanding relative to the presence of contamination and its likely extent. A second measure of the quality of a contaminated soil volume estimate, in the case of BAASS, is a comparison of the best guess volume estimate with the conservative volume estimate. For Maywood, the conservative volume estimate is more than double the best guess. The data available for the site indicate that the preponderance of this difference is attributable to uncertainty about the exact depth of known contamination. The observed depth of contamination based on sample results and DHG data often varied significantly among adjacent soil cores. In the case of Maywood, where a significant amount of remediation has already taken place, a third measure of the quality of the volume estimates is the retrospective estimate of contaminated soil volumes for areas that have already been remediated [2]. The USACE is currently comparing the latest BAASS volume estimates for those areas with records indicating what was actually removed. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

SNP Discovery with EST and NextGen Sequencing in Switchgrass ( Panicum virgatum L. )

Although yield trials for switchgrass (Panicum virgatum L.), a potentially high value biofuel feedstock crop, are currently underway throughout North America, the genetic tools for crop improvement in this species are still in the early stages of development. Identification of high-density molecular markers, such as single nucleotide polymorphisms (SNPs), that are amenable to high-throughput genotyping approaches, is the first step in a quantitative genetics study of this model biofuel crop species. We generated and sequenced expressed sequence tag (EST) libraries from thirteen diverse switchgrass cultivars representing both upland and lowland ecotypes, as well as tetraploid and octoploid genomes. We followed this with reduced genomic library preparation and massively parallel sequencing of the same samples using the Illumina Genome Analyzer technology platform. EST libraries were used to generate unigene clusters and establish a gene-space reference sequence, thus providing a framework for assembly of the short sequence reads. SNPs were identified utilizing these scaffolds. We used a custom software program for alignment and SNP detection and identified over 149,000 SNPs across the 13 short-read sequencing libraries (SRSLs). Approximately 25,000 additional SNPs were identified from the entire EST collection available for the species. This sequencing effort generated data that are suitable for marker development and for estimation of population genetic parameters, such as nucleotide diversity and linkage disequilibrium. Based on these data, we assessed the feasibility of genome wide association mapping and genomic selection applications in switchgrass. Overall, the SNP markers discovered in this study will help facilitate quantitative genetics experiments and greatly enhance breeding efforts that target improvement of key biofuel traits and development of new switchgrass cultivars.

09 BIOMASS FUELS↗

Adaptive Time Step Control for Multirate Infinitesimal Methods

Multirate methods have been used for decades to temporally evolve initial-value problems in which different components evolve on distinct time scales, and thus use of different step sizes for these components can result in increased computational efficiency. Generally, such methods select these different step sizes based on experimentation or stability considerations. For problems that evolve on a single time scale, adaptivity approaches that strive to control local temporal error are widely used to achieve numerical results of a desired accuracy with minimal computational effort, while alleviating the need for manual experimentation with different time step sizes. However, there is a notable gap in the publication record on the development of adaptive time step controllers for multirate methods. In this paper, we extend the single-rate controller work of Gustafsson [ACM Trans. Math. Software, 20 (1994), pp. 496-517] to the multirate method setting. Specifically, we develop controllers based on polynomial approximations to the principal error functions for both the "fast" and "slow" time scales within multirate infinitesimal (MRI) methods. We additionally investigate a variety of approaches for estimating the errors arising from each time scale within MRI methods. We then numerically evaluate the proposed multirate controllers and error estimation strategies on a range of multirate test problems, comparing their performance against an estimated optimal performance. Through this work, we combine the most performant of these approaches to arrive at a set of multirate adaptive time step controllers that robustly achieve desired solution accuracy with minimal computational effort.

97 MATHEMATICS AND COMPUTING↗

Spatio-temporal Estimates of Disease Transmission Parameters for COVID-19 with a Fully-Coupled, County-Level Model of the United States

Sandia National Laboratories has developed a capability to estimate parameters of epidemiological models from case reporting data to support responses to the COVID-19 pandemic. A differentiating feature of this work is the ability to simultaneously estimate county-specific disease transmission parameters in a nation-wide model that considers mobility between counties. The approach is focused on estimating parameters in a stochastic SEIR model that considers mobility between model patches (i.e., counties) as well as additional infectious compartments. The inference engine developed by Sandia includes (1) reconstruction and (2) transmission parameter inference. Reconstruction involves estimating current population counts within each of the compartments in a modified SEIR model from reported case data. Reconstruction produces input for the inference formulations, and it provides initial conditions that can be used in other modeling and planning efforts. Inference involves the solution of a large-scale optimization problem to estimate the time profiles for the transmission parameters in each county. These provide quantification of changes in the transmission parameter over time (e.g., due to impact of intervention strategies). This capability has been implemented in a Python-based software package, epi_inference, that makes extensive use of Pyomo [5] and IPOPT [10] to formulate and solve the inference formulations.

42 ENGINEERING↗

Challenges to Evaluating Ecological Risks in Performance Assessments - 18424

Performance assessments (PAs), whether probabilistic or deterministic, have largely focused on evaluating human health effects of radionuclides. In the past, it was asserted that protecting human health would automatically protect the environment. This blanket assertion has been questioned, so tools and methodologies to directly estimate non-human biota dose have been developed and made available since the early 2000's. For example, the Biota Dose Assessment Committee (BDAC) guidance led to the development of the RESRAD-BIOTA (RESRAD stands for Residual Radioactive materials) software to implement this guidance. The BDAC was a US Department of Energy (DOE) led working group with members from academia and consultancies in addition to DOE technical staff. Parallel efforts have been in place in Europe with the development of the ERICA tool (Environmental Risk from Ionising Contaminants: Assessment and Management). More recently, interpretation of the requirements of US Department of Energy (DOE) Order O458.1 have led some waste management facilities to include ecological risks in addition to human health effects in their PAs. We have recently developed methodologies to assess ecological risks within probabilistic PAs (PPAs) for two Federal facilities. One of the challenges in applying the non-human biota assessment models to PPAs is that ecological risk assessments following US Environmental Protection Agency (EPA) or DOE guidance start with protective screening assessments before moving into more site-specific ecological risk assessments. Conservative assumptions bias the screening assessments and greatly overestimate the risks posed by contaminants. However, although the screens are biased they can be useful in ruling out model spatial domain areas or types of biota from requiring further assessment. In some cases, the results of the screen could entirely rule out ecological risks from further assessment. For a more refined and unbiased estimate of dose to non-human biota, one would typically conduct site-specific studies to better define exposure through the food web or adverse effects based on eco-toxicity studies of environmental, and generally weathered, contaminant mixtures. However, for PPAs that consider future site conditions after the loss of institutional control, such site-specific studies are not possible. Rather, site-specific future ecological risk models use probabilistic inputs that result in a distribution of population doses or exposures and calculate the distribution of possible adverse effects to ecological populations. This approach considers both radionuclide and chemical inventory. For both Federal facilities, the model spatial domains include riparian and aquatic environments located downstream of the waste management site. These riparian and aquatic environments represent linear exposure elements for some wildlife populations and require special consideration of exposures to individuals and to populations along such spatial domains. At a minimum, wildlife foraging should be focused along the axis of such water bodies. Another challenge is how to incorporate ecologically relevant spatial scales into models that have contaminant transport as the primary focus. Riparian and aquatic environments also have threatened and endangered (T and E) species, which necessitate greater levels of protection given their rarity. A final challenge for conducting ecological risk assessments is how to evaluate T and E species in the future. This might be addressed by evaluating potential for impacts on individuals of feeding guilds with species that are currently threatened or endangered. We also have to consider how ecological populations might change as a result of climate change. It is a more difficult problem to predict what species might become rare in the future, and how to best assess future populations of threatened or endangered species. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Scientific Open-Source Software Is Less Likely to Become Abandoned Than One Might Think! Lessons from Curating a Catalog of Maintained Scientific Software

Scientific software is essential to scientific innovation and in many ways it is distinct from other types of software. Abandoned (or unmaintained), buggy, and hard to use software, a perception often associated with scientific software can hinder scientific progress, yet, in contrast to other types of software, its longevity is poorly understood. Existing data curation efforts are fragmented by science domain and/or are small in scale and lack key attributes. We use large language models to classify public software repositories in World of Code into distinct scientific domains and layers of the software stack, curating a large and diverse collection of over 18,000 scientific software projects. Using this data, we estimate survival models to understand how the domain, infrastructural layer, and other attributes of scientific software affect its longevity. We further obtain a matched sample of non-scientific software repositories and investigate the differences. We find that infrastructural layers, downstream dependencies, mentions of publications, and participants from government are associated with a longer lifespan, while newer projects with participants from academia had shorter lifespan. Against common expectations, scientific projects have a longer lifetime than matched non-scientific open-source software projects. We expect our curated attribute-rich collection to support future research on scientific software and provide insights that may help extend longevity of both scientific and other projects.

Malviya Thakur, Addi [ORNL] (ORCID:000000022681999↗

Design of a modular digital computer system DRL 4 and 5

Design and development efforts for a spaceborne modular computer system are reported. An initial baseline description is followed by an interface design that includes definition of the overall system response to all classes of failure. Final versions for the register level designs for all module types were completed. Packaging, support and control executive software, including memory utilization estimates and design verification plan, were formalized to insure a soundly integrated design of the digital computer system.

Source record↗

Status Report on Transfer of Physical and Hydraulic Properties Databases to the Hanford Environmental Information System - PNNL Remediation Decision Support Project, Task 1, Activity 6

This document provides a status report on efforts to transfer physical and hydraulic property data from PNNL to CHPRC for incorporation into HEIS. The Remediation Decision Support (RDS) Project is managed by Pacific Northwest National Laboratory (PNNL) to support Hanford Site waste management and remedial action decisions by the U.S. Department of Energy and their contractors. The objective of Task 1, Activity 6 of the RDS project is to compile all available physical and hydraulic property data for sediments from the Hanford Site, to port these data into the Hanford Environmental Information System (HEIS), and to make the data web-accessible to anyone on the Hanford Local Area Network via the so-called Virtual Library. These physical and hydraulic property data are used to estimate parameters for analytical and numerical flow and transport models that are used for site risk assessments and evaluation of remedial action alternatives. In past years efforts were made by RDS project staff to compile all available physical and hydraulic property data for Hanford sediments and to transfer these data into SoilVision®, a commercial geotechnical software package designed for storing, analyzing, and manipulating soils data. Although SoilVision® has proven to be useful, its access and use restrictions have been recognized as a limitation to the effective use of the physical and hydraulic property databases by the broader group of potential users involved in Hanford waste site issues. In order to make these data more widely available and useable, a decision was made to port them to HEIS and to make them web-accessible via a Virtual Library module. In FY08 the original objectives of this activity on the RDS project were to: (1) ensure traceability and defensibility of all physical and hydraulic property data currently residing in the SoilVision® database maintained by PNNL, (2) transfer the physical and hydraulic property data from the Microsoft Access database files used by SoilVision® into HEIS, which is currently being maintained by CH2M-Hill Plateau Remediation Company (CHRPC), (3) develop a Virtual Library module for accessing these data from HEIS, and (4) write a User's Manual for the Virtual Library module. The intent of these activities is to make the available physical and hydraulic property data more readily accessible and useable by technical staff and operable unit managers involved in waste site assessments and remedial action decisions for Hanford. In FY08 communications were established between PNNL and staff from Fluor-Hanford Co. (who formerly managed HEIS) to outline the design of a Virtual Library module that could be used to access the physical and hydraulic property data that are to be transferred into HEIS. Data dictionaries used by SoilVision® were also provided to Fluor-Hanford personnel who are now with CHPRC. During ongoing work to ensure traceability and defensibility of all physical and hydraulic property data that currently reside in the SoilVision® database, it was recognized that further work would be required in this effort before the data were actually ported into HEIS. Therefore work on the Virtual Library module development and an accompanying User's Guide was deferred until an unspecified later date. In FY09 efforts have continued to verify the traceability and defensibility of the physical and hydraulic property datasets that are currently being maintained by PNNL. Although this is a work in progress, several of these datasets are now ready for transfer to CHRPC for inclusion in HEIS. The actual loading of data into HEIS is performed by CHPRC staff, so after the data are transferred from PNNL to CHPRC, it will be the responsibility of CHPRC to ensure that these data are loaded and made accessible. This document provides a status report on efforts to transfer physical and hydraulic property data from PNNL to CHPRC for incorporation into HEIS.

54 ENVIRONMENTAL SCIENCES↗

NASA Software Engineering Benchmarking Study

To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.

Rarick, Heather L.↗

Technology Estimating 2: A Process to Determine the Cost and Schedule of Space Technology Research and Development

As a leader in space technology research and development, NASA is continuing in the development of the Technology Estimating process, initiated in 2012, for estimating the cost and schedule of low maturity technology research and development, where the Technology Readiness Level is less than TRL 6. NASA' s Technology Roadmap areas consist of 14 technology areas. The focus of this continuing Technology Estimating effort included four Technology Areas (TA): TA3 Space Power and Energy Storage, TA4 Robotics, TA8 Instruments, and TA12 Materials, to confine the research to the most abundant data pool. This research report continues the development of technology estimating efforts completed during 2013-2014, and addresses the refinement of parameters selected and recommended for use in the estimating process, where the parameters developed are applicable to Cost Estimating Relationships (CERs) used in the parametric cost estimating analysis. This research addresses the architecture for administration of the Technology Cost and Scheduling Estimating tool, the parameters suggested for computer software adjunct to any technology area, and the identification of gaps in the Technology Estimating process.

Cole, Stuart K.↗

Understanding and Predicting the Process of Software Maintenance Releases

One of the major concerns of any maintenance organization is to understand and estimate the cost of maintenance releases of software systems. Planning the next release so as to maximize the increase in functionality and the improvement in quality are vital to successful maintenance management. The objective of this paper is to present the results of a case study in which an incremental approach was used to better understand the effort distribution of releases and build a predictive effort model for software maintenance releases. This study was conducted in the Flight Dynamics Division (FDD) of NASA Goddard Space Flight Center(GSFC). This paper presents three main results: 1) a predictive effort model developed for the FDD's software maintenance release process; 2) measurement-based lessons learned about the maintenance process in the FDD; and 3) a set of lessons learned about the establishment of a measurement-based software maintenance improvement program. In addition, this study provides insights and guidelines for obtaining similar results in other maintenance organizations.

Basili, Victor↗