Investigating an Industry Practical Model for Confidence in Expert Cost Estimation of Feature Enhancement Requests

Abstract

MONTE is a mission-critical system used by NASA for navigation and design of deep space missions. It has been used in over 40 missions over the last 18 years and is continually evolving. One of the most difficult problems in managing the maintenance of MONTE is estimating cost. For pragmatic reasons, cost estimates for implementing new features are provided by expert judgment. However, to properly manage our resources, we must have a high degree of confidence in these estimates. Historically we used the traditional three-point “best-to-worst” triangular distribution to obtain confidence. Unfortunately, this approach does not provide confidence that match reality. We have discovered that the relative error in expert judgment cost estimates of new features is modeled very well as an exponentially distribution. Now we use this to obtain cost estimates with confidence that is practical, justified, and matches our reality remarkably well. This work presents our investigation of the error in expert judgment cost estimates and how we currently make practical use of it. We also investigate the importance of having experts performing the estimates by comparing their error distribution characteristics with non-expert estimators. We establish the necessary assumptions and conditions for which the model applies so it can be used more generally.