Calculating QA Costs – Service Planning Costing (SPC)

The series on Costing Models Includes Back In Costing (BIC), Agile Anarchy Method (AAM), and Just Test Something (JTS) Method. The intent here is not to claim one model as king, but rather to evaluate the potential benefits and pitfalls of relying on one model exclusively. My hope being that by sharing these approaches, QA organizations will evaluate their current models and perhaps find room to tune them for greater excellence.

Service Planning Costing is done by calculating the desired number of test cases, then multiplying them by time needed to complete these and the costs for this time. To this you add any additional project costs. It requires a full understanding of the project before costing can be done, and detailed project planning and scope.

SPC Model Inc. is costing a planned project. The new Project will have 50 new web services, each averaging 5 functions in the service. SPC Model Inc standards require a minimum testing one positive and one negative scenario only per service function. By multiplying these numbers together, SPC Model Inc plans require 500 test cases. The project plan calls for 5 expected development cycles or code drops. The security team will need to run their set of tests and the performance team theirs. They also need to do a final lab and production implementation test. Its estimated their current team skills are at a level were each test case will take an average of 12 min to develop and complete and the corporate rate is $50 per hour for QA resources. Finally they plan on purchasing a new tool, 2 lab machines, training and need to pay for hiring and on-boarding new employee. The Service Planning Cost (SPC) is seen to be $63,000 for this project as per below.

SPC Base

The SPC model is focused on breaking the testing down into as many small pieces as possible and assigning a value to each piece. Variations are possible using averages across the project or breaking the project down further into stages. For instance, First release may have less test cases or services, Security test could be a separate calculation or hourly rate could include training and hiring. The more granular, the more likelihood of accuracy, but the greater the risk of inaccuracy. Numbers like release cycles and time per test can inherited through by reviewing previous similarly projects or through running short pilot. Whatever numbers are used, it is still based on plan and plans can go wrong.

Advantages.

  1. Although some of the numbers may come from previous projects, the actual costs are based on real target project scope.
  2. The model supports organization test coverage standards or objectives. The number and extent of test cases is planned for positive, negative, load, security etc.
  3. Very easily communicated and aides understanding between on-shore and offshore teams and management, ensuring all parties work to the a defined plan. This is one of the reasons for this model to be popular with outsourced organizations to reduce project risk and clearly define the scope to prevent scope creep.
  4. Timelines, gates, KPI and milestones are known
  5. Inefficiencies can readily and clearly be identified and costs easily understood. For instance the cost of an additional code drop.

Disadvantages

  1. The line items can be very subjective, hence vulnerable to padding or underestimating. A very small mistake in any line item is compounded and can result in significant over/under costing. For example change the individual test case time by 2 minutes average.
  2. Does not allow for much flexibility in changing the plan. For instance, a particular service may perhaps warrant additional testing or an additional requirement or unforeseen services added.
  3. Still dependent on other parties delivery. What if a extra code drop becomes needed due to a previous issue not being fixed correctly
  4. Relies on a totally flexible environment. A testing team cannot always be expanded or contracted in real time.
  5. Does not support agile development model well.

CONCLUSION

Service Planning Costing (SPC) is not a silver bullet, or exact science and it does not prevent misuse through padding to target a desired price or outcome. It can however be a extremely valuable tool for analyzing expenses, identifying inefficiencies and managing KPI. These KPI can be used one project to the next, and “tuned” over time to plan better.

I will be covering more around these KPI and on understanding better how an organization, or individual,  may focus on the SPC model to “Dial In” for excellence in future posts and at my TASSQ presentation.