Calculating QA Costs – Backing In Costing Method (BIC)
The series on Costing Models Includes Service Planning Costing (SPC), Agile Anarchy Method (AAM), and Just Test Something (JTS) Method. The intent here is not to claim one model as king, but rather to evaluate the potential benefits and pitfalls of relying on one model exclusively. My hope being that by sharing these approaches, QA organizations will evaluate their current models and perhaps find room to tune them for greater excellence.
Very few organization use 100% one model or the next. These models are neither exclusive or of a rigid scale. Rather most organizations will use one model as primary model, and perhaps have 1 or more secondary models. Placing them somewhere on a scale between one or more of these models.
Backing In Costing Model (BIC) or baseline in model is uses past measurements to determine future costs with very limited desire to changing Process, Coverage or anything for that matter. Similar, yet not to be confused with regression testing, a baseline from previous year/s is used to work backwards to costs future projects.
For example, last year BIC Model Inc. had 10 QA staff that delivered 2 large projects to the relative satisfaction of everyone. Last years cost, were based on the previous year and I no one remembers how those were calculated. They established a baseline, and set the expectations within their organization and to their customers. The process was settled, roles understood and number of production errors accepted. The entire cost allocated to QA last year was $600,000. That is $300,000 per large QA project and average of $60,000 per QA staff (The hourly QA costs, although rarely used are seen as $30.36).
This year BIC Model Inc. has a global executive requirement for 10% reduction in workforce and costs. However they have 2 similar size projects and a 3rd project that the management accepted is ABOUT 50% the size of the other two. This years QA will be fortunate to have an increased budget to $ 675,000. BUT “Don’t consider this a new baseline, next year we drop back to $540,000 from last year” . The amount is calculated by taking the baseline $600,000 -10% annual savings target =540,000 x 25% for the additional project. BIC Model Inc. decides to spend this additional money by adding 2 fresh new recruits to QA at $37,000 a year to offset the additional workload expected.
The BIC Model is focussed on previously accepted and predictable costs and about maintaining same Status Quo. Usually effected by annual cost reductions efforts, but rarely is the team successful as in the example above raising the budget.
It’s often characterized by management seeing little benefit to training staff or updating tools, as these bring change and possibly new costs. Vocal supporters for streamlining process, there is usually much resistance to change. Until some accepted baseline measurement like usual amount of production errors, fails to meet expectations. The motivation then, is often to streamline process to maintain the previous baseline.
Advantages
- Costing is based on real corporate experience and history. There is no unexpected surprises like additional corporate expense sharing like health or retirement plans or desk space lease etc
- It’s a easy sell to executives, they expected the costs and there is no need to explain percentage coverage calculations, number of services, test coverage, release cycles, emerging technology concerns etc
- People tend to know their roles and relationship between development, business and QA is usually mature.
- It’s very subjective regarding size of projects. In the example above, how similar in size are these projects really? This allows for a certain amount of exaggeration.
- Software maintenance and regression testing costs are usually well understood
Disadvantages
- Backing in to how much testing will be done, based only loosely based on the amount of testing required. It’s very subjective regarding size of projects. Time per test case development, number test cases, number of software releases etc can often be ignored
- The focus is on maintaining Status Quo and not on improvement. There is often little progression, changes to process, promotion, training, introduction of new tools and skills development.
- The baseline is often dated. Rather than re-calculating the baseline and to better understand the time per test case development, number test cases, number of software releases etc. after each project, the baseline can be years old.
- Bad process, habits, practices, people etc. become part of the baseline to be protected.
- Corporate costs cutting initiatives, like 10% example In BIC Model Inc. can eventually be the “straw that breaks the camels back”. On the other hand, QA is constantly living under the threat cuts, and needs to show that each year is equal or larger than the previous. Annual negotiation becomes and constant battle for survival balanced against expectations on the delivery of quality. Subjective numbers, not backed by detailed baselines, can become very inflated.
Conclusion
Taking a baseline at the end of each project, to understand the impacts of new process, people, tools etc and to ensure that your calculations for any future project is good practice. Evaluating one project vs another. Were the BIC model can rapidly fail, is if the detail or level of the baseline is poor or the baseline becomes outdated. Backing into the amount of testing to be done based vs. using these baselines to calculate forward using a more detailed costing model.
The second in this series covers Just Test Something (JTS) Method.
Did I miss an advantage or disadvantage. Please feel free to comment below. My next costing model will be posted shortly.
