More Detailed look at Service Planning Costing Part 1
In an earlier series, I went over a number of costing models that I have come across, that are used by different organizations. Service Planning Costing being one of the more detailed and specific, and often used by outsourcing companies. What is most interesting around service plan costing, is that a change of a single iteration, or minute in testing time can have major impact to the total price for the project. That said, I have seldom met an organization that has statistics or KPI on test times.
To understand this better, lets look at Testing as just one part of the Software Development Life Cycle. The majority people experience higher testing costs during product maintenance than development, yet at that point expenses have settled and are generally known. During the testing phase however, there are many more unknowns.
During software development testing, there are a number of different levels or test types. The first and most common test is the basic unit test, testing a function in the service to ensure that it does what it is intended to do. Each time the code is revised, a unit test is usually run to ensure that nothing unexpected has resulted. At some point, code is consolidated in a “release drop” and Integrated testing is done. Integrated testing, ensures that each unit interoperates with the others. Systems Testing is the next level and is around testing the entire system. This includes things like security testing, Performance testing under load, and other specialized tests to cover different environments. At each of these testing levels and each test iteration is usually compared to the previous baseline, in what if normally referred to as regression testing. The final testing level is user or business acceptance testing. Depending on the complexity of the application and the development approach and the maturity of the organization, there can be anything from 5-50 Testing level iterations. For example, in Agile environments, the sprints are usually shorter, yet need to include each of these test levels, while in waterfall, a company may only begin integration testing once all unit testing is completed.
Now each test Iteration at each testing level follows a process. A Test plan is developed, and the expected outcome along with intended inputs is created. This then is executed in a given environment, and the outcome is compared and analysed against what was expected. Any unexpected results or differences require further analysis. Afterwards a test report is completed, detailing what was done and the result The test cycle can be brief, in a unit test or lengthy as it may be in acceptance testing. In a single web service, there may be 10-20 function and each function may need to be tested. Possibly with a positive and negative or some other boundary requirement. The result could be hundreds of test cases for a single web service. Thousands of test cycles can make up a single test level and each test level can be repeated with each code drop or systems change. Since employee salaries are the largest costs in software testing, its makes estimating these times essential.
Since each of these processes is repeated exponentially, a small variance of a few seconds allocated to each test cycle or to the number of test levels in costing, can result in huge impacts to the costs. This is especially difficult to manage, when environments, code releases and other parties invested n the process may not appreciate the costs impacts.
This touches on core issues with QA and testing costs. I will be speaking more on this Tomorrow at TASSQ meeting in a workshop, and in future posts. Drilling down on some thoughts on how to reduce the costs and increase the accuracy of costing. In the mean time please feel free to comment below.
