2013 - ST3PP

1. SOAPSonar – Installing and Getting Started

So new you new to SOAPSonar. Perhaps you joined a new team that uses it, a student, or you downloaded a trial and trying to get started. Here is a quick tutorial on the install and UI.

Installation

After downloading the latest version (request a no obligation 14 day trial here), install by right click and run as administrator. If you don’t install with Administrator rights, you should get warnings.

Once installed you should be presented with the registration screen. Please enter your name, company and email. If you are behind a web proxy, please first enter the proxy settings. These can be taken from your browser settings. Go, File, Settings and Preferences and select the Global Proxy Settings tab. enter your organizations proxy settings here before registering. Alternately you can select the Manual Activate button Option.

If you doing and evaluation or your company uses Instance based licensing, enter your license key then select activate key. Instance based licensing can only be loaded on one machine per license key. Note if you using Instance based pricing, the number of days still on your license is shown.

If your company uses Floating licenses centralized license server is used. The server is used to check-out or check back the license only and does not require persistent network access. Licenses server is only accessed at the time the lease is granted or released. The server is not a file server and need not be a dedicated machine, but requires a static IP address and a small windows application. SOAPSonar can then be loaded onto as many machines as you would like, but only one machine can use a licensee at a time. If you using floating licenses, select the button for Use License Server and enter the details. (or hostname) and open port. Request New License Lease and select the Key Type and how long you wish to check the license out for. Select Request License and if there is an available Floating license of the Key Type requested, SOAPSonar will activate.

The User Interface

SOAPSonar UI follows the usual windows convention

File – File related tasks like new project, load project and save project with Settings and Preferences
Mode – SOAPSonar 4 pillars of QA (functional), Performance, Compliance, Vulnerability (security). Note the mode can also be changed in the right corner.
Tools – Various tools from key management to traffic capture.
Library – List of Automated Data Sources and vulnerability definitions that can be used.
Updates – To check for updates to the latest release. SOAPSonar does not require manual scripting. If manual scripts have not being used, upgrades should have no impact on test cases. It is highly recommended that you stay current to avoid known issues and get the new features.
Registration – Discussed above
Simulation – To launch a CLOUDPort generated run-time virtualized mock service to test against
Agents – For downloading and configuring remote performance load agents.
Help – Yes there is a help file.

Now each of the 4 pillars (QA, Performance, Compliance, Vulnerability) can be set to

Project View – For the configuration of your Project Tree or Test Suite Groups for automating your testing.
Run View – For running automated tests projects
Report View – For those who actually want to see the results after a test is run.

Conclusion

You should now be ready to start testing. If you have a project and feeling ready, you can start that.

Else we are working on developing Tutorials. These will be very basic tutorials to highlight a few frequently used features that should not take more than a couple hours to complete.

Cost of Versioning and API or Service in the Data Economy

The Data Economy is booming, and much is being written about Software “eating the world”. many companies however, have not formalized a strategy on Developing and Versioning their API’s. In a recent Forbes article “Collaborate to Grow Says Deloitte Global CEO Barry Salzberg” MIT Sloan graduates Jaime Contreras and Tal Snir are quoted as saying “the peer-to-peer exchange of goods and services – is being called the next big trend in social commerce, and represents what some analysts say is a potential $110 billion market.” Last month, Information week did as entire special issue on the “Age Of The API“, the enablers of the Data Economy.

This exponential growth in API’s is however creating significant versioning concerns and many organizations are beginning to consider their strategy for API versioning as their current strategies become insupportable. For example, the business needs a REST version of an existing SOAP service for mobile access. Should the migrate the entire service and all the client Consumers of the service to a new REST API and end of life the existing SOAP service? Or perhaps develop new API and leave the SOAP service in place? Whatever the reason is for the change, is the best strategy to create new, update the existing, replace the existing or do something else? How many of these changes can they manage in a given period of time and what are the costs? Read More

PERFORMANCE TUNING MOBILE API – CLIENT

In my first post in this series, I highlighted the need to isolate and break down the user experience into logical and measurable portions to used as a baseline.

The client often gets the most focus the device in the users hand at the end of the chain of factors influencing performance. As it’s at the end of the chain, it is the sum of all the others and unarguably the user’s final experience. This being said, the client impact on performance is only the time added from the moment the device receives a complete message, to display. Or the point of submit till the time it leaves the device. It’s may not be hard to identify when a mobile application is giving a poor user experience, but QA needs to also identify why. Read More

PERFORMANCE TUNING MOBILE API – NETWORK

A mobile application by default has a network component. The portion from the phone to the Ethernet card of the API servers. Who has not seen the US commercials “can you hear me now?” Canadian Wireless Service providers spend significant effort to plan their network coverage, identify poor performance, do capacity planning and ensure signal coverage. This includes crowd sourcing, BI, using tools and even driving around. Wireless networks are however not static and everything from the number of tree leaves, to time of day, affect the signal strength and capacity for a given location (Wireless bandwidth is limited and shared per cell frequency and cell coverage). Add to this nearly 10,000,000 square km of geography we have in Canada and you can understand the enormity of testing the network.

PERFORMANCE TUNING MOBILE API – API Themselves

To non-technical minded people and API is often a “a program running on a server somewhere” and rarely considered impacting to user experience. Web Services API’s Provide the information responses after doing the necessary calculations to requests made by the mobile application that populate the information on the screen. The client is all about presentation, and usually does little computation. Its the API and not the client that is doing the heavy computational tasks and hence can have greater effect on the user experience. Sure you can throw more computing power at it, but this does not always work.

Depending on the design, each request to by the mobile device to Consume an API, could respond by Providing multiple fields. Say the API is a customer record API. A request by the Client, would result in the API Providing the entire customer record even if only a small part of this response is needed. That means if the application needs to display the customer number, yet the API Provides the entire customer record, the entire record would be transmitted to the device, which would drop everything but the customer number. On the other hand, any screen on the client, can request more than one API. Say a second API Provides order history for a given customer number. The client application has a screen that displays customer number and the last order made. It would need to first request for the customer record API. The API will Provide the entire customer record. The client would then send the customer number as part of its request to Consume order history API, and the server would then Provide a response for after doing the necessary computation to generate the order history. This computation process could rely on a external DB or CRM system (what we call a enabler). The client would then populate and display the page with just the portions needed. This workflow we call a Chained service, the response from one request, being used as a request for another.

The time taken for a API to respond, includes this logic or computation that can involve look-ups on other systems (enablers). Identity validation, DB lookup and even external systems like partner shipping systems, or foreign trading systems. Robust and Sustainable API should be kept small, lightweight and client independent, to ensure their modularity and re-usability. This was not always the design criteria, and many older services are monolithic and tightly coupled with the client. API Gateways are often used to mediate and create new lighter weight services for Mobile applications. Adapting protocol and message format and creating virtual partial API, to strip unwanted traffic off the network portion. These gateways can offer caching and performance improvements, but can also be sources of latency.

The rapid growth in mobile application development has resulted in many new technologies, and emerging standards. Newer, lighter weight protocols like REST are generally used vs. more mature, heavyweight SOAP. New encryption methodologies like elliptic curve are common since it requires lower client CPU processing. New Identity formats like SAML and OAuth are used to address identity in the cloud and mobile arena. These new emerging technologies, are often still early or pre-standard development and relatively immature. Furthermore, the skills of developers and QA in these new standards are very limited and in extremely high demand. When last did your team get training on one of these emerging technologies, or have they simply learned these as they have developed your mobile application? It is unlikely that a business can expect the same level of maturity and quality in mobile applications as perhaps they may in other more traditional development and the fault density will probably be far higher in new mobile application development.

API Performance = User Experience – (Client, Network, Enablers)

From Client and Network posts, we already know the Network and Client performance impacts. The same test case run locally to the API gateway or server provides the performance for API + Enablers.

I constantly recommend that performance testing be done earlier in the testing and development life-cycle. Since SOAPSONAR same test cases can be used for functional and performance testing, performance testing should start as each service is validated functional on a service by service basis. SOAPSonar will report on each individual request and response time for each service, including each step in a chained service. A client application may only show the end result of the chain service or the service that responds slowest. Testing via the device may give user experience, but provides little information as to what or which service is slowing things down.

An important part of performance testing is understanding the impact of load on performance. This is usually done right before production cutover, and often leaves little time for time consuming rewrites. The result is often over architect hardware or network to compensate. SOAPSonar can use the same test case, with Virtual Agents to generate load (including across physically distributed load agents). Reporting on the pact on performance at a given TPS or understanding at what point the system will begin failing. By defining a success criteria in the test case to fail tests that take over a given time, can help identify individual services that start failing under load or when running a regression test.

SOAPSonar can detail the performance of each request made to each API and the response time, hence identifying any particular API’s which may not perform well. If these API are supported by an Enabler, identifying the poorly performing Enabler vs the API can requires additional isolation.

PERFORMANCE TUNING MOBILE API – ENABLERS

The logic behind API’s can involve look-ups on other systems. API’s also address things like identifying the user, authentication, security, encryption, message signing etc many of these reside on other centralized or shared systems. A trend in the Data Economy, SaaS and Cloud services is to do mash-ups of API’s, either Consuming API’s outside of your organization as part of your application, or Consuming, refining then Providing them it turn. Collectively, I refer to these as Enablers, services that support, yet not exclusively for your application or under your control. The vast majority of the performance issues I have being involved in was due to some Enabler or other being slow or failing under load.

Yearly Archives: 2013