Manual testing is not an option any more

Today the number of service and handsets to be tested is increasing always faster:

Existing methods and tools fail to keep up:

• Radio network testing is focused only on the transport layer of the application and not on the service and content. Real user experience is not tested and only a very limited subset of handsets can be covered by these testing solutions.
• Physical integration with real handsets suffers from difficult deployment, limited reusability and scalability, very complex maintenance, lack of troubleshooting information and very limited monitoring support. Furthermore real handsets are not made for continuous monitoring and may behave inappropriately.
• Handset agents tested are compatible with only a limited set of phones, and suffer from complex maintenance and limited troubleshooting information. Also they suffer of results of limited statistical value.

Services and content testing become carrier’s main headache because most of the problems encountered by customers are now caused by the content and not the network.

In other words, even when the content is correctly received, problems can be encountered: the handset cannot display or play it correctly; the content does not conform to what the user expected to receive or with what the provider expected to transmit.

Examples of customers’ issues:

• The soccer goal picture I downloaded is truncated: I cannot see the ball!
• My new downloaded Java game won’t start, message says: “missing API”
• I received an MMS from my wife but it is blank
• My WAP portal home page says “Sorry, service unavailable, please try again later”
• I requested a MP3 from Madonna and received the lastest Eminem tune
• A “Warning Adult Content” message has not been transmitted before the adult video streaming starts.

Ensuring successful content validation on a large scale

Background
Biggest i-Mode operator outside Japan. The mobile content team helps service providers to give the best possible user experience.

Challenge
Ensure quality content and services on all available i-Mode terminals, knowing that more than 300 services are now available on the i-Mode portal and that manual testing has thus become impractical.

Solution
Zandan Mobile QoE service was therefore deployed to monitor: the availability, the performance, and content validation interoperability with i-Mode handsets. 52 criteria have been tested from handset interoperability to respect of design guidelines.

Benefits
ARPU has been increased for the operator and its content providers. The operator’s teams are thus able to concentrate on providing more value-added offerings for the service providers.

Mobile QoE daily validation

Background
One of the biggest French operators has Value Added Services customers growing rapidly. The marketing department was looking for a solution to improve the QoE of its customers.

Challenge
Ensuring QoE of content quality on selected 2.5G and 3G services was the main issue ; manual testing was not able anymore to keep up with demand, and no troubleshooting information was available to help solve problems.

Solution
Zandan Mobile QoE service was deployed to monitor availability and performance, to test delivery from the users’ perspective and to deliver more error identification and diagnostic ability.

Benefits
Zandan's scalable model was able to handle a growing frequency of test campaigns, even during nights and over weekends. After deployment, a faster and more reliable process was delivered. Effective troubleshooting became possible thanks to the available technical information.

From an operations nightmare to increased availability

Background
Our client is one of the largest telecom operators in Europe with over 42 million subscribers and several international subsidiaries.

Challenge
The difficulty was to manage different versions of the portal for each country and find centrally defined criteria and targets for availability and performance. Once again, the manual testing was not able to guarantee a comprehensive control of all services.

Solution
Zandan Mobile QoE service deployed to monitor portal page, second-level page, resource page (download), MMS P2P, MMS Application to Peer. More than 40,000 tests were completed per month.

Benefits
MMS availability increased 25% after only a few weeks of operation and several poor QoS areas were identified and fixed.

Testing the QoE of a complete range of services

Background
One of the leading mobile operators in Russia whose current network is the result of the merger of several networks.

Challenge
The existing automated Voice & Network testing solution was not enough to keep up with complex Content & Services. Definition of KPIs by region was needed and user access restrictions were mandatory.

Solution
Zandan Mobile QoE platform deployed in 5 regions of Russia with a probe per region to monitor: WAP Portal, SMS P2P, MMS P2P. More than 19,000 tests were completed each day and reports were customized for different categories of users.

Benefits
Business decisions started to be made on the basis of reliable test data. The identification of problems become clear, as was their source: radio network / core / content & services. Alerting and root cause analysis ensured more rapid intervention and problem rectification.

From 52% to over 90% availability in weeks

Background
A major content provider was experiencing a dramatic increase of traffic since being put on the Vodafone Live! Portal.

Challenge
Its challenge was to assure conformance with SLAs with its portal customers that were becoming critical.
Previously their testing was done randomly by technicians using real handsets. Some music services had 52 % availability.

Solution
Zandan Mobile QoE service was deployed in 8 countries to test 2.5G and 3G services with a 24/7 monitoring of the music services.

Benefits
The availability quickly rose to over 90%. The main cause for poor QoS was discovered: a server was always failing to recognize phones.
Our customer found the ability to guarantee and prove compliance of SLAs with its portal customers

From months to weeks to deliver new platforms

Background
One of the biggest mobile operators in the world with several subsidiaries all over the globe. The Global IT Group is in charge of deploying new platforms.

Challenge
The manual testing costs were sky-rocketing. The delays in delivering new versions of the mobile portal continued to increase. Monitoring was not integrated with testing and troubleshooting.

Solution
Zandan Mobile QoE platform was deployed to perform different types of testing:
- Before going to production: unit testing, load testing, acceptance testing, compliance testing
- During production: real time monitoring, real time alerting, QoS Measurement
- Route cause diagnostic and troubleshooting

Benefits
Delivery of new platforms went from months to weeks and manual testing costs were cut by more than 80%.
Problem anticipation was improved, allowing a decrease in ongoing maintenance and support costs.

Deliver unparallel levels of reporting and benchmarking to marketing and IT teams

Background
One of the biggest mobile operators in the world with several subsidiaries all over the globe. The Global IT Group is in charge of deploying new platforms.

Challenge
Marketing and End-to-End performance teams had no way to see the big picture anymore. No one knew exactly the quality of service, and how it compared over time and to competitors. Departments were silos with no means to communicate the same info easily.

Solution
Zandan Mobile QoE platform was deployed in 25 countries to test 2.5G and 3G services with service-oriented and pre-defined reports to benchmark handsets, networks and services.

Benefits
Quality of Experience managed on a global scale allowing to leverage the best practices of each subsidiary. Better measurement and understanding of customer experience was made possible easing and accelerating the detection and troubleshooting of problems.