With the imminent arrival of 5G services, network providers are vying to provide the best possible services and consumer expectations are rising. In order to monitor changes to service delivery and relative customer satisfaction it is vital for operators to have independent, accurate and robust data that reflects the current market and is comparable across multiple territories.
It’s fair to say that umlaut’s 2020 benchmarking framework for telecom services is both the jewel in its own crown, and a global “best-in-class” example of a cross-industry benchmarking survey. umlaut constantly adjusts its testing methodology and scoring models in line with technical developments. This year, more than 200 mobile networks in more than 120 countries are being evaluated according to umlaut’s unique scoring methodology, which is increasingly recognized as the industry’s gold standard.
Introduction to umlaut
umlaut is a leading global, full-service, cross-industry, end-to-end company that offers advisory and fulfilment services to clients all over the world. Headquartered in Aachen, Germany, it has a team of over 4,300 consultants and engineers working to develop and implement innovative solutions to today’s ever-increasingly complex technology challenges.
In the telecommunications sector, umlaut (previously known as P3 group AG) provides independent technical and management consulting services including network planning, end-to-end optimization, security, Quality of Service and Quality of Experience testing, international benchmarking, device testing and acceptance services. umlaut addresses all industries with connectivity needs.
Their clients include network operators, equipment vendors, device manufacturers, public safety organizations and regulatory authorities around the world.
The global acceptance of their methodology is based on its two-pronged approach of Network Testing (NT) and User Experience Testing (UET) which is based on crowdsourced data of actual mobile phone usage, rather than interviews with users. For the collection of crowd-data, umlaut has integrated a background diagnosis process into thousands of diverse Android apps.
Network Testing
umlaut’s 2020 methodology had to incorporate 5G, and still cover the performance of legacy services. The aim is to grade every operator with the “umlaut score”, which is a unique measure for management and supervisory boards to compare their networks with others all over the globe and enhance strategic decision-making. To achieve this, and to support the related “Best in Test” award, a set of common mandatory tests have to be included for every provider in every territory.
MNOs (mobile network operators) need to be able to obtain objective measures of network quality, to substantiate claims and to ensure that they can deliver the required service performance.
User Experience Testing
Customer experience is also a key metric for MNOs.
Some user research is conducted on a controlled basis, including checking reception strength and quality in moving vehicles. This is to ensure comparable procedures are followed, thus delivering comparable results.
For the collection of crowd-data, umlaut has integrated background diagnosis processes into thousands of diverse Android apps.The apps are recording data on network coverage, time taken to connect to sites, to download content, quality of connectivity with the network provider, and many more. Data is collected 24/7, every day of the year. According to umlaut, this data collection is compliant with the GDPR and includes no personal user data.
Outcomes
Combining the NT and UET scores allows creation of an overall leader board of the best service providers. Here are 2019/2020 results, as an example.
Most recently, the benchmark survey also had to cope with the extra mobile usage caused by the pandemic lockdown in most countries. This caused higher levels of remote working, or for those that were furloughed it created more opportunities, and an appetite, to download entertainment.
Identifying changing patterns of mobile phone usage showed telcos how the amount of data being accessed changed, the extent to which users switched to WIFI, and many other aspects. Communication apps continued to be used heavily, apps for home office and home schooling showed an increase in usage, while the use of shopping apps declined. Network operators can use such findings as an important foundation for strategic planning.
umlaut’s data also helped governments, city authorities and the healthcare sector. It showed that many German residents from Austria, Switzerland, France and The Netherlands returned home, while many from Poland and Romania, including a high proportion of harvest workers, remained. The data also indicated that people were spending less time in public areas, and people were moving slower in city centres because fewer of them were driving or using public transport.
In summary
In an exclusive comment for Crowdsourcing Week, umlaut CEO Telecommunication Hakan Ekmen said: “Today, we measure and evaluate the telecommunications networks in 120 countries. So we assess the digital infrastructures worldwide – such as mobile and fixed networks – and make them usable for every customer.
Our benchmarks are regarded as the de facto industry standard. Thanks to our consistent methodology, the results are comparable across the countries. Network operators consider them to be the most relevant assessment of their network’s performance.”
umlaut’s aim is to achieve a comparable best-in-class benchmark research status in other business sectors.
0 Comments