To succeed in today’s global market and create positive brand awareness there is an increasing need for organizations to improve user experience of their websites, apps and other software. Helping companies to launch their websites and mobile apps to the public rapidly and reliably is a growth sector. In order to keep up with the volume of testing at the required fast pace, on-demand crowdsourced research through crowd testing platforms (also called crowdsourced testing) obtains real user feedback to ensure software reliability at an incredibly fast speed.
How crowd testing works
Crowdsourced testing platforms provide their client organization users with on-demand access to thousands of vetted testers. They are recruited on a global basis to provide a round-the-clock service. The high numbers of testers available also means the organizations launching new products or services can specify particular audience sub-groups they may be targeting, or specific languages. Importantly, the testing is carried out in the real world, by various types of people under a multitude of circumstances and on a vast number of devices, rather than under homogenised test centre conditions by employees.
The organizations pay the platforms a fee for the work, and the platforms post the work requirements. Their crowds of testers provide swift results because each tester wants to find the bugs or weaknesses first to receive payment. Project management teams of employees at the platforms collate all the actions in a report, and the platforms handle payment to the individual testers. Payments are based on the results they achieve, and not the time they spend doing any work. It is therefore down to each tester to improve their personal speed, accuracy and reliability to earn more from their time input. As a side income, some testers earn a few dollars for each bug or other software weaknesses they identify on a microtasking basis. Others who fully devote themselves to the work can reportedly earn over $2,000 per week.
Market size and structure
The global crowdsourced testing market size is forecast to rise from USD 1.5 billion in 2020 to USD 2.3 billion by 2026. It is moderately competitive and consists of several major players. Few of the notable players are able to dominate the market. Here are five of the top platforms that have come to our attention.
Crowd testing platforms
Testbirds
Testbirds was founded in Germany in 2011 by Georg Hansbauer, Philipp Benkler and Markus Steinhauser. The company has over 100 full-time employees at its headquarters in Munich, Germany, with more at offices in Amsterdam and London.
Testbirds’ crowd testing services offer a comprehensive, high-quality testing service that is adaptable to the individual needs of clients with a focus on the quality assurance and usability of digital products. It doesn’t matter if it’s apps, websites, Internet-of-Things applications (IoT) or the newest tech innovations. Crowdtesting platforms rely on unbiased real users, chosen to match any specific target group. Testbirds has used a total of over 600,000 testers from all around the globe, testing on over a million devices in a real environment.
They provide some exclusive services:
- Merged QA (Quality Assurance) and UX (User Experience) through their Bugability program
- Customer Journey Testing to ensure the online and offline world mesh together seamlessly
- Language Testing for chatbots, language assistants or conversational AI
- The opportunity to apply Testbirds’ processes among an organization’s employees or customers, through Bring Your Own Crowd.
Testbirds’ global client list of 250 organizations includes large scale multinational groups as well as small- to medium-sized enterprises and start-ups. BMW, Audi, Deutsche Telekom, Allianz, The Independent newspaper and Western Union trust Testbirds’ to optimize their digital products.
Applause
Applause is the worldwide leader in crowd testing and digital quality. Software is at the heart of how all brands 
engage users, and digital experiences must work flawlessly everywhere. With crowdsourced testers available on-demand around the globe, Applause provides brand owners with a full suite of testing and feedback capabilities. This approach drastically improves testing coverage, eliminates the limitations of offshoring and traditional QA labs, and speeds time-to-market for websites, mobile apps, IoT, and in-store experiences.
Thousands of leading companies including Ford, Fox, Google, and the Dow Jones rely on Applause as a best practice testing platform to deliver high-quality digital experiences that their end-users can trust.
GlobalAppTesting
As their name suggests, they focus exclusively on app testing. Though any service provider that includes Facebook, Google, Microsoft and Instagram among its clients has to be doing something right!
From launching in 2013, they claim a crowd of 40,000+ vetted and ranked testers operating in over 105 countries to provide fast response times in a multitude of languages, supported by robust automation tools. Their web and app testing solutions are engineered to integrate seamlessly with clients’ DevOps or CI/CD tools to enhance their existing workflows.
They encourage their testers to play to their strengths, stretch themselves, and become the best version of who they can be.
Their vision for the future is autonomous testing augmented with humans. This will allow enterprises and SMBs to deliver high quality products with minimal testing effort.
Testlio
Testlio was founded in Estonia in 2012 and is now headquartered in the San Francisco Bay Area. It claims to be the original networked crowd testing platform, though launched after Testbirds. Only 3% of tester applicants are accepted into the Testlio network, yet they still have a global crowd of over 10,000 validated testers in more than 150 countries – it’s a massive pool of some of the best Quality Assurance testers in the business. Their US team works mostly from home offices with clusters in Austin, the Bay Area, and Southern California.
A point of difference, contrary to most of their competitors, is that Testlio pays its testers by the hour, not by the bug. They focus on QA testing, and finding bugs is a by-product of the testing. They say this means higher quality issue detection and thoughtful reporting to improve overall defect management. Clients can communicate with the testers, if they want to.
Its clients include American Express, CBS, Microsoft, the NBA, Netflix, and SAP. Between them Testlio’s clients collectively serve nearly 2 billion users.
UserTesting
User Testing’s USP is that it is a video-first crowd testing platform where clients can see and hear the experiences of real people as they engage with their products, designs, apps, processes, concepts, or brands. Do faces scrunch up in frustration or light up with joy?
Voice tone and intonation, plus body language, can deliver so much more depth of understanding than just written reports on how people react with and to a new app, social media content or IoT devices. Clients can see their apps being used and compare the experience to using a competitor’s. Testers who represent a selection of a core target audience can be sent physical products for clients to see their interaction when following through on set tasks. Individual testers can even be contacted for deeper one-on-one conversations with them.
User Testing also uses machine learning to provide transcriptions, written reports and sentiment analysis. Clients include Samsung, Adobe, ebay, Expedia and Wayfair.
Three more crowd testing platforms
Here are brief notes on three more, as a bonus.
- UNGUESS (formerly known as AppQuality) is a tech-platform headquartered in Italy that brings the collective wisdom of its TRYBER crowd to the Quality Assurance, User Experience and Security decision-making processes, fast, and at any time needed.
- Hundreds of companies including Adobe, Oracle and Solarwinds use Rainforest QA to automate their QA testing process and easily integrate it with their development workflow via a simple API.
- Ubertesters also focuses on Quality Assurance, testing mobile and desktop apps, websites, live stream services and IoT devices.
Crowdsourced testing can also be used by B2B companies, as this other Crowdsourcing Week article explains. Are you a crowd tester, or perhaps you work at an organization that uses crowd testing? We’d really like to hear from you.
0 Comments