Crowdtesting, also known as crowdsourced testing, is a modern approach to software testing that leverages a diverse and distributed group of individuals, known as the “crowd,” to perform testing activities. Unlike traditional testing methods that rely on dedicated in-house testers or contracted professionals, crowdtesting taps into the power of the crowd to discover bugs, evaluate usability, and provide feedback on quality assurance, user experience, and usability testing for websites, apps, chatbots, IoT and business applications. Our thanks to Tobias Brunner, head of marketing at Testbirds, for his market insights that we are sharing with you.
Testbirds was founded in 2011 and is a world-leading crowdtesting provider, headquartered in Munich in Germany. It has over 100 employees and works with more than 250 clients worldwide, including large scale multi-national groups such as BMW, Audi, Deutsche Telekom and Western Union, alongside small- to medium-sized enterprises and startups, mostly based in Europe.
How crowdtesting works
Crowdtesting platforms, which are the service providers, recruit and onboard a large community of testers from various backgrounds, locations, and devices. These testers may include professionals, enthusiasts, or everyday users with different levels of technical expertise. Testbirds has access to over one million testers around the world. This network is big enough for tests to be carried out by specific groups of people meeting demographic profiles specified by the client, on a variety of devices, and testing is carried out 24/7.
Photo by Glenn Carstens-Peters on Unsplash
The software or application to be tested is divided into specific test scenarios or tasks. Tasks can range from exploring different features, attempting specific actions, or trying to identify bugs or issues. Crowds of testers, using their own devices and setups, perform the assigned test scenarios based on the provided instructions. They record their findings, such as bugs, crashes, usability problems, or any other relevant observations. These reports can include steps to reproduce the issues, device and software configurations, and screenshots or videos to illustrate the problem.
Crowdtesting platforms typically have a team of experts who review the bug reports and feedback from the testers. In performing a quality control function, they validate the reported issues, prioritize them based on severity, and provide additional information back to the developers, such as steps to reproduce the bugs.
Based on the received feedback, developers can make necessary improvements and fixes to the software. This iterative process continues until the software meets the required quality standards or specific objectives.
Crowdtesting can also be applied in B2B scenarios, as explained in another Crowdsourcing Week article.
Crowdtesting’s advantages over traditional testing approaches
With large and geographically distributed crowds of testers, crowdtesting covers a wide range of devices, operating systems, and configurations, ensuring comprehensive testing coverage.
It also allows for quick scalability by involving hundreds or even thousands of testers simultaneously. This enables faster testing cycles and quicker identification of bugs, or any other issues.
The feedback and insights it generates are from real users, not employees, reflecting diverse user perspectives and identifying usability issues that may not be apparent to a limited in-house testing team.
Crowdtesting can be more cost-effective than hiring a dedicated testing team or maintaining an extensive in-house infrastructure for testing purposes. It offers flexibility in terms of payment models, allowing organizations to pay for results such as the number of bugs found, for example.
Major players in the crowdtesting sector
Applause is a leading crowdtesting company, offering a wide range of testing services, including functional testing, usability testing, and exploratory testing. They have a large global community of testers and cater to multinational clients across various industries.
In addition to providing a similar range of services, Testbirds offers a diverse crowd of testers, known as “Testbirds Nest,” who cover different languages, devices, and locations to ensure comprehensive testing coverage. Testbirds often finds itself competing directly with Applause to win customers, and it frequently comes down to a client decision over payment terms.
Testbirds stands out for using a different payment structure to most of their competitors. Rather than provide a subscription service, which means clients pay a monthly fee regardless of whether they use any services or not, Testbirds operates on a tokenized basis through its Birdcoins. Clients effectively pre-pay, but only use the tokens for services they actually use rather than for retainer fees. Technically, Birdcoins expire at the end of a three year lifespan, but in practice they are rolled over rather than burned. It is a company brand pillar to offer as much flexibility as possible to encourage clients to burn Birdcoins so that they will buy some more. Testbirds keeps in touch with clients about how they can use their Birdcoins to pay for services, and it also means clients cannot say “we don’t have any money available for that right now.”
Rainforest QA is another on-demand crowdtesting services provider, allowing clients to leverage their community of testers for functional and regression testing. They focus on delivering fast results and offer integrations with various testing tools and platforms.
Test IO offers crowdtesting services for web and mobile applications, focusing on usability, functionality, and localization testing. They provide comprehensive test reports and actionable insights.
Global App Testing also offers crowdtesting services for web and mobile applications.
UserTesting specializes in remote usability testing, allowing clients to gather real-time feedback from users worldwide. Their platform provides video recordings of user interactions and feedback to help improve the user experience of digital products.
Testlio offers end-to-end testing solutions, including functional testing, localization testing, usability testing, and regression testing. It is known for its strong project management capabilities, assigning a dedicated project manager to each client to ensure smooth communication and coordination throughout the testing process.
Ubertesters offers crowdsourced testing services, using a large global community of vetted testers with devices, to allow a flexible, fast, and cost-effective way to scale an in-house testing team and launch bug-free, digital products that are optimized to clients’ needs.
Testbirds’ future opportunities and challenges
Flatten out seasonal demand
Quarter 4 is historically Testbirds’ busiest time of the year. To avoid bottlenecks later this year it has added more project managers in a new office in Leipzig, which operates alongside its other European offices in London and Amsterdam. Now it needs to keep them busy during the rest of the year.
There are two new products available to help achieve this.
- A self-service facility to create simple surveys of perhaps 10 questions on a topic, define a target group of testers, and receive realtime results.
- Unmoderated Usability Tests. These are user tests with video support. They can be executed on a self-service basis, though do require some degree of knowledge and familiarity with Testbirds’ systems. Some of their competitors already offer this service.
There is also the possibility of some brand advertising to stimulate demand for their services.
Launch in the US in 2025
A solid 18 months from mid-2023 to the end of 2024, and fresh investment, will allow Testbirds to enter the US market. They will only do this if they have the budget to afford the required resources to set up an office, create a project management team, and fund marketing support.
Crowdtesting could be a valuable tool for businesses, projects and individuals operating in any of the BOLD Awards 33 international categories. Entries can be submitted right up to the end of 2023, though the cost of processing entries will rise during the year. Early submissions can be returned to and updated any number of times before the cut-off date of December 31st, 2023, and each submission can be entered in up to three categories. Two assessment stages include a round of public voting to create shortlists, followed by scrutiny by a panel of judges. The award ceremony for finalists will be a gala dinner in Venice, Italy, in March 2024. You’ve got to be in it to win it, check it out now!
0 Comments