Crowdtesting offers a flexible, cost-effective, and efficient approach to software and app testing, enabling companies to access a diverse pool of testers, obtain real-world feedback, and improve the quality of their products and services. As a result, it has gained popularity in the ever-evolving landscape of software development and quality assurance. Analysts estimate that software testing costs companies around $45 billion per year, with spending expected to grow 5% annually to be a market worth $90 billion by 2032 (source: Global Market Insights, March 2023). We thank Emeka Obianwu, Vice President, Alliances & Acquisitions at Testlio, for sharing with us his insights and experience in the crowdtesting platform sector, and what he believes makes Testlio stand out from the crowd.
A personal challenge
Emeka Obianwu, Vice President, Alliances & Acquisitions at Testlio
Emeka has been part of Testlio’s leadership team for two years, though crowdtesting is a relatively new space for him as he spent the previous two decades working in a number of different technology and services sectors. The appeal of testing is that he finds it an exciting space because there’s so much demand for it. Within the overall testing sector, he finds that crowd testing is a disruptive force that is a more efficient way than traditional outsourcing or insourcing can ever match.
As a provider, the Testlio crowdtesting platform differentiates itself as to what a managed crowdtesting experience needs to involve, and what signal driven testing is capitalizing on. It operates with clients on much more of a relationship basis than a transactional one. It gives Testlio an opportunity to raise the bar on how it informs its clients with more intelligence on where to test, when to test, and so forth. As part of the leadership team, Emeka has to keep the company on its course, which he explained further.
About Testlio and what sets it apart
Testlio launched 10 years ago, and is very much a managed testing service that leverages a crowd. It certainly regards other testing service providers including Applause, Global App Testing and Testbirds as direct competitors, though Testlio believes its three-part solution offering sets it apart with some notable differentiation:
- Client Services
- A crowd network of freelancers
- Its platform.
Testlio serves some very large brands in the market, and big brands tend to have big plans. It may take a number of years for certain ideas and plans to come to the marketplace. Testlio’s operating model aims to similarly have multi-year relationships with those brands. Even short-term crowdtesting projects are regarded as a milestone towards a multi-year relationship with any particular client. The way in which Testlio invests in and thinks about its crowd of testers, its platform and its Client Services is based on that operating model, which has cost structure implications.
1. Client Services
Applause uses a subscription model, and Testbirds has its client ‘park’ money with them when they buy its Birdcoin tokens to pay for services at a later date. Testlio also uses a subscription model. A perceived downside of this for clients is that they still pay their regular subscription in the months when they don’t use the service much, or at all.
However, Testlio has an ethos of working as a partner with their clients, anticipating their requirements, and proactively going beyond the brief.
They provide each client with an Engagement Manager (operating as a project manager), a Test Manager and an Account Manager. Their aim is to embed themselves in their clients’ business, becoming more of a trusted business adviser than just a service provider. Their platform is also more open than any of their competitors, and it allows their clients to integrate their testing activities and findings into their own systems.
2. Testlio’s crowd of freelancers
Testlio has always prioritized the quality of its crowd over the size of the crowd. That’s been consistent since Day One. One of Testlio’s founders had been a freelancer at Applause, and one of the design flaws she saw was thinking of a crowdtesting platform too much as a marketplace, with the emphasis on who can deliver the biggest crowd, and then using that as a cornerstone of marketing to enterprise customers.
Testlio’s alternative is to set a high standard for its crowdtesting recruitment, and build a crowd that buys into the concept of long term customer value, not just deliver short term transactional value. Consequently, Testlio has only a 3% acceptance rate of freelancers that express interest in being part of its network.
It doesn’t matter how good someone is as a freelance tester if they don’t buy into Testlio’s multi-year relationship model and commit themselves to long-term client engagement.
3. The Testlio platform
Testlio’s founders began by asking “how do we make our services an extension of the modern DevOps infrastructure?” DevOps comprise a set of automated practices that are an amalgamation of strategies, principles, and resources which help improve an organization’s ability to produce applications and deliver services.
Clients may select any one of a number of DevOps tools based on varied criteria. Testlio’s crowdtesting platform is the most flexible one in the sector as it can integrate with seven different categories of DevOps tool sets to support its clients.
Beyond this, Testlio has an integration with a third party app ratings and reviews and app statistics provider that’s mapped to the digital workspaces in which Testlio operates on behalf of its clients to test their applications. Through monitoring third-party reviews, Testlio can at times find itself providing clients with early warnings about issues before they have recognised they exist.
Recent developments at Testlio
Testlio ran two experiments using generative AI technologies in April and May 2023. The outcomes indicated that quality assurance can be better and faster using a combination of expert quality professionals and AI-powered machines. Broadly, Testlio’s experiments show that generative AI can make written instructions and information more understandable, leading to more efficient testing and better quality assurance results. Generative AI can also reduce time spent writing, reviewing, and editing core instruction content, which is critical to successful quality assurance and quality engineering efforts. With these findings, Testlio has embraced commercial-grade generative AI capabilities via its software testing platform and services. It will use new efficiencies to help its clients release better products faster.
On May 31st, 2023, Testlio announced it is partnering with crowdsourced cybersecurity provider Bugcrowd to offer “comprehensive quality and security testing services that capitalize on the unique efficiency benefits of crowdsourced testing.” The alliance builds upon existing market momentum for crowdsourced testing as a solution to address both a skills gap in the market, and fulfill challenging product verification scenarios that require a diversity of locations, devices, users, and transaction types.
Further challenges
Emeka sees Testlio’s biggest challenge as prioritization of their crowdtesting roadmap. What’s the next integration they want to build, not just to deliver operational efficiencies to their clients, but to turn them into actual signals that can be leveraged to better inform their clients?
Other notable challenges include making sure that these signals are actionable for Testlio’s clients. One of the things that they are spending a lot of time thinking about is not just data sources and filters and how to correlate some of this information, but what does it need to look like?
Crowdtesting can also be applied in B2B situations, as a case study in another Crowdsourcing Week article explains. Have you used or worked for a crowdtesting provider, either as an employee or a freelancer? We’d like to hear about your own experiences in this sector.
0 Comments