From Bugs to Brilliance: How Crowd Testing Elevates Software Quality

Crowd testing enables products, including B2B, to be tested in real world situations, by a crowdsourced on-demand global workforce, using multiple devices.
From Bugs to Brilliance: How Crowd Testing Elevates Software Quality - Crowdsourcing Week

Written by Clive Reffell

Oct 17, 2023

Data shows that, on average, 40% of users abandon a website if it takes more than 3 seconds to load (source: Toptal). Yet at the same time, digital product and service providers are continually up against time to bring their offerings to the marketplace ahead of their competitors. It can be difficult for an in-house team to test sufficiently if products are delivering the required levels of both quality and usability while in a race against the clock. A growing solution is to use crowd testing: have products (including B2B crowdtesting) tested in real world situations, by a crowdsourced on-demand global workforce, using multiple devices. Working under a wide range of real-life conditions, they give 24/7 feedback based on diverse levels of customer experience and expectation.

To get an idea of the commercial importance of getting it right, it has been estimated that slow websites account for a massive $2bn global loss of annual sales. Beyond fast, sites also have to be secure and accurate to build customer confidence and satisfaction, free from bugs, spelling errors and mistakes in translation

UNGUESS is a European crowd testing service provider, with its headquarters and a GigaTestFactory in Italy that supports thousands of companies internationally. Their early involvement in client projects, using their own crowd of testers, can save time and resources by From Bugs to Brilliance: How Crowd Testing Elevates Software Quality - Crowdsourcing Weekspotting functional product deficiencies and identifying bugs at an early stage. Using a feedback loop such as this can avoid costly redesigns further down the track.

A feature of UNGUESS’s level of customer service is its highly vetted and deeply engaged crowd of more than 50,000 trustworthy and reliable testers. They bring proven skills and fresh eyes, free of any potential corporate mindset that could influence an in-house team of testers. TRYBERs, as they are known, are from more varied backgrounds and cultures and bring greater diversity. They also sign non-disclosure agreements to ensure confidentiality for each client.

With such a large crowd of real-user testers, qualitative testing can be carried out in almost any country around the world, on virtually every possible configuration of device and operative system in a time span no longer than a week. The importance of this for an e-commerce company is that at every release, a predetermined set of users tests the new website functionalities in different languages and on different devices, immediately identifying functionality issues, and bad translations.

These real-world testers also raise usability issues that can enhance product development and create a better user experience. Usability testing through the rapid interactions of such a large crowd make it possible to identify, with a robust degree of quantifiable certainty, where significant numbers of users would struggle or make mistakes.

Notable UNGUESS clients include the sports goods retailer Decathlon, the household appliance brands Braun and Candy, tv station Sky, the Pirelli e-bike rental service, and Toyota. UNGUESS was also the first company in Italy to pioneer the use of crowdsourced testing methodology, and the first company in Europe using its power to optimize varied subjects at once including quality, security, and user experience of digital products and services for businesses. Here is an example of B2B crowdtesting.

B2B crowdtesting of a billing system

Here is how the integrated industrial group Growens, provider of the popular mailing system MailUP, worked with UNGUESS to use crowd testing. Growens also develops technologies for predictive marketing, mobile messaging, and content creation. It has a strong presence in Europe and North America, though in total it operates in 115 countries with more than 26,000 business customers.

When Growens approached UNGUESS they were using an internal service for managing the purchase, invoicing, and renewal of their services across five business units. The system had developed over the years, and handled purchases made from both a direct e-commerce channel and from a sales network.

From Bugs to Brilliance: How Crowd Testing Elevates Software Quality - Crowdsourcing Week

Image source: UNGUESS

Growens wanted to transition to an external market service for subscription management which required the progressive revision of several software components. At the same time, it was vital to avoid service disruptions or alterations to the user experience. They needed to know their clients around the world would not have problems using the new software.

In an almost year-long project, UNGUESS supported the client with a systematic approach to mapping all the paths involved in migrating to the new system. At each stage, over a hundred various contracting and billing methodologies had to be functionally verified. It was necessary to test both the APIs (Application Programming Interfaces) and the front-end across different types of users.

The project was managed using a combined approach of test automation and crowd testing. For the test automation, a test manager from UNGUESS coordinated Growen’s five business units. Three different automated testing machines were built in Robot Framework and integrated into existing development pipelines, with a focus on continuous quality.

At the same time, several crowdsourced testing sessions involved approximately 150 users working with over 250 different combinations of browsers and operating systems in various contexts. The crowdtesting sessions lasted an average of two to three days each, providing a significant time boost to UAT (User Acceptance Testing) activities and functional verification.

Validation of crowdsourced testing

The use of automated testing machines ensured quick and accurate execution of test cases within the development pipeline, and raised more than 200 bug reports. Crowdsourced testing provided real-world user feedback, and over 500 bug reports uncovered potential issues in diverse environments. The combined effect meant the client’s development team was able to save over 30% of their time dedicated to testing and quality assurance activities, which freed up resources to focus on development tasks and accelerate the overall project timeline. It also opened up possibilities to enhance user experience and enrol ethical hackers.

This B2B crowdtesting case study is one of an UNGUESS collection that covers their work with mainly consumer-facing brands and businesses. Find them all at

About Author

About Author

Clive Reffell

Clive has worked with Crowdsourcing Week on sourcing and creating content since May 2016. With knowledge and experience gained in a 30+ year marketing career based in London, UK, he operates as an independent crowdfunding advisor helping SMEs and startups to run successful crowdfunding projects, and with wider social media and content marketing issues.

You may also like

How to Best Use Crowdsourcing in Retail and E-commerce

How to Best Use Crowdsourcing in Retail and E-commerce

Crowdsourcing can be a powerful tool for retail and e-commerce businesses to leverage the collective intelligence and creativity of their customers, plus the broader online community. Few brands have exploited its benefits to the degree of the LEGO toy company. It...

Speak Your Mind


Submit a Comment

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.