Crowdtesting can be a valuable approach for testing and improving Internet of Things (IoT) devices and applications. IoT involves interconnected devices and systems from everyday household items to industrial machinery, that are connected to the internet and are collecting and sharing data. This innovative technology can involve complex interactions, and ensuring their functionality, security, and usability is critical. It is not only transforming the way we live and work, but it’s also significantly altering the approach to software development.
With more devices connected to the internet, there’s a greater risk of cyber threats. As a result, software developers are now prioritizing cybersecurity more than ever before. The multitude of devices connected to the internet also mean it’s vital that they can seamlessly communicate with each other. This has increased the need for interoperability through software that is more flexible and adaptable, capable of functioning across different platforms and devices.
These factors all add to the importance and complexity of thorough testing of software quality and user experience before IoT products are brought to market. What’s at stake is not only the successor failure of individual products, but global brand reputations.
Crowdtesting IoT developments
Crowdtesting of IoT devices brings numerous benefits. Notably, this includes using a diverse group of testers from various locations, which can help assess how an IoT product performs in different real-world usage scenarios, rather than in controlled conditions.
Crowdsourced testing, as it is also referred to, offers scalability through involving a large number of people using interconnected devices in multiple configurations. This means the process can verify compatibility with different devices, operating systems, and communication protocols. Using a high number of testers can also simulate various loads and stress scenarios to ensure the system’s reliability and performance under pressure. If required, it can be carried out almost simultaneously in various geographical locations around the world.
The crowdsourced testers can provide valuable insights into the user interface, mobile apps, and overall user experience for developers to make necessary improvements. The usability and user experience of IoT products are crucial for commercial and reputational success, and identifying weakness sooner rather than later means they are easier, and cheaper, to put right.
Testers can also identify vulnerabilities and potential threats by reporting bugs and simulating various cyberattack scenarios, and verify that update processes do not disrupt the functionality of IoT devices.
To apply crowdtesting effectively to IoT development requires careful planning and management of the testing process, with defined objectives and adequate access to the IoT devices and systems being tested. There are a number of specialist crowdtesting platforms available to provide suitable groups of testers, and structure and manage the whole process. Here is an example of IoT crowdtesting by a major domestic appliance manufacturer.
An IoT Crowdtesting Case Study
Haier Europe is a European leader in the domestic appliances market through its brands Candy, Hoover and Haier. In 2020 the Haier Group 
launched a washing machine with a single button control, meaning it can only be used if connected to a smartphone. hOn is Haier’s proprietary one-stop app to smartly manage and make the most out of Candy, Hoover and Haier appliances.
Delegating full control of a washing machine to an app was a brave choice. The process is to tell the app what the clothing to be washed is made of, photograph the pile of laundry that needs washing, and the app will set the Candy Nova to the correct programme. It was fundamental to ensure the quality of the app, the usability of the interface, and the interaction of the app with the household appliance.
Haier Europe engaged UNGUESS, a European crowd testing provider based in Italy, to focus on:
- evaluating the overall experience (clarity, usability, completeness of the range of programs offered) on the app and its connectivity
- analysing the user experience on the available features (list of programs, snap and wash, smart-home skills, and more)
- collecting user feedback on different sections of the app: maintenance of the washer, the resolution of problems related to its use, etc
- understanding the users’ preferences on features and sections
UNGUESS’s response was based on three stages of selecting appropriate testers; ensuring the quality of their testing experiences; and qualifying their feedback with a response to suggested improvements.
Tester selection
Testers were selected in various countries through an initial profiling to ensure they matched Candy’s target audience brand, an approach consistent with Haier Group’s value of pursuing Zero Distance to Consumers. They were then able to use the product in their daily lives, discover the features, and interact with the app as real end-users on numerous occasions.
Tester experiences
For results based on authentic experience, testers selected from UNGUESS’s community received a Nova washer at home. The testers’ preferences, opinions, and suggestions after using it were collected through surveys. The data that was collected provided both quantitative and qualitative information about the app experience offered by hOn, and the overall experience with the Nova washer. There were three main clusters of results:
- general experience on the app and connectivity,
- features (list of programs, smart-home skills, preferred washing cycles, etc.),
- sections (notifications, statistics, maintenance, etc.).
The questions presented to the testers allowed the Haier team to investigate the Net Promoter Score (NPS). This is a customer loyalty and satisfaction measurement based on how likely consumers are to recommend a product to others.
Tester feedback on improvements
Areas of improvement in the user experience were singled out and it was possible to identify more directly the most appreciated features that users would like to give more prominence and access.
Proposals for improvements and further developments that were identified through the tester survey results were then fed back to the testers to check if they were consistent with their needs.
Client summary
The final words are from Alessandra Carobene, Senior Product Manager IOT & Connectivity at Haier Europe, who summed up the benefits of crowdtesting the award-winning hOn app:
“The collaboration with UNGUESS gave us the opportunity to move on from laboratory tests, pre-development and post-development one-off interviews and usability surveys to a much more practical and experienced analysis, able to capture facets and give users the possibility of developing an opinion over time.”
0 Comments