The number 1 issue facing the entire wave of crowdsourcing companies today is quality. Everyone can setup a service that is cheaper and has a faster turnaround time, but quality is holding the industry back.
In some recent discussions, we came up with a simple framework for thinking about the problem and building a roadmap of features and policies to address the concerns. I think this framework is a useful starting point – the features and policies you build in each one of these areas will vary greatly, but it helped me structure my thinking. Also, this is for paid crowdsourcing companies, not the unpaid user participation type models. I don’t believe it applies as well to unpaid models.
——-
Attract
It all starts here. I’ve frequently argued that it’s “easy to find people that want to make money, and it’s hard to find people that want to spend money.” I still believe that’s true – but it’s missing the important qualifier that it’s still very hard to find high-quality people. Everyone knows this since hiring is probably the #1 challenge for almost any startup. Your company needs to be attractive to the best workers in the world. Or if not necessarily the top 1%, at least the top 10%. If your site or marketplace are viewed as a “race to the bottom” – then that’s what you’ll get. I think it’s critically important to consider the saying As hire As, Bs hire Cs. Same applies in crowds although not as directly. If an A player is browsing around the site and sees mostly Bs, then they will not be interested. Think about what works to attract employees in the “real world” – do the same thing in the online world. Perks for workers, positive culture, big vision, fair compensation… read the Netflix culture deck. See what Facebook and Google do to attract the best developers in the world. Why should it be any different in the online world.
Screen
Ok great – so you’ve proven that people can make money on your online platform and you’re attracting a lot of good candidates. Now the job is to decide who to let in. Many online marketplaces or crowdsourcing sites simply let everyone in the door and then build systems to try to identify the best. But what can you do on the front end before the individuals become part of your network? There are lots of different ways to test candidates and a vertical approach works best here in my opinion. Have a category-specific test for your applicants. Utilize LinkedIn / Facebook connect to screen. You all know how to screen for potential employees, it’s not that expensive and you can do it for your crowd as well. Or perhaps someone will start a crowdsourced candidate screening company. J
Optimize
Assuming you now have an active network or crowd performing tasks, you should be gathering a lot of data. The types of data you’ll want to capture also vary by industry, but you can imagine a host of performance metrics. Let’s take a crowdsourced design contest. I might want to capture metrics for each individual designer such as time to submit a design, # of designs submitted to unique contests, % of designs that received input from buyer, community ranking of design, % win rate, CTR on the actual designs submitted, time on page of submitted design, etc. You’ll also no doubt want to have a user feedback system. Then the point of the “optimization” theme is to get the most work to the best workers. You can do this by preferred placement of the people, preferred / early notifications, job quotas, limits, etc.
One important part of the Optimize theme is to make sure that you have true individual metrics. Metrics about a design firm / BPO company’s performance isn’t really that helpful.
And regarding user feedback systems – I hate 5 star systems. They end up becoming terribly biased, but that’s a subject for a different post.
Expel
You don’t need to worry about this one right from the start, but it will become important over time and as you scale up. The lowest performers need to be removed from the network. A big company doesn’t keep their lowest performing employees. There is the constant lifeboat test performed in some companies. The highest performing consulting companies have an up or out policy. In theory, paid crowds should operate along similar principles. It is always a little bit more delicate in the crowdsourcing world, but you have to do it. Low quality poisons everything about the marketplace and results in dramatically increased procedures and costs in QA, customer support, etc. Just get rid of the bad apples. You can do this based on the same metrics that you capture in the optimize theme.
Retain
By now you should have a nice high quality crowd. That crowd is now your most valuable asset (and by the way, you should truly own your crowd, don’t completely rely on somebody else’s crowd). You need to work to retain your asset and make sure you’re getting as much of their attention as possible. With any freelancing type situation, you always have the benefit as the payer that you only pay for time worked. Which means you can make things very cheap for the end consumer, but the flip side is that freelancers go where the money is. So, one of the most important things to manage is utilization rate for your crowd. Make sure the most important people are earning a good amount of money and make sure there is a path for the middle tier to start making more and growing their business. Take surveys, measure NPS of your crowd, and of course, provide exceptional customer service.
0 Comments