When I first heard the story of John Henry as a very young child, I found it puzzling. I couldn’t understand how anyone could honestly believe that somehow the human spirit could triumph over machinery at manual repetitive tasks. Later, I came to understand that this story was an allegory for society’s struggle to accept marginalization in value of human labor as a result of the modernization of the industrial age. I didn’t understand the story as a child, because the superiority of mechanized labor over human labor was well accepted and sewn into the collective consciousness. Today, we are reliving the John Henry story. The new battleground? AI (primarily Machine Learning Algorithms) vs Crowdsourcing. The buzz around the Human Cloud is that we can harness the intuitive pattern recognition and judgement abilities of people that machines just cant effectively replicate. Sadly, I think in large part, this is simply reliving the seductive, but ultimately hopeless story of John Henry. We are already starting to see quantitative studies to show how this movie ends. That said, I’ve found some compelling niches for crowd sourcing that complement the use of AI:

• Crowd sourcing is very effective for ad hoc jobs where the cost to automate dramatically outweighs the cost of acquisition and QA on the crowd sourced model.   
• Likewise, it is well suited where the domain contains a graduated value return as a function of quality. 
• Crowd sourcing can also be effectively used as an audit function to discover holes and exploits of your automated algorithm (eg to discover that people are gaming your moderation system using #u€&!ng symbols….). 
• Crowd sourcing is a great training source for machine learning algorithm based solutions.  If you can’t beat the machines, at least you can work for them….