Thomas Edison is credited with the saying that “Genius is 1% inspiration and 99% perspiration”.
Many of the most successful web 2.0 companies understand this intuitively and it is reflected in their product management. Although they all have a general vision for their product, it does not spring full formed from their minds. Rather, they build A:B test harnesses to explicitly test their hypotheses on live users. They don’t ask their users what they want, but rather they watch what they do. They try multiple versions of everything (title text, call to action copy, buttons versus links, number of screens in signup etc) and they let the data decide the direction of the product. They’re not driven by philosophy, but by the scientific method.
Examples of companies that take this approach include many of the standout viral growth companies of the current generation, including RockYou, Slide, Plaxo, LinkedIn, Facebook, Tagged, Flixster and many more. (Disclaimer: Lightspeed is an investor in both Rockyou and Flixster)
For game design, the equivalent would be the trend towards metrics driven development.
Raph Koster wrote up the Master Metrics presentation given at GDC by Dan Arey and Chris Swain from USC. They talk in part about Microsoft’s approach to metric driven design:
MS User research group… using heatmaps. When a project goes thru MS, 3 people from the user research group assess the gameplay experience. They are a real thought leader in this area.
1. usability testing – can user operate software
2. playability, does user have a good play experience
3. instrumentation, how exactly is the user playing, using tracking software
This is the first year that they are talking about this stuff publicly, the Wired article (Ed Note: Halo 3: How Microsoft Labs invented a new science of play), etc. Here’s a picture showing black dots on the Halo map. So dense on deaths that there is no info. So let’s tie it...