The modern world is dominated by statistics. This will surely come as a surprise to no one. Statistics control countless aspects of our modern lives, and are applied as a matter of course to almost everything we do online. They are the backbone of many aspects of 21st century life, without us necessarily being aware of it. Stats can answer many questions when applied and interpreted with care, and many aspects of society would simply break down without them. But is statistical analysis always the best way to answer every question it is applied to? I’m not so sure.

Statistical analysis, the required study of which many of us barely lived through in school, is based on three things: practice, percentage and comparison. Numerous algorithms can be applied to these three parameters to produce many different types of results, depending on the application. And this works very well for certain things, where a purely mathematical calculation and prediction is appropriate. But there are some areas where pure maths, while yielding useable results, might not produce the best, or most accurate answer. In these cases the application of some knowledge of human psychology might be more appropriate. Let’s take a look.

In the area of sales and marketing statistics are relied on almost exclusively to determine almost everything about selling products. But in the entertainment industry, following statistical advice produces a failure as often as a success.

In music for example, how many times have you asked yourself, “Why did they release THAT song as the single? It’s the worst song on the album.” The reason of course, is that statistically, people have bought music produced in that style. The record label is not really concerned with the content or quality of the song because that is intangible, and the statistics algorithm cannot measure it. More obvious aspects like style and rhythm can be quantified by maths, and can therefore form the basis of a marketing strategy.

In film, the same applies. The reason there are so many formulaic sequels and blatant clones being made, is that audience response to the style of film can be measured statistically, whereas subtleties like the quality of plot, acting and character, cannot. Trust me, every producer wants to make good films, but they HAVE to make profitable ones. With huge amounts of money at stake, most are not in a position to take a chance. When a truly great film does appear, you can be sure that it fits into a statistical marketing niche that probably has nothing whatsoever to do with its success.

I believe that both the music and film industries suffer from their reliance on statistical marketing strategies.

In the case of music, releasing a weak song as the lead single can unfairly bias both the public and critics against an album. This can also even have the knock-on effect of undermining the artist’s confidence, causing them to question their direction and motivation. Would it not be much better to examine the psychology of why people like certain types of music, why they react to certain melodies and harmonies and how rhythm effects the body in different ways? Music is both unfathomable and intangible – this is what makes it wonderful – but there have been many scientific studies on how music effects the mind and body. These document the psychological processes that produce pleasure when listening to music. As the appreciation of music is almost completely subjective, it seems to me that a scientific study of an actual psychological process might be a better platform to base a marketing strategy on than a mathematic algorithm, or some A&R guy’s personal opinion. Just a thought.

In the online worlds of social networking, retail sales and community collaboration, statistics are used almost everywhere to create user profiles. These profiles can contain sales history, product preferences and ratings, amongst other information. For the most part these profiles provide useful information, both for the retailer, if applicable, and the end user. But as in the examples above, sometimes the use of statistics in profiling doesn’t actually produce the correct results.

For example, let’s look at the music-based community sites, iLike and Last.fm. Both of these examine your music listening habits and form profiles based on these, which can then be shared with the community. They both have clients that examine your local music listening habits and use this information to form part of your profile. There are many similarities between the two, but iLike uses a singular method that paints a pretty good picture of how statistics-based profiling falls down.

iLike allows you to download an application that puts an iLike pane onto the iTunes player window. The first time you launch the iLike pane – and only the first time, it’s important to note – it goes through your entire iTunes library, catalogues every track and creates your profile based on this. The statistic it uses to establish your favourite artists is the number of songs you have in your library by any given artist.

The problem with all this is that the number of songs by any given artist does not at all designate MY favourite artists. It is possible, in my case at least, to have an enormous number of tracks by artists I seldom listen to. For example, I have 21 albums by one artist I that I rarely listen to. This artist is now listed in my iLike profile as my very favourite artist, and friends are being suggested to me based on this. These favourites are permanent and cannot be removed! My actual favourite artists don’t even make the list because apparently I don’t own enough tracks to reach whatever threshold has been designated in the software.

If I’m going to build a profile on a social site I want the profile to reflect what I like right now, and frankly, this may change at any time. Unlike iLike, Last.fm builds your entire profile based on what you listen to, either on the site or in iTunes, and on tracks you specifically designate as “loving”. This is actually psychological profiling because it is basing it’s statistics on what you actually do, not on random quantities. Amazingly, iLike tracks what you play as well and also allows you to “love’ songs, but none of this information is used to build or alter your favourites, the very thing that is establishing your identity in the community! I find this incredible.

So lets move on to some statistical success stories. The “People who bought this also bought this” recommendation engines used by Amazon, iTunes, etc. work extremely well. They work because they guide you in a direction and then let you make up your own mind. They are not establishing absolutes, they are only offering suggestions. And more often or not these suggestions pay off. And this is because an element of psychology is being used in encouraging you to do what someone else has already done.

And now, at the risk of sounding like a complete Apple fanboy, having already mentioned iTunes several times, I would like to very briefly talk about the single greatest psychological marketing success story. Countless people in the blogosphere have speculated for years on how Apple manages to consistently come up with innovative products that no one else seems to think of, and sell them to consumers who previously didn’t know they needed that product. The answer is so simple that it amazes me that I’ve never heard anyone say it.

Apple applies psychology when developing products, and never statistics. They do not look at markets and see what people are buying, they look into people’s heads and see what they want. There is no mystery about it. Anyone can do it, but most large companies rely on traditional statistics based marketing strategies because they are safe.

So there you have it. Once again I have tossed some stuff up in the air and let it fall, drawing few conclusions myself. Perhaps it will get people thinking, or perhaps give them another reason to think I’m a jerk. Somebody always does. Have fun.