Posts Tagged ‘Statistics’

Market Research on Trial: Do you plead science or art?

Published on May 15th, 2012 by Jack

Historically, disciplines have been placed into the categories of art or science. Recently, Simon Mansell of TBG Digital claimed that advertising has moved from being an art to being a science. Market research – with its underling principals being held in the social sciences – it could be argued, is traditionally a scientific discipline. However, is market research now going in the opposite direction to advertising – turning from a science into an art?

The Case for Science

The fundamental case for market research being a science revolves around how many traditional scientific principals are still utilised within it. Traditional statistical methods are still a mainstay in any quantitative research project – correlation, key driver analysis, data reduction – all rigid, mathematical processes.  Does the continuing fundamental role of such processes mean research is defined as a science?

Whilst scientific thinking is undeniably rooted in market research’s past, highly scientific disciplines are coming to the fore as potential ‘big methods players’ of the future – marketing sciences and neuroscience for example. Is this a sign science is here to stay?

The Case for Art

Market research output has recently undertaken a sweeping new direction. Infographics and data visualization are being used more regularly to tell research stories. Compared to research of old, these stories are far more visually appealing and require an artistic mind to construct – indeed, many research agencies now employ graphic designers. Is it a case of designer in, scientist out?

The traditional, science based research journey – simply put – goes from objectives to method to analysis to conclusion. However, research is now a lot less linear as we refine methodology, alter concept/NPD offerings, feed back to marketing and product teams, re-test hypotheses and then generate ideas. This journey requires far more intense designing and increased creative thinking to reach the end goal. So, are design and creativity – concepts one associates far more readily with art vs. science – a sign researchers are becoming artistic?

The Verdict

Science in research is likely here to stay. Scientific thinking will feasibly remain at the heart of researchers’ ideas, whether this is statistical thinking, social psychology or cognitive and behavioural sciences. That said, artistic forms of designing and reporting research have bettered market research output greatly in recent years. Methodologically, both science and art have a great deal to offer going forward with disciplines such as neuroscience and semiotics. The reality is science and art will likely become more intertwined in the future of market research – allowing market researchers to get the most from both areas. The jury, therefore, you could say, is out…

 

Research Resolutions

Published on Jan 7th, 2011 by Jack

The findings of the RSM “State of the Industry” survey reported out at the end of 2010 may have left some researchers feeling anxious about what 2011 may hold for the industry – a continuation of market researcher’s track record of being “consistently wrong”, prolonging our over optimistic outlook and the forward momentum of the much feared  DIY researcher. Conversely, RSM’s findings could be an excellent source for some 2011 research resolutions.

Method Matters

It is a mis-conception outside of the research community that anyone can put a questionnaire together, acquire some respondents and generate some data. As researchers will know, asking the appropriate questions to the correct people is not as easy as this. In years previous to 2011, the complexity of these fundamental research skills may have been under sold to research buyers. These buyers now take such methodological fundamentals for granted. Now in 2011, this should be the year where researchers start to communicate how integral (and complex) these underpinning research principals really are to the success of a study. By doing so, the mis-conception regarding these skills can be turned into an appreciation of their value.

Research IS Sexy

Hans Rosling recently stated that statistics is the sexiest subject in the world. I admit, statistics are sexy. However, even a quantitative researcher has to admit that the creative, artistic world of qualitative research, with its projective techniques and colourful schematics, is far superior in this domain compared to quantitative work. What does this give market researchers overall? A titillating industry. Maybe 2011 should be the year we position market research in this way. Presenting quantitative data in colourful, interactive ways combined with fully utilising the creative freedom allowed in how qualitative findings are shown whilst telling a story to fuel business decisions can turn the personification of research from that of an old man drawing a black and white bar chart into an artistic, original thinking twenty something.

Criticism Where Criticism Is Due

DIY research is one of MR’s biggest threats. This was highlighted by numerous articles in industry publications in 2010. 2011 needs to be the year where these criticisms are backed up by evidence and the anti-DIY perspective laid out to research buyers and clients. By doing this through parallel surveys and such the like we can show that the research agency products available today are of a higher quality and can add greater value vs. DIY offerings.

On your bike…….or on your calculator

Published on Jun 23rd, 2010 by Jack

On the 2nd July three members of the Northstar London office (Matthew, Jack and Chris) will be partaking in the 2nd Marketing Industry Triathlon relay race (see old news for further details). The organisers of the event have billed it as “a great networking opportunity for the marketing world to unite in healthy, fierce competition”. Please note the highlighted words – fierce and competition. To this end, Team Northstar will be looking to climb as far up the overall rankings as they can. I know what you are thinking………what on earth does this have to do with statistics? In the marketing world, statistics are commonly used to drive policy and decisions with a view to gaining a competitive advantage over rivals. Why not use the same statistical methods to gain a competitive advantage over our industry peers in a sporting context?

The following analysis was compiled on the basis of the results from last year’s Marketing Industry Triathlon relay race. Our aim is to find out which of the three triathlon disciplines (swimming, cycling and running) is key to our end position and thus the discipline in which we need to optimise performance to keep ahead of the pack. All of the analysis is based on overall finishing position and the individual (not cumulative) positions within the three disciplines.

Firstly, we have to identify if there is a relationship between the overall finishing position and the individual positions in the swim, cycle and run. Correlations run between overall finishing position vs. individual swim position/individual cycle position/individual run position yielded the following results:

There is a strong relationship between the overall finishing position vs. the finishing position(s) in all of the disciplines. However, the relationship between the overall finishing position vs. the position in the cycling leg is significantly larger than the relationship between the overall finishing position vs. swim and run positions.

So we now know that there are relationships between the finishing places in the individual disciplines and the overall finishing position, but surely it would be better to know the importance each discipline has on where you finish? Yes it would, and on that note please cue a Shapley Value regression analysis…

A Shapley Value regression on the importance of the swim/cycle/run position against the overall finishing position derived the following results:

This shows that the position on the cycling portion of the triathlon is considerably more important in determining the overall finishing position than the position in the other two disciplines, essentially meaning that the cycling leg of the triathlon is where the race will be won or lost (hopefully the former!).

That said, this is not a foregone conclusion. Many triathletes talk about the “4th discipline” within a triathlon – the transitions – i.e. going from the swim to the bike and the bike to the run. Throwing these into the Shapley Value regression mix as it were provides the following output:

Whilst the length of time spent transitioning is relatively short compared to the time in the water or on the track, it still goes a fair way in determining the overall finishing position, soaking up variance mostly from the cycling element of the triathlon.

So what does all of the above number crunching mean for Team Northstar on the 2nd July? Well, a “tri”-ad of tips based on the above would read as follows:

  • Performance within the cycling leg will be the key driver for triathlon success
  • That said, this is not to detract from the roles of swimming and running in our end position as both yield a significant degree of importance with regards to the overall finishing position
  • Relay triathlon is a team sport, with our performance in the transition zone accounting for 19% of importance in determining our overall finishing position

Constant Sum Scaling

Published on May 24th, 2010 by

Used to measure the spread of opinion, constant sum scaling involves the distribution of a pre-designated amount of points to a series of characteristics. Analytically, the key measure derived from this data is the mean amount of points designated to each characteristic.

Basic Statistics – I beg your pardon

Published on May 8th, 2010 by Jack


The remit of quantitative research analysis spans from the basic (pie charts representing responses to a yes/no question) to the complicated (logistical regression). The trick for researchers is to make the complicated as understandable as the basic. Three key factors (no pun intended) which can contribute to making your statistics understandable and digestible are; audience, explanation and actionability.

Audience

Essential to any research is the audience to whom it is intended, and even more so when advanced statistical analysis is involved. Prior to going into your presentation and talking about your mind blowing latent class segmentation model, REMEMBER THIS, the people receiving it may not have a clue as to what you are talking about. Generally speaking, statistics intimidate people, and as a result they keep their distance! Sophisticated number crunching is not supposed to bog down the decision-making process but rather be a catalyst for it. In the end, relationships and meaning found out with complex statistical techniques are of no use as a catalyst unless they are communicated properly to the audience. When trying to find a method to communicate your statistics, consider this, God speaks to all of us through crosstabs!

Explanation

Explaining complex methods to the layperson can be a challenge for researchers, but is best done by a single slide outlining the theory of the method as opposed to the mechanics, that is if you don’t want to lose your audience. For many audiences comprising of CEOs, Marketing Directors, Product Development Managers and the like, it is the ‘spirit of the law’ that they are after in order to adequately conceptualise the topic.  And if you can show them how result translates into percentages, you will reinforce and strengthen your argument.

Actionability

Now that your audience knows what your analysis entails, it is imperative that you translate it into real and tangible outcomes, i.e. what it means for their brand.  For example, telling your audience the correlation between customer service and customer satisfaction is 0.7 will generally mean nothing to them. Telling them that if they increase customer service standards by x amount customer satisfaction will increase by y amount is much more meaningful. Key ways to do this include; not presenting unessential elements of your analysis which may confuse the audience, where possible reconfigure results into meaningful output such as percentages, relate the numbers in your presentation to the brand under investigation, add in contextual information from secondary research sources (articles, books, etc.) and make the slides showing your statistical wizardry aesthetically pleasing as well.  In the end, many people are visual learners so a well-laid out and supported story will go a long way with clients.