ARE YOU LOSING OUT ON VALUABLE CUSTOMER INTEL? - PART 2: How To Choose The Correct Type Of Survey Q


In my last blog post, I shared the tips I’d picked up about writing survey questions. That got me thinking about writing in general, its purpose and the myriad styles and approaches that different writers use.

Most writing…like journals, stories and even this blog…informs or provides information. However, surveys, like exams and some types of letters, ask for information, rather than give it.

When you create writing that informs, you can start at the beginning and go free form or start at the end and document the journey. However, when your writing asks a question, you can’t know what the end will be.

And if you start at the beginning you risk drifting off in the wrong direction…which is a longwinded way of saying when you approach writing a survey or any enquiring piece, you need to start in the middle.

START IN THE MIDDLE…

When I say ‘start in the middle’, I mean you need to start by thinking about how you want to analyse your data.

Do you need to find out how many of your respondents are male or female? Or do you need to know more than that…for example, how many of these female respondents work full-time?

Most likely you’ll want to know how your respondents feel. Do they think a product is poor or excellent? Do they agree or disagree with a particular statement?

Sometimes you may even want to channel your inner stats geek and find out whether there is any correlation between the different responses. For example, to what extent does the number of sales staff affect the total number of sales?

Where is the sweet spot…in terms of numbers of sales people? Or…to put it in slightly geekier terms…at what point is the correlation between sales and staff at the highest and most efficient point?

The consummate stats geek will probably get a twitchy urge to do some regression testing which would enable them to predict the behaviour of a dependent variable (e.g. “sales”) when an independent variable (let’s say “GDP”) fluctuates.

All this probably sounds a bit daunting, and probably, if you’re planning on conducting regression testing for the first time, you should probably get someone trained in research methodology to give you a helping hand.

But for those of you without a data scientist close by, I’m going to give you a leg up. I’ll start at the beginning and by the end of this journey you should have a fair idea of how to select your questions…maybe not like a virtuoso…but at least with insight and confidence.

Have patience, grasshopper…

TYPES OF QUESTIONS

The first thing to know is that there are two different categories of question, the open ended question and the closed ended question.

Open-Ended Questions

Open ended questions don’t give respondents a choice of answers; rather the respondent is invited to write an answer in their own words. These answers require individual analysis and coding and are especially useful for exploratory research.

Open-ended questions can also reveal unexpected ideas, which means that the extra analytical effort required is often rewarded.

Closed-Ended Questions

Closed ended questions are questions that include a set of pre-determined responses or a scale.

They’re not standalone questions but a question/answer package, which is used when you have a fair idea of the likely answers. It is possible to use both styles in the one question as shown in Example 1 below.

In this blog I intend to deal with closed ended questions as these are the ones most used by online survey tools and the ones that’ll get you into trouble if you try to analyse the data in a way that the data doesn’t support.

CLOSEED-ENDED QUESTION TYPES

There are essentially 4 types of closed ended questions.

These ‘types’ package together a question/statement and a pre-determined set of answers also known as a ‘scale’. Each question type generates data, which can be analysed up to a certain level of complexity, with nominal data being the most basic and ratio data being the most sophisticated.

Just to make it interesting, these questions can be linked, layered and combined in any number of ways. But for now, let’s just jump in at the basic level of measurement – Nominal.

Nominal /Categorical Questions and Measurement

Nominal questions generate data that is the simplest to measure and compare. They ask a respondent to categorise themselves or their preference for certain products and services.

This question type includes dichotomous questions (questions that have only two possible answers e.g.: Yes/No or True/False).

The pre-determined answers to nominal questions have no particular order of importance (known as rank order) and are basically names, labels or identifiers.

The data collected from these types of questions support the most basic analysis and are really only sufficient for counting responses, allocating percentages and performing basic cross tabulations (comparing the data between two questions).

The following are 2 examples of nominal questions sets.

The data analysis on nominal questions might be simple, but writing the questions still requires the care mentioned in my previous blog. It’s worth re-iterating that precision, objectivity and knowing your audience are key.

Nominal questions may require a simple answer, but they can also cover sensitive topics and sensitive questions need care.

A question such as ‘What is your gender?’ may seem straightforward, but with our increasing awareness and understanding of LGBTQ identities the responses will need to be exhaustive and may need to be framed as follows:

‘Other’ could also be replaced by ‘Prefer not to answer’.

Knowing your audience means you know which response would be more sensitive and therefore easier for your respondent to answer.

Keep in mind that sensitive questions, unless they are absolutely essential, are best left out of surveys.

Ordinal Questions and Measurement

An ordinal question gives the respondent the chance to choose an answer from a ranked or ordered range of answers. It’s useful when you want to know how respondents feel about an issue or product.