Surveys are a great way to kick start discussions on various topics, from company policies to social issues, but the process of creating an engaging survey is actually a lot more complex than we realise. Not only does it involve formulating thought-provoking questions and statements, it also requires good organizational skills and a solid command of the English language (or any language that you’re writing in). In other words, the process of crafting a good survey isn’t just an art, but a science as well. Here’s where OPPi comes in to help.
If you have a survey to craft or just want to ask better questions to get better responses from your participants, here are seven tips for making this daunting task a lot more manageable.
Depending on the target audience of your survey, the language and tone of your questions will differ. As much as we should try to craft questions in proper English, it’s always best to speak the language of your target audience to enhance comprehension and prevent survey abandonment due to confusion.
For instance, if we’re intending to put out a survey among the low-income community in a certain block of flats to better understand their needs and the efficacy of aid that they’re receiving, we would avoid using jargon or big words, and break each question down into more digestible statements. So instead of asking “To what extent have the monthly monetary handouts been effective?”, we would ask “Is the monthly allowance useful?”.
Of course, if you’re intending to survey a very specific group of people about a niche topic (like the use of technology in surgery, or the future of digital performances in theatre), using jargon might be unavoidable. In this case, we recommend providing a standardized definition so that every participant will be on the same page in understanding the language used in the survey. The same goes for abbreviations — always type them out in full to avoid any misinterpretation (i.e. World Wide Fund for Nature instead of WWF).
The choice of words used and the way a single question is phrased can alter the meaning of an entire statement. It can also reveal the underlying biases of survey organisers and shape the responses of participants, whether intentionally or unintentionally.
In an example taken from Pew Research’s 2005 study, 51% of participants said that they favoured “making it legal for doctors to give terminally ill patients the means to end their lives,” but only 44% said that they favoured “making it legal for doctors to assist terminally ill patients in committing suicide.” The use of the word “suicide” in place of “end their lives” adds a slight negative connotation to the statement, and evokes a more emotional reaction from participants. It might even be seen as a more offensive or biased way of phrasing the same question, and subconsciously influence participants to vote against the statement.
Apart from being objective, there are a number of other common phrasing problems we should avoid:
Double-barrelled questions are questions that touch upon one or more issues, but only allow for a single answer. Often, we tend to craft questions or statements like these unknowingly because our minds work so fast that we tend to lump related issues together for the sake of efficiency. However, it can be very difficult for participants to pin a definitive answer to these questions, and also run the risk of being misinterpreted.
For instance, instead of asking “How effective do you think the government’s environmental and social policies are?”, we should split the question into two so that participants can provide separate answers—one for environmental policies and another for social policies.
In that same vein, when it comes to phrasing questions, statements, or writing anything in general, double negatives should always be avoided as they are likely to result in confusion among participants. For example, instead of asking “Do you think that we shouldn’t disallow same-sex marriage in Singapore?”, the statement can be shortened and simplified to “Do you think that we should allow same-sex marriage in Singapore?”.
Sometimes, in our bid to add context to a particular question, we end up forming leading statements instead. These statements might be seemingly innocuous, but they actually manipulate participants into responding to the question in a certain way.
For instance, if participants are given the statement “85% of Singaporeans agree thats sex work should be recognised as work” before being asked “Do you agree that sex work should be recognised as work?”, participants would be a lot more likely to agree as that has been shown to be the majority or more popular opinion among Singaporeans. This leading statement however, is not crucial to participants’ understanding of the question, and should thus be omitted entirely to prevent biased results.
There is a difference between opinions and factual statements, which is crucial when designing a survey. A factual statement reflects something that is proven to be true, while an opinion is based off of a certain belief or feeling, and is neither right nor wrong. When presenting statements to participants in a survey, we should always avoid factual statements as they will not result in any meaningful or actionable insights.
For instance, in a survey on participants’ perception and preferences towards local hawker food, a factual statement could be “Do you agree that chicken rice is commonly found in local hawker centres?”, while an opinion could be “Do you agree that chicken rice is one of the more appetizing options in local hawker centres?”. For the first question, there isn’t much of a debate that can be had because it can be proven that chicken rice is commonly found in hawker centres, whereas for the second question, a discussion surrounding the different opinions about the taste and presentation of hawker centre chicken rice can be held as everyone would have different palates and food preferences.
Consistency is often said to be the key to success, both in life and in crafting effective surveys. In this case, what we mean by consistency is using the same terms instead of alternating between synonyms throughout your survey.
Say you’re conducting a survey among parents regarding the screen time of children. If you choose to use the term “children”, then stick with that throughout the survey. If you choose to say “kids” instead, then stick with that.
A consistent survey not only reads more professional, it also avoids confusion, especially when using bigger terms or words.
Context is always important in helping us to better understand a given question or statement. Thus, how you organize questions in your survey is crucial as the previous question sets the context for the next, which could influence the way participants respond over the course of the survey.
According to the Harvard University’s Programme on Survey Research tip sheet on question wording, a good survey should:
- Open with an introduction to set the overall context for the survey.
- Start with general, broader questions that are easy for participants to answer.
- Have more sensitive questions placed towards the end of the survey.
- Have a randomized order of questions if questions asked are phrased very similarly to improve data collected.
Introductions to surveys need not be very lengthy, though it would be good to touch on the driving force behind the survey, and what you hope to achieve with the results of the survey. You could also mention a bit about your organisation or company and why this specific survey topic matters to you.
Broader questions serve as good opening questions as they ease participants into the flow of the survey, and also help provide additional context for the questions to come. For instance, in a survey about environmental policies in Singapore, asking general questions about climate change and the increase in attention towards the environment in parliament would get participants thinking and prime them to better respond to more specific questions about individual policies in place later on in the survey.
Sensitive questions should be placed towards the end of the survey after a sort of rapport has been built through participants answering broader, more generic questions, just so that they will feel more inclined to share their thoughts and perhaps even elaborate on them. Much like a conversation with a friend, we should always open with some form of small talk first before diving into the deeper, heart-to-heart topics to avoid scaring the other party off.
When asking a series of very similar questions, it would be good to include them in a more randomized order so that participants won’t tire of seeing the same phrasing consecutively, and will be more inclined to read the question thoroughly as opposed to skimming over it. It might seem like a small detail, but think about it, if you see a question that starts with “To what extent do you agree that the government’s policy on…” five times in a row, you’d stop paying as much attention to the actual question, wouldn’t you? This is also why with OPPi, we introduce multiple-choice questions after every three statements so that participants can take a short mental break from more thought-provoking statements.
For closed or multiple choice questions, it’s always best to present participants with all possibilities. According to UX Booth, the number of choices should be kept relatively small, but include all possible choices and be balanced both ways. Options like “not applicable” or “prefer not to answer” would be good for more sensitive questions, while an “others” option would be great for learning about alternative choices that might not have been represented.
Take, for example, a question about total monthly household income. The options given should range from low (perhaps $500-$1000) to high ($8000 and above). As financial circumstances might be a sensitive topic for certain individuals, a “prefer not to answer” option should be presented if these results are not a crucial factor in the analysis of the poll results. An “others” option would also be helpful to accommodate any participant that might fall outside of the given categories.
Survey fatigue is what happens when participants become bored or tired of responding to your survey, and either abandon the survey midway or respond to the remaining questions without really thinking it through. This is a very real problem that both survey participants and researchers can relate to, which is why it’s so important to keep your survey short and sweet.
If your survey topic is a more interesting or unconventional one, you might still be able to push it a little further and have a slightly lengthier survey, but if your survey is a very intellectually taxing one, we recommend paring it down and carrying out follow-up surveys later on if needed. Don’t forget, a few complete responses is always better than many incomplete ones. In OPPi, we constantly advocate that each engagement session should generally be kept within 30 questions.
It might not seem like a very big deal, but pre-testing surveys with a small sample size of your target audience can be very helpful in helping to sieve out holes in the survey that you might have missed. Approaching a survey as a designer and as a participant is very different, and often, when we spend too much time attempting to craft a good survey, we tend to lose sight of the small details that participants would be more attuned to.
For example, when setting a survey, we would have all of the necessary contextual information and prior knowledge of the given topic, and might thus not be as detailed in our explanation of the questions. However, when participants come into the survey with a fresh set of eyes, the way that they interpret the questions will determine how successful our wording of each question is. If participants express confusion after the survey, we are then able to edit our phrasing or the order of questions before sending it out wholesale to our target audience. And if participants react positively to the survey, then congratulations, you’re ready to send that survey out into the world of your target audience!
To sum up, the process of crafting a solid survey is not a short or easy one, and it takes a lot of practice and tweaking along the way. This list is by no means an exhaustive one, but they are some of the more important things to take note of when crafting questions or surveys of any kind. But always remember, a clear, unbiased survey will reap the best insights.
At OPPi, we strive to help you conduct complex conversations and measure what really matters. Contact us now or sign up for an account with us to try out our AI-powered engagement methodology today.
How do we make the right decisions at the right time for the right issues and people?
In the time of home-based learning and online games, how can we protect our children from the dangers of the cyber world?
In the battle to eradicate gender inequality, where do we begin?
Find answers to common questions about OPPI.
With the free Pathfinder plan, you can ask a maximum of 20 questions per engagement session. There’s no limit with paid plans.
At OPPI, ensuring your safety is our utmost priority, reflecting our commitment to creating a secure and trustworthy environment. We are proud to hold the Cyber Essentials certification, issued by The IASME Consortium Ltd, which underlines our adherence to rigorous cybersecurity standards. This certification (ID: 8565506d-ded4-4983-9e1d-52d0c7b421b5) is a testament to our proactive stance on security, continuously updating our policies to safeguard your information and peace of mind.
OPPi starts analysing data immediately, but that information only becomes meaningful once you’ve heard from a few respondents. You’ll be able to view an analytics report on your session once you’ve had seven responses.
Yes. All data in your free account will be upgraded to the premium account.
Yes, if you have a Trailblazer or Explorer plan. If you have a Pathfinder plan, you’ll need to upgrade to view the raw data.
Yes, you can do that. We’ll need to customise your package and for that you’ll need our Explorer plan. Contact us so we can explore what you need and quote you.
Lots! Stay up to date by following us on Facebook, X, Instagram and LinkedIn, or send any suggestions or feedback to hello@oppi.live.
