Mobile optimisation is the pervasive solution where an online survey is rendered on a mobile screen using a function in the survey software. While mobile optimisation allows respondents to access online surveys via mobile devices, it does not necessarily preclude sample bias if the survey design did not apply mobile-first strategies. High dropout rates and lower response rates due to surveys designed for PCs rather than mobiles may result in skews in the data or poor data quality.
From February to April 2016, TEG Rewards ran 200 online surveys that were either mobile optimised or designed using mobile-first principles. All surveys were up to 15 minutes long and had a minimum of 100 completes to be considered in the analysis. The following are insights based on response rates by device, demographic and survey design.
Response rates by device
On average, completion rates by device were as follows: 63 per cent desktops, 10 per cent tablets, and 20 per cent mobiles. In contrast, the completion rates for surveys that utilised mobile-first principles, conducted using mobile devices, ranged from 25 to 37 per cent, and the dropout rate was less than desktops: eight per cent on desktops versus three per cent on mobiles.
Typical comments included:
- ‘The layout wasn’t that good compared to other surveys.’ Female 25-29, 5 min, mobile-rendered
- ‘Awkward to complete on mobile phone.’Male 20-24, 5 min, mobile-rendered
- ‘Great survey that worked well on my Windows phone.’Male 40-44, 5 min, mobile-first
Response rates by demographic groups
For a nationally representative sample, the difference in the dropout rates between PCs and mobiles (both mobile optimised and mobile first) was not significant. Surveys targeting 18-39s resulted in more than 20 per cent of the sample completing via mobile, with a lower dropout rate compared to desktop completion. Surveys targeting mothers, particularly with those with babies and toddlers, resulted in more than 25 per cent completing the survey via mobile, with no dropouts, compared to an average five per cent drop-out via desktop.
Surveys targeting those aged 60 years and over resulted in seven per cent completing via mobile with a 14 per cent dropout rate, compare to 71 per cent via desktop and a dropout rate of only three per cent. This highlights that mobile is not the best for all demographic groups.
Comments included:
- ‘Love this survey! More like this please!’ Female 20-24, 5 min, mobile-first
- ‘Easy to navigate on phone. Questions were simple to answer.’ Female 35-39, 5 min, mobile-first
Response rates by survey design
Surveys designed with simple scales (single or multiple response) had 25 per cent completion via mobile and two per cent dropout rate. Surveys with grid questions or that required scrolling, on average took 14 minutes to complete, with a 10 per cent completion and 20 per cent dropout rate via mobile.
Comments included:
- ‘Simple and easy to read on my mobile. No issues with pages not loading or cutting off some of the questions options for a change!’ Male 30-34, 10 min, mobile-first‘
- The survey was really annoying on my mobile – had to slide right-left all the time.’ Male 25-29, 5 min, mobile-rendered
- ‘Great content for the survey but it was way too long.’ Female 20-24, 5 min, mobile-rendered
- ‘Quick and simple – thanks!’Female 30-34, 10 min, mobile-first
- ‘Good questions – not many “emotional” questions – it’s only utilities!!’Male, 35-39, 5 min, mobile-first
In conclusion
In terms of overall performance, mobile-first surveys, by far, have the best response and completion rates across all devices. Respondents rated all 200 surveys based on their experience and on average; mobile optimised surveys received a rating of 3.9 out of five compared to mobile first surveys rating 4.4. By comparison, on average, PC-only surveys received a rating of four out of five.
Top 5 tips for mobile-first surveys:
- It can take 1.5 to 2 times longer to complete an online survey on a mobile device. The Mobile-first surveys should be completed within two minutes, allowing for up to 10 questions.
- Be brutal and only ask what is necessary and if possible, append what you already know.
- Keep questions text succinct. Instead of asking, “Which of the following would you say best describes your ultimate holiday destination?”, use “Ultimate holiday destination?”
- Limit scrolling by keeping answer option texts as short as possible rather than using wordy descriptions.
- Limit the number of options. Grid questions should be avoided and if it must be used, consider each option as a question.
Lisa Salas, head of marketing and operations, TEG Rewards
Recent Comments