Ofcom study offers skewed take on mobile phone satisfaction

Source: Ofcom

The study - entitled the "Consumer Mobile Experience" - was carried out using a novel research method, with network performance and satisfaction levels being taken directly from 6,632 volunteers via a mobile app they'd downloaded.

Using this app, Ofcom were able to glean a variety of instructive findings, from differences in 3G and 4G performance, to how network congestion results in slower download speeds.

However, the report has been criticised by some analysts for using only "self-selected" data, which may have distorted the results, insofar as only certain kinds of mobile users were likely to voluntarily participate.

Peak performance

Perhaps the most significant fact to emerge from the report is that reliability levels of the UK's mobile networks are generally very high, although there's unsurprisingly some variation between urban and rural areas.

For example, less than 1% of voice calls made in the UK are dropped because of loss of service, with 90% of customers being satisfied with the performance of their network when it comes to making a call.

Similarly, more than nine out of 10 mobile downloads are successful, with the individual rates of 3G and 4G services showing similar levels at 92.4% and 95.6% respectively.

That said, such percentages waver slightly when placed in a rural setting, as shown in the diagram below:

Source: Ofcom, Consumer Mobile Experience, June 2017

The rural-urban divide

Yet even more pronounced than the rural-urban differences in voice call satisfaction were the differences in overall satisfaction:

Source: Ofcom, Consumer Mobile Experience, June 2017

Aside from being consistent with past research on rural customer satisfaction, what's interesting to note here is that the jump in satisfaction levels from 3G to 4G customers is smaller for rural areas than it is for urban.

With urban customers, their satisfaction increases by 10 percentage points when they move from a 3G to a 4G tariff.

However, for rural customers, the jump is only 8%, even though the proportion starts from a lower base of satisfaction and should therefore be more accessible to an increase.

This suggests that 4G networks in rural areas just aren't quite dense or widespread enough to provide the kind of boost to download speeds that might be expected.

As such, rural customers might need to think twice before switching to a 4G contract, or at least wait until the UK's operators have fulfilled their coverage promises and made rural mobile data a bit more reliable.

Wi-fi trumps 4G

They'd also do well to reflect on another one of the report's major findings, which is that even customers with a 4G contract use wi-fi more than their mobile signal to connect to the internet.

Source: Ofcom, Consumer Mobile Experience, June 2017

As the above chart reveals, 4G customers use wi-fi to connect to the web 69% of the time. Just as interestingly, when they do connect to a mobile broadband network, the limited availability of 4G means that they often connect to a 3G or even 2G network instead.

Perhaps this would explain why even 4G customers in urban areas aren't always satisfied with their mobile service, with only 73% either "fairly" or "very" happy.

However, it's at this point that a word of caution is needed, since certain analysts have criticised Ofcom's research for not being "scientific" enough.

For instance, RootMetrics - who conduct their own regular tests of mobile network performance - have said that the report "doesn't come close to painting a true picture of mobile performance in the UK".

Their general manager for Europe, Scott Stonham, elaborated on this verdict by adding, "Crowd data is inherently flawed from a scientific perspective because it relies on self-selecting - rather than random - data collection".

That Ofcom's report relies specifically on users who were willing to participate - rather than on a random selection - suggests that its sample isn't truly representative, and may include a disproportionate amount of customers who either have a below-average or above-average service.

Conflict

Also, RootMetrics' criticism of the reliability of the report is strengthened by the results of another recent Ofcom study, called "Comparing Service Quality".

With regards to mobile customers, it put the "satisfaction" level for operators as whole at 92%, 23 percentage points higher than the corresponding rating provided by the Consumer Mobile Experience report.

Source: Ofcom, Comparing Service Quality, April 2017

If nothing else, this contradictory result shows how the multiplicity of different reports can sometimes have the opposite effect of helping the customer make informed decisions.

Instead, their sheer number and conflictual nature can have a tendency to confuse people, so that in the end, we might possibly be advised to trust our own impressions of our network's quality, at least before those of anyone else.