Interview with Dave Page, CEO, Actual Experience – specialists in Digital Experience Quality
For several years, I have been writing about how broadband services need to become more outcome-led. Thankfully this has become a less lonesome task over time. As such, I had the pleasure of interviewing Dave Page, CEO and Co-Founder of Actual Experience, who are based in Bath in the UK.
They are specialists in Digital Experience Quality, the science and art of measuring and managing complex digital supply chains. Dave kindly shared his insights into an outcome-led future for broadband services.
MG: How did Actual Experience get into the business of Digital Experience Quality?
DP: I personally have 25 years of involvement in enterprise and managed service provider communications. Over that period, I have seen an increasing mismatch between the end customer (the ‘buy side’) and the supply side.
Customers who had bought a service would often complain, whilst the supply side reassured both itself and the customer that the lights are green, so everything is fine, and we’re doing a great job. Ultimately this isn’t true as the buy side is complaining the service is not fit for purpose.
We are leaving the traditional enterprise world where you used only a few applications in a closed environment, and where every component was delivered by a single IT group. Now we have IT plus hundreds of suppliers in the supply chain for your online experience. This fragmentation of supply means each supplier can easily believe they are doing well; yet the buy side still says “it’s not usable!”.
Fifteen years ago I met Prof Jonathan Pitts from Queen Mary College, University of London. He is a “Professor of Queues” and their mathematics. I loved what was is doing as a left-field academic working on the optimisation of networks and applications. He had a novel way of dealing with traffic, whilst the rest of academia dismissed his idea.
His insight was to determine what effect your queue algorithms have on the voice of the customer for real-time audio, which is to say, what is its effect on what someone really hears. By doing this he was able to gain efficiency without degrading perceived quality. This focused on the only thing that’s important – whether the customer thinks it is fit for purpose.
How do you define Digital Experience Quality?
We think of it as the voice of the customer, who is the ultimate arbiter of value. Quality is something that is experienced in the physical world, and we all make assessments of a service’s quality for ourselves. It is not in the gift of the supply side to define the quality – they can think whatever they like.
It comes back to fitness for purpose. Most people wouldn’t buy a Rolls Royce to take their kids to school. That’s a misfit for purpose. They also wouldn’t use a Fiat 500 to race in Le Mans, as that’s not fit for purpose. What is fit for purpose is a subjective judgment of the buy side.
Quality matters because the brand of any product is made up of design plus quality, and all digital products have a quality. Whilst the design component of the brand is the customer journey, the quality is about the consistency and repeatability of every step of that journey. That consistency is both longitudinal for each user – do you get the same quality when the journey is repeated – as well cross-sectional over the whole user base.
I believe that digital products should be like modern cars or phones or TVs: made to a consistent level of quality. That means digital service quality has to be seen as part of the brand’s customer experience management. It is not just a technical network performance measure.
In the physical world we have ways of managing the variability of product quality, such as Six Sigma and Kaizen. This has been transformative in its nature. For instance, it was unimaginable 30 years ago that car makers would use the voice of the customer to optimise their supply chains, whereas that is now considered ordinary.
In the digital world, networks are only a part of the supply chain: there are also devices, data centres, servers, plus layers of applications and databases. These are outside the scope of network, and have to be factored in. We are just beginning that journey towards quality management that is the established norm in the physical world.
What’s the problem that enterprises have with digital experience quality?
The interesting thing is that if you talk to most enterprises, they say they already have a “voice of the customer” for their digital products and services. What they are actually talking about are net promoter score (NPS), customer satisfaction (CSAT), and other ways of asking customers about products.
This data is gathered over large bases of customers, often hundreds or thousands, spending millions of dollars per annum in the process. As a result, the brand thinks it has the voice of the customer. The problem is that these NPS and CSAT scores are historical, delivered well after the fact, on a monthly or quarterly basis.
Meanwhile, you have a grumpy customer for, say, your online banking. You hand the complaint off to the supply side, but it’s far too late: the digital world moves in microseconds. Whatever caused the bad experience and poor NPS has long-since gone. You are then left with a colossal exercise to relate the grumpy customer to its cause.
So brands are trying to do great things, and offer great experiences from great suppliers. They collect NPS data, but can’t connect that voice of the customer to any improvement process. The data is all too late. Relating cause and effect in a fragmented supply chains – where you don’t have visibility of many parts – is practically impossible today.
This is true on consumer services like broadband too, not just for enterprises. Evaluating experience can’t be done same way as the physical world. It’s not just that you can go into the factory to inspect it; you can’t even guess which factory to go to!
Enterprises implement quality concepts such as NPS, but with digital products they are working 6, 9, 12 orders of magnitude in time away from digital manufacturing. Whilst cars are discrete objects, digital quality is continuously manufactured in real-time: each video frame or audio slice is made and delivered in milliseconds! We can’t ask people, it being too slow and too late.
This is why, in fact, we all experience variability, a lack of quality, in our daily digital lives, at home, on the phone and in the office. That’s where Actual Experience comes in.
How would you define the current approach without digital experience quality?
“It is what it is!” – that is the outcome of the current broken approach. It has until now simply been to hard to adopt the quality management techniques that have been proven to be so vital for process flow manufacturing.
Today we see a growing enterprise need for improved voice of the customer scores, and to also take cost out of the supply chain. Yet there is a “breakage” between the enterprise and its supply chain, as you can’t apply those well-proven quality management principles. Enterprises are repeatedly told by suppliers that “there’s nothing much we can do”.
Yet everyday we all experience videos that don’t stream, glitchy audio, banking sessions that stall, and online grocery shopping where the pictures of bananas don’t load. And suppliers think it’s all going great! In my experience, it’s like they have “Jekyll and Hyde” characters: they defend their fantastic job in their supply-side persona, yet readily relate their own grim online personal experiences from the buy side.
It seems that we are in a similar position with online services that American auto makers faced in the 1970s. They knew their products were lacking, and often broke down with no warranty. It was common to have a tool box in the trunk. After all, “it is what it is”, and there was a puddle of oil on every driveway. If you looked up and down the street, the cars were all alike in terms of quality.
Then the Japanese cars came, ones that were both better and cheaper. They had warranties, and bits didn’t fall off. This changed the automotive world. When these cars started to arrive, it seemed to happen overnight, but they had been doing it for a decade. Now the biggest manufacturer in the world is Toyota, a Japanese company.
We’re about to face the same transformation in digital services. I believe that what took 30 years in the automotive world will take more like 3 years in the digital world. We will see brands paying rapidly increasing attention to quality, fitness for purpose, and the resulting voice of the customer business metrics.
Successful brands will take an “outside-in” viewpoint, which is all about the customer. If they don’t do that, they will be beaten by other brands who are more ruthlessly consistent and repeatable. Telling customers “you got what you got” won’t wash.
The network provider doesn’t own the problem: they are but one of many businesses and technologies in a long digital supply chain. Since there are many parts, and they know overall quality is important, they try to optimise it using various tools in the network layer. However, they are optimising it in complete isolation to the service’s value to the brand, and that’s fundamentally the problem.
Network engineers can often be quite brilliant, as networks are arguably more complex than any other factory in the digital supply chain. The systems they manage are both bigger and more distributed than any other piece of digital infrastructure. But they are not being optimised for the things that matter to businesses and their brand values.
If you are BMW, you improve quality for one reason only: to improve the brand and increase sales. Network teams are working with weak proxies for that brand value, and often wrongly optimise the network for experience and cost.
That wasn’t such a big deal in the past when the vertically-integrated network service was the entire application and business. As IT and networks have technically disconnected, they separated the network provider from the end user value that they used to be very connected to. The slow pace of this disconnection has fooled people.
We now face the task of reconnecting them with the digital experience quality idea. This applies to both improving the customer-facing brand and internal staff productivity. The idea of quality is an important enabler, as it allows the supply side and buy side to join in a common language around the only thing that’s important, which is the customer.
The goal is to then optimise around voice of the customer and hence business outcomes, not QoS or QoE per se. This will have a revolutionary effect on the quality of digital products and services.
How is Digital Experience Quality making a difference?
At Actual Experience, we’ve ported supply chain management ideas into the digital world. A good example is the work we did for a global publisher.
They had an internal publishing application system hosted in each country. This publishing application was utterly critical, since it drove the publishing lifecycle. Failure would mean possibly missing publishing deadlines in a ruthlessly competitive industry.
Their aspiration was to save a lot of money going to a global centrally-hosted model. They imagined what might happen if they pressed the switchover button: how can they know they will still meet their publishing deadlines?
In this case the “customer” whose voice we care about is an internal one for the staff. Will the productivity still be there, or will be cost saving vanish from missed deadlines? The supply side was saying “yes, no problem!”.
The Board were in a holding pattern. They wanted to make the transformation for financial reasons, but they couldn’t quantify the risk. We looked at the in-country apps and the voice of the customer metrics they had. We then compared these to a pre-production version of the hosted cloud application.
It was immediately easy to see that most countries would be OK, and they could transfer over with at least the same (if not better) quality. Meanwhile, there were some countries (including one major one) where they would not have been able to meet publishing deadlines.
To migrate safely they wanted it to be “right first time” and “fit for purpose”. If these couldn’t be met, then they wouldn’t migrate. That’s exactly what BMW would do. If you have a quality problem, do not ship the car. Yet with digital quality we tend to throw the problem over the fence to the customer every time.
In the case of the publisher, for the countries in the problematic region, they bought up another hosted instance provided the service more locally. They proved that was fit for purpose and then swung that into production so that all publishing was being run in a cloud application.
This case study highlights the basic ideas of “voice of the customer”, “right first time”, and “fit for purpose” that are presently lacking in digital supply chains.
How well is the broadband industry meeting the digital quality needs of enterprises?
Over the last few years, we have established beyond doubt that the current offers are not truly fit for purpose. The UK regulator, Ofcom, recently commissioned a report from us focused on SMEs and status of broadband. We measured broadband services from a fitness for purpose perspective, not speed. In doing so, we looked at what small businesses actually do: things like popular business cloud services, thin-client over VPN and video conferencing.
When we compared the broadband capability against that “fit for purpose” test, we showed that there was a huge amount of variability and inconsistency. This was both between ISPs, as well as from different locations of the same ISPs.
For example, we found that one particularly well known cloud service suffered from massive variability at a national level. When we correlated that digital experience quality to our analysis of the supply chain, and found a probable cause in how ISPs were peering to Internet.
This matters because the digital economy of Britain is being constrained by these issues.
Digital quality is an issue for consumers, too. They have different purposes: Twitter, Netflix, working from home, banking, kids’ homework, accessing government services – all of which are fundamentally important. Again, when we analyse these needs against a “fit for purpose” benchmark, there is a significant shortfall in terms of capability.
An anecdote, but a typical example of the problem, is one broadband customer I know. This is a small business involved in video-based training at home. They had ordinary 10mbit DSL, and were getting 8-9Mbit/sec in speed tests. The service was perceived to be OK. They then went to cable, wowed by advertising of much higher speeds.
For both the SME and consumer, they think that if it’s good at 8 or 10 megabits, image how great it would be at 50 or 100 megabits. Of course, you know the ending: they got the faster service, and suddenly they struggled to do online video training. The quality inconsistency meant it didn’t work well, despite scoring highly in speed tests.
What are the most important factors affecting digital quality?
I think the leading issue is the quality metrics. This whole discussion is about supply side and buy side using two different sets of metrics. That means network and other technology teams are disconnected from what actually matters.
If you look back 25 years ago, network teams were absolutely connected to the business and customer. One digital factory provided everything. The subsequent fragmentation has led to useful metrics at system level, but they are insignificant to brand and productivity. We need to reconnect them to that language if these suppliers are to stay relevant.
How do you see this sector evolving over the next few years?
The work we have been doing with Ofcom has brought voice of the customer ideas into measuring “Digital Britain”. We have been signalling the need to to explore new ideas, and industry has got to switch over to a quality-based approach.
If I am a “best effort” ISP and I am competing with Martin’s quality-assured ISP (God help me), I can only offer them more speed for less. You offer 100mbit/sec, I’ll offer 150 for same price. The speed idea is simply not connected to whether a digital product is fit for purpose or not. It’s a different thing!
Some of our customers are big ISPs, and they foresee a shift away from speed-based metrics. They are asking how to get more traffic into a network, but with less money to fund infrastructure investment. The quality idea is different, and recognises you can’t optimise broadband for everything at once. It’s like buying one car that’s good for the school run, Le Mans, and driving through the desert! Simply doomed.
If you could send a postcard back in time to yourself as you founded Actual Experience, what advice would you write on it?
I would write… Trust your instincts more. Building a business from scratch is like raising a kid. All kids are unique so you have to work out what works for your kid. In that respect, some commonly held business wisdom can be unwise for your business. Play the hand of cards you’ve got, not the one you hoped for.
For the latest fresh thinking on telecommunications, please sign up for the free Geddes newsletter.
Read It Now
