Episode 45 Antonio Maiorano, Director of Sales and Business Development at BARE International, talks about the best practices to measure and evaluate customer experience using mystery shopping. He explains how mystery shopping and crowdsourcing can help you achieve different objectives and mentions one of the biggest challenges CX leaders face - evaluating objective feedback.
Episode 45
Episode Summary
How does mystery shopping impact customer experience?
Mystery shopping is a tool that helps businesses evaluate customer feedback objectively. In practice, there is usually a person who acts as a customer and evaluates the services they are being performed at. However, this process can only be successful if staff members are not aware of this examination, hence the word “mystery”. This technique can provide you with a clearer perspective on your customer journey and a better understanding of customer feedback.
The challenge is to evaluate the standard
Capturing and understanding customer needs and expectations have become a major centre of interest in the last ten years. This process, known as the Voice of the Customer (VOC) provides you with a large amount of information from real customers. However, even the most sophisticated business technology comes with its flaws. On one hand, collecting a large chunk of unfiltered and subjective opinions generates a rich scale of customer feedback data. On the other, marketers might run into a problem when evaluating what the standard is. If that is the case, mystery shoppers are proved to be a helpful tool as they know what to look for and trained to examine and evaluate objectively.
The rise of crowdsourcing
Crowdsourcing is an innovative process that allows to complete tasks by a network of individuals. This type of customer feedback data collection can be conducted via a smartphone or a mobile app. The main difference between VOC and crowdsourced mystery shopping is, that the latter is not done by real customers. Unlike traditional mystery shopping which is not managed in real-time and can be costly, crowdsourced mystery shopping offers data gathering quickly and immediately. Does this mean the end of the traditional approach? Both techniques have their advantages and disadvantages. The first one offers an evaluation of more complex tasks. Therefore, it needs to be done by mystery shoppers who undergo proper training. This process leads to increased cost and takes more time to bring results. The second is cheaper and managed by a large group of individuals. However, the complexity of tasks considerably decreases (e.g. “Are the employees dressed properly?”).
This article summarises podcast episode 45 ” Evaluating CX: Mystery Shopping or Crowdsourcing?" recorded by CX Insider. For more information, listen to the episode or contact Antonio on his LinkedIn profile.
Written by Valentina Svobodova
Full episode transcript
Antonio: In the last five to ten years, the voice of customer became really something that a lot of companies wanted to have. Voice of customer is, you know, collecting surveys directly from real customers. But what is the problem of that is that real people have real opinions and unfiltered opinions and subjective opinions. And maybe you don't need subjective, because if you have to evaluate a standard, you need objective. You need someone that actually was trained to recognize the touchpoints and not just go there and give a subjective opinion, but give something objective.
Valentina: In today's episode, Adam and I invited Antonio Mirando, an expert in evaluating and measuring customer experience. We will primarily talk about mystery shopping, crowdsourcing and how you can use these tools to deliver customer experience excellence. Enjoy the episode and don't forget to let us know what you think.
Valentina: Hello, everybody, and welcome to another CX Insider podcast episode. This is Valentina speaking as always, and today I am joined by Adam and Antonio Maiorano. Hello, Antonio. Thank you for coming on our podcast.
Antonio: Hello Valentina, hello Adam. Thank you for inviting me.
Valentina: Antonio, I can see on your LinkedIn profile you say that your whole life you've always been trying to balance the Apollonian and Dionysian within you. Are you trying to say you're a demigod?
Antonio: But you say that I didn't. No, it's just something from my education. I just wanted to to remind myself where I come from because I had classical studies in Italy. We have classical high school in which we studied Latin and Greek. So it was really an important part for my evolution. And I just wanted to remind myself. So the point is that, however, I have worked in market research and then I went to work for good experience. So I handled data which are pretty practical thing. But on the other side, I really have I really like to talk with people. I do like to improvise and be creative. So I'm always trying, even in my work, to put the two things together.
Valentina: Interesting. Yeah, I just I just Googled it and it says that actually Friedrich Nietzsche popularized this this term, but maybe we should start off by addressing something more relevant than whether you are a demigod or not. And that is how did you actually how did you actually get involved in customer experience?
Antonio: Yeah, as I say, that I started working in market research companies well around five, six years ago, and then I moved to BARE International. We are we are a mystery shopping company, but we are also a customer experience consultancy company. This is how we define ourselves nowadays. And I started working as an account manager, mainly in retail and automotive. And after two years, I moved to the business development department because, you know, I like to talk a lot. So that's the best way to interact with with my clients to really talk about things I'm doing right now. And from January, I'm the European sales director. So, yeah, now I also have my team and we are we are trying to mainly consult the prospects and current clients on how how to better understand the customer experience and and what kind of tools to use to understand that.
Valentina: One of the things that I would like to focus on today in today's episode is actually mystery shopping. It is one of the things that that your company, BARE International, is is the industry leader. So how would you say that mystery shopping? How would you say that? What kind of impact it has on customer experience? And maybe to add to that a bit as a as a as another question is what kind of impact do you think it has on employee experience?
Antonio: Well, you know, when Bare International started as a mystery shopping company thirty six years ago, our founder and current president started in doing mystery shopping for hospitality because, you know, mystery shopping started mainly as a hospitality exercise. I mean, it's pretty understandable. Hotels and restaurants are the ones that face more customers and need to have a really excellent customer experience to be able to walk in and reach their goals. And then it evolved because this is the point, I think that that from from that industry, it evolved to other industries. Other industries took the example. So I think that mystery shopping, it's it's still nowadays a very quick and efficient method to look at the processes. So your processes regarding customer experience and how to improve them. And when I talk about processes and I talk about also maybe touchpoints of your customer, of the customer experience of of your company, I'm thinking of the locations, the physical locations in case it's a restaurant dealership or, you know, a shopper, but also the employees. Because we train our employees, we train employees to also face the customers with certain standards. And we want to understand one, if those standards are correct, because might not be correct.
Antonio: And you always have to keep that in mind that, you know, when we design our training, when we design the customer journey, maybe we can make some mistakes. We have to be humble in that. So we need to check. They are correct what we thought is correct and in case to adopt it and of course, also to see how the employees react to those standards because it might not work in some countries, for example, this is what I was discussing in a conference some some weeks ago, that things as far as the customer experience really change and also the evaluation of the customer experience really changed country to country. If we just look at Europe, southern countries like Italy, for example, they tend to give higher evaluations while, you know, when you go to the Nordic countries, the valuations are lower. So if you have a worldwide company, maybe and you have the same standards, maybe, maybe the results will be different. But it doesn't mean that that more than Denmark shops are doing worse than your Italian shops.
Valentina: Hmm. That is super interesting. Why do you think it is that that customers in southern Europe tend to give higher evaluations?
Antonio: Think is mainly cultural. I think it's it's coming from the fact that on the side, culturally, Southern cultures are more available, more talkative, more open. And these, of course, it's it's. It goes also in the way that the employees are, for example, pitching and upselling like that, and maybe they don't follow the rules strictly, so maybe they are not following your your standards strictly, but they get results that maybe other cultures are following more the standards, but maybe they have less less interactions.
Adam: And they were just to touch on that. Antonio, you may or may not know this, but for something else, the question, is it quite common then the organizations, let's say retailers or in automotive, would have a different kind of customer experience strategy depending on the country?
Antonio: Yeah, that is not happening most of the time. So I understand why, because, of course, if you have if you have a global if you have to have a global strategy and have that job. But I think that. It's also important to read the data in the correct way, so it's also important who is reading the data and for them to understand that. So if they understand that, you know, and they don't apply the scoring to the way, then you don't need to have different strategies. On the other side, I think that yes, it's it's true. It's it's an important point. Maybe in the future, this should be also an improvement points for for the for the customer strategies of the future and the leaders of the future not to have, you know, one size fits all strategy, but to adopt based on geography.
Valentina: Mm hmm. Yeah, that's interesting. Just to get back a bit to the mystery shopping here where we started. I would like to ask you, so how on one hand, how would you say that organizations could use mystery shopping to improve their CX? But on the other hand, are there any loopholes that may lead to do have a negative effect on CX?
Antonio: Every every tool has, you know, has their flaws. So, of course, there are. I think that the main way to utilize mystery shopping is that you have something that is objective. So, you know, in the course of the years, different tools have been deployed. And in the last five to 10 years, the voice of customer became became really something that a lot of companies wanted to have was of customer needs, you know, collecting surveys directly from real customers so it can be done via QR code, via tablets, though in these days it's difficult via email. You probably stumbled upon one of those in your life. And that was the idea, was that, yeah, we want to have real feedback. So feedback of real people. But what is the problem of that is that real people have real opinions and unfiltered opinions and subjective opinions. And maybe you don't need subjective because if you have to evaluate the standard, you need objective. You need someone that actually was trained. And this is what we do in international, because one of our big standards is the fact that we train all the evaluators for all our programs. And it's not just that, OK, yes, you go and do something. It's really webinars and continue evaluation process and internal evaluation of the evaluators really to really give them the tools. They give the evaluators tools to recognize the touchpoints and not just go there and give a subjective opinion, but give something objective. So is it something that maybe a customer would not notice? But it's really important for me then I need to do a mystery shop.
Antonio: And regarding the the loopholes. Yeah, of course. And this is why I think that this is for for example, why as we are moving from field work agency, we moved at least 10 years from only doing fieldwork agency to be a consultant because. There is no tool that is perfect, so you need to adopt the tools based on your needs and you need to adapt your projects. You cannot run the same mystery shopping project with the same survey over and over for years because you know, human beings, you know, their evolution, our evolutionary traits is that we adapt. And we realized in this last year, you know, we adapt so quickly to something so huge as a pandemic. And of course, also your employees will adapt to a survey that is there since five years. They will understand what is what and who is who. I know, for example, our famous British coffee shop chain. And, you know, in their mystery shopping, it's an exercise that they do weekly and they ask allergens. A mystery shopper has to ask about allergies. And of course, after years, employees learn that. And basically, if you go, you're not a mystery shopper. But if you go there and and you ask if I am allergic to pistachios or whatever, then they will just give you a free coffee, because just in case you are a mystery shopper, it would be happier because you receive the free coffee you can do is what you have to understand what is happening. You have to keep the pulse of what is happening and change. You change it every two or three years, probably.
Adam: That to be honest with you Antonio, I mean, I worked in retail for quite a long time, at a branch manager level or a store manager level, and you did learn, you did very quickly calculate if you could work out it was mystery shoppers. And and it was it was funny because it used to be linked within the earnings of the staff as well. How how high you scored on a mystery shop. And it wasn't a common occurrence. It happened maybe once a month. And I remember I used to be a case of, oh, I think this person's got a camera in their handbag. And it really wasn't a fair impression of how every customer is served, because he was he would identify the mystery shoppers very quickly and try and make sure that every single bit of your script or whatever you're doing was with of tip, which gave him a false reading or a false impression, I guess, employer, which is not really what they want to achieve. So I was going to actually ask, did you think then we kind of answer this anyway, but do you think that mystery shopping has changed over the years and especially do you think it's changed during this pandemic?
Antonio: Yeah, thank you for the question. We are changing, you know, I also see that what we are requested to do, it's different from the past. Because of course, also that the customer experience managers are learning these lessons, so. What is the request, the main request is to have something a bit more adaptive. So to have the possibility to change, for example, periodically the surveys, and I think that is a very important point and on our side, that's Bare international. You know, for us, the point is to have employees that are trained also to intercept those needs from and maybe anticipate. You know, so read the data and see that, OK, if you're constantly getting in all your stores ninety five percent probably we need to change this because probably that is not work. And regarding the shift, we could just we are conducting evaluators survey worldwide. We already collected more than four thousand replies with our database, but they are customers and we will use that to to have really interesting data. And one thing, because I was reading the partial data now, for example, one really interesting. During this last year, of course, people change their tendency to to have personal interaction in grocery. So compared to before the pandemic, now most of the people are more inclined to not have it. So, you know, it's fine not to have the cash, but other points that other industries are remaining steady as, for example, financial products. So still, people want to have personal advice because in some in some industries, when buying some products, you need to have that personal touch.
Adam: Absolutely. Yeah.
Antonio: So maybe that will also change the way that we do mystery shopping. And we see the customer experience
Adam: Is interesting about the personal touch because, you know, I've seen lots of surveys and things happen over the pandemic. Regarding your what do retailers what do retail banks see as the most important thing for them, for the CX? And there's always various options, ones along with processes or applications and websites, et cetera. What are the options is nearly always regarding staff and that personal touch. And every survey that I've seen that always seems to come top, that personal interaction and the human touch, should we say so? Yeah, it's very interesting to hear that you're saying that as well.
Antonio: Yeah, and probably it will change because, you know, the long term effects of the pandemic, I think it will be that more and more people maybe do not want to be in a crowded space. So maybe you need to organize a bit better your your your locations, your shops or offices because you need to have more appointments. You need to have, for example, more organization in that sense and that change the customer experience. You know, I was talking with one of our major clients in retail, and what they were saying is that, yeah, we went from having, you know, on average, the footfall of, I don't know, 50, 100 customers at one moment in some countries to have five people, but you still have five employees. So that means that it totally changed. Even if you're just buying, choose it, totally change the approach how to reply to a question, what time you give to a customer.
Adam: One of the biggest questions really for me, and I'm sure our listeners going to know this is an organization like Bare International or any organization that you guys are the experts in this. And I'm guessing from the research that you do for your clients. There's obviously a huge amount of data that is produced and data is key. And everything we have now seems to collect data. But I suppose one of the toughest parts that we have with that is actually how to action that data. How do you recommend or in your experience, how can organizations use the data that's gathered for mystery shopping or from surveys and put that into action? And how how how do you suppose what do you suggest on the.
Antonio: Yeah, there are different things that you can do, so first of all, going back to collecting the data. I think that an average manager is handling, I would not say a random number, but really a lot of tables. Sure. So the point for as a as very international, but I think also as in general, when we are working, when we are working with other businesses, we want to make the life easier and more complicated for for for the managers. If I'm giving, you know, I don't know, 50 page PowerPoint in which I list all the countries in which I did my research and extremely deep dive into each point. You know, the manager will never have the time to. Now, of course, the point is to, first of all, have different layers. So if you need something quick, you have to have a live dashboard which which we offer and live dashboard means that immediately you get what you need, that I feel that. I just need I just need to look at Greece to Athens for my shop in Athens. What is happening there? What was the last call? OK, I got that and I'll check the or what is the average score in at the moment? That is a quick thing. And then when it comes to the overall analysis, not have really tons of data, but just just a few key points.
Antonio: And this is why as a consultant, we are also offering business intelligence services because we have our own team that is looking at the data. And of course, we always say that to our clients. You say, of course, if you have a team, you know, you have to analyze that. You just don't have to get it done. So if you have your own team, that's fine. We will give you the data and your team will analyze it. If you don't, then we should do that. But don't get just a that when you want to get just the data. And because maybe you want to do something more on the on the regional or branch or store level, then what you need to have are action plans. So to have some key points, this is also a tool that we offer, for example, and to highlight some key points that if they are applied in the wrong way, for example, then you need you as a store manager, need to act immediately. Otherwise, you know, your score will go down or your score will always be. On a lower level and this kind of tools needs also to activate the more extended manager managers area because it is important for for everybody to have the pulse of what is happening, but also to make changes really quick.
Adam: Yeah, absolutely. I mean, that makes perfect sense to me. And I suppose what you're saying is with young people like yourself, with this kind of data that we're in now with data being available, it's all about breaking it down, making it more understandable at a level that you need to report on rather than look at the bigger picture. But I really like the point you made there about actions and giving organizations the ability to get the data quickly rather than waiting for like a monthly report or whatever. You know, giving them information is not real time, is it? You know, you can't get real time information.
Antonio: Well, we for us, for example, the real time means it depends on what is the tool, because if it's actually customer satisfaction, it is real time, because if it's a survey that that it comes from from a customer, then when they submitted it, something submitted. And you can see immediately. When it's mystery shopping, of course, there is a process so that the quality and we have. Three to five business days to finalize the report. But yes, it's like as soon as it's finalized, you can see that
Adam: And you can break it down by storage. And that's very helpful. And I can see how.
Antonio: But I think that that is the point to help big organizations, because, of course, you know, if you are owner of a small restaurant chain, ten shops, ten locations. Then, of course, you can go and look one by one. And it is important. But if you are a customer experience manager of a company which has, I don't know, 200, two thousand locations, that is just impossible.
Adam: Well, you actually mentioned as well on a previous conversation, you and me had, Antônio, regarding something called crowdsourcing. And one of the questions I've had was, do you think that crowdsourcing is going to replace the kind of traditional mystery shopping that we've all we've all done for 15, 20 years?
Antonio: Yeah, I don't think so. That's the answer. Just to give a bit of context, because maybe. So what is crowdsourcing? It's something yeah. It's something in between mystery shopping and the VOC. So crowdsourcing is actually gives the possibility to people that have a certain app which can be our app or the, you know, the company's apps, like, for example, the brand, the brand has its own app and there is the crowdsourcing function in there. And this person will receive a pop up notification when it's near a store or a location and will be asked to reply to five/ten small questions. The main difference from the voice of customers that these people are not. But the the main thing that you can get from that is that you, for example, you can reply to simple questions that are mystery shopping questions, but gather data much more quickly because there is no there is no is more live, because when you gather data, we will accept it in seven to 72 hours and then you will receive it. And what kind of questions? You can reply, what are your employees dressed properly, for example. You know, you can just reply to that question. Did they try to upsell something? Did they offer you the complimentary coffee? Is the new Christmas campaign out? I just take a picture. You know, it's very quickly. I just take a picture. I don't need to edit that as Bare international. So you receive the data much more quickly. And and it's very simple way to have important questions that maybe you need to be answered. In like one or two weeks and have them.
Adam: Sounds good, sounds very clever.
Valentina: Yeah, I wanted to ask you about that, by the way, can you can you hear my laptop, the fan of my laptop? Can you hear me? Oh, no, no. Good. Because I think it sounds like I'm launching a spaceship. Seriously. My microphone is right next to it.
Antonio: So we are going into the future of customer experience. So that's the song.
Valentina: I wanted to ask you one of the you said you mentioned earlier that one of the one of the biggest benefits of mystery shopping is that it gives you objective evaluation. And now maybe with crowdsourcing, you know, it's way, way, way more people, more data. But can you ensure the objectivity of the evaluation as well? Because I assume these people don't go through the proper training as they do if they did mystery shopping, right?
Antonio: Yeah, exactly. The point is that, you know, we as a company, you know, we as consultants, we have to ensure that. So where the objectivity from mystery shopping comes from, you know, trainings, webinars, etc, etc. here comes from the type of questions. You cannot ask certain questions in a crowdsourcing scenario. You cannot ask did you feel welcome? Or you can ask. But it will be a subjective reply. The point is, focus it on, you know, things that are black or black or white. So is something that or not is something proper. Properly shown or not, it's very quickly it doesn't need that much training. We also need to be in our system because that depends if it's in our system, these are our evaluators. So still they have the the level one mystery shopping, training. But I think that the most important way to have objectivity is with the question.
Valentina: My last question for you would be, as we're getting to the end of the episode, is and I always ask, what advice would you give to aspiring sex leaders?
Antonio: You know, for me, it's very important, first of all, that we need to explore always to explore new solutions, because at the end of the day, it's a trial and error process. So we cannot know at the beginning what would be the right solution. And in this sense, it's also important to push the company, your company, out of the comfort zone. And this because mainly, as also we said today, customers are evolving. So the customer experience has to move with them, not just as things were 20 years ago. And this is what also is that we are doing. And the last point, which, you know, it seems like an obvious, but it is important to listen to your customer and it is important not to be afraid to ask, because I know from experience that there are companies that are afraid to ask. Because I understand that it can sound scary because maybe the replies will not be as happy as you think, but, you know, in the personal and the business life, we have to take the that even if they are negative because we need to build them our future. So, yeah, really listen to the customers and what they have to say.
Valentina: Thank you, Antonio
Antonio: Sound and thank you to you.
Valentina: Thank you for listening. I hope you enjoyed this episode, and if you did, don't forget to, like, share comment or subscribe to the podcast on your favourite channel. And we'll see you next time.
