Forget everything you know about QA

Kim Wagner, Head of Quality at PartnerHero, wants you to forget everything you’ve ever heard about Quality Assurance (QA) programs. In her mind, traditional QA programs used by most companies and BPOs are biased, incite fear in employees and - perhaps most damning - do little to help customer support teams improve the quality of customer interactions.

So what’s the solution? During Kim’s five years as a leader for Airbnb’s Trust & Safety and Community Support Quality Programs she threw out the traditional QA handbook and built something for modern companies. Kim’s quality program is a three-step, holistic process that identifies the people, process and product issues that lead to quality issues and then works cross-functionally to address the identified problems.

First we’ll dive into what is wrong with traditional QA programs, next we’ll take an in-depth look at what a modern quality program should look like.

What are the main issues with traditional QA programs?

At a traditional call center a quality assurance or management team is tasked with listening to a certain number of previously recorded or live calls per agent per week. The agent will then get an email and coaching session to go through “what they did wrong.” Needless to say agents typically dread this process - it is decidedly bad for team morale and culture.

Kim reports that one of the biggest issues with standard quality assurance programs is bias in sampling. “In an ideal world you could QA everything. Obviously we can’t do that so it is important to pick a representative sample size. One example of this going wrong is when companies only pick tickets that got bad CSAT scores to QA. An agent might take 1,000 tickets and get one or two bad CSAT scores. Those tickets get reviewed and the agent gets pounded for it. It’s not representative and it's not fair. BPO’s with this type of quality program are passing the buck, blaming agents for bad performance when the first step should be looking inward at processes and products to understand what led to a bad CSAT score.”

To Kim, the fundamental problem with typical QA programs is that, “they exist for the sole purpose of checking a box instead of being used as a real opportunity to solve problems and help employees learn and grow.” That is what Kim has solved with her new approach to quality.

Quality Assurance for Modern Companies

The building blocks for a modern quality program are in the People, the Process and the Product.

People: Human interaction is at the heart of most support interactions so it makes sense to look here for quality issues. But before you dive into grading tickets there are a few steps to take to make sure you get it right.

1. First take a step back to understand the slice of data you are looking at. Maybe you just want to do an overall audit of the quality of responses or maybe you want to audit adherence to a specific policy. Maybe you want to audit for brand voice accuracy. Before you start evaluating tickets you should know what you are looking for.
2. Next is deciding how to pull the data. With what frequency will you be looking at that slice of data? How much do you need to pull to get a representative sample? Is it clean?
3. Once you’ve managed step one and two you create a rubric so you can consistently grade tickets and be sure you get the data you need to answer your initial question (overall quality, adherence to a policy etc). The rubric is key to having consistent, quantitative data at the end of the project.

Process and Training: Once you’ve gathered some results from the People phase you can do the analysis to spot trends - where the issues are occurring over and over again - regardless of the agent. This allows you to address the root cause through process or training. Instead of just telling an agent that they did something wrong and moving to the next task, why not sync with the training to make sure it is clear? Maybe the training is missing an important step or isn’t up to date. Having a consistent feedback loop between quality programs and training is crucial to maintaining quality on your team, especially as you scale (see more about this in an interview with Robyn Barton, our Director of L&D). Not to mention that it is a great way to make sure your training materials don’t gather dust in a corner. Kim says, “there are lots of companies where training and quality are at odds with each other but really they should be working in lockstep. For example, ‘we noticed this batch of new hires didn’t really get this, can you touch on it more in the next round?’”

Product: The last step is to look cross-functionally, beyond quality and training to product and engineering. Perhaps customers are upset because the shipping dates in their confirmation emails are wrong. A customer service agent can of course answer their question and get them the right dates but the root cause of that issue should be rectified through product fixes. This is where your quality team will build cross functional communication resulting in product changes to improve customer experience.

Quality as a Service at PartnerHero

Since joining PartnerHero eight months ago, Kim has been implementing her quality program across our partners. Below are the steps she takes to make sure our partners are running high quality programs:

1. Understand what quality means for your business - what are customer expectations? What are the business objectives?
2. Build a system for measuring quality as she defined it in the first step: how are you capturing data? What tools are you using?
3. Design a rubric that can be used to grade interactions relevant to your service or product.
4. Capture data and do a thorough analysis of drivers for your quality errors.
5. Provide stakeholders with clear recommendations based on insights.
6. Implement a closed loop feedback system with the training team.

Kim’s passion for quality comes from seeing first hand the incredible impact that a quality program can have on business. “No matter what you sell there is an opportunity to have customers feel great about it. Good quality programs have the power to be that magic wand that can come in, diagnose what’s going wrong or set up systems that can turn your customers into brand evangelists.”

‍