VRTAC-QM Manager Minute: Using AI so VR Staff can Better Navigate All the Requirements - Washington General


Episode Artwork
1.0x
0% played 00:00 00:00
Feb 01 2024 30 mins  

In the studio today are Cassie Villegas, Outgoing Interim Director of Washington General, and Sven Akerman Jr, a contractor from Outlook Insight with Washington General.

You can find out how Washington General empowers staff with an integrated AI tool that does the heavy lifting when researching policies, regulations, and RSA requirements, freeing staff to focus on providing quality services.

Listen Here

Full Transcript:

{Music}

Sven: Our team was presented with a challenge from DVR. They had a 767 page customer service manual, but it was really kind of difficult to find answers quickly.

Cassie: You can go right in and find your answer, and I found it in 0.2 seconds rather than two hours. Coming through all of the different policies or regulations.

Sven: I see this more as enablement capabilities as opposed to replacement capabilities to where, like Cassie was saying, get back to focusing on what you really want to do, not what you have to do.

Cassie: Now, I don't think our staff could live without it. If we tried to pull it back now, there'd probably be mayhem.

Intro Voice: Manager Minute brought to you by the VRTAC for Quality Management, Conversations powered by VR, one manager at a time, one minute at a time. Here is your host Carol Pankow.

Carol: Well, welcome to the manager minute. Joining me in the studio today is Cassie Villegas, outgoing interim director Washington general, and Sven Ackerman, junior contractor from Outlook Insight who is working with Washington General. So thanks for being here, you guys. Sven, how is it going in Washington?

Sven: Well, things are fantastic heating up and, uh, well, not heating up. It's actually getting wetter. But it's a beautiful time to be in business for Washington.

Carol: Excellent. And, Cassie, how are you doing? I know you're the outgoing interim director. Hopefully you're going somewhere good.

Cassie: Yeah, absolutely. Things are winding down here for me in terms of VR, but starting to wind up for the next step. So yeah, pretty excited.

Carol: Good for you. Good for you. Well, thanks for joining me. We are going to have a very interesting conversation today about artificial intelligence or AI as it is commonly known. And artificial intelligence has been all over the news this summer and fall. I think about the Hollywood SAG-AFTRA strike. CEO Sam Altman with OpenAI. And in a nutshell, artificial intelligence is a simulation of human intelligence processes by machines, especially computer systems. And so when I was preparing for the podcast, I started making a list of AI in my life. I'm like, okay, what things are considered AI and it really is all around us. And I know we think it's this other big thing there, but it's here, it's everywhere right now. And I thought about my virtual assistants Siri and Alexa, the facial recognition when I go to the airport, I use CLEAR. So they're looking at my eyeballs to, you know, get my identity spam filters. You think about the algorithms in your Google search, driving my car with driver assisted technology and so many more. And I think there's a tendency for people to kind of go to that dark place. They conjure up all the dark things I could do. And you think about machines are becoming humans, and you look back at movies like The Terminator or War Games, Space Odyssey, and there are definitely valid concerns. You know, we've heard in the news as of late where individuals are cloning your voice, you know, and they're sending it to your grandma and asking for a ransom note or something like that. So as with all things that are new, you know, there's always this balance. And so when I think of AI, I always think of things like it being really cutting edge, which, sorry to say, I'm not always associating with VR, you know? And much to my surprise, I come across this article about how you guys are using this in Washington. So we have to dig in. I'm super excited. So, Cassie, can you tell our listeners a little bit about yourself and your background and how did you come to VR?

Cassie: Yeah, so I'm currently, as you said, outgoing interim director. I've been the interim director for Washington General since July of 2023. Prior to that, I've been our deputy director. I've worked with our community rehab programs, their CRP's. I've been a tribal liaison. I've done some of the DEI work. And prior to coming to Voc Rehab, I worked for the Independent Living Council for the State of Washington. So I got my foot in the door with VR through Independent Living. But before that, I've worked in trio programs with students trying to help them obtain education, higher education, and that career exploration piece that's very similar to what we do in VR. So I've been around not very long. And of course, like you stated, I am outgoing. I'm leaving. We have a new interim director that I've handed some power over to or transitioning right now. So yeah, good things on the horizon for me, but also for VR.

Carol: Very cool. It's always fun to find people's pathway into VR. We always have our foot in somehow. So that's cool that you came in through the IL world. So also, can you tell us a little bit about Washington General and how many staff the agency has and like how many consumers you all serve?

Cassie: Yeah, absolutely. So in the state of Washington, we have 39 counties. Our state is broken up into three regions. We have about 41 offices across the state, and we have about 340 staff in the last year on cases, we've served just over 7000 customers. That does not include youth and students who are not on caseloads through those like group services, through pre-employment transition services. But that would probably well over double that 7000 number if we included them in that count.

Carol: Well, absolutely. And I have two brothers that are out in Washington, so I know about the complexity geographically of your state it is very different. And while something may seem like it is an hour away, it really is not as you're hitting the mountains or which way you're going. And big, big differences in very rural areas of the state, a lot of geographic complexity. So, Sven, let's go to you. Tell our listeners a little bit about yourself and the company that you work for.

Sven: Sure. Well, I'm the founder and chief executive for Outlook Insight. We're a social services focused company that leverages modern technology, really helping those who help others. We're proudly celebrating our 25th anniversary here on December 22nd. So just four days away from 25 years old, my background is in technology management and implementation, with a focus really on Microsoft Azure and Microsoft 365 solutions. But I've been working with vocational rehabilitation agencies since 2009, when I started working at a company that built VR case management software, and since then, I've been proud to have delivered well I've been part of the implementation of over 26 VR case management systems and had the pleasure of traveling all over the country, meeting with executives and staff from VR agencies, kind of understanding the mission, what they do, how our technology helps them, and then branched out onto our own kind of rekindling the fire of the entrepreneurial spirit in 2019. And since then, we've been working with a variety of different agencies, including Washington DVR, Washington DSB, DSHs at large, and others, really helping them drive new technology capabilities to streamline their operation as we go along.

Carol: Very cool. So you bring a very unique perspective. You can see technologically some of the gaps that we have to as a system for sure. So how did you get into this AI space?

Sven: Well, you know, it was just a curiosity back in like December of 2022, you know, as the OpenAI ChatGPT conversation started to explode, we started looking at new ways to automate, you know, just some of the routine work we absolutely fell in love with generative AI. It's really, really spent that first quarter of the year just understanding the myriad of different open source, fee based, large language models, how to use them. And over time, we really kind of settled back into our roots a bit and decided that the Microsoft Azure platform was going to be the one that we wanted to focus on, really just kind of recognizing that's where our background and experience in technology is most prominent. And then the advanced capabilities offered a lot in terms of what we could do with it. Well, not to mention that it's generally acceptable by most state agencies to be working in Azure. So with that, it was interesting. Two things really happened agency related at about the same time. The first thing that happened was our team was presented with a challenge from DVR. They had a 767 page customer service manual, and it's a really a vast collection of guidance, policies, procedures. But it was really kind of difficult to find answers quickly in there. So they asked if there was a way to better access that content, make it more accessible and available to folks. So we accepted that challenge, and we decided to take some of our AI curiosities and apply it to this space. And we loaded a model along with their customer service manual. We also loaded it full of the RCWs and wax of Washington State law. We threw in there all the RSA policy directives and TAC publications as well as the federal statutes, and that really established a full corpus of data.

Sven: And then we started to ask questions, kind of refining the prompts to optimize the responses. And that's when it all started to come together in terms of forming great answers for Washington DVR related to their customer service manual. But the second thing that happened was really interesting. You know, working in technology, I help a variety of folks, especially as people are onboarding, sharing knowledge where I can. And one of the newer staff person who was hired to take over the technical roles of someone else who's going to be retiring, came up to me and they had a technology policy question, and I was like, well, don't trust me on this answer. Let's go find the authoritative source. Let's go out to the Washington OCIO policy website and see if we can find the answer there. And I got to tell you, you know, after 20 minutes of searching, we were coming up empty handed. And eventually we wound up finding the answer that we were looking for in one of the sub sites there. And I thought, man, this is just too hard. I mean, how can a new worker be expected to remain compliant when the compliance guidance comes from so many places? So that was frustrating enough for us to take all of the OCIO policies in Washington state, load them into a model on our own, and see can we get to those answers much more quickly? And we were just thrilled and delighted with the approach. It really helped that new employee get to answers very quickly, and we have since been kind of socializing that experience with others. So there's how we got in there.

Carol: Tell us a little more about this technology. How does it really like, how does this all work?

Sven: Well there’s not a ton of mystery to it, although we're not going to get into any of the complexity about how AI and machine learning works. But really the intent of this solution is really to. Help workers find information to make the most productive with their compliance driven environments. So ultimately, what we're intending to do is to drive them to authoritative content based off of just human language prompts and questions. Kind of like you would ask a coworker a question or have a discussion with them. And so the way that it works is, you know, at the agency level, the content that is of interest for exposure through this model is compiled together, and we pull that data into a search index. So we index all of that content in a way that we can then do what's called a semantic search across it, basically finding answers based off of intent of the question, not just keywords. And that delivers back a very rich response in terms of the authoritative content that comes back. And then we take those results, and then that's where we start to leverage the large language model, the Azure Open AI large language model, and we summarize the content of that search.

Sven: Okay. So we searched the content that the agency knows we have a result set. We summarize that in just a human readable form. We present that back to the user. And then we also include in their citations back to the original document that was uploaded and ingested. By doing it that way, we're able to quickly show the user. Here is a summary of what has been discovered with some context and understanding around it. Let them drill into the authoritative content to really see you know, what are the specific words that are of legal authority. But then we take it a little bit farther in that since it's not just a search retrieval system, using that large language model, we expose the ability to do things like drafting emails based off of the context of the conversation you're having, or explaining things in new ways, like through telling stories or simplifying concepts down to perhaps a more easily understandable level for folks you're trying to communicate with.

Carol: I thought the storytelling was kind of hysterical because for disclosure to our listeners, Sven shared the link with me to access. So I'm in there asking it questions, and then it says, do you want this presented in another form? So obviously, you know, I put in like, hey, how many days do I have to determine eligibility? And so of course, then this whole thing comes back, it links to all of the policies. It gives you any RSA guidance on it. It's got the regulatory citation. And then I'm like, tell me the story. You can click the button. You know, tell me a story about this. And so then it's like, oh, then there's the story about Bill or whatever. And it does this whole thing about to put that in the context, I thought that was pretty cool. I am like, I've never seen anything like that. Now. The story got a little wild there for a bit as it goes through it, but it was pretty fun. I thought that was super interesting way to think about it. So, Cassie, I know Terry Redman started this initiative with Sven and his company. How were you involved in all of this?

Cassie: Yeah, so as the deputy under Terry, I was hearing a lot of really cool things from Terry, from Sven hints that something big was coming. We're working on this AI tool. It was really excited when Terry decided to step away, and I got to bump up into interim to get the full picture and to, like, really know what was going on behind the scenes. Just personally, I'm excited by innovation and technology. I am a millennial. I've, you know, had a computer in front of me my entire life, basically. So I'm like, let's use new tools. So I inherited the project, was really excited when I got to finally see the demo from Sven. And I'm like, let's go, let's get this out there to our staff as fast as we can. And that's what we did. We kept it moving. We added to the list of to-do's, but this was the top of that to do list. And here we are. We've got this really great tool.

Carol: So when did this all roll out?

Cassie: So we soft launched it in early October just to get hands on it from some of our like program managers. And then we had the hard launch the full as available to everyone. At the end of October of 2023, we had an all staff in-service event for two days. We brought Sven and his team in to introduce it to all of our staff, to train them on how to use it, to kind of give them that background on the purpose and how to ask appropriate and meaningful questions, how to check the sources and just start easing people into using it. So we've had it up for about two months or so now. So yeah, we had a really quick timeline and our staff are using it daily now.

Carol: So how is that rollout going? What's the reaction from the staff?

Cassie: I think now it's pretty positive. There's still people that are discovering it. You have new folks or people who maybe just they didn't think it pertained to them. We had a program manager in our headquarters office. I think it was just what last week Sven at our holiday party, they pulled you aside and you left the party to go show them how to use the tool, because they just didn't have a need for it until someone said something and they were excited to see how it worked. But at first I think there was a lot of nerves around it. I, like you, kicked us off at this conversation as scary to some, so there was a little bit of convincing, hey, this is a really beneficial tool for your day to day. It did take a little bit of show and tell, demonstrating the tool, showing the benefits, reinforcing the benefits to everybody. Why it’s a good thing? But now I don't think our staff could live without it. If we tried to pull it back now, there'd probably be mayhem.

Carol: Well I can imagine I think about all the new counselors coming in, new techs, you know, all the different staff coming in and you get your orientation and, you know, you're reading policies and procedures and it's just all so much. There's so many dates and all these, you must do this and this and this. I think this would be fabulous because everybody gets assigned a mentor. You have a way that you're getting trained, but you don't want to 40 times a day, go back and ask them like, oh gosh, I forgot. Like what was the requirement around this? If you could just type your question in and you get the answer. I find for me at least, that helps me remember for the next time, you know, because I've done that research, I've gone in and seen it then and I'm able to like, retain that even better. So I'm sure. Are you hearing that like from new staff especially? I would think this would be like an amazing tool for them.

Cassie: Yeah, absolutely. I think there's many benefits to the tool. So like you said, new staff coming in, maybe not embarrassed. They don't want to ask the same question four times. Sometimes you need the context to retain it. Maybe you hear the old tropes from staff who've been around. You know, we've got the MythBusters kind of thing that needs to happen. Sometimes you can go right in and jump in, find your answer, and make sure to share it with your peers. Actually, that's a myth. This is what policy says. And I found it in point two seconds rather than two hours coming through all of the different policies or regulations that we have to work within. So there's benefits there. There's also benefits when it comes to talking to our customers. If we don't have a clear understanding of our own practices or policies, it's a challenge to explain the why to each other, but also to our customers. So there's a benefit that it's right there in front of your fingertips. Really easy to access when you're in meetings with customers to explain the why. And then like you've talked about already, the storytelling piece. So storytelling helps us learn. It's helping us as VR professionals learn, but it also helps our customers have a clearer understanding of services and the whys. And I think that sometimes the work that we do is really hard. So we get a little bit of cognitive fatigue, or if we're constantly having to think differently to explain things to our customers, it can take away some of the brainpower we need for the more challenging parts of our day to day. So you just go into this AI tool and say, tell me a story about this particular policy, and I can explain it to the person sitting in front of me in a way that they're going to understand, and I'm keeping that cognitive load off of me and just using the tools so I can redirect that where it's actually needed, where maybe the tool doesn't have the ability to help me with that actual direct customer service.

Carol: I know folks have that tendency to always go to the dark place, you know, like, did you have staff feel like you guys are just trying to get rid of us? Like, now we got this and you aren't going to need us anymore?

Cassie: Yes. And you know, I did have a call from a labor representative saying, tell me about this tool. How does it impact the day to day? We don't want you to replace VR counselors. And we're like, absolutely not. This is a tool to enhance critical thinking and professional judgment and to make the day to day work of our counselors easier so they can get back to what they're here for, which is that direct vocational rehabilitation counseling. We know that there has been a lot of changes over the years, WIOA and others that have put an extra burden on staff who feel that maybe they're being removed from that direct service that don't appreciate it. So this is just another tool to allow them to spend their time where it really needs to be spent. And that's with our customers. So instead of spending hours looking through the regs, trying to figure out, can I do this, can't I do this, why they can go directly to it, type in their questions, take something that maybe used to take hours, complete it in a few minutes, and then move on to, you know, the true meaning of vocational rehabilitation.

Carol: Yeah, I want to put a big exclamation point on that, because I know we have a lot of our listeners that keep going, like, what are other states doing? How can we get counselors back to counseling and not having to do all this other stuff? This just seems like another tool in the toolbox that could be super amazing for your colleagues across the country. So, Sven, I want to ask you, a lot of times when you're developing some kind of new technology or a new way of doing something, it can take a long time. How long did this take from kind of start to finish? When Terry approaches you with the idea for you to deliver the final product.

Sven: So again, we started we started our research about a year ago, and once we kind of cut our teeth on just what was real, what was not where we were, and we're not going to focus, that was one of the bigger decisions that we had to make. We wanted to choose something that would be most impactful, but also most acceptable by everyone. We had started to build the base platform for delivery of this generative AI solution, like, say, the middle of Q1 last year. And by like the middle of May, we actually took it on a road show at a conference here locally and showed it to a group of peers, a couple hundred people at a tech conference. And so we already had kind of the foundation in place. But that foundation was guided heavily because the conversations I started having with Terry were well before that. I mean, it was probably September or October of last year, you know, marinate on it a little bit and then. And come around December. It's like, let's see what we can do with AI on it. So we took it on as a personal company endeavor to bring a product into the space that could quickly and easily be onboarded for state agencies. And so by the time we got the go ahead from Cassie and her team, I think let me see. That was a Thursday. And by Monday we had the model fully loaded up and available for that soft launch she was talking about. So it only took a few days. And since then we've continued to refine that process. We can deploy now, you know, typically within 24 hours of having an order have the solution in place and at least core knowledge there to start to inspect and to do QA on.

Carol: That is crazy. I'm like shaking my head. I know our listeners can't see it, but I'm like, that was quick. Like, you did this all really fast.

Sven: You know, It's been a bit of a whirlwind and I can't tell you that. It's been with a great sleeping nights. Um, so there's been a lot of nights and weekends. I know we're still working full time other jobs, but even just time slicing it in, it's been a passion. When Cassie talks about the need for counselors to be delivering the services that they were really hired to deliver, I can speak directly to that. My brother in law's developmentally disabled here in Washington state. He benefits from DVA and other services, and I know how important it is for those counselors, those people who are serving the community to get the mundane, hard stuff out of the way so they can do the actual brain work and really focus on that service delivery and mission. Because for every minute that I can help save through technology, that might be one more minute for my brother in law. So it's been really kind of cool to think about it from that perspective. And as we've been going through, you know, it's really led to other things like our absolute commitment to total accessibility. We're currently undergoing a WCAG2.2 AA certification audit, and we're hoping to have that wrapped up probably in early January. It's accessible right now, but it's going to be unbelievably accessible here. Once we wrap up a couple of issues that they've identified, it's gone quick. But it's been long. It's been a good kind of long, but it's certainly been rewarding.

Carol: Good on you. I think that's really cool. And I'm super glad you're really focused in on that accessibility. That will also help the colleagues across the country be able to access that. Now, Cassie, I know you've talked about some really nice benefits for your staff. Have you seen any other ways that's making your program better? I know lots of states are really interested in rapid engagement or if you've seen any impact. I know it's only been two months, but is there seeming to be some impact on your processes?

Cassie: Yeah, I think so. Right now it's all anecdotal and I wouldn't have like very hard concrete data with just the timeline that we're in. But I do think that it is having a positive impact. If we look at the amount of time alone, not even direct customer service, but the time it's taking our staff to find answers to their questions, we're a large program were spread all over the state. We've had a lot of turnover. So people that are maybe in the field don't always know who to contact at headquarters with their questions. So if you're looking at just the amount of hands a question can go through before there's an answer, in addition to having an FTE sitting in like a policy position expected to respond to all of the questions that they're getting, I mean, we're saving full FTEs on this tool. Our policy manager used to basically do this nonstop. They would just be responding to emails all day. It didn't give them the opportunity to dive deep into the real, meaningful work that they needed to do to improve process and policy for our program. It was just simply answering questions to staff in the field, and we've already seen that decrease significantly. We do know that customers that their questions are getting answered quicker, our staff are able, like I said earlier, to provide simple terms, plain talk for some of those answers that probably would have gone back and forth multiple times with customers. So we are seeing an increase there. I do think it does tie really nicely into this focus in rapid engagement that we're seeing across the country.

Carol: That is super cool.

Sven: Tying into that Cassie, one thing I can share from the back end is, first of all, when this rolled out, this is a voluntary use tool. It was just a resource that was made available. Nothing was taken away. Their SharePoint is still there, their customer service manual is still available for them to use as they have, but just through organic adoption. Since we soft launched this on October 10th, there have been over 10,000 request and response cycles that have gone through our system. Over 20 million AI tokens have been used up from an organic growth perspective. That's one of the leading indicators of something that's useful for folks.

Carol: That is super cool. Holy cow. I know Sven too you had a really interesting perspective about what's happening with the whole knowledge base over this next seven, eight years or so. You want to talk about that?

Sven: Yeah, one of the things we were looking at, I mean, anytime you're looking at a product, you're looking at the market and the market going forward. Right. And I found it curious, looking at some of the data that came from the Office of Financial Management, O.F.M., where their calculations, just based off of the age of the current population of state workers, some 25 to 40% of the staff in Washington state are going to retire over the next 4 to 7 years. They're just reaching that age and it's going to happen. So what that's going to do is it's twofold. One, it's going to have a direct impact on availability of senior folks who have been around for a long time and know so much they're going to walk out the door and their knowledge is going to walk with them. So you have that compounded by a whole new crop of new folks coming in. Right. And where are they going to go to get those answers? And how are they going to maintain the mission in the absence of having that the proverbial shoulder to tap, if you will? That was one of the things that we were looking at, thinking, oh my gosh, this is actually a huge problem not just in Washington state. This is a national problem. It's the baby boomer generation is reaching retirement age. And it's like, okay, holistically, I think in terms of just government continuity, I think solutions like this need to be made available to folks. And with any luck, we'll be able to keep the service levels where they are or even improve on them as we go forward. But I don't think it's going to be an accident. I think it's going to have to be purposeful.

Carol: Absolutely. Yeah, Cassie?

Cassie: Yeah, I was going to also add, just as a little story piece, our incoming interim director was in voc rehab a long time ago. It's been a while. So she's being reintroduced to VR and she's been here all of maybe a week now, maybe two weeks. And she told me I've been using the tool every time I'm in a meeting, and I need to know a definition of something, or I'm not really sure of the answer. I just like, can hop in a virtual meeting, open it on my second screen and find the answer. So I think being spend to what you're saying about new folks coming in, it is absolutely a game changer for the transition of generations in and out new staff. VR is very complex, we all know, and it's just a tool that has a solution that's now that we're using. It seems so simple for all of these really complex problems. We're solving really complex problems with an AI knowledge interpreter tool.

Carol: Yeah, I thought it was absolutely amazing. I just know as I went in and played around just a little bit, I'm like, oh my gosh, within five seconds you have this whole array directly to your policies, procedures, any of the RSA sub-reg guidance, all of the, you know, citations that you needed. Here it is. And then you can get it laid out for you in a very understandable way, which is super helpful for the work. I know if our listeners are going to be interested in this and getting some more information, so what would be the best way for them to do that? So Sven, do you have some contact information?

Sven: We do. We're just now starting to build out a website with more details. So organic conversational growth is happening in real time. So it's like okay we can't just keep answering them live. We have set up a page, up until now, remember, our focus has been on delivering the value of the tool itself. But we have a new marketing director and a team that's actually starting to work on these things. So we do have a website put up its outlookInsight.AI. And if you go there, you'll find some information about the tool, an ability to contact us. We have a contact us link. And we're starting to publish webinars that will be available for folks to register for. If they want to sit in on a demo and get more information that way. It's not fancy, it's in its infancy, but it's a start, and it's better than having to answer emails and phone calls the whole time.

Carol: Yeah, very cool. And Cassie, is there anybody that would be good for someone to reach out to at Washington General?

Cassie: Yeah, we could give you the contact information of Jack Fruitman, our IT director, and his email would be Jack j a c k dot period dot fruitman fruit [email protected].

Carol: Excellent. I really appreciate that. And was there any other last things either of you wanted to share?

Cassie: I mean, I just want to plug that these types of tools can have a really positive impact in really simple ways. Like I just said, it seems really complex now that we're using it, Sven, it's complex, more complex for you than it is for me on the technical side of things. But for me, it's a really simple tool that does solve a lot of large issues. So I think it's really important that we're all just embracing the advancement of technology, and we're accepting of these innovative solutions. You know, it can be scary. It can be dark, but we can also use it to our benefit, and we can use it in a way that helps us really support those who need our services the most. And we can get back to true VR counseling with this tool. And I hope to keep adopting and seeing the adoption of tools like this in the future, especially for VR.

Sven: I'm going to echo that. The application of AI, it really is, for all intents and purposes, in its infancy. Machine language has been around for a long time. A lot of the foundational components have been in use for decades. But all of these pieces coming together now with these language models is novel. It really does promise the power to simplify our lives and really improve the quality of what we do. I see this more as enablement capabilities as opposed to replacement capabilities to where, like Cassie was saying, get back to focusing on what you really want to do, not what you have to do to make that happen. And, you know, there sure are some unresolved concerns. And, you know, those can't be ignored. But what the right application to the right use cases, technology really doesn't have to be scary. It can be super awesome, and it can really empower your teams really from day one.

Carol: Very cool. Thanks for being with me. And Cassie, best wishes to you. And Sven, Thank you, I'm sure you're going to hear some folks reaching out to you. So thanks to both of you, I hope you have great holidays.

Cassie: Yeah. Thank you, you too.

Sven: Thank you so much, Carol.

{Music}

Outro Voice: Conversations powered by VR, one manager at a time, one minute at a time, brought to you by the VR TAC for Quality Management. Catch all of our podcast episodes by subscribing on Apple Podcasts, Google Podcasts or wherever you listen to podcasts. Thanks for listening!