Exploring Caption Companion: A Tool for Real-Time Captioning Without Internet

[Music]

>> SHARI: Welcome to This Week In Hearing. I’m Shari Eberts, co-author of Hear & Beyond: Live Skillfully with Hearing Loss. Ever been to a conference or lecture or life event that was hard to follow but no captions were provided? This is sort of a rhetorical question if you’re a person with hearing loss. Even at hearing industry events there aren’t always captions available. Our guests this week are trying to make this situation a thing of the past with their product Caption Companion. Caption Companion is a portable 8-inch tablet that provides realtime AI captioning, and it comes with a remote microphone to capture audio accurately, and offers unlimited captioning without requiring an account or internet connection in most situations which sounds very nice. So here to tell us more about the product are Margaret Downs, the Executive Director of Premier Visual Voice and the creator of Caption Companion, and Mathew Call, Premier’s Director of Technology. Thank you both for being on the podcast today.

>> MATHEW: Thank you for having us.

>> MARGARET: Thank you.

>> SHARI: So before we dive into Caption Companion, I would love to learn a little bit more about both of you. I know that accessibility for people who are deaf or hard of hearing is really personally important to each of you. So I was hoping you could each share a little bit of your backstory and what inspired your work on Caption Companion. So let’s start with Margaret first.

>> MARGARET: Hi, I’m Margaret Downs. I started in 2009 and I was about 43 years old. I was actually looking for a job that I could do that would fit within my realm of needs. I am legally blind so I was actually looking for you know some type of a job that I could do from home. And I came across closed captioning. Um, went to training for it. It took me about two years and um, that’s actually how I got started. I thought, you know, how cool is this that me being legally blind, and actually helping people who are deaf/hard of hearing, and also other people. But yeah I just… With my barriers and challenges that I go through every day with being visually challenged um, it, yeah it just kind of it was like a no-brainer, so… I never looked back. I love what I do, love helping people. And um, I am still currently providing services. So I do it every day.

>> MATHEW: I want to jump in. When you said you found closed captioning, what, do you type out the captions and add them to videos? What do you mean? [CHUCKLING]

>> MARGARET: Well I found it and I was kind of I guess uninformed because I thought to myself, you know, they presented it to me as such and I was like okay what is it? I know I’ve heard of it but you know, what is it? So, researched, and it was — I could do it live, I could do it post production, and there were all these, you know, different opportunities and that’s kind of how that came about.

>> SHARI: Well that’s part of the issue with captioning and just as an accessibility tool, is that a lot of people don’t know about it and so it’s not utilized perhaps as much as it could be if there was more education and more awareness about it, so um I can totally understand that life experience that you had. And so Mathew, what’s your — what drove you to be involved in this project?

>> MATHEW: Well I’m a SODA (Sibling Of Deaf Adult). So my youngest sister was born deaf. Our whole family used Signing Exact English growing up. And then my bachelor’s degree is in Sociology, that’s back in 2003. But I’d studied Spanish and lived in Central America for a while. So while I was working as a Spanish/ English interpreter, that was here in Sacramento, California, I decided to go back to school for a degree in American Sign Language interpreting. And then later became certified. We weren’t really — We didn’t know much about Deaf culture when I was a kid, but as an adult, my sister’s very culturally Deaf, and so we learned ASL and delved in. And then my brother, he was also a certified ASL interpreter. He had an entrepreneurial spirit, right and he had his own company. It was, you know, early 2000s when video remote interpreting was a new thing. And he was tech savvy. So he said you, trilingual interpreter, you come work for me. So I’m like, let’s do it! And we we started this business and he said we should add remote captioning to our list of services. And I said let’s do it! A friend of mine, a mentor who was also a stenographer and interpreter, he introduced me to voice-writing, which is what I was trying to get Margaret to say. We are voice-writers! [CHUCKLING]

>> MARGARET: Sorry! [LAUGHS]

>> MATHEW: We are respeakers. We, you know, of course we worked with steno-writers, we worked with voice-writers, we worked with offline, post production. And, I never looked back. I ended up getting certified as a captioner too. So it started to take off, and we were getting a lot of clients. Now this is how I met Margaret, right? We’ve been colleagues for many years. But this is also where Caption Companion has its roots, so I kind of wanted to move into that because this is before the pandemic, almost all of our clients were on-site and the captioner was remote. So we needed a reliable way for the remote captioner to hear audio from inside the room. That proved to be far more difficult than we ever expected. You think, oh just join a Skype meeting (back before they cancelled Skype). Or join a Zoom or join a meeting. Well you’ve got a vast myriad of laptops and different manufacturers and different operating systems and the microphone inside that device sometimes was horrible quality. And then of course you’ve got the person speaking walking around the room. And so you never knew if you were going to get good audio. The worst irony in our industry is when the remote captioner, whose job it is to provide an accommodation for hearing loss, has worse audio than the person sitting in the room who is deaf or hard of hearing! And that happens far too often. And then of course our captions come out terribly when you don’t have a good audio feed. So we started providing clients with a wireless microphone thinking that would solve our audio problems. Well it was a big improvement but some clients still experienced issues like difficulty connecting our mic to their device or software issues or unreliable internet at the site location. And we finally realized that as long as customers were using their own hardware, there would inevitably be problems. Some companies might be okay with that, right? After all, the issue’s not our fault, it’s on their end. We weren’t okay with that. We needed an all-in-one hardware solution dedicated to captioning that we could package up, send to customers, and then all they had to do was turn it on and tap a button. That way, they could use their own laptop or phone in the same way as everyone else in the room, instead of having to use their device for captioning. So that led to a lot of remote CART success for us. I mean that was really key to our growth. And it created a much more seamless experience for our customers, consumers, and clients. Well once ASR started to become pretty decent, we even added a way to get AI captioning on those tablets so that customers could choose whether they wanted to use a human captioner or automatic captioning. So after building up the company for a number of years, we were acquired. At the time, we saw it as a good thing. So I went over with the acquisition. But that company decided to head in a different direction and had no interest in selling a captioning tablet. That’s when Margaret approached me and she said well if they’re not going to pursue that idea, then I will. And she put together a team of experts. And for couple years, they really went to town on the design and functionality and testing of the device. We’ll get into this later, but when it managed to pass a certification exam that only human captioners had been able to pass, that’s when I knew I needed to join forces. She saw the wisdom of working with technology rather than fighting against it. That’s how Caption Companion came about.

>> SHARI: That’s terrific. It sounds like you guys are a powerful team with sort of complimentary skills and, you know, you can push each other to keep moving things forward which is always good to have in a business partner, right? Somebody who inspires you to keep moving forward, so that’s wonderful.

>> MATHEW: Absolutely.

>> MARGARET: Yeah, definitely.

>> SHARI: So tell us a little bit more about your product, Caption Companion. So you know, what is it, and what are some of the key features that can make this useful for people who have hearing loss?

>> MATHEW: Yeah the biggest thing is it requires no internet, no account, no login, no subscription, no user data collected, and no limits ever. And it comes with its own wireless Bluetooth SmartMic and the accessories you need to ensure a good audio feed. Because remember, audio is what truly determines the accuracy of captioning whether it’s provided by a human or AI. If you’ve got stellar audio, your accuracy will go way up. And that’s — the whole design of the device was around audio quality. It’s very versatile. It can be used in virtually any type of scenario. Tech support is always included. Even though internet is not required for the AI captioning in several languages and dialects, you can securely connect the device to the internet to get — to connect to a prescheduled human remote captioner. There’s 18 languages and dialects available offline, and 81 available online. And of course it’s same- language captioning only. It’s not a translation from one language to another device.

>> SHARI: That’s terrific. I remember in the early days of the pandemic when we would have AI captions on Zoom I definitely had to fight to get those but really was only English not even Spanish or any other languages so having that many languages available for transcription it’s really pretty amazing.

>> MATHEW: Yeah, Zoom, oh my goodness, those were fun times. We actually worked with them, with their accessibility team. It was — that’s an interesting side story. But yeah, there was a lot of effort to get Zoom to finally allow AI captioning in all meetings without different pay walls or different levels or the host having to enable it. I remember your efforts during those years.

>> SHARI: Absolutely, it was definitely a team effort. I think all of us with hearing loss recognized this as just such a critical issue for the community and so everyone was really trying to work with them as much as possible to push them in that direction. And we got there eventually!

>> MATHEW: Yep!

>> SHARI: So it was good to see that. So I think Mathew, you mentioned that, you know, for captions to be useful whether they’re live or AI, they have to be accurate and they have to be well synchronized with the speaker. Talk to me a little bit about what testing you’ve done to verify the quality of captions that you have on Caption Companion.

>> MARGARET: Well exciting news, like we discussed, Caption Companion passed the NCSP certification by the Global Alliance, which has been passed only by humans previously so it’s the first ASR product to pass the test. The testing is very rigorous. So what it does, it tests for not only word-for-word testing but it tests capitalization, punctuation, sentence structure; all of this stuff has to do with the accuracy. We know that in captioning, if you don’t put a comma in the right place, sometimes it uh, it just throws the whole meaning off. So this is the test that Caption Companion passed. So um, anything else you want to add, Mathew, to that?

>> MATHEW: Not really, I mean generally when you see accuracy rates for ASR, they’re using what’s called the Word Error Rate model, WER. It’s easy to get in the high 90 percentages if you don’t have to put any punctuation, if you don’t grade meaning, if you don’t grade capitalization. That’s the secret, like dirty secret behind big tech. “We have this much…” “Look how accurate our ASR is!” Well yeah, try the NER model. Take the NCSP and then see what your percentage is. So that’s what our focus was. We need it to be as accurate as human captioners. That’s what our background is, that’s what we did for a living. So we knew whether we were — and it’s not perfect. Just like human captioners make mistakes, AI still makes mistakes. There’s plenty of room for improvement, But at least it gave us a threshold that we could say all right this has been externally validated. And it’s exciting! We’re always looking to make it better.

>> SHARI: Well congratulations on that. It’s a good — an important milestone. Is there a way that consumers can look that up for various captioning providers, can they look at that rating or that number? Where would they find that?

>> MARGARET: I think it’s on the, the Global Alliance website, isn’t it?

>> MATHEW: Yes, it is. Just go to speechtotextcaptioning.org and you can click Find a Service Provider and it has all of the certified captioners. Members too. You can be a member without actually taking the test, so it has a list of anyone who wants to be in the directory. So anybody can look that up.

>> SHARI: Fascinating, thank you for that information. That’s awesome. So I’m sure you have lots of examples of where it’s been used successfully. Maybe you can share a couple of those with our viewers.

>> MARGARET: Oh my gosh yeah. One word: everywhere! So number one, disaster relief. It would be very useful with the no internet required, long battery life, you know, I mean you just literally, you turn it on and there it goes. It just starts captioning so that is kind of like a number one thing. Your workplace, interviews, job interviews, churches, Bible studies, conferences, seminars, webinars, trainings that you may have, Caption Companion can connect to your computer, your cell phone, a landline, and it captions the audio coming from those devices. Let’s see, anywhere else… doctor’s appointments, over the dinner table, any everyday conversations that you have. I mean it just kind of — it’s almost like where CAN’T you use it.

>> MATHEW: Yeah that’s the right question!

>> SHARI: I’ll reword it for next time, yeah! [CHUCKLING]

>> MARGARET: That would have been a shorter list!

>> SHARI: No that’s, that’s terrific. I guess really like how creative can people be, to find all the uses for it. So say you are holding a conference or you’re a venue and you need to provide captioning for a large group of attendees. Can you use Caption Companion that way? Can the captioning be put on a larger screen or how would you use it to provide captioning for an event like that?

>> MATHEW: Yes it can! You can connect it either through a wired adapter that acts as an HDMI input for a large screen monitor or projector or video mixer. And you can have open captioning, you can have full screen text only, or you can overlay it on top of whatever else you’re showing. Or you can treat it in in more of a “closed captioning” fashion where a venue might purchase several of them and if someone needs one, they check it out like you might check out an assistive listening device. Then they can have captioning at their seat. So yeah, either way it works. And we’ve done it — venues have done it both ways for conferences and conventions.

>> SHARI: That’s terrific. And so you just have to make sure that whoever the speaker is, is using that paired microphone so that you get the good audio I guess right?

>> MATHEW: Yes and also comes with an adapter to get a direct feed from the soundboard or mixer board. So you can get it that way. There’s multiple ways to get audio input. We always work with the organizers to make sure that that’s set up properly, but yes.

>> SHARI: Perfect. So if people are listening and they are in charge of venues, right, and maybe they’re wondering how they can provide the best communication access for deaf and hard of hearing patrons. I mean what’s the right answer? Is it captions? It’s my go-to. I really prefer captions, but I know that there are a lot of different opinions out there. Is there any research that you’ve come across that shows whether that’s true for most people with hearing loss? Are captions the preferred method of accessibility? And, you know, can captions be considered as an assistive technology that meets the requirements of the ADA for these venues and meeting planners?

>> MATHEW: Yes, I mean, we all know how visual sign language interpreters are and they are very important when it comes to people whose sign language is their primary language, ASL for example. But of course if you don’t know ASL, that’s pretty useless. And it turns out that a lot of culturally Deaf are proficient in English and they also can benefit greatly from captions. So yes as a matter of fact, the Committee for Communication Access in America conducted a large survey to investigate consumer use and preferences for assistive communication technology. See how they use the blanket term “assistive communication technology”? Under that is assistive listening. That’s just one way. It was specific to venues in the United States, and they released a report of their findings in January 2024. And I love to read this verbatim, this is directly from their report which you can find. There’s no paywall. It’s ccaa.name. ccaa.name And it says: “Results: If only one technology was to be available, the first choice of respondents was CART or captions, with hearing loops the preferred assistive listening solution. Action by venues to address the various complaints about the assistive communication systems they provide could increase their utilization.” There were all kinds of problems with, I mean I know we’re speaking to the choir but you go anywhere and you ask for an assistive listening device. Well first someone has to know where it is. Then is it working? Then is it charged up? With captioning, it’s there. Maybe you forgot your hearing aids, maybe you don’t know what program to use, maybe they’re not working. Doesn’t matter! As long as you can see, you’ve got captioning. So it really does represent the most universal solution for [communication] accessibility. Now there’s not a one-size- fits-all. We know that, right? There is a time and place for every other type of accommodation. But yeah, when that large survey result came out, it was pretty much confirming what we — those of us in the industry already knew. But at least now we have some external validation that yeah, CART and captioning needs to be more in the forefront of the people that organize these venues, that set up.

>> MARGARET: And you know I think that captions assist many people, you know. We actually, we’re advocates of course for captioning and accessibility and it should just be there, you know, at a venue. Or if you’re an event planner, then, you know, your attendees, your participants don’t have to come to you and say, “Oh you know I need this accommodation” or what have you. Because speaking as a person who needs accommodation, you don’t really want to ask all the time. Or you know you kind of feel that you’re just bothering someone or “No I really don’t need it” or you know, “I’ll be all right.” You just think that way. But if these, at least public places would just, “Here it is. There it is.” “We have the technology and it’s there now.”

>> MATHEW: Yeah and I think that’s what Caption Companion does, is it creates a very simple solution. You just drop it in. Unlimited offline, never have to think, never have to pay again. Whereas before, they had to really figure out, “All right. How are we going to do this? What software solution is going to work? How do we integrate it? How does it work?” There was a lot of tech variables that were barriers to entry, you know? A lot of friction. So hopefully this will eliminate a lot of that.

>> SHARI: Well hopefully a lot of venues are going to be learning about this product and just captioning in general. I mean, I am a — you know, you’re speaking to the choir here, right? I love captioning. Especially open captioning. And I find that so many people in the audience benefit. So my husband has, you know, pretty normal hearing. But I always see him looking over at the captions himself as well. So, you know, I think helps everybody.

>> MARGARET: Yep.

>> MATHEW: Absolutely.

>> SHARI: So I did want to ask you though, because there has been a lot of forward motion in captions. Both Apple and Android now have captioning built into the operating systems. Facebook has captioning, and PowerPoint even has captioning built into its product. So is there something that really sets Caption Companion apart from these other options which don’t require a user to carry around an extra device with them?

>> MARGARET: Well the number one thing is the audio that Caption Companion has. And again, we all know audio is the key to the to the captions. So we have actually one of our beta testers who is an engineer, deaf engineer, and very techy, you know, nothing really gets past him — using the ASR, you know, the AI captioning in this and that. Then tested Caption Companion. And due to the audio quality, the hookups, the way it’s crafted and designed, the captioning is so accurate. He could not believe that just audio would increase accuracy of captions as much as it did. So I would say that is number one. You know there’s no internet, there’s no monthly fees, there’s no sign up, there’s none of that with Caption Companion. It’s a one-time cost, You can run captions 24/7. The implementation of it, the way that it is crafted and designed, designed by people with many many years of experience in the field of captioning. It does address the barriers that we have come across with providing the remote services as Mathew mentioned. I think that another thing is a feature that it has, the custom words you can add. And the way that the text is displayed, you can scroll through the text. If something is missed and you want to go back, you can review it that way. So there’s that feature. And the fact that we have 24/7 technical support, customer service. So we are always here to help with implementation. You know, if you’re at a venue, if you’re anywhere, I mean we’ve got virtual chat, we’ve got texting, we’ve got — so anyway we are here to help.

>> MATHEW: Yeah I mean there’s a time and a place for using your phone quick like just speak into the app. But any time you have to use it for a prolonged period, it becomes a real drag to use your phone for that. I mean you can’t use your phone for anything else. Then you’ve got incoming messages and interruptions and then it’s draining your battery and internet. And of course whatever you’re using might not be free and unlimited. So that’s when it becomes a lot — a lot nicer to have a dedicated captioning device, when it’s beyond just the quick pull out your phone.

>> SHARI: That makes a lot of sense. So if consumers wanted to buy this or venues even, they wanted to purchase one, how would they do that and what’s the cost?

>> MARGARET: Just go to premiervisualvoice.com the Caption Companion page. We have a secure cart and they can order right there. Very simply place your order, pay for the order. Normal cost is $1599 but we are offering a March Madness special. So throughout the month of March, Caption Companion is $1399, so $200 discount. And you can contact us if you need volume, you know, if you need 10, 20 of them at a time you know for larger venues or conferences and that kind of thing, you can always contact us.

>> SHARI: Excellent. All right so as we are wrapping up here, is there any other final thoughts that you wanted to share? Really appreciate you being here.

>> MATHEW: Gosh, no, we’ve covered it all. I mean it really is an integrated hardware plus software package specifically crafted for realtime captioning backed by decades of knowledge and experience in captioning accessibility. So it —

>> MARGARET: We’re just — I’m sorry I stepped on your talking! [LAUGHING]

>> MATHEW: No, fine! [CHUCKLING] I was going to say it’s — it’s in a new category. A lot of people are like well you know is it compliant with this and this and this and this? Well all those are data security. There’s no data! It doesn’t leave the device. It doesn’t even have to connect to the internet. So it’s kind of a new category. A lot of people aren’t sure how to quantify — how to categorize it. It can be compared to speech-to-text apps but it can also be compared to assistive technology but it can also be, you know, it fits everywhere. Where does it not fit?

>> MARGARET: Right. And we’re really, we’re so excited about it and we’re just so looking forward to getting it to the people who need it and can use it.

>> SHARI: Awesome! Well thank you so much Margaret and Mathew for being on the podcast today and sharing all these exciting updates.

>> MARGARET: Thank YOU.

>> SHARI: And like you mentioned, people can learn more at premiervisualvoice.com and on the Caption Companion page. Thanks again, this was a great conversation.

>> MARGARET: Thank you Shari.

>> MATHEW: Thank you Shari, it was wonderful.

[Music]