Episode 44: COVID-19 Exposure Assessment Tool

Lisa Peña (LP): A new data-driven tool developed by scientists determines the risk of COVID exposure at events, meetings, in your workplace, anywhere. We'll tell you how this tool analyzes different scenarios to manage risk next on this episode of Technology Today.

[MUSIC PLAYING]

We live with technology, science, engineering, and the results of innovative research every day. Now, let's understand it better. You're listening to the Technology Today podcast presented by Southwest Research Institute. Transcripts and photos for this episode and all episodes are available at podcast.swri.org.

Hello and welcome to Technology Today. I'm Lisa Peña. More than two years into the pandemic, scientists are working from all angles to keep the public safe. Scientists from SwRI's wholly-owned subsidiary, Signature Science LLC in Austin, have developed the COVID-19 Exposure Assessment Tool, C-E-A-T, or "ceat."

Signature Science president and CEO, Brian Schimmoller, who led the development of CEAT, and director Dr. Molly Isbell are here to tell us about this useful and innovative risk management tool. Our first guests from Signature Science on the podcast. Thanks for joining us, Brian and Molly.

Courtesy of Signature Science LLC

The interactive CEAT PDF allows users to fill in specific information to determine whether a particular gathering is a low to very high risk event. Factors that influence the assessment include mask use, distance between people, size of the group, duration of activity and community spread. Developers are now working to create a more user-friendly version of the tool.

Dr. Molly Isbell (MI): Thank you.

Brian Schimmoller (BS): It's good to be here.

LP: All right. So to start, will you give us a quick introduction to Signature Science? Tell us about it.

MI: I can start, and Brian, if you want to chime in. Signature Science is a multidisciplinary scientific consulting and services company. Most of the work we do has something to do with signatures, that's where the name Signature Science comes from. It has to do with signatures associated with chemical, or biological, or other threats to Homeland Security, to military applications, to the public health.

The work we do includes developing and building chemical sensors, we provide quality assurance support and performance monitoring for major government sampling and analysis programs that detect biological and chemical threats. We perform DNA analysis, both for law enforcement applications and other government applications, and hand in glove with that is the development and the use of advanced bioinformatics tools to mine the DNA sequencing data. So that's just a variety of the kinds of work that we do related to chemical and biological signatures and threats.

BS: I'd like to also add that we've been part of the Institute since the company was founded in 2001. And both Molly and I have been with Signature Science since its founding, and we've worked with the institute on different programs as well, either as a subcontractor to the institute, or the institute was subcontracted to us.

LP: All right. So you closely work with Southwest Research Institute on a number of projects, but today we're talking about a Signature Science project, the COVID-19 exposure assessment tool CEAT. What is CEAT? How do you describe it?

MI: So CEAT is an interactive tool that allows users to enter information about a scenario. Typically it's when you're going to have people gathering together, it was really developed initially for use with Signature Science. We were thinking about bringing people back into the workplace.

So it allows users to enter information about the number of people that might be in a gathering, information about the venue, is it outside, is it inside? What's the size of the room? What's the ventilation of the room? How far apart are people are going to be? How long is the event? And it takes all of that information and synthesizes it together to come up with an estimate of the exposure risk associated with that event. That's kind of the short answer about what CEAT is.
 

Signature Science President and CEO Brian Schimmoller led the development of CEAT. He says the tool was first created to assess the risk of COVID in the workplace and to safely bring back staff to the office. Since its creation, uses for the tool have expanded to include any gathering.

BS: I'll add in that when we were first starting to think about CEAT, and work on CEAT, it came from wanting to, ourselves, understand the risks that our own employees would have coming back into the workplace. So that was sort of the early thought about it is, let's develop something that we can use for ourselves to understand how to manage the risks for our employees, whether they're working in our laboratories, or working in our office spaces or meeting rooms.

LP: And how is risk presented with this tool? Does it spit out a percentage, or how does a user see risk once everything's figured into the tool?

MI: Yes, it's actually a ratio, or a relative risk. So rather than being a percentage risk, really what we do is we identified a scenario that OSHA had described as being a high risk scenario. So this high risk scenario back in kind of the spring of 2020, OSHA said if you have two people sitting in a 10 by 10 foot room, sitting three feet apart, speaking for 15 minutes without mask, one of whom is known to have COVID, the other one doesn't. That is a high risk scenario.

And then what CEAT does is it calculates the risk in a scenario that we define based on how many people are in the room, how big the room is, what the activity level, how long the activity is. It says, "What's the exposure risk in this particular setting relative to that high risk setting?"

So it's really just a ratio. So if that ratio is equal to one, then we say that's exactly equal to this high risk scenario. So you're high risk. If it's ten, or a hundred, or a thousand, then you're jumping into the height of very high risk. If it's less than one, then we're saying that you're less than that high risk scenario. So it's really a relative risk value that it gives.

LP: OK. So the user ends up with a low to very high risk categorization once all that information is put in. OK. So how does the tool calculate this risk? You touched on it a little bit, but how would someone use it?
 

Signature Science Quality Assurance and Data Science Director Dr. Molly Isbell is part of the CEAT development team. She says scientists have modified the tool over time due to changing variables like new virus strains and the introduction of COVID-19 vaccines. CEAT is a flexible tool, adjusting for changes in infection and vaccination rates.

BS: So I'll start with that one. The first thing we do in doing the calculations is try to determine what the amount of the COVID material, we'll call it, would be in the air in the space? And so that's really been being driven by the ventilation rate in the space, the emission rate that could be happening from people in the space of COVID particles, or COVID aerosols, and then the size of the space.

So that that's the first thing we do is try to figure out how much material could be in the space. And then from there, we look at the effect of masks from two perspectives. One perspective is the effect of masks in halting any emissions that might occur if someone's wearing a mask but is infected, and the other way we look at is from the effect of someone wearing a mask and the mask protecting them from breathing in the emissions of others.

LP: So we're two years, more than two years, into the pandemic now. Do you feel like this tool is still useful, or why do you feel it is needed at this point?

BS: It's interesting that as we've been developing this tool, you have the sense as you're working on something like this over time is that well, maybe this isn't so useful anymore, because maybe we've gotten to the point in the pandemic where the pandemic might be behind us. But we are continually, in some cases, surprised that this thing tends to hang on and still has relevancy even today within our workspaces. We continue to be concerned about COVID transmission.

And like many other workplaces, we are seeing some cases, not necessarily coming from our workplaces, but among our employee population. So we are continually concerned about that, and have used the tool to help us inform policies, even in the last few weeks here in our offices at Signature Science.

MI: And one of the things that I think is pretty useful about the tool, even still, is the fact that it allows you to take all these different factors into account kind of all at once, which is what everyone is trying to do intuitively anyway. It's "OK, well, there's been an increase in the local community, or I'm starting to see more cases in my workplace, are people in my workplace becoming infected?

How do I take that into account? How do I take that into account if I've only got three people in a room? How do I take that into account if I'm in a bigger sized room, or based on what the activities are, or how long it's going to be?"

All those different factors play in together and affect each other. And that's one of the things that I find really useful about the tool, is that it pulls all of that together in a very systematic way to say, "OK, now what's the overall risk taking all of that into account?" Because sometimes it's really hard to sort through just in your brain.

BS: The other thing I want to add is beyond COVID, there are other respiratory viruses. And even today, in our discussions with our health and safety team, we have a situation where an employee has a respiratory virus that isn't COVID. And by having this tool, and having thought through all the dynamics associated with the transmission of respiratory viruses, we're able to apply the tool's capability, potentially, to those viruses as well.

And we're looking forward to finding opportunity to add new viruses, and expand the tool's capability in that way.

LP: OK, so it's flexible. It is called the COVID-19 assessment tool, but as you just mentioned, it can be used probably for flu, maybe a cold if someone wanted to use it for that, is that what you're kind of seeing for the future of the tool? Expanding it to different areas beyond COVID?

BS: That's right. It would certainly need to be expanded, it doesn't currently do that. But a lot of the underlying capability and subparts of the tool would apply to other respiratory diseases, too.

LP: All right. So we've talked a little bit about how to use this tool in the workplace, but in what scenarios do you feel that it is most useful?

BS: So my perspective on using it is as a planning tool is the way that it is most useful. And one of the scenarios that we've seen in our own workplaces, we have training programs where we have clients come in for training, and we can use the tool to think through various mitigation strategies for those training events. You know, how can we keep both our employees and our clients protected as best as we can, with being informed by the tools?

So we can look at two different scenarios. One where, for example, we looked at scenarios where we move training events into larger spaces to see what the impact of that might be. We looked at scenarios where we limited the number of people in the training, to see how that might affect the risk, as well as adding masking and ventilation strategies.

So we were able to look at various options and decide which would be the most prudent option for us to take to provide the service that our clients were looking for, as well as to do it safely.

LP: And what industries do you think would benefit most from this tool?

BS: Well really it can work across many, many industries. So anything from office work to educational scenarios. We've even seen it used by NASA in their trying to manage their workplace risks.

MI: There's also a group that's using it, and I don't know the name of the group, but a group is using it to plan their choir practices. So there's some choir director who's conducting various choir rehearsals, and that's obviously a scenario you're worried about. People are singing and breathing in close proximity to each other. And I know that she's used the tool to figure out how to do that most safely, where to do it and for how long, and how many people at one time.

LP: So if you're planning any kind of event, meeting, even just hanging out in the same room with a group of people, this tool sounds like it can be useful for just about everyone. How accurate is it, and how do you determine the accuracy of the tool?

MI: Yeah, that's kind of a complicated question to answer. Because the bottom line, it's sort of accuracy is simple. It says that if you're in a high risk scenario, does the tool tell you you're in a high risk scenario? If you're in a low risk scenario, does it tell you you're in a low risk scenario? That's sort of the definition of, "Is it giving me an accurate answer?"

But it is complicated to assess the accuracy, because it's not like you can, like you can with a lot of other things, go out and conduct a controlled experiment to put people in a room and see if it turns out to be a high or low risk scenario.

One of the things that we did was to look at various publications. There's been a lot of different publications about different transmission events that have occurred, and we identified, I think it was eleven different transmission events where there was enough information in the publication for us to plug the data into CEAT to say, OK, if we had been planning this event and known in advance the size of the room and the number of people that were going to be here, and when it was happening, and what the rate was in the community at the time, what would CEAT have told us?

And in all eleven of those cases, it turns out that, sure enough, CEAT said that you were in a high to very high risk scenario. And those were associated with transmission events. So that was at least a demonstration that the tool is giving some reasonably accurate answers.

The flip side is a little harder, and something we're still kind of looking at how to assess, which is, what if you're in a low risk scenario, because no one's really publishing papers about low risks, and you're like, we had this event, and we did these measures, and no one got infected as a result of the event. We don't really quite have as much publications to work with. So that's probably a little bit more anecdotal from the low risk scenario.

In terms of just kind of how the overall answer is giving. One of the things we have done, though, is to look at different components of the tool. And this is probably getting a little bit more in the weeds, and Brian might want to jump in a little bit on this too, but one component of the tool, really, is taking into account sort of the air dispersion. How much might be circulating throughout the air in different scenarios?

And so there have been published papers where they do, in a room, measure CO2 in a room, based on the size of the room, and the ceiling height, and the numbers of people in the room, and how long were the people in the room where they're measuring the CO2. Kind of as a proxy for how much virus particles might be being emitted. And we have taken that portion of our model and compared it to some of those published studies to understand whether or not that portion of our model is giving accurate answers.

So we've been able to take different pieces of the model. Same thing with masking. We look at what's been published with respect to mask effectiveness, and then that feeds into the model. So components of the model have been validated to be accurate. It's that final answer is the harder thing to do, except to use the published events.

LP: Could this be an app one day for all of us to download and use? I mean, it sounds pretty user friendly. You just put in some data, and you are given a load of very high risk scenarios. So do you see it being an app?

BS: Certainly. It is available now for anyone to download, number one. But it is not an app in the sense of it's operating on your phone, or iPad. But you can use it on your computer by simply downloading it and using the PDF format of it.

But in terms of developing an app, that is something that we are in the process of working on. And we're looking for partnership opportunities to do that as well. And we're working with several other organizations to do that.

LP: So what we can do is we will make that available on the web page for this episode, where you can find the PDF. So what factors have you discovered are most effective in lowering risk?

MI: So that's really a-- I think it's a "it depends" answer. And it's actually one of the big reasons that we developed the tool to begin with. Because it's really the combination of factors that I think affect the risk in a given scenario. And some of those factors are in your control, and sometimes they're not. Sometimes you can change the number of people, or you can change the venue, you can move to a bigger room, you can reduce the amount of time. Sometimes you can't change those things.

And so I think what drives what you're going to do in a given situation depends on what you can control, and what you can't control. And that's, again, the whole reason CEAT was born, is to sort of take those factors into account to a systematic way to figure out what's optimal for your scenario.

So I think all of the factors matter. But the dials you turn in a given situation depend a lot on what it is you need to do, and what your constraints are. Obviously the rate in the population is a big driver, right? So as the incidence rate in the population increases, you start to see these spikes, that has a big impact on what other mitigation measures you needed to put into play. Or if you even need to put any mitigation measures into play at all. So that's a pretty big one.

LP: Let's talk a little bit about the tool's flexibility. When those numbers change, the tool is able to change right along with those changes.

MI: Yes. And in fact, that's why we built that into the tool. And this is actually another thing, as the pandemic progressed, the tool also evolved. Right? So when we first started developing the tool, vaccinations weren't available. But then, suddenly, vaccinations became available, and we realized, "Hey, if we don't put vaccination as a factor into this tool, we're not getting a very accurate answer." So we need to understand what the vaccination rate is in the population, we need to understand, is this a group that, maybe if you're 100% vaccinated, that actually is going to change the risk of this particular group?

So we had to build that in. We also had to build in testing protocols. So what if, before an event, you require everyone to be tested? How does that change the answer? And that wasn't something that was available when we first developed the tool. In addition, when we first developed the tool, it was just COVID-19. That was what it was now. We've had multiple variants since then, and we've had to then build that into the tool as well to say, "OK, now if it's the Omicron variant, how much more transmissible is that than the original variant?"

So we have had to evolve the tool to accommodate these different changes as the pandemic has progressed and evolved. And we've taken all that into account now.

BS: So Lisa, you were asking a few minutes ago about the factors that make the most difference--

LP: Yeah.

BS: --in the result. And one of the-- really two of them stand out in my mind is the effect of masking that makes a very big difference in the result, assuming that people are wearing good masks. So N95 masks that are being worn well. That makes a very big difference in the results, and in the public health guidance, I think is consistent with that as well.

The other one that makes a big difference is whether an event is indoor or outdoor. And that's sort of an intuitive result as well, and really, the model is consistent. It shows that outdoor events are much, much, much lower risk than indoor events.

LP: So have either of you used the tool to figure out if you're going to attend an event or a meeting, or travel to a location? Have you used it personally to make a decision?

MI: I have used it for a couple of different events that I was going to go to. And if anything, it was kind of more just to convince myself that sure enough, given the size of the room that we were going to be in, and the number of people, that it really was safe. So I've used it personally a little bit. But more than that, I think, probably the most that I've actually used it has been kind of more in the workplace setting when we're talking about having a meeting in a room, or something like that. But I have on a couple of occasions used it. I don't know, Brian, if you use it personally or not.

BS: So I've used it with two weddings that happened in my family, my son and my nephew both got married earlier this year in January and February. So I was running the model for those events, and it helped sort of inform, at least, my own behavior. And I show the results to my family and extended family, and suggested they take certain actions as well.

MI: Did they listen to you, Brian?

BS: In some cases.

[LAUGHING]

LP: And that's a real world example of how the tool can be useful. So how did those events end up shaking out? Was there any COVID transmission that you know of?

BS: There was some COVID transmission in one of the events.

LP: And you wore your mask, so you were safe?

BS: Correct.

LP: And the tool helped you decide to do that. That's really cool. That's great.

MI: I actually attended one event that I had run the tool ahead of time, and kind of convinced myself that it was going to be a relatively safe event, given that people were going to be reasonably well spaced, and it was a huge room. And it was a few days later, one of the women came back and said, "Hey, I just want you guys to know, the next day found out that I had COVID." And it turns out no one else in the room was infected. So I think she had it before the event.

LP: Yeah.

MI: And no one else-- and so I felt kind of good about that, you know? So it's pretty anecdotal.

LP: That's great. Real world examples of how it's working. So what is the next step to getting CEAT out and in use? We talked about the possibility of an app, but what does the future hold?

MI: Well, I think that is what we're working on. We are working on a web-based version of the tool. We're also working on a somewhat simplified user interface, when we developed it, we said we want to make this easy to use. And I think it is relatively easy to use for someone who's used to thinking about these kinds of things, who kind of understands the different factors.

But it can be intimidating. It's not something that my great aunt would use to plan the next family gathering, for example. She would look at that and say, "I don't know how to fill this." Or even someone just kind of needing to make a faster decision. So we're looking at trying to make a somewhat simplified user interface, and make it available on the web, so it doesn't involve having to download this PDF file. And yeah. That's probably, I guess, the next steps.

And I think as Brian mentioned before, the other thing that we're looking at is beyond COVID. Can it be used for other respiratory diseases that are transmissible via respiratory routes? And I think that we can. It's going to take a little bit of work to modify it for other diseases, but I think it can be modified for that, and I think that would make it an even more useful tool kind of generally.

BS: And we've also partnered with the American Industrial Hygiene Association. And as part of that partnership, we let them use the tool for one of their programs that they were doing with the CDC. And then we're also looking to develop a more simple version of the tool that would sort of have less styles and numbers to be entered that would be more easily used by a layperson, let's say, to assess their exposure risks.

LP: So if we go back to the app, or maybe it's a web page where you tick some boxes, but user friendly, it sounds like, is the future for this tool. So when possible, we do like to get to know the people behind the science. And Brian, you have a background in meteorology and weather. So do you use that for the work you're doing now with Signature Science? How does that come into play?

BS: Sure. I do, in fact. Most of my work in Signature Science has always involved the measurement of chemicals or other threats in the atmosphere, or air in general. Whether indoor or outdoor. So to some degree, this CEAT project is an outgrowth of that, and my interest in meteorology has always been coupled with an interest in air pollution meteorology as well.

I'm the sort of person that's fascinated by smoke plumes, for example. You know, I could just stare at those, and be mesmerized by them. And think about the science and the engineering, and all the dynamics that are happening when you see a smoke plume coming from smokestack, industrial stack, it doesn't matter. I'm fascinated by that, and always have been. And have worked on lots of different projects that involve both the modeling and measurement of air pollution.

LP: So I think next time I see a smoke plume, I'm going to look at it a little differently. So Molly, you also have a really cool hobby that does have some parallels to your work. Tell us about it.

MI: So I love to sail. I just enjoy sailing in general. I love being on the water with the wind in your face, it's just rejuvenating. And sort of the purity of sailing, where you're only using the wind and the sails to move, I mean, I love that feeling. But what I really love is sailboat racing. I do a lot of sailboat racing. And when you're racing, there's a lot of different things that you have to really focus. There's a lot of things you have to take into account.

Some of these things are physical, some are mental, some are within your control, some aren't. It's the boat handling and the maneuvering, and trying to figure out the fastest way to get around the course, and you're taking into account the fact that the wind is always shifting, and what it's doing right here is not necessarily what it's going to be doing later, or what it's doing in a different point of the lake. And trying to take all that into account, and all those factors into account. So there's at least a link there between the fact that you're having to think about and manage all these different factors to move as quickly as you can, that's sort of similar to what we're doing with CEAT. There's all these factors you have to take into account to make decisions.

But the other thing I really love about sailboat racing is the teamwork aspect. You've got your helm's person, the person that's driving the boat, you've got someone else that's usually calling tactics. You've got someone that's trimming the jib, someone that's helping with the tacks and the jibe, some people really, we call them ballasts, but they're helping to keep the boat balanced, which is actually pretty important when you're trying to make a boat go fast. And it's really one of the things that is really enjoyable about the racing on a team in a boat, is everyone knows their job. You trust each other to do the job, you anticipate what's going to happen next. You know enough about each other's jobs that when things go wrong, because they sometimes do, you're able to just step in and help each other. So that makes it really fun to sailboat race.

I really feel like that at work a lot, too. I feel like I work with really excellent people who are smart, they're competent, we all have our roles. We trust each other to do our roles, we work well together. Sometimes things go wrong. Sometimes it's something in our control, maybe we executed a maneuver badly, or we made the wrong tactical decision. Sometimes it's outside of our control. But we always know, really, what we're aiming for, which is to do good science on behalf of our customers. And we're able to kind of come together as a team and work it out, and get to the right solution. So again, I feel like there's at least that connection between the work that I do and the sailboat racing.

BS: So on this CEAT project, it was a team effort as well, where Molly and I were chiefly involved in the development of the tool. But we work with some other folks outside of Signature Science as well that were involved in helping us use the tool and apply it in certain cases. And then also write a paper that not only talked about the technical aspects of the tool, but its context in sort of the public health and epidemiology communities. So just wanted to recognize some of those other folks that worked on it.

And those included Nidia Trovão from the National Institutes of Health, and Afshin Beheshti from KBR NASA. And then also Som Dutta from Utah State University. So each of those helped work on the program. And then other folks who were using the tool at NASA, Chiefly Benjamin Heck who used it at NASA's Ames Research Center.

LP: All right. So whether sailboat racing or developing the CEAT tool, it really is a team effort. And your team has done great work here, it's been great to learn a little bit about both of you. And of course, thank you so much for telling us about the CEAT tool. It's a great way to arm the public with knowledge. And I think we're all still trying to navigate gatherings during COVID, and this tool really takes this hard data and shines a light on the big picture.

So I can't wait until it's an app, I will be one of the first to download it, or go to the website, and click in those boxes to see how it works. So again, thank you for your time and for telling us all about it, Brian and Molly.

MI: All right, thank you.

BS: Thank you, it's been fun.

And thank you to our listeners for learning along with us today. You can hear all of our Technology Today episodes and see photos and complete transcripts at podcast.swri.org. Remember to share our podcast and subscribe on your favorite podcast platform.

Want to see what else we're up to? Connect with Southwest Research Institute on Facebook, Instagram, Twitter, LinkedIn, and YouTube. Check out the Technology Today Magazine at technologytoday.swri.org. And now is a great time to become an SwRI problem solver. Visit our career page at SwRI.jobs.

Ian McKinney and Bryan Ortiz are the podcast audio engineers and editors. I am producer and host, Lisa Peña.

Thanks for listening.

[MUSIC PLAYING]