Math the vote pt II: Polls and predictions

While at Northwestern University, Dr. Alexandria Volkening and her colleagues developed compartmental mathematical models to predict the 2020 US elections. What's a compartmental model? How do you teach kids about math modeling using Finding Nemo? Which is better: the Parmvir poll or the Alexandria poll? We ask her about this and more.

Listen:

Since this recording, Alexandria has moved to continue her work at Purdue, but you can always follow what she’s up to on twitter @al_volkening and learn more about her on her website.

The featured track for this episode comes thanks to Woody & Jeremy and you can dive into so much more of their stuff on bandcamp. We all need a little something funky in our lives.

Watch:

You can find our full length conversation with Alexandria on YouTube. Subscribe now if you like your to see our faces as we talk.

Episode transcript

[Background intro music playing is "Green Dress" by Woody & Jeremy]

Parmvir: All right, friends. So, um, for those of you who don't know 2Scientists already, we are a podcast. And thanks to the pandemic, we are also now video podcast. You can see us, uh, live and in person. Um, and we are here today for the second part of our two parter which we're calling math the vote. And, uh, last week we heard from Thomas Weighill who was talking about some really cool stuff using geometry and gerrymandering.

And today our guest is Alexandria Volkening, who's going to talk to us about how to predict elections through polls, and another mathematician. I suppose it shouldn't be too shocking. So, can you tell us what it was the got you into math in the first place? What inspired you to take that on as your, your academic career?

Alexandria: That's a good question. I would say I entered college, very convinced that I wasn't going to do math or physics because I thought that I hadn't, I hadn't heard of applied math before. Uh, and I had a really good teacher, John Zweck at UMBC who said that mathematicians and medical doctors meet each other, uh, and that you could make an impact on these applied problems.

And so that was the first time, I think, sophomore year that I had heard of applied mathematics. And so that's what got me, I'd say, interested in applied math and it was started out in biological applications. It's more social science applications now.

Parmvir: Very cool. So where was it that you went to study to do these things?

Alexandria: I went to for undergrad. I did my undergraduate at University of Maryland Baltimore. Uh, which is right outside of Baltimore in Maryland. And then for graduate school, I went to Brown university and applied mathematics and I graduated in 2017, spent two years at the mathematical bio-sciences Institute at Ohio state.

And now I'm at Northwestern.

Parmvir: And so were you studying the same kind of things in each place or has your work kind of evolved over the years?

Alexandria: So in, in graduate school, My main focus was on fish patterns. So you can think of like salmon or clownfish, like Finding Nemo, have these patterns on their skin and the patterns form, uh, from pigment cells.

And so humans also have one type of thing went to on their skin. Um, but it's, it's kind of similar to how you can think of birds, uh, flocking together and interacting with each other, the cells in the skin interact together and they form these patterns. My work initially in graduate school was on, uh, understanding how did the pigment cells interact to form the patterns on fish.

But I think one of the benefits of modeling is that you can apply it to other problems too. So I do things like, like election forecasting. I do things, uh, with social movements on Twitter and intracellular transport. So it's kind of broadened it.

Parmvir: Absolutely. And actually, this is what intrigued me about your work.

We'll talk a bit more about the research specifically later, but what is it about your work that allows you to apply this idea to so many things that to maybe the outside observer, they look very different.

Alexandria: Hmm. So I think that's also why I got into math. So when, when I was in undergrad, I, I felt kind of like, maybe I was going to med school, maybe at the last one, maybe I'd go to grad school.

And so you kind of keep your doors open as much as possible. And math has allowed me to be like very broad. Uh, and, and so I think mathematical modeling is just looking at a problem. So really learning more about it. And then just building up a model that could describe that in a mathematical language, that kind of tool building a model can be applied to lots of different problems.

So one is just modeling, I'd say. Yeah.

Parmvir: Um, another thing I noticed on your, your website, cause we scouted you out and you we had a look at your CV. You do a lot of outreach work rather than, I mean, a lot of people are very dedicated to the academic careers and there are probably still scientists, mathematicians, folks who don't talk too much to people who are not other specialists in the field.

And, um, so you've talked to everybody from like kids in school to seniors, to, uh, teaching in correctional facilities. What is your drive to do that?

Alexandria: I, so I it's something my parents involved me in always as a child. Uh, we did, we always did volunteerism and outreach and I think. College, it transitioned from, I mean, I used to teach, like I taught fitness at senior citizen centers in college.

And at some point you just realized that maybe you should use kind of the mathematical skills you have to do outreach as well. And cause that's something that you can bring. Uh, so that part of it is just, I enjoy outreach and this is something I can bring to it. Um, I think. It's for me, it's important to, as a woman in math to also just be visible in the field.

And so to that, to use that, to go into a classroom and to have a. Because usually when I go into the classrooms, students will call me miss so-and-so and they're like, I couldn't be like, oh, actually, you know, I went to school for a long time. And what that means is it's, it's a different title. Um, so I think there's some power in using that platform.

Uh, so I really enjoy it. It's good. I mean, it's like this. So what do you do the 2scientists podcast?

Parmvir: Well, it's essentially to humanize the people behind the work. And also, as you said to demonstrate that we come from different backgrounds and we don't necessarily look like, I think there is still the stereotypical example of scientists in the white lab coat.

And clearly that's not you. I mean, me when I bothered to wear a lab coat, I, I guess kind of, um, but probably Asian woman is not top of the list for people. So yeah, it's for people to have real life examples

Alexandria: I think, I think there's also, I mean, it's, it's, it's a challenge in and of itself to actually find a way to explain like something that you're working on to an audience.

So when I go into, I do, I like to try to find ways of explaining my research on specifically on fish patterns, to children's like kindergarteners and like, you have to. You have to think about the words you're using and put them in a way that makes sense. Uh, and so that's just, I think an interesting challenge in and of itself.

Parmvir:  I have to say that the advantage of working on something like clownfish is it's probably something that catches kids imaginations quite quickly, right?

Alexandria: Yeah.

I think, I think, yes. I think that when you're working with, when you have a, you know, a room of like 20 or 30 kindergarteners and they're all screaming. And so some of them are just, you're just trying to re reach one child. But, uh, in, in that kind of scenario, some of them get, you know, some of them will just hear math and fish and what in the world, like just connecting those components and showing them applied math earlier.

Some of them will only get fish. And we do talk about Nemo when I go into classes and some of them will see this, you know, woman in stem, and for some reason, she, you know, she was able to come in and speak. Right. So they, I think they get different things out of it, but the fish stuff that's helped. I agree.

You mentioned that you, you sometimes wear a lab coat, so you do both math and biology?

Parmvir: I do no math whatsoever. I do as little math as possible!

Alexandria: But your work is on neuroscience?

Parmvir: Yes

Alexandria: Okay. That I feel like that's a field that has brought in a lot of mathematical models in, into it as well.

Parmvir: Absolutely. I mean, the last, I have to say since I moved to the US um, I didn't know, this was a thing at all. Uh, applying the kind of models that people are talking about. And then I realized as we moved on, as my colleagues were asking people to, for example, um, model the electricity within a neuron to see if the shape of that represents the kind of ions that move in and out during an action potential, so when a nerve cell fires. I think that's the first time I started to see work that kind of David and his colleagues do apply to neuroscience. And I'm kind of curious as to how your work works on your neurons specifically because the neurons I work on are among the longest that we have in the body. And I know that the kind of things that you model, um, look at transportation within those nerves. So how do your models map that out?

Alexandria: So, first of all, how long is the, what does long mean?

Parmvir: Ooh. So in humans, the sensory nerves can be as long as, I guess, if you're really tall human, they can be a meter.

Alexandria: That's pretty cool. So, so what I do that has this kind of in this vein of neuroscience, so one of the things is actually, so intracellular, transport is one, but I also do, um, uh, looking at social movements on Twitter and we actually model it like an integrate and fire neuron model. So the idea of some excitement building up in a person related to some, um, feature in the news. And then once that excitement reaches some threshold, you fire off a tweet. And then you'll go back,

Parmvir: That's so cool!

Alexandria:  Kind of fun. Uh, but for the intracellular transport stuff, That's a kind of a new application for me, it's this idea right, for those who don't know, um, neurons have a kind of, yeah, you can think of them as having roadways inside them.

And so you have to, the neuron needs to transport things from one side of the cell to the other side of the cell and with these meter along neurons, which I don't think are the ones that I look at, um, you know, you have to, you have to transport. And so there are these proteins that you can think of like cars.

It's, it's really like hooking onto the road. And transporting a cargo across the neuron. So we just model, how does that transportation happen? What would happen if the transportation broke down? What'd you get a traffic jam inside your neuron? That wouldn't be good. Things like this.

Parmvir: What information do you get from, I assume you work with biologists in order to make these models, or do you extract information from their papers or how does it work?

Alexandria: Uh, initially I think that first you have to just put forward a model that is probably too simple is the reality. Uh, so it's simple. It kind of gets you into the field and then you start to look around and look at the literature and get a better sense of thing. So, uh, for the intracellular transport work in neurons, I'm not working with an experimentalists right now.

I do work with experimentalists for my, my zebrafish work and that slowly. Uh, over years, anytime you can call, anytime you find a biologist, who's willing to talk to you. It's, uh, it's invaluable because they can speed everything up and help you figure out what the interesting questions are.

Parmvir: And so does this also extend to the work that you're doing now in terms of modeling social sciences?

Because this is, I guess this is where the, the polling work comes in, right?

Alexandria: So with the, the election forecasting work, This is our we're kind of stepping into the field. Uh, and so right now we don't have social science collaborators, my experience, I think it's, and it might just be because I come from a math biology background, but I think it's sometimes easier to cross the disciplinary divide between math and biology.

Uh, so I am excited to cross this disciplinary divide between math and social science and getting some social scientists, collaborators in the, in the future. But not yet. That would be extremely cool though.

Parmvir: But what inspired this project then?

Alexandria:  The project that we're working on on election forecasting was, so it started with, uh, a few different folks, Daniel Linder, who is at Augusta university, Mason Porter at UCLA and Greg Rempala who's at Ohio State and it came out of the 2016 elections, uh, where I think we all remember that there was this huge, like all the forecasters were were giving Clinton a big chance of winning. Right. And we can think about what this big chance actually mean, uh, is a separate question, I think. Um, but, uh, we, we realized you have all these different forecasts or some people were giving her a 99% chance of winning others were giving her a 70% chance of winning. And we just didn't understand it. We didn't understand what those, where those numbers were coming from.

Uh, and we wanted to. Bring a mathematical modeling perspective to it. And really, I think the motivation was partly just to understand the process better ourselves, uh, that was kind of the first thing,

Parmvir: One of the most famous examples. I think if you're really nerdy about these models and politics in general is obviously Five Thirty Eight, so for anybody who doesn't already follow them, they, as I understand it, they look at polling data from various different sources. And so we've listened to the podcast where they talk "Model talk" and they essentially try and describe how they kind of tweak um, I guess the algorithm. So how does, what you do relate to what they do? Can you explain, do you know what they do? Cause I have no idea. Like they talk about the model, but you know, to me it means very little.

Alexandria:  So what with Five Thirty Eight, does my understanding is, so first, the one thing that they are extremely good at aggregating polling data and they, they create these publicly available Excel documents that are what our model is powered on of just the polling data from all different polling houses. So this is what a polling organization is, or a polling house is the group that collects polls and, uh, what Five Thirty Eight does. And I think what they're particularly well known for is that they don't treat all the polls equally. They assign different polling organizations or different polling houses, a grade.

So you can be like an eight plus polling house or a C minus polling house. And, uh, they, based on that they will weight certain polls more or less importantly, in, in their work. Uh, so that's one thing that they do. Another thing they do is account for the idea of, um, elections being correlated in different states.

So the idea is that if you're off and you misidentify, who's going to win in Wisconsin, then chances are, you're also going to misidentify who's going to win in Minnesota. So you're kind of gonna hit missing a whole bunch of states at once. This can be because of something like misidentifying what a likely voter is when you construct the poll.

Uh, so they do this, but their approaches are based on statistical modeling. So it's taking in all these polls, weighting them in different ways. What we're using is a dynamic mathematical model that again, uses the polling data, but it's kind of projecting forward. Uh, what's going to happen using differential equations.

Parmvir: Okay. So literally, what is it that you take from the data that's out there? How does it get plugged into your model and then what is it that it spits out at the end?

Alexandria: That's a good question. So what we take in is, so there's two types of data that are commonly used an election forecast. One is polling data and the other is fundamental data, which is meant to be how we're fundamentally making our decisions. That's things like. The economy or, uh, you know, what party we identify with, or if we think that the candidates are good speakers or there are various other things. So we, our model is fully based on polling data. We don't use the fundamental data and what we do is basically take technical and data, and then we average it by month.

So you just get one single polling data point average. And we actually use models that are also used to study, uh, infectious diseases like the flu, uh, they're called, so you may have seen some of these in the news things like: susceptible, infected, susceptible models. These are models that at their core are just ways of taking people in grouping them into compartments. Uh, so you could be someone who is in a susceptible compartment to a biological disease for someone who is in the infected compartment for disease, right? And then you prescribe rules for how do you change compartments? So you might expect if you're a susceptible person, you interact with an infected person, you can change to infected.

So that, that is actually, that is the type of model that we use is if you are a Democratic or a Republican voter, you can interact with an undecided voter and make them change what group they belong to. So that's, that's the kind of structure of our model and we combine it with polling data to understand what are the influence between the vote.

Parmvir: So I guess you've kind of answered David's question, which is you use compartmental models. What are they?

Alexandria: Right. And it's at its very core a compartmental model is grouping people into compartments or, or groups. And prescribing rates for how you change what group you belong to. And it's, it's been applied for, you know, it's applied for biological diseases. It's also applied for social contagions things like how quickly you'll will adapt a new technology, uh, and now to election forecasting. So the other thing you asked was just what comes out of the models. And I think that there is like all different, so if you go to Five Thirty Eight's website, you'll see that there are all different ways that they're visualizing what comes out of their models, right? There are things like electoral vote distributions. There are things where we just say, what are the margin of victory is in a, in a state, uh, for, so that would mean like how much is the Republican or the Democratic candidate winning by? Uh, there's also just, we think with this chance, so-and-so, is going to win the election, right?

So all of these different things can come out of a model and that's, they come out of our models as well. One thing that we haven't focused on as much. What the percent of undecided voters is in each state. And I think that's an interesting thing to think about too. Not just the margin, but who hasn't decided yet.

Parmvir: Yeah. Um, particularly important.

Alexandria: Especially in Florida!

Parmvir: Yes. Yes. Indeed.

Alexandria: Not in Illinois.

Parmvir: Yes. This is true. So following up on that, David says one can make many decisions, taking the polling data and applying mathematical, the mathematical approach that you use, what guides your decision process when making those decisions?

Alexandria: So I'd say we, across our model, uh, we chose to be as simple as possible wherever we could. So the reality is that when I say we use this model that is also used to study biological diseases to forecast elections, that in, in and of itself is a simplification, right? Because those are not, those are very different things, you know, we don't just interact with a Republican voter and change our opinions. It's a whole, whole bunch of things. There's news coverage and maybe peer pressure and all sorts of things can, can interact together. So it's very complex, what's actually going on and for the data, we also chose as simple, a way as possible with handling the polling data.

Uh, so we don't weight it at all. We, we just average it by month. We don't weight more recent polls as, more strongly. You might expect that more recent polls might be better polls than polls from a year ago. We don't account for the fact that some polls are partisan polls. Some polls are framed in terms of likely voters or some polls are framed in terms of registered voters.

So the, the difference there is actually most of the polling data that you see that's publicly available is not actually the raw polling data. It's data that's been weighted and adjusted. And when a pollster calls you and asks who you're going to vote for, they make a decision about whether or not you are likely voter, right? And so whether they're going to include you in their data sets. So we don't necessarily usually it's proprietary what they define as a likely voter. So those are all adjustments that you can make to the polling data butwe just read it all the same. We, so it's all equally good. Uh, which is a simplification cause I'm sure it's not all equally good.

And, and in terms of, right. So it was just, it was always the simplest approach possible was that's how we started this.

Parmvir: Can you just clarify what you mean by the term weighting?

Alexandria: So for instance, if you have a two polls, uh, so you have a poll from, from you Parmvir and a poll from me, uh, if we know that in the past, you know, 10 elections your poll has been better at forecasting the election than mine. Then we might say, well, it's not, we're not just going to average the two of our polls. We're actually going to say yours is better. We should trust yours more. So maybe we multiply yours by 0.7 and then we multiply mine by 0.3. So we just kind of, we say you are, this yours is better we should, we should trust it more. Uh, so that, that would be weighting it in different ways. And that's what, what Five Thirty Eight does. I think that those are really important things to do. And that's what we want to do. Something we want to consider in the future to see does that improve the forecast, uh, in our model. But, but for now it's just simplest choice. That's, it's kind of a basic modeling, uh, philosophy that you go with the simplest thing and you see how that works and then you build complexity for them.

Parmvir: So David says if Parmvir polls are better, why not use those straight away?

Alexandria: Well, okay. So, so, okay.

Parmvir: It's, this is a thing now, Alexandria polls and Parmvir polls are a thing

Alexandria: [laughs] They're only slightly better, okay.

Parmvir: [laughs]

Alexandria: So I think that's, that's also a good question. I think part of it is just that you want also to have, uh, the more polling data that you're using, in some sense, you're getting more samples of people. So it is a question of whether or not you actually want to throw out all the polls and it's not always clear.

You know, it's a 70, 30 divide here. I think that the reality is when you're dealing with elections is something we realized it's hard to even judge forecasts because it's kind of, I mean, there's not a ton of elections that happened in the U S it's a pretty infrequent thing. So, you know, if it only rained every, you know, two or four years it's just a harder to judge a forecasters accuracy, right.

In terms of the weather. If and so I think that's part of it that it's, when it's hard to figure out which polls are are better. Um, and then the reality is some states are pulled at ton. So a state like Florida is going to be pulled a ton. Some states are pulled very infrequently, uh, some states aren't pulled at all. And so if you don't use the polling data that's available for a state, because it's not, not necessarily the best data that then you have no data.

So do you get very different responses from people in different states with regards to the polling information? Do you know?

Well, uh, the polling data we use is, is aggregated by Five Thirty Eight. And so it's the publicly available data, I think you mean in terms of response rate in terms of how they

Parmvir: Yeah. Who participates and cause, I guess you also have to adjust for things like demographics. Um, if a given state is a lot older, I think this is, this is one of the problems that people are incurring now. Right? People don't answer the phone anymore. I certainly don't answer the phone anymore. I let Google filter people for me before, I have to, to, uh, contact another human. Um, I guess this, this is not necessarily your work or is this something that you consider when you're creating your models?

Alexandria: I think that that is a really important research question and identifying what's going on with like behind the scenes. So this is, this is not my work right now because we just take the publicly available, polling data as is, but behind that step. So before those Excel documents become, become a thing, it is true that there's all these issues of what is your sample size? What is the mechanism that they're using to create that sample?

So. Uh, some polls are done online. Some polls are done by cell phone. Some are done by landline, right. They're, I read about a poll that was done over a video gaming console. So that's just choosing very different parts of the population. Right. And so all of those things are things I think the polling houses will take into account when they weight their polling data before we behind the scenes, before we actually see what comes out.

Parmvir: So David was saying, have you tried different versions of your model other than the one that you have put on the website? So are you, um, I guess he's talking about github is where you've been sharing all your info?

Alexandria: Gitlab,

Parmvir: Gitlab.

Alexandria: Okay. We have, uh, we have only done this disease transmission model so far. We've started to look at. If you were to only use. So if you were only using likely voter polls, so what would happen if you cut out some of the, some of the polls, because you think the registered voter polls are maybe worse, does that improve the forecast?

And what we're seeing is that for the subset of elections, that we've done this analysis for that it improves it in some states and in other states, not so it's hard to draw conclusions and we want to see if you look up. You know, the, the four years that we have for elections that we have polling data for, uh, does it, does it approve it consistently in Minnesota for example, if you use likely voter polls rather than registered voter polls and does it consistently not improve it in Ohio, we want to tease that out. But at this point it's unclear. I think there's a lot of questions left and that's part of the reason we put the work on Gitlab as we, as we wanted to encourage other people to get involved too, because we're not election forecasting experts so it's, it's really, it's a way of putting it by using the disease transmission models. We're putting it in kind of a multidisciplinary framework that we hope will be familiar with other people as well.

Parmvir: Yeah, this is, I think this is the, the interesting thing about your work is that it's so different from what up until recently biologists were doing, which was, you know, they they'd hoard all of their stuff and then they'd publish it 18 months later, at which point, you know, the field's probably already moved on to some degree, whereas I feel like um, certainly within the modeling world, the data and the information is much more transparent. Right?

Alexandria: I think it's also, I guess it's very different too, because my understanding is when you're a biologist, you're putting much more effort into these experiments that can take so much longer. And I mean, there's what stresses me out about a biologist's job is that they're dealing with equipment that is so expensive, I couldn't imagine. This is, I'm a person that packs two shirts in my conference bag anytime I go, for when I inevitably pour coffee on my shirt.

Parmvir: [laughs]

Alexandria: So my coffee cup, just never goes on the same desk as my computer. Cause I am incapable of that. I think, I imagine that's part of the reason that mathematicians can have a little more transparency is that they they're things aren't taking the same amount of time in the same way as biological experiments.

I don't know if you have thoughts on that.

Parmvir: I, I think my feel was more that people are so worried that they're going to get scooped, that they were not prepared to put things out there, except COVID seems to have changed that, um, for better or worse, I guess. I mean, There's so much information coming out now that it's it's to some degree, it's worrying to see what people had just prepared to put out there without, um, uh, kind of the number of replicates to probably make these fair assumptions.

Alexandria: I think even for me, so we put our first forecast stuff in, in 2018, right before the midterms, just on ArXiv (pronounced archive), which is a pre-print server. And I found it extremely stressful. Usually when you're working in biological problems as a mathematician, I think it's like you put out this paper and you know, you wait a couple of years for them, like for the experimentalists to tell you why you're wrong.

And so it's not like, there's a delay and, and then you can build on it. And, and I think, uh, for the election stuff, it was terrifying to be immediately right or wrong. And I can imagine with the COVID stuff that, that is just, I mean, it's an entirely different level of importance.

Parmvir: Quite, I mean, I know colleagues in the field who are distraught at their own colleagues now, people who are working in immunology and virology saying, just stop please. Um, but getting back to the polling. So David says, hypothetically, now it's November 4th or fifth or sixth, or whenever the final results are in. Um, how will, you know, if you got it right. Or does it take a few elections to have enough data to prove or disprove your methods?

Alexandria: That's a, that's a great question. So what I like to say is so Five Thirty Eight last, last election, so 2016, gave Clinton, right? It was about a 70% chance. And then Trump about a 30% chance. And so people will say they got it wrong.

And I think the reality is that is like, if we all were to take a quarter and flip it twice, If we flip it twice and we got two heads, we wouldn't be in utter shock. I mean, if we flipped it a hundred times and we got a hundred heads, then when we would be in shock, but, but really that's, so there's a 25% chance of flipping a quarter twice and getting two heads.

And, and so in that sense, the reality is a 30% chance for jump is, was high in 2016. So I think part of the issue is that when we all look at these forecasts, and I do it, I look at it and I just look at the color of the states and I want to know, is it red or is it blue? And you kind of, you go through it and you count them up.

And you're like, okay. And, and the reality is what you should be looking at is the, the confidence interval. So what is the range of the forecast? Uh, so in many of our forecasts, we have a Democrat, uh, when projected. And so the bars are all blue. Um, but the reality is if you look at the percent of undecided voters that are in those states, it's actually higher than the margin of victory.

So it's entirely possible that that could be upset. So I think what you need to do is look at a forecast, just like you said, David, you need to look at it across many different election years. And it's challenging because there's only so many election years that have data available, uh, and, and polling data available.

But what you'd want is to be able to put forward you know if you put forward four forecasts, you'd hope that, you know, if you're forecasting that there's a 25% chance of a certain candidate winning, you'd expect to get it right about three or four of those times.

Parmvir: So I really love your coin toss analogy, because that is, I think that the best demonstration, you know, people, as you say, they, they like to look at the maps and they like to see, okay, is this going to go blue or is this going to go red? Whereas by saying that it's like, yeah, of course, like if you flip the coin three times, it's likely to happen. Um, and I think that a lot of people's perceptions of whether these things work well or not is based on, um, you know, not being able to understand what the statistics mean.

Right. So you give percentages to people and it, it doesn't easily equate, especially when things are emotive. You know, you tell somebody that they have an X percent  chance of being afflicted by a particular disease. Like how do you take the emotion out of that and just deal with the numbers that you're given?

Alexandria: Yeah. That's a good point because the numbers also mean, I think very different things, right? Because if it's a 20% chance of raining tomorrow versus like a 20% chance of getting in a car accident tomorrow, like it all would like exactly what you're saying, the way we interpret these things is quite different.

Parmvir: He also said what results could come in in the election that would make you say yes, we got this one wrong.

It's an interesting way of phrasing it, David.

Alexandria: [laughs]  I don't, I don't, if anything would make me go, "yes! We got this one wrong", quite like that. Um, I think so there are different ways of judging forecast accuracy. Just looking at the colors of the states and just saying, did you identify the right winner? And in a state, I would say that is the easiest way to judge forecast accuracy, but it's probably the least meaningful.

Uh, I think what is often more meaningful is looking at the vote margin. So if you, if you are, for instance, projecting that Trump will win a state by this two percentage points. And we, he ends up losing by you know, 0.5%. So that's actually a pretty, you were pretty close at your forecast. So if you're looking at those differences in the vote that if our differences are small, that would mean we have a pretty good forecast this year.

If the differences are quite large, if even if our forecast for a given state, is that Trump will win by, you know, 20 percentage points if he ends up winning by 50 percentage points, then that was perhaps a bad forecast because we really missed, there was a huge error in the, in the vote margin there, even though we identified the correct winner. So I think looking at vote margin is important and in, more generally just using this as, so right now, polling data for is 2004 through 2016. So we'll pop 2020 in there as well. And see what was our accuracy across those years. We want to be able to say when we're 75% confident that we in fact get 75% of those races correct, that would be good.

Parmvir: So on the subject of, um, kind of outcomes. So you've been talking about how you do these things. Um, what's the why behind doing this? So we're, we're kind of curious. I mean, we were talking to, um, Thomas last week because he's part of a group who is studying gerrymandering in order to be able to work with local groups to help, um, establish fairer redistricting. Um, so on that note, David says, we know you've given a lot of presentations recently on the basis of this work. Have you been contacted by non-academics looking to, um, try and get some understanding or maybe gain an advantage through these predictions?

Alexandria: I did get to meet a former Senator that came to a conference talk once, which was totally fun and, um, unexpected. But, uh, so no, not at this point. I think the reality is this model is simplified. It's it's going to come out in a, in a couple of days, um, in, in published form, uh, I think really our motivation is more mathematical with this problem where what we want to do is kind of invite a larger audience to engage with election forecasting. And just because we, I think as part of it, it just, we didn't understand the process particularly well when we started and we wanted to just bring a new perspective to it. So I think hopefully what it will do is raise questions within the mathematical community and maybe other communities as well.

So I'd say hopefully the next research that comes out of this, well, we'll have more of a real world impact thing.

Parmvir: Um, so you you're quite, um, happy to share various bits of the information that's coming out on your Twitter account. Um, have you been contacted or, uh, you know, has anyone spoken to you on Twitter about your results and made any comments as to what they think? Um, particularly people who are, you know, and they're not experts in the field and they don't maybe understand how the models work?

Alexandria: Most of the audience so far has been an academic audience. I gave, uh, one of the conference talks I, I gave up last year was, it was really cool because I so I did a poster on elections and, uh, a couple of folks came by and you kind of ask what their background is to get a sense of how to pitch the presentation.

And it turned out that they were actually family members who were with a mathematician at the conference. And so this was like the poster that was accessible. And, uh, that was, cool to kind of cross the, that, so it wasn't an academic community, but it I'd say most of the case in Twitter is predominantly academics so far and mathematicians based.

Parmvir: So I was going to say, let's leave this on a positive note, but I was wondering, uh, do you have any like coping mechanisms to deal with the influx of election based news? Or just switching things off.

Alexandria: Yeah, I will. Netflix is good. Um, coffee and tea is good. I think I have. How about your Parmvir? What is your...

Parmvir: uh, actually I, currently I am drinking something which is a pineapple upside down sherbet wheat. It's ridiculously all the things and it's very sweet. Um, yes, a lot of drinking and watching political TV shows that make us wish politics was better.

Wonderful. Well, thank you so much for your time, Alexandria we really appreciate it. And um, we look forward, at least David we'll look forward to seeing your paper when it comes out and maybe he'll translate it for me.

Alexandria: Thank you both.

Parmvir: Cool. Thank you. And Jill also says thank you. All right. Take care. Bye.

[Musical interlude]

Alexandria: So. Before I, I knew better as a senior. I once gave a research presentation in a cheerleading uniform, because no one told me you weren't supposed to do that. Um, I had to go to a cheerleading game afterward and I didn't want to waste time changing my outfit.

[Musical outro]