Twitter ‘isn't really making all the details clear’ to Elon Musk: Bot expert

Kaicheng Yang, a computer scientist on the spam bot detecting team at Indiana University, joins Yahoo Finance's Alexis Keenan to discuss the bot dispute between Twitter and Elon Musk and how Twitter is measuring and addressing spam on the platform.

Video Transcript

[MUSIC PLAYING]

ALEXIS KEENAN: Welcome back to "Yahoo Finance Live." I'm Alexis Keenan, taking the reins from Brian Cheung here for a minute to talk about the escalating legal battle between Twitter and Elon Musk. Now, Musk took to Twitter on Saturday to double down on his counter lawsuit against the social media company. He alleges that Twitter's SEC filings are fraudulent-- that is if, a big if-- the company underestimated its fake and spam accounts. Musk said his $44 billion merger should proceed if Twitter hands over its method for sampling and confirming those accounts.

So here to help talk with us and help us really understand this bot detection issue is a PhD candidate, computer scientist Kai-Cheng Yang from Indiana University. His bot tool is called the Botometer. And it is now at the heart of Musk's and Twitter's claim. So thanks so much for being with us.

KAI-CHENG YANG: Hi. Very glad to be here talking about our research, actually. Very excited.

ALEXIS KEENAN: We're glad to have you, Kai-Cheng So first I want to just start out by really clarifying that Musk is citing your tool, the Botometer-- this is a publicly available tool-- to back up his claims that Twitter's estimates on fake accounts and spam accounts are actually more egregious than what they represent in their SEC filings. But I want to talk to you about the difference between your tool and what Twitter is claiming. Your tool is about detecting bots, is that right, automated accounts?

KAI-CHENG YANG: Yes, that's our definition. And that's why we built the tool.

ALEXIS KEENAN: OK, and so how is that different, though, from spam accounts?

KAI-CHENG YANG: I think Twitter has made it clear that they are focusing on spam and the false accounts. And my understanding of their definition about spam accounts is that those kind of accounts would send repeatedly all different kinds of information, trying to promote some website or some product or some cryptocurrency to people, kind of annoying, right? But you can achieve those kind of goals through bots, of course. But also you can have real people control those accounts. In my opinion spam accounts has an overlap with social bots, which is what we detect. But also it's not entirely the same thing.

ALEXIS KEENAN: OK, so let's just talk about those bots and isolate those. In 2017, your group did give an estimate. And it said that as for bots that you thought between 9% and 15% of active Twitter users were, indeed, bots. So do you still believe that estimate is true today?

KAI-CHENG YANG: Yes. So our group did those kind of estimations back then. But I do want to mention that we've been upgrading our tools constantly. So the Botometer today is different from what we have before, right?

And also the situation on Twitter has been changing quickly, because Twitter also, they have been doing a lot trying to remove bots and other inauthentic accounts. So actually, I think they drive to the bad actors to change their behavior, to change their accounts. So I am not sure that estimation is still accurate today, unfortunately.

ALEXIS KEENAN: OK. So, Kai-Cheng, do you believe that there is any way for Twitter to give Musk the information that he wants, which is information to analyze the number of bots and/or spam accounts? Do you think there's a way for them to do that without giving up personal information about its users that it says are proprietary?

KAI-CHENG YANG: Actually, I do think so, because based on my understanding, what Elon Musk wants is how Twitter did the analysis. And also the data provided by Twitter to Elon Musk, I think, to some extent is sufficient to perform such analysis. But also my understanding is that Twitter isn't really making all the details clear to Elon Musk. And also it's not clear to the public. Yeah, I think there is a way to do that.

ALEXIS KEENAN: OK, and Twitter in its legal filings, it criticizes your platform, perhaps your methodology. But do you believe that the way that your tool is designed, the Botometer, does it do what it's meant to do?

KAI-CHENG YANG: Of course, I think our tool is doing what it's intended to do, right? And Twitter actually have been saying similar things for a while, that us as outsiders could not build a tool that's really perfect in detecting bots or other accounts, because they have some private information that we don't have access to. But still, based on our understanding, based on our research, we do believe we are doing a pretty good job.

ALEXIS KEENAN: And last, I just want to ask you if you think that Musk's account-- he's saying that he believes that the prevalence of fake and spam accounts could be up to 10%, which would be double what Twitter has said in its SEC filings-- do you think that Musk's contention here could be true? Or do you think he's way off the mark?

KAI-CHENG YANG: I don't have an estimation by myself. But I do want to mention that based on my understanding, Elon Musk is using our tool to get such a number, right? And I'm sorry I have to be a little bit technical here in order to explain this, right?

Our tool works, if you give it account, it will give you a score. If the score is higher, it means the account is bot-like. If the score is low, it means it's human-like. But it's a score. So in order to have a number of percentage of bots on Twitter, you have to choose a threshold, right?

And that's, I don't know how Elon Musk did it. And technically you can choose any threshold you want and to get any result you want. So that's my understanding right now. Elon Musk didn't make it clear how they choose this threshold to me.