TikTok’s AI avatars could briefly make problematic content

We’re in a world where it’s getting harder to believe what we see on the internet. The advent of AI avatars is making this worse. TikTok is testing something like this, but it might have hit a bump. TikTok’s AI avatars could briefly make problematic content.

The ByteDance-owned company was swift to jump on the AI bandwagon and bring its own AI tools. Now, the company is working on its Symphony Avatars. These are AI-generated avatars that are meant to give you the illusion that they’re actual people. As if it wasn’t hard enough to believe what you see on TikTok already. Right now, the tool is still in testing with a limited number of users.

TikTok’s AI avatars briefly made problematic content

This issue was first seen by CNN reporter Jon Sarlin. He was able to gain access to the tool using their personal account. That was the first clue that something was askew. This test is supposed to be limited to an exclusive group of users. So, common users shouldn’t have access to it.

No guardrails

The rabbit hole gets deeper, as the user saw that the videos produced weren’t watermarked. AI companies are encouraged to have some sort of watermarking system in place to let people know when content is AI-generated. This way, people won’t be deceived when they view the content.

Reciting terrible text

For the cherry on top of this problematic sundae, TikTok’s Ai avatars were able to make some rather questionable content. The reporter was able to get an avatar to recite Hitler’s “Mein Kampf” and Osama bin Laden’s “Letter to America”. In case you don’t know, those are pretty bad things to recite.

So, there were no guardrails on what could be produced and no way to tell that it’s fake. It looks like TikTok is in hot water, right? Well, not really. Again, this feature isn’t publicly available. The company said that a “technical error” made the pre-production feature accessible to some common users. Also, the problematic content wasn’t posted to the actual app. It appears that it only made it to the preview stage.

According to a statement from the company, “If CNN had attempted to upload the harmful content it created, this content would have been rejected for violating our policies.” So, it appears that it wouldn’t have been much of an issue.

TikTok has pulled access to common accounts and is working on making sure that the feature doesn’t produce videos that would get anyone canceled. So, when this feature does launch, it will hopefully be safer.