This isn’t something I was noticing (hey, I follow more than 5,000 people), but Twitter acknowledges on its own blog that “some users are seeing tweets from users they do not follow.” Needless to say, such an issue could create an awkward experience when viewing your homepage, or the “all friends” column in a client like TweetDeck.
With Twitter currently facing some fairly significant spam and malware issues, it’s possible that this bug could lead to you seeing some fairly annoying stuff in your Twitter stream, though at this point it's unclear how widespread the issue is and if there is any pattern to the tweets that are finding their way into user’s timelines.
Twitter says that they “anticipate that it will take several days for us to completely resolve all of the symptoms of this issue.” In the meantime, if you’ve been experiencing the issue, let us know in the comments and we’ll update here with any pertinent details.
Update #1: Lorand R. Minyo writes "I've noticed this a couple of times, this "glitch" seems to be affecting DM as well, since I've received messages from people I clearly wasn't following." In other words, if Twitter thinks you're following them, they'll be able to DM you.