In a big open world game that allows players to randomly kill each other at will, there has to be some kind of penalty to killing innocent players to stop the game turning into an endless large scale deathmatch.
Rockstar did something a little different to punish continued random killings in GTA Online, and general jerkiness could end up with some terrible consequences.
The first thing that can happen if you continue to be a jerk by quitting games, killing players and ruining other's experiences on GTA Online is that you'll start to be joined with other jerks just like you. If you keep it up, you'll be stuck playing in the time out corner of GTA Online with all the other naughty criminals who can't refrain from messing with other players.
If you decide to start playing nicely, you'll be freed after two real days, and after that point you'll be initiated back into average people society.
You won't be able to play with your friends after you get the 'bad sport' title, and to make things even worse, you'll have to wear a dunce hat so that everyone around you knows you've been a bad, bad kid.
In my opinion this is a great feature for the game, and so far in my experience, I haven't come across too many 'bad sports' that continue to try and ruin my fun so it must be working quite well.