Following on Twitter means that a person is interested in receiving your tweets.
So, if you're following someone on Twitter, you will see their tweets in your message stream. However, if they are not following you, then they won't see whatever you tweet in theirs.