Even before the iPhone got GPS capabilities it could tell you where you were through cell phone tower data—a useful trick if you were trying to figure out where you were on a map. And the introduction of GPS has given the phone a whole slew of useful location services (we're particularly fond of using the feature to track our bike rides). However the fact that the phone can tell you where you are does not make it cool that since iOS 4 was released last June, iPhones have been keeping a secret file of everywhere their owners have been—a "feature" we like to think of as the "suspicious spouse's new best friend."
The unfortunate news was uncovered by two researchers (one a former Apple employee) doing work visualizing mobile data. And if you don't believe it, they've gone and made an app you can use to see what info your iPhone has stored on you. The file with the location history lives on any iPhone, as well as any computer that it has backed up on, and is easily accessible unless you choose to encrypt your backups. On the plus side, as far as anyone can tell the file is not being shared with any other programs or even Apple. It just sits there.
What is the purpose of this cookie trail file? Apple has so far been mum on the issue, but the researchers who uncovered it speculate it may be for some future "feature."
Meanwhile, in other iPhone news, Reuters is saying that they have it on good authority that the iPhone 5 will be shipping in September. Hopefully by then you'll be able to decide what information your iPhone stores on your whereabouts.