My cat loves chasing things - a few flies got into the apartment, and he's been going crazy for, like, two hours trying to run them down. So I got to thinking, the flies are sort of hard, because they have an advantage with being able to fly, but what if I got some crickets and let them loose, so he could chase them down? And why stop there - would it be okay for me to buy some white mice and let them loose in the apartment?
Don't let anything loose in your apartment that you don't want to see chewed up and deposited in front of your bedroom door. Cats are great for catching mice, but they're also great at chewing them up and leaving them for you as gifts, which... well, ew.
Cats are actually astonishingly good at keeping themselves occupied, so if you run down to your local pet store and find some little (fake) mice and throw them to your cat, he'll be occupied for hours. (Catnip mice are especially fun.) Or, even better: we found these remote control mice -- all the fun of real mice, but without the mess!
So, we'd advise against letting live things loose in your apartment, unless you really want crickets hopping around in your underwear drawer.
Related: Ask Gothamist on how to catch unwanted mice in your apartment.