This is one of my favorite places on Earth. If you like to fish at all, you should look into a charter trip. The Keys are situated on a living reef, so the opportunity to target just about any species or format that appeals to you is there, fishing the reef, wrecks, the Gulf or the Gulf Stream. The Gulf Stream swings in very close to land here, so targeting the big pelagics is a reasonable thing to do. The living reef, just six miles south of Key West itself, is home to scads of different species, including yellowtail snapper, one of the best-eating fish anywhere, and one that's found only along a living reef. The backcountry in the upper keys is home to many species more. You can go big-game fishing and target marlin or tuna, you can go reef fishing and fill a cooler, you can go backcountry fishing, away from the bigger seas, and enjoy a great day catching all sorts of things.
I fished with Capt. Tony Murphy out of Key West last year and we caught nine different species in a half-day trip, bouncing from the reef to the flats in an effort to see as many different fish as possible. If I were planning my first fishing trip out of the Keys, that's the trip I'd pick.
Here's a story and photos I did about the day for OutdoorHub.com
All of Key West is not Bourbon Street. Duval Street, in Key West, may be Bourbon Street, but the rest of the key is just a key, a nice little tropical town. That said, Key West does not define the Keys overall. There are many other places to stay and things to see.
This is a marketing website, but it's run by a contact and acquaintance of mine and it's pretty comprehensive for the Keys overall.