I've been there several times and the French Quarter isn't that bad usually. New Orleans has a lot to offer but it depends on why you are there. They have great food, the zoo and aquarium are good and they have excellent local scenery and architecture.
I personally never had problems with criminal elements like pick-pockets, muggers or drunken revelers looking for trouble.
You missed Mardi Gras but the French Quarter still has plenty of boobs (they have some nude bars/shops) if you're looking to see that.
The French Market is also interesting. Lots of souviners, food, and random stuff you can buy there.