Having lived in the Garden District many years ago, I'll throw in a few recommendations as well, because there is just no reason to go to New Orleans and not have an amazing time! And, for the most part, it's all about the food.
I have to disagree with Venchka on one thing (sorry Wayne) - I don't know about the Bourbon House or Redfish Grill, but we had the absolutely worst service ever at Palace Cafe and I would never, ever go back. Just way too many excellent restaurants to choose from as is without it.
That said, you want the real deal po-boy's, then his recommendation of Domilise's is on the money. More hole-in-the-wall joints recommended further down.
Commander's Palace is always a classic - especially for Sunday brunch - which, to me, is the meal that should always be devoted to a Commander's visit. And, last time I was there (post-Katrina) they had 25 cent martini's at lunch during the week. Given how my husband's oral argument went before the 5th Circuit earlier that morning, it was definitely a martini lunch for him and a bargain for me.
The cemetary across the street is interesting, but as others have said, be very careful. It doesn't matter what time of day you are there, people have been robbed in broad daylight.
Wandering around the Garden District is always one of my favorite things to do, as is wandering down Magazine Street. In the GD, Anne Rice used to live @ 1st and Coliseum (huge house) and Jefferson Davis (Pres of the Confederacy) died at 1st and Camp (I lived a block away on Camp & 2nd, not that that is a tour highlight itself :-D).
As previously said, lots of cool shops and restaurants on Magazine - the traffic is a drag, but I've walked it several times with camera in tow.
The Zoo is a great choice if you have kids along - and I don't know if they still have it, but you used to be able to take a paddlewheel from the Riverfront to the Zoo, which kids love. I haven't been to the Aquarium since Katrina, but it was great before that, so I hope it has recovered well by now.
Another fun thing to take kids to see (or even if you just love Mardi Gras) is heading over the bridge and visiting Blaine Kern's Mardi Gras World - they do the floats for the parades and they have lots of cool stuff to see in their warehouse.
Blaine Kern's Mardi Gras World - New Orleans, LA
I also love just riding the street car up and down St. Charles. As Venchka suggested, Camellia Grill is right on the line, just after St. Charles turns into Carrollton and if you aren't digging the chili cheese fries, it's other specialty is pecan waffles. And yes, if you go late at night, you'll see every strata of New Orleans society.
The Camellia Grill
Around the corner from Camellia Grill, you'll find Brigtsen's - in a little house right off the river. Always a treat!
Brigtsen's Restaurant Award Winning New Orleans Restaurant
To me, the D-Day Museum is a must see. It's fascinating and you could easily spend a good part of a day there. I also love to just wander the Quarter and the Fauberg Marigny neighborhood, but again, be aware of where you are and when you are there! And, no stop is complete without beignets @ Cafe du Monde, no matter how touristy it is. I always stop in this gallery - I like his work a lot and, in fact, have a print of his in my house:
Photo Works New Orleans
Another absolute institution in New Orleans is Galatoire's - with everyone waiting in line for it to open for brunch or lunch and sipping bloody mary's from the corner bar while standing in line. You will feel like you stepped back in time by generations:
Galatoires Restaurant New Orleans, Louisiana
Live music? Check out:
Tipitina's - New Orleans, Louisiana or Rock 'n Bowl (it opens to the store link, but you can get to the events from here):
store .
My top food picks:
Restaurant August - to me, hands down, my favorite upscale restaurant in town - love Chef Besh. It's an elegant restaurant, the food and service are always top notch. Pricey, but worth it. In the CBD (Central Business District) on Tchopitoulas, basically across from the Windsor Court Hotel.
Restaurant August | Chef John Besh | New Orleans Lousiana
Bayona - in the Quarter. Chef Susan Spicer always has a menu of simple, but perfectly prepared choices. A small, but pretty dining room. Great service. Not as pricey as Restaurant August, but not inexpensive.
Bayona Restaurant
Delmonico's - on St. Charles. To me, it is the standout of Emeril's restaurants. It's old school New Orleans. I also like Emeril's in the Warehouse District, and they are very different restaurants - Emeril's setting is edgy, warehouse, Delmonico's is classic. NOLA is, to me, the tourist restaurant in his group.
Restaurants : Emeril's Delmonico | emerils.com
Parkway Bakery - for lunch - always one of my favorite "joints" - although it's been discovered by the Food Network fans.
Parkway Bakery & Tavern Worth the drive there!
Parasol's - another well-loved classic hole-in-the-wall.
Parasol's Restaurant & Bar | Parasol's Bar & Restaurant
Cheezy website aside, the best burger around is here:
New Orleans Best Hamburger - Port Of Call - New Orleans Restaurants
I assume you have a hotel reservation? If not, I can make some recommendations there as well. Have fun, report back and take lots of images.