The Bastille area touches the 4th (Marais), 11th and 12th arrondisments so gives access to a range of styles, from honkey-tonk to contemporary gourmet. Just to the north of the monument, along Boulevard Richard Lenoir, one of the best open-air markets is spread out on Thursdays and Sundays. Here is a nice appreciation, although rather dated: http://www.davidlebovitz.com/2006/02/the-sunday-mark/
The Saint-Martin canal cruises start near there too, much different than the Seine boats. The nearby promenade of plants gives strollers both landscape on the old elevated rail line and artisanal shops on street level, making for excellent strolls to Gare de Lyon. Transportation connections are pretty good and the further east and north you go, the more you will see authentic inner-city neighbourhoods.
To check out urban transport, whether bus or subway, look at the interactive map on http://www.ratp.fr/en/ You will see that Bastille is served by three separate Metro lines, streamlining your commute to city attractions.