I love the bridge of love locks (they may have taken it down?), and also the bookstore by Notre Dame. I think it's called Shakespeare Books (if you face Notre Dame's entrance, it's to the left across the bridge).
However, I also think Paris is really over-rated, and that you miss out on so many cool things in France if you stay in Paris. I love Brittany (I think this is a "roads less traveled" tourist destination - would not have gone, but my partner is from there), Normandy, Nantes, La Rochelle, etc. there's so much more to France than Paris, and - frankly - I've liked the non-Paris parts of France best. To me, Paris is just where the plane lands.