I've got some slight experience with the RX350, more though with an RX400H.
They're nice and all. But I find the driving experience to be a bit boring. Mind you I'm used to driving my M-Sport, my modded truck, or my buddies '93 M5. All of which make most vehicles seem rather boring.
So if you're not looking for an exciting driving experience I can't think of any other negatives offhand other than I think that any Lexus is a tad overpriced. But that's my opinion as I'd just rather get the regular Toyota version of a vehicle than pay the premium for the Lexus version.