When I have a passenger in the car I am almost always a worse driver, especially in street traffic. I'm fine on the interstate, for long trips and all.
But somehow the presence of another person--who is of course a driver and a sort of tacit copilot--causes a certain nervous boldness, a tendency to take corners a bit harder, stop a little shorter. I certainly don't want them to think I drive too slowly, that I can't get them there on time.
Sometimes I ask not to talk when I'm driving. Occasionally I'll explain that I'm a slow driver. Sometimes that makes my passengers fidgety. Maybe it's not my speed.