Yes, you are 100% right. A lot of my shots are taken quickly - like this one where I ran out of my car on a freezing cold day an the way to work. I shot this by hand just off the highway and as a result I get curvature. The wide angle only makes the curvature more pronounced when you are not EXACTLY level due to a vertical vanishing point.
Every shot taken with a tripod has turned out really well - no seams in the stitch but in some instances I don't have time to set up the tripod so I have to guess.