Hopefully you've got him figured out.
Your RIGHT, the more work you give him the more endurance he builds up before tiring out. This takes time, he needs to learn to " stand your ass still, because I said so".
It's easiest to start on the ground. Use any word[whoa,stand,quit] and as soon as his feet move, put him exactly back where he was. Use the same teaching method once mounted. Start with the easiest for him, somewhere he's not anxious and after a work. Make it progressively harder[mentally] before work/longer/around other horses. It is a slow process, but once he learns it, you will have a horse that will hang out on a loose rein.
PS-Try not to get on and directly ride off. Remember horses learn through consistant repetition. And nobody said that you actually have to "ride" after tacking up and mounting. Don't give up.