Sometimes our duty takes us away from those we love. We don't love them any less, naturally, but we have a duty to be something greater than ourselves. I have often had to sacrifice my personal wants for duty and I've had to sacrifice my personal comfort to aid my family. It depends on your motivation.

( This is something that doesn't require her to be from the same world as Jared - she thinks these sorts of troubles plague anyone. )

My motivation is to make the world as good as it can be for all those in my kingdom, including my family, so that without me they'll still have a good life. Perhaps that's your brother's motivation. No less frustrating for you, naturally, but perhaps that's what he thinks.