You are the standard other people argue in front of. Emotion enters your court as evidence, not as verdict, and you will rule against your own preference if the logic lands there — which is why your judgment carries weight nobody voted on.
The cost is the bench. Presiding is a lonely seat, and the discipline that makes you fair can make you unreachable — people stop bringing you the messy, unreasoned things, which is most of what a life is. Not everything wants judging; some things want witnessing.
The King of Swords is the clearest air in the deck. Keep the rigor — it is needed — but come down from the bench on purpose, regularly. An opinion held warmly is still an opinion. The people who respect your mind also just wanted your company.