We study how the powerful perceive power abuse, and how negative experience related to it influences the appropriateness judgments of the powerless. We create an environment conducive to unfair exploitation in a repeated Public Goods game where one player (punisher) is given a further ability to costlessly subtract money from others (victims). We find that punishers who choose to abuse their power rationalize their behavior by believing that free-riding, while forcing others to contribute to the public good, is not inappropriate. Victims of such abuse also start to believe that punishers' free-riding and punishment are justifiable. Both punishers and victims are not aware that their beliefs are adjusting in this way. In addition, subjects assigned to the role of power, regardless of how they use it, think that outside observers share their beliefs about the appropriateness of their actions. All these observations are explained by the Belief in a Just World hypothesis, which states that people rationalize any wrongful acts in order to maintain a coherent picture of the world that is orderly and lawful. Our findings demonstrate the fearsome capacity of humans to exculpate abusive behavior by themselves and others.