That's just how it is, especially if you're a good person.
And it happens even more if you're a Christian.
It doesn't matter what you do, people are just that way.
I guarantee whatever you think you're going through, you've never been as spitefully accused as I have.
I have yet to meet anyone who has been through the crucible of the shear and utter ineptitude of people as I have. Everyone has at least someone who thinks good of them, someone they can at least trust to some extent. I do not, other than those who are obligated. So, I guess there must be some who don't even have their family.
However, what I've been through is of no fault of my own because I have yet to succumb to actually treating another badly or bowing to the pressure to harm others just to make it lighter on myself.
I attribute that strength to Christ.
When the Bible says the meek shall inherit the earth, it's not talking about simpletons oblivious to the world around them. It's talking about those who suffer daily at the hands of common everyday people because of how they're spitefully accused, mistreated, gossiped about, and even through it all, they insist on retaining their meekness and refusing to disparage another.
It's hard, and it will be even a psychological challenge, but you must understand that others are generally foolish and are easily convinced of nonsense.
You will find less baseless judgement leveled against you in unspoken words and gossip when you're not around in others like you, but in others like you, you will also find it more difficult to find a friend, but once you do, you will find a more loyal type of friend if they are not too weak.
I find that most who come to be my friend are far too weak and will inevitably succumb to the foolishness of those who wish to cause me problems.
This is why the Bible says "vengeance is mine saith the Lord," because anyone who goes through this sort of thing comes to a point where they could torture and kill those who wish to hurt them, but God's justice is far more effective in teaching people the right things.
Some will come to this realization and be sorry, some will see the suffering they inflicted first-hand, and feel some of the pain they inflicted on others while some, the truly cursed, will never feel the reality that they are wrong, will never see the pain they inflicted till they die, and then they'll learn it for eternity. I feel for them the most because they'll learn everything for eternity without a single thing upon which to say, "please, make it more lenient for this reason or that." They will have nothing to point too because they were never made to suffer it themselves while still on this earth.