Rethinking "The Man"

I've continuously heard and caught myself referring to "the man" as the problem in the world.  Of course, "The Man" is a vague anonymous reference to a body-less entity created by my subconscious through gathering evidence from every bad decision every rich white guy in a suit has ever made.
I'm beginning to see that I've been making an assumption, which has really caused suffering in me. I've always seen "The Man" being the problem, that there is literally some type of person who is innately wrong, deep down evil, and out to blow up the world.  I've associated the personhood of "The Man" being the problem... that if we could just get those people out of power, that we could put some so called opposite person in there and that we'd have a shift.  I agree that we'd have a shift, but I'm starting to see that my obsession of "The Man" as the problem and not "the perspective" being the problem is having me suffer as I write off people as being evil or good.  In my head I'm in the middle of a battle and so I'm always in defense, always looking for better footing.

Poorly scanned image comes from my book of diagrams.

When I approach the world always in defense, you bet I'll find evidence that there's a war going on. I've turned internally and have even been in defense of myself, not being certain of my own standing.  Over the last 3 years I've actually deactivated myself on numerous occasions in fear of myself, in thinking that deep down that I'm an evil person.  I fit the bill for "The Man" with my privileged status in many areas.
This view of the world isn't working for me; I know that for sure.
As I approach the world in defense, I enroll people around me to approach the world with caution.  My fear and defensiveness might even look like aggression, as much of the fear in the world looks.  I've created a battle in my head, which perpetuates battles going on in the world!
What were to happen if I could step back from my defensiveness and trust?  What if I could get off of my fear and acknowledge that there are no evil people, only others being defensive just like me?  Fighting war with war isn't working.  Me being scared of my own shadow isn't working.  For the sake of peace, for the sake of a peaceful mind... I'm lowering my gun before "The Man" does.