That’s father than I would take it personally

Because they’re supposed to be responsible to and represent the people who voted for them. Irrelevant to this situation.

It’s different because there might be some people who you always end up downvoting their comments and you don’t even realize it’s the same person because you don’t look at user names and there aren’t really that many prolific commenters here so they will see that you’re always downvoting them and assume malice in a way that they wouldn’t otherwise because they’re not going through your comments, they’re only looking at theirs.

For example, this comment just a few down:

I’m mildly curious if the single down vote I get on ~70% of my comments is from like one guy I pissed off at some point. At the same time I don’t care enough to work around the system, so maybe it works?

They immediately assume that if one person is downvoting them it is malice instead of maybe that one person just disagrees with their comments for other reasons and isn’t even looking at user names.

Edit: I just noticed you did the same thing in your comment too. There are not that many users here, just because someone is consistently downvoting your comments doesn’t mean they are following you around, there isn’t a lot of content and people who spend time here see a lot of it. Maybe they just disagree with every comment you make that they happen to see and have no idea or care who you are.