it’s a bit slippery slopey but yes i suppose so. if your downvoting sprees amount to being abusive as the admins or mods define it then on their terms your account should be banned, ip banned, whatever. i don’t think there’s any voting based behaviour that gets you in irl trouble, obviously posts inciting violence, sending credible threats or other behaviour that gets you noticed by your regional authorities is another kettle of fish, but none of that seems relevant to up/downvotes being visible
it seems from a very brief search that likes and dislikes (see link below, i assume they translate to up and down votes) are the extent of what is available so a more nuanced slashdot or steam review type rating is unlikely to be viable.
in any case the ability to upvote and downvote feels like a core differentiating feature to this kind of forum and inbetween measures are unsatisfactory. upvote and downvote anything you like, and everyone can see you doing so, would be an improvement imo on the current implementation.
at least it may be possible in a future version to allow or disallow voting behaviours on a community rather than instance basis?
https://www.w3.org/wiki/ActivityPub/Primer/Like_activity