The thing we can do is defederate. You can talk to your admins about regular interactions with problematic users on specific instances and see where it goes. If it's a specific community in general, you can report it to that instance's admins (something I've done that resulted in getting a pedo community removed). This is how you "punish" instances and communities that refuse to enforce good user conduct.

But more broadly, I don't support downvoting in general, because that leads to lazy commentary and brigading behavior. No thanks. I enjoy being on an instance without downvotes, and I am not interested in a system where "the mob" decides what is valid or invalid, acceptable or reprehensible. Mobs are stupid and gullible.

Ultimately, you need to stop thinking about Lemmy as a single community. We can do what we need to do as users, mods, or admins to protect each instance, but what makes the whole thing good and healthy is the tentative agreement between each part to operate in good faith. It's not our job to decide what "health" is for all of Lemmy/the Fediverse, and the minute it becomes that, what makes the Fediverse great will have been lost to the thing it despises: centralized authority.