The phrase "diversity hire" implies that they were not hired for their abilities. It attacks the person looking for a job rather than one making the hiring decision. So yes, it is a slur.
The purpose of phrases like "diversity hire" is to dehumanize and marginalize without ever having to learn anything about the person you don't like and you won't bring about inclusion by embracing those tactics, but it is noticed and called out, but it is done as a conversation and against the individual doing the hiring, not against a group of people who were hired and did nothing wrong.
At least that has been my experience.
It's a screen shot from this "flour in hairdryer" prank. Skip to about the 1:30 mark to see where the prank goes wrong.