I personally think it isn't wise to use an actor of any race in substitute of another, if that character's race is part of the story. The only reason I could think of to change the character's race, gender, status, etc. would perhaps be to tell a different story, but then it should be renamed and be a different story. But if a character's race, gender, status, etc. is tied to that character's story, then it shouldn't be discarded frivolously.
From what I see, I feel that a lot of the disconnect is based on whether people find an attribute (in this case, race) important or not as part of the character's story.
But magic doesn't have any grounding or association with the world as we know it. Neither does aliens.
We can only use the world as we know it as a frame of reference for a story.
I'd argue in lbaudia's example that it is confusing if in 12th century Mongolia, there was some Nordic guy walking around, I'd imagine there to be a backstory of some kind. If there wasn't, then that would definitely be an example where I'd be annoyed at white people in fiction.
I thought a great example of casting was the TV show "The Expanse". To be able to cast someone as specific as Bobbie Draper so well - these studios have no excuse to whitewash as they do except laziness.
