I think your confusion is in what that EXCLUDE is doing. It is NOT ignoring filters. It's just saying not to group by Location when aggregating AVG([Rating]). When you filter out all but one location, AVG([Rating]) and { EXCLUDE [Location (Loc)] : AVG([Rating]) } become equivalent, because with either calculation, you're averaging for all points in your filtered partition.
As a result, your condition for receiving an A will always be true if there's only one location. (Check the math: X > X - .1X → X > .9X)
Here's a different way to get what you're after. Make a calculated field (I'll call it Location Filter):
LOOKUP(ATTR([Location (Loc)]),0)
Then trash your Location filter and replace it with that field. We're doing something sneaky here - we're making the exact same filter as we had before, but we're disguising it as a table calculation (by using LOOKUP()). Tableau doesn't execute table calculations until after it's created the filtered partition, so we've tricked it into letting us use every location while still just examining one.