Why is a x b.b =0 correct

Since the dot product would give us a scaler, this means that the cross product would have to come first. If you do a x b, this gives you a vector (say v) that is perpendicular to both vectors a and b. So we have:
v • b

Well v is perpendicular to b (and a). Remember that two perpendicular vectors yield 0 as the dot product based of the the rule that x • y = |x| |y| cosθ. So we have:
a x b • b = v • b = 0

What you have is correct.