At that point in the code the only thing that appears to be different between dir and olddir is olddir is normalized, while dir isn't [i]necessarily[/dir] normalized (it's left up to the function that calls ReboVec).
If the calling function is passing a non-normalized direction (i.e. self.velocity), that would probably do it. Dir might be so large that the traceline is starting very far away, behind another wall or inside a solid (i.e. the ceiling or a wall behind, or whatever is 16,000 units away). The fix given is appropriate, since ReboVel doesn't specify in the comment above it that dir should be normalized (other than the variable being named dir, and directions are typically understood to be normalized; not everyone knows that though). It might also be appropriate to move this line:
olddir = normalize(dir);
to above the if statement so olddir can be used instead of dir in the if statement's traceline too.
I'm just not sure why only the floor had the problem. O_o