E46 M3 SMG missing shifts

Happens every once and a while. Driving normal, shift from 3rd to 4th - the gear doesn't engage, and the rpm meter goes up like you would reving your car in neutral. The car engages the gear about 5 seconds after. What could be causing this?