It usually moves because the servo receives some strange imput from the reciever. If you look at the signal with an osciloscope you can see irratic pulses for shorter or longer perriods of time. The servo wil move if it preceives a pulse, valid or not. If it receives random pulses (noise) it wil move iradically.
I would guess that when you switch off the transmitter the receiver slowly ajusts the gain of the receiver unitil all it gets is static or it reaches some limit. Thats where the noise comes from.