Otherwise, we'll have incorrect scrolling when we switch hardware
devices without switching virtual devices. For example, on a ThinkPad,
scroll using the touchpad, move the eraser mouse, and then scroll again:
the deltas will be wrong. This also matches what GTK+ does.
https://bugzilla.gnome.org/show_bug.cgi?id=689258