Keyboard switching overrides a user specified xmodmap keymap

[Ivan Martyushev]

I have a configured keyboard input sources switching in the menu:
    System Settings → Keyboard → Shortcuts → Typing → Switch to next source: Alt + Shift L & Switch to previous source: Shift + Meta L
Also I have a manually launched command:
    xmodmap ~/.Xmodmap
The ~/.Xmodmap contains settings to remap few numpad buttons:
    keycode 81 = Prior KP_9 Prior KP_9
    keycode 89 = Next KP_3 Next KP_3
    keycode 90 = Insert KP_0 Insert KP_0
    keycode 91 = Delete KP_Decimal Delete KP_Separator
The remapped keys works as expected as long as no keyboard input source switching occurs.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: unity-control-center 14.04.3+14.04.20140410-0ubuntu1
ProcVersionSignature: Ubuntu 3.13.0-24.47-generic 3.13.9
Uname: Linux 3.13.0-24-generic i686
NonfreeKernelModules: nvidia
ApportVersion: 2.14.1-0ubuntu3
Architecture: i386
CurrentDesktop: Unity
Date: Thu May 8 11:54:11 2014
ExecutablePath: /usr/bin/unity-control-center
SourcePackage: unity-control-center
UpgradeStatus: Upgraded to trusty on 2014-05-06 (2 days ago)
usr_lib_unity-control-center:
 activity-log-manager 0.9.7-0ubuntu14
 deja-dup 30.0-0ubuntu4
 gnome-control-center 1:3.6.3-0ubuntu56