In eeschema, if you're zoomed in to just a small portion of the sheet and try to use the scrollbars to slightly reposition the viewport, the viewport will sometimes instead jump by a large amount left (if using the horizontal bar) or upward (if using the vertical scrollbar). In other words, if an errant jump occurs, it always jumps to a much smaller X or Y coordinate.
By a "large" amount, I mean (for example) that if I happen to be working at the right edge of the sheet, it may jump nearly to the left edge, or indeed even beyond the left edge so that it's outside of the sheet altogether.
I have been dealing with this particular problem for many months, but have been unable to characterize exactly when this happens. It happens, perhaps, 10% of the time. I will note, however, that I typically use a grid size of 25 mils rather than the default 50, and sometimes use a grid size of 5 mils for a very small bit of editing, and that the jumps subjectively occur more frequently if zoomed in a lot at a small grid size. At any rate, whether it's objectively more frequent then or not, it's the time when it's most annoying, because it's the most time-consuming to try to find the patch you were working on before it happened, so it definitely stands out the most in my memory.
Version info:
Application: kicad
Version: 5.1.0-rc2-unknown-4612175~81~ubuntu16.04.1, release build
Libraries:
wxWidgets 3.0.2
libcurl/7.47.0 OpenSSL/1.0.2g zlib/1.2.11 libidn/1.32 librtmp/2.3
Platform: Linux 4.4.0-53-generic x86_64, 64 bit, Little endian, wxGTK
Build Info:
wxWidgets: 3.0.2 (wchar_t,wx containers,compatible with 2.8) GTK+ 2.24
Boost: 1.58.0
OpenCASCADE Community Edition: 6.8.0
Curl: 7.47.0
Compiler: GCC 5.4.0 with C++ ABI 1009
Build settings:
USE_WX_GRAPHICS_CONTEXT=OFF
USE_WX_OVERLAY=OFF
KICAD_SCRIPTING=ON
KICAD_SCRIPTING_MODULES=ON
KICAD_SCRIPTING_PYTHON3=OFF
KICAD_SCRIPTING_WXPYTHON=ON
KICAD_SCRIPTING_WXPYTHON_PHOENIX=OFF
KICAD_SCRIPTING_ACTION_MENU=ON
BUILD_GITHUB_PLUGIN=ON
KICAD_USE_OCE=ON
KICAD_USE_OCC=OFF
KICAD_SPICE=ON
It appears to me that it's very easy to reproduce this problem immediately after using the "Zoom to Selection" feature, if you use it to zoom in by perhaps 5X. Just using it to zoom in by a small amount like 2X doesn't seem to trigger the problem.