harfbuzz could be the most likely suspect. However, also wily has 1.0.1-1build2 - could you test the issue on wily? If it's broken there too, I could try to help by forward-porting old harfbuzz to xenial. But if it's not broken there, then it's probably not harfbuzz and also not freetype or fontconfig (same versions in wily and xenial of all them).
The possible suspects below, from qtbase build dependencies.
package | vivid | xenial
fontconfig | 2.11.1-0ubuntu6 | 2.11.1-0ubuntu6
freetype | 2.5.2-2ubuntu3.1 | 2.5.2-4ubuntu2
harfbuzz | 0.9.37-1 | 1.0.1-1build2
harfbuzz could be the most likely suspect. However, also wily has 1.0.1-1build2 - could you test the issue on wily? If it's broken there too, I could try to help by forward-porting old harfbuzz to xenial. But if it's not broken there, then it's probably not harfbuzz and also not freetype or fontconfig (same versions in wily and xenial of all them).