Firstly, it does not look like you are using the official raspberry pi kernel for Xenial. Your kernel is 4.1, and the Xenial kernel is 4.4, so it is possible that the kernel you are using does not have the required apparmor bits that Ubuntu needs.
As for the error, does journalctl contain any more information WRT brltty, and does systemctl status brltty also contain any more information?
Firstly, it does not look like you are using the official raspberry pi kernel for Xenial. Your kernel is 4.1, and the Xenial kernel is 4.4, so it is possible that the kernel you are using does not have the required apparmor bits that Ubuntu needs.
As for the error, does journalctl contain any more information WRT brltty, and does systemctl status brltty also contain any more information?