Unable to find bluetooth device on RPi3 running Ubuntu Core 16

[Chunsang Jeong]

When tried to find/init bluetooth device from rp3 ubuntu core image by using bluetoothctl and bluez snaps, it returned "No default controller available" all the time.

I'd tested it both from stable and daily images, but just got same failure.

http://cdimage.ubuntu.com/ubuntu-core/16/stable/current/ubuntu-core-16-pi3.img.xz
http://people.canonical.com/~ogra/snappy/all-snaps/daily/current/

[Oliver Grawert (ogra)]

can you try:

sudo modprobe bcm203x

and see if it starts working then ?

[Chunsang Jeong (chunsang)]

@ogra,

 "sudo modprobe bcm203x" didn't make any changes, even with modprobing btbcm.

[Oliver Grawert (ogra)]

this is weird, i clearly see:

[164699.723513] Bluetooth: Core ver 2.21
[164699.723667] NET: Registered protocol family 31
[164699.723683] Bluetooth: HCI device and connection manager initialized
[164699.723712] Bluetooth: HCI socket layer initialized
[164699.723732] Bluetooth: L2CAP socket layer initialized
[164699.723779] Bluetooth: SCO socket layer initialized
[164699.759345] usbcore: registered new interface driver bcm203x

in my dmesg output ... whicgh to me looks like the device should be available

[Jim Hodapp (jhodapp)]

@ogra: have to tried connecting a BT device to the BT controller?

[Oliver Grawert (ogra)]

looking around i stumbled over the patch below, that avoids to set the speed before loading the firmware in hciattach. we dont seem to carry this patch in:

https://git.launchpad.net/~snappy-hwe-team/snappy-hwe-snaps/+git/bluez/tree/tools/hciattach_bcm43xx.c?h=bluez/5.37

patch:
https://github.com/OpenELEC/OpenELEC.tv/blob/6b9e7aaba7b3f1e7b69c8deb1558ef652dd5b82d/packages/network/bluez/patches/bluez-07-broadcom-dont-set-speed-before-loading.patch

[Oliver Grawert (ogra)]

@jim, i see the same issues chungsang sees, even with the bluez package from edge that actually had teh bluetooth-control imterface (which we need connected on the pi to access /dev/vhci)

[Konrad Zapałowicz (kzapalowicz)]

@ogra, anything we could do to help you debug this issue?

FYI: we will update the bluez to v5.44 in the next two weeks.

[Oliver Grawert (ogra)]

@konrad, well, that patch looks like it could be relevant, could we include it for a test ?

[Konrad Zapałowicz (kzapalowicz)]

@ogra, The hciattach is not used on RPi at all therefore this patch will not affect anything. Could it be that we are missing a proper driver or DT entry?

[Oliver Grawert (ogra)]

@konrad, so i talked to paolo on IRC, seems we have a slight prob here with BT and serial sharing the UART (and we default to have a serial console enabled for IoT use cases). so this needs a bit more research on the gadget and kernel side.

[Konrad Zapałowicz (kzapalowicz)]

@ogra,

Right, read your comment and researched a bit. It seems that there are two UARTs on RPi:

* UART0 (Bluetooth /dev/ttyAMA0)
* UART1 (Software mini-UART /dev/ttyS0)

It also looks - correct me if I'm wrong - that currently we disable the UART0 by routing UART1 to /dev/ttyAMA0 in order to get a reliable serial console. The UART1 is a crap as the baudrate is derived from system clock. As I understood we can have one or another - never both (working reliably).

There were however evidences of people using Bluetooth and serial at the same time. It looks that it requires "core_freq=250" lock in the /boot/confi.txt.

Do you think that we could get away with it? Would that kind of sw-uart for serial console usable at all?

[Oliver Grawert (ogra)]

we do set core_freq=250 and enable_uart=1 in our config.txt today and afaik the serial console doesnt work if i drop enable_uart=1....

[Oliver Grawert (ogra)]

just verified ... keeping core_freq=250 and dropping enable_uart=1 completely kills all output on serial

[Oliver Grawert (ogra)]

so digging deeper there are dtoverlay=pi3-miniuart-bt and dtoverlay=pi3-disable-bt overlays but no matter which combo of enable_uart, core_freq and either overlay i use, as soon as i enable them no boot is possible at all anymore. (note that we use other overlays successfully in teh default setup). i fear i have to defer to paolo, perhaps we are missing a kernel config option here ...

[Konrad Zapałowicz (kzapalowicz)]

@ogra thanks for following up

[Paolo Pisati (p-pisati)]

The kernel is fine, i've tested it on a xenial image:

-add to config.txt:

dtoverlay=pi3-miniuart-bt
core_freq=250

-grab raspberry's bluez and bluez-firmware packages:

https://archive.raspberrypi.org/debian/pool/main/b/bluez-firmware/bluez-firmware_1.2-3+rpi1_all.deb

and install both

-reboot your board, then:

$ dmesg | grep tty
...
[ 4.589607] 3f201000.uart: ttyAMA0 at MMIO 0x3f201000 (irq = 87, base_baud = 0) is a PL011 rev2

the PL011 tty is the one we want, and it's the ttyAMA0:

$ hciattach /dev/ttyAMA0 bcm43xx 921600 noflow -
bcm43xx_init
Flash firmware /lib/firmware/BCM43430A1.hcd
Set Controller UART speed to 921600 bit/s
Device setup complete

$ hcitool dev
Devices:
        hci0 B8:27:EB:BA:51:89

Turn on bluetooth on your phone and make it discoverable:

$ hcitool scan
Scanning ...
        A4:B4:C8:6E:0D:5A My Android phone

A couple of notes:

1) the bluez-firmware contains the adapter fw files, so it's mandatory (though it install the files in /lib/firmware while our hcitool search for stuff in /etc/firmware)
2) I've tried to reproduce it with our bluez package/hcitool, but we are missing something - so i extracted all the relevant patches from raspbery's bluez package, applied to the ubuntu one and tested again: it was a bit better, but i was still getting a timeout during fw loading, so for this example i decided to use raspberry's deb - ironically, after the first timeout, if i rerun hcitool, this time it worked fine and the adapter was configured succesfully

[Paolo Pisati (p-pisati)]

Here is the bluez package i tested:

https://archive.raspberrypi.org/debian/pool/main/b/bluez/bluez_5.23-2+rpi2_armhf.deb

[Oliver Grawert (ogra)]

@paolo: this is all ancient bluez 4 stuff i fear ...

[Oliver Grawert (ogra)]

could we just ship the fw files in the pi2-kernel snap ?
i'll happily push the deb to the image PPA where the kernel snap Makefile could pull it from ...
iirc there is already some special casing for pi2 in the Makefile code that installs the raspberrypi-wireless-firmware deb from the PPA.

[Paolo Pisati (p-pisati)]

Makes sense to include the bluetooth firmware files in the kernel snap.

[Oliver Grawert (ogra)]

@paolo:

bluez--firmware is in the PPA now, can you add it to the:

PACKAGE :=

line for pi2 in the Makefile ?

[Paolo Pisati (p-pisati)]

@ogra: done, and i kicked a new build of the rpi2 kernel snap.

[Oliver Grawert (ogra)]

now i see bluetooth in lsmod right after boot when not changing anything in config.txt ...

but still no working BT:

ogra@pi3:~$ sudo bluez.bluetoothctl
[bluetooth]# agent on
Agent registered
[bluetooth]# default-agent
Default agent request successful
[bluetooth]# scan on
No default controller available

adding dtoverlay=pi3-miniuart-bt to config.txt still gives me a completely black screen and no boot though (only the red LED comes up, no output anywhere)...

[Oliver Grawert (ogra)]

ok, i got a little further, seems the bluez snap doesnt really have access to ttyAMA0 so i had to re-install it in --devmode ... now i get (still without the dtoverlay though, since this does not boot at all):

ogra@pi3:~$ sudo hciattach /dev/ttyAMA0 bcm43xx 921600 noflow -
bcm43xx_init
Cannot open directory '/etc/firmware': No such file or directory
Patch not found, continue anyway
Set Controller UART speed to 921600 bit/s
Device setup complete
ogra@pi3:~$ sudo bluetoothctl
[NEW] Controller AA:AA:AA:AA:AA:AA pi3 [default]
[bluetooth]# agent on
Agent registered
[bluetooth]# default-agent
Default agent request successful
[bluetooth]# scan on
Failed to start discovery: org.bluez.Error.NotReady
[bluetooth]# quit
Agent unregistered
[DEL] Controller AA:AA:AA:AA:AA:AA pi3 [default]
ogra@pi3:~$

[Oliver Grawert (ogra)]

seems the /etc/firmware dir is actually hardcoded in

https://git.launchpad.net/~snappy-hwe-team/snappy-hwe-snaps/+git/bluez/tree/tools/hciattach_bcm43xx.c?h=bluez/5.37

[Oliver Grawert (ogra)]

using a patched hciattch (with the speed setting patch from above as well as a fixed FIRMWARE_DIR pointing to /lib/firmware):

ogra@pi3:~$ sudo hciattach /dev/ttyAMA0 bcm43xx 921600 noflow -
bcm43xx_init
Flash firmware /lib/firmware/BCM43430A1.hcd
Set Controller UART speed to 921600 bit/s
Device setup complete
ogra@pi3:~$ sudo bluetoothctl
[NEW] Controller B8:27:EB:1E:54:7E pi3 [default]
[bluetooth]# agent on
Agent registered
[bluetooth]# default-agent
Default agent request successful
[bluetooth]# scan on
Failed to start discovery: org.bluez.Error.NotReady
[bluetooth]#

so this time i get a proper MAC and the device seems to be initialized, but bluetoothctl is not able to scan

[Oliver Grawert (ogra)]

the respective lines from dmesg:

[ 32.367423] Bluetooth: Core ver 2.21
[ 32.367486] NET: Registered protocol family 31
[ 32.367491] Bluetooth: HCI device and connection manager initialized
[ 32.367509] Bluetooth: HCI socket layer initialized
[ 32.367521] Bluetooth: L2CAP socket layer initialized
[ 32.367544] Bluetooth: SCO socket layer initialized
[ 32.400549] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 32.400565] Bluetooth: BNEP filters: protocol multicast
[ 32.400583] Bluetooth: BNEP socket layer initialized
[ 54.875017] audit: type=1400 audit(1491234097.112:36): apparmor="ALLOWED" operation="open" profile="snap.bluez.hciattach" name="/dev/ttyAMA0" pid=1311 comm="hciattach" requested_mask="wr" denied_mask="wr" fsuid=0 ouid=0
[ 54.875348] uart-pl011 3f201000.uart: no DMA platform data
[ 59.299551] Bluetooth: HCI UART driver ver 2.3
[ 59.299569] Bluetooth: HCI UART protocol H4 registered
[ 59.299574] Bluetooth: HCI UART protocol BCSP registered
[ 59.299579] Bluetooth: HCI UART protocol LL registered
[ 59.299583] Bluetooth: HCI UART protocol ATH3K registered
[ 59.299588] Bluetooth: HCI UART protocol Three-wire (H5) registered
[ 59.299751] Bluetooth: HCI UART protocol Intel registered
[ 59.299816] Bluetooth: HCI UART protocol BCM registered
[ 59.299821] Bluetooth: HCI UART protocol QCA registered
[ 59.376975] Bluetooth: RFCOMM TTY layer initialized
[ 59.377009] Bluetooth: RFCOMM socket layer initialized
[ 59.377041] Bluetooth: RFCOMM ver 1.11

[Paolo Pisati (p-pisati)]

Can you try with hcitool?

[Konrad Zapałowicz (kzapalowicz)]

@ogra

[bluetooth]# scan on
Failed to start discovery: org.bluez.Error.NotReady
[bluetooth]#

Try 'power on' before you 'scan on'

[Oliver Grawert (ogra)]

WHEEE !

ogra@pi3:~$ sudo mount --bind /home/ogra/hciattach /snap/bluez/current/usr/bin/hciattach
ogra@pi3:~$ sudo hciattach /dev/ttyAMA0 bcm43xx 921600 noflow -
bcm43xx_init
Flash firmware /lib/firmware/BCM43430A1.hcd
Set Controller UART speed to 921600 bit/s
Device setup complete
ogra@pi3:~$ sudo bluetoothctl
[NEW] Controller B8:27:EB:1E:54:7E pi3 [default]
[bluetooth]# agent on
Agent registered
[bluetooth]# default-agent
Default agent request successful
[bluetooth]# power on
[CHG] Controller B8:27:EB:1E:54:7E Class: 0x500000
Changing power on succeeded
[CHG] Controller B8:27:EB:1E:54:7E Powered: yes
[bluetooth]# scan on
Discovery started
[CHG] Controller B8:27:EB:1E:54:7E Discovering: yes
[bluetooth]#

[Oliver Grawert (ogra)]

note: this is without any overlay dtb loaded, using the standard Ubuntu Core config.txt

so we need the patched hciattach and a systemd unit that calls it like above to initialize the controller.

@konrad: can you include:

https://github.com/OpenELEC/OpenELEC.tv/blob/6b9e7aaba7b3f1e7b69c8deb1558ef652dd5b82d/packages/network/bluez/patches/bluez-07-broadcom-dont-set-speed-before-loading.patch

and change FIRMWARE_DIR to /lib/firmware in our build as a start ?

[Nikolay (2xl)]

I did the same patch - modify firmware patch and speed initialization patch... and I got bluetooth working on RPi!
I used my own app to test it - https://github.com/Nikolay-Kha/BluetoothAudio
hcitool scan
also works fine. bluetoothctl starts, but I cannot type anything there, don't know why.

Patch, that I did for bluez 5.37 is in the attachment, prebuilt hciattach is also there.
I took and install firmware from here - wget https://archive.raspberrypi.org/debian/pool/main/b/bluez-firmware/^Cuez-firmware_1.2-3+rpi1_all.deb

to start bluetooth I used this commands(script in the attachment also):
sudo ~/bluez/tools/hciattach /dev/ttyAMA0 bcm43xx 921600 noflow -
sudo ~/bluez/tools/hciconfig hci0 up

But, there is always BUT... I experience very weak signal. My bluetooth headset which works normally with 5 meter distance with my phone, doesn't even connect to board with 1 meter. It works only when I put headset right on board. Probably it's just a hardware issue of my RPi board instance... or some system setting, I don't know.

[Nikolay (2xl)]

oops, I forget the attachment that I promised. It's here.?field.comment=oops, I forget the attachment that I promised. It's here.

[Nikolay (2xl)]

And I experiences quite similar issue, that I fill here - https://bugs.launchpad.net/snappy/+bug/1679432

RPi establish SCO connection with my headset and I can send SCO packets to it (with dragonboard I can not even send them), and actually here sound from headset. But I cannot read SCO packets from headset... though I can do it with external USB dongle.

hciconfig shows:
$ hciconfig
hci0: Type: BR/EDR Bus: UART
 BD Address: B8:27:EB:EF:57:6F ACL MTU: 1021:8 SCO MTU: 64:1
 UP RUNNING
 RX bytes:32155 acl:487 sco:0 events:740 errors:0
 TX bytes:11882107 acl:392 sco:228259 commands:225 errors:0

sco RX counter is zero...

[Oliver Grawert (ogra)]

@Nikolay: this bug is about generally making BT work on the pi images by default, your problem is very specific, would you file a new bug for it instead (so we can close this one once the driver works at all)

[Konrad Zapałowicz (kzapalowicz)]

@ogra, yes I will include

[Konrad Zapałowicz (kzapalowicz)]

@ogra, changes pushed to snappy-hwe-snaps/+git/bluez

[Nikolay (2xl)]

@ogra, yes, sure, I created report here - https://bugs.launchpad.net/snappy/+bug/1679747

[Oliver Grawert (ogra)]

@konrad: 5.44-1-dev from the edge channel works fine now as long as i use --devmode ... not using devmode gets me a denial:

[ 48.909501] audit: type=1400 audit(1491387062.644:27): apparmor="DENIED" operation="open" profile="snap.bluez.hciattach" name="/dev/ttyAMA0" pid=1501 comm="hciattach" requested_mask="wr" denied_mask="wr" fsuid=0 ouid=0

we have two options here, either bluez starts shipping a plug for the serial interface and we start providing access to ttyAMA0 through it via the gadget snap or the bluetooth-control or (better) bluez:service get permission for it ...

i guess either way we need to involve the security team here

[Jamie Strandboge (jdstrand)]

For the slot Permanent policy I think it is fine to allow the bluez interface access to /dev/ttyAMA0. ofono, ppp, et al are in similar situations.

That said, if I understand this bug correctly, the most correct solution would seem to be for the pi gadget snap to declare the serial-port interface, eg:

slots:
  pi-bluetooth:
    interface: serial-port
    path: /dev/ttyAMA0

such that the bluez snap would do:

apps:
  bluez:
    daemon: simple
    plugs:
    - serial-port

The question of which to do is I think dependent on how often this situation occurs. If there are many gadgets that have different /dev entries, then having gadget yaml expose this seems the right choice-- the bluez snap need only 'plugs: serial-port' on any device that has a gadget that exposes it and it is connected (in this way there are no snapd changes required for new devices/etc-- gadgets are updated to expose the correct device and bluez picks them up as it gains support for the device).

If the pi is the only gadget out there that does this, then I think it also makes sense to go the gadget route.

Adding to the permanent slot policy probably only makes sense if /dev/ttyAMA0 specifically is a common access across a wide variety of devices. Even then it could be argued the gadget route is the way to go, but in this particular case putting it in the permanent slot policy could remove a small stumbling block for gadget developers.

[Konrad Zapałowicz (kzapalowicz)]

@jdstrand @ogra

This depends on how the Bluetooth chip is wired on hardware level. If it is using UART like in this case it all further depends on how the UARTs are wired. It might be /dev/ttyAMA0 however it might also be /dev/ttyS0 or even /dev/bluetooth - whoever writes the driver has the luxury of choosing.

Whatever we do here it will be only applicable for devices using /dev/AMA0 for BT.

The definition shall be in the gadget snap then.

[Oliver Grawert (ogra)]

@korad, well, i would assume that 90% of the IoT boards we will support will have such a UART based setup, so i would lean towards having generic serial tty support in the BT rules without having to define it for each single board in the gadget... i.e. simply by adding rw support for /dev/tty[A-Z]*[0-9] to the apparmor rules ...

else just installing the bluez snap on devices that didnt take that setup into account in their gadget proactively will simply not work.

[Oliver Grawert (ogra)]

also ... keep in mind that we will additionally need to ship a systemd unit caring for the hciattach call that will be equally specific and will likely live where the hciattach tool lives (in the bluez package), we can not ship this in the gadget without having the tool available. i think we need to generalize on both levels here (systemd/apparmor) to make the bluez package work OOTB on such HW.

[Gustavo Niemeyer (niemeyer)]

Jamie's suggestion seems spot on for the time being. This is really something for the gadget to expose, and it should be made in an explicit way. It can be a regular expression as suggested because then you cannot assign a particular snap plug to a particular serial port anymore.

The upcoming hot-plugging feature will take care of making the process less bothersome, which I think is your concern there, Oliver.

[Gustavo Niemeyer (niemeyer)]

Sorry, it CANNOT be a regular expression. I wish Launchpad allowed edits.

[Konrad Zapałowicz (kzapalowicz)]

@ogra going back to this after a while, what is the status - is it being included in the gadget snap?

[Oliver Grawert (ogra)]

with todays edge image the interface is available on the pi3 ... we now need a device specific hciattach systemd unit in the bluez snap (since that ships the hciattach binary, we cant ship this in the image)

ogra@pi3:~$ snap interfaces|grep blue
:bluetooth-control -
pi3:pi-bluetooth -
ogra@pi3:~$

[Oliver Grawert (ogra)]

here is a proof of concept script+systemd unit ... http://paste.ubuntu.com/24537569/
using this setup i have properly working BT after boot *if* the bluez snap is in --devmode

my proposal forward would be to:

 - ship a similar script inside the bluez package
 - make it a daemon so we get a systemd unit for it
 - ship an interface connection for pi-bluetooth in bluez' snapcraft.yaml

the script allows for adding other hardware to the case statement for other boards we encounter ...

indeed with the "pi3:pi-bluetooth" approach we have in place now this means that the bluez snap now needs to define all possible interfaces for all boards in the snapcraft.yaml to allow hciattach access to the different tty devices on different boards.

[Konrad Zapałowicz (kzapalowicz)]

@ogra

A few questions to make sure that I understand:

I have seen that you have added /dev/ttyAMA to the serial interface of snapd. Any other changes on this side such as discussed previously modifications to gadget snap?

Now you are also suggesting a unit that would hciattach to the desired serial port node. Preferably shipped as a part of bluez snap.

[Simon Fels (morphis)]

Btw. you guys should see if the more modern btattach utility does the job instead of hciattach too. See http://manpages.ubuntu.com/manpages/xenial/man1/btattach.1.html for a few details. Sooner or later btattach will replace hciattach as the way to attach serial lines to the kernel bluetooth stack.

btattach is much smarter as it only sets the line discipline for a serial port and leaves everything else up to the kernel.

Only supporting btattach would simply the whole implementation as we just have to provide a serial port plug on the bluez snap the user can plug any serial port to and then (via interface hooks) the bluez snap will figure out what to do with it, without having to know anything device specific. The relevant UART options (speed, noflow, vendor protocol) should be all specified via the connecting slot. Can we set arbitrary attributes on a slot definition and implement with that our own meta data needed for such a thing? Something like this would be possible then:

gadget:
  bt-serial:
    interface: serial-port
    path: /dev/ttyUSB0
    speed: 115200
    flow-control: false
    hci-uart-protocol: h4

The plug side in the bluez could then just figure out everything on its own by inspecting the attributes of the connecting slot.

[Konrad Zapałowicz (kzapalowicz)]

I like @morphis idea

[Oliver Grawert (ogra)]

well, the btattach tool would still have to be shipped and executed by bluez.

how would the additional info added in the interface data get handed over to the btattach tool ?
how would we execute it ? i admit it looks a lot better and bluez wouldn't have to define each and every board implementation.
but it also means a new interface from scratch or at least some significant changes to the serial-port interface to deal with the extra values and on the execution side it wont change much (you will still need a systemd unit in bluez and some script to hand the options to btattach (unless you want to patch btattach itself to pick up the options where the interface provides them (but will have to carry and maintain that patch then)).

i definitely like the more generic name, and will rename the pi3:pi-bluetooth interface to pi3:bt-serial to get us around the "has to have every board in snapcraft.yaml".

for the rest ... the question is how much effort these interface changes or re-implementation of a new takes ...