BLE Connection to device which does not advertise ...
# help
z
Copy code
import ble

DEV_NAME  ::= "O2Ring"
SCAN_DURATION   ::= Duration --s=3

find_by_name central/ble.Central name/string:
  central.scan --duration=SCAN_DURATION: | device/ble.RemoteScannedDevice |
    if device.data.name and device.data.name.contains name:
        return device.address
  throw "no ring device found"

main:
  adapter := ble.Adapter
  central := adapter.central

  address := find_by_name central DEV_NAME
  remote_device := central.connect address
  print remote_device.address
  
  services := remote_device.discover_services 
  //0x0d    BLE_HS_ETIMEOUT    Operation timed out.
  //   EXCEPTION error. 
  // NimBLE error, Type: host, error code: 0x0d. See https://gist.github.com/mikkeldamsgaard/0857ce6a8b073a52...
  //   0: ble_get_error_            <sdk>/ble.toit:980:3
  //   1: Resource_.throw_error_    <sdk>/ble.toit:811:5
I'm trying to communicate with a ring-based pulse oximeter which does not advertise any services. when calling
discover_services
I receive a NimBLE timeout error. I know the service UUID, and the UUID of the characteristics I'm interested in. Can I skip straight to exchanging write/notify data from the characteristics? Service data from LightBlue screenshots are attached
f
I don't have a device to test this on, bit maybe it works if you provide the IDs during discovery. @mikkel.damsgaard might know exactly what to do.
m
The BLE protocol requires you to discover the services. They do not need to be advertised to be discovered.
One thing we havent done quite yet, is pairing on BLE on esp32. So if the oring requires the device to be paired, then this is something that could easily happen.
If you have the possibility to try it on mac, then the mac can pair and you can then see if it works.
z
I do have this currently working on macOS in a Python program, let me post some snippets!
https://github.com/z3ugma/o2r
Copy code
def on_detection(self, device, advertisement_data):
    if device.address not in self.devices:
      name = device.name or device.address
      uuids = device.metadata["uuids"] if "uuids" in device.metadata else None
      if self.verbose > 4 and device.address not in self.pipe_down:
        print(f"Considering {device.address} {name} {uuids}")
        self.pipe_down.append(device.address)

      valid = False
      if uuids is not None and BLE_MATCH_UUID in uuids and BLE_SERVICE_UUID in uuids:
        valid = True
      else:
        # We might not have the list of UUIDs yet, so also check by name
        names = ("Checkme_O2", "CheckO2", "SleepU", "SleepO2", "O2Ring", "WearO2", "KidsO2", "BabyO2", "Oxylink")
        for n in names:
          if n in name:
            if self.verbose > 1:
              print(f"Found device by name: {n}")
            valid = True
            break

      if not valid:
        return

      print(f"Adding device {device.address}")

      dev = O2BTDevice(address_or_ble_device=device, timeout=10.0, disconnected_callback=O2BTDevice.on_disconnect)
Copy code
async def _go_get_services(self):
    if self.disconnect_pending or not self.is_connected:
      return

    # services = await self.get_services()

    if self.manager.verbose > 1:
      print(f"[{self.name}] Resolved services")
      for service in self.services:
        print(f"[{self.name}]\tService [{service.uuid}]")
        for characteristic in service.characteristics:
          print(f"[{self.name}]\t\tCharacteristic [{characteristic.uuid}]")
          for descriptor in characteristic.descriptors:
            value = await self.read_gatt_descriptor(descriptor.handle)
            print(f"[{self.name}]\t\t\tDescriptor [{descriptor.uuid}] ({value})")

    for s in self.services:
      if s.uuid == BLE_SERVICE_UUID:
        for c in s.characteristics:
          if c.uuid == BLE_READ_UUID:
            asyncio.ensure_future(self._go_enable_notifications(c))
          elif c.uuid == BLE_WRITE_UUID:
            self.write = c
@mikkel.damsgaard afaik it doesn't require pairing. With Python using
bleak
we call
connect
only
self
in this context refers to an instance of
BleakClient
Copy code
class BleakClient:
    """The Client interface for connecting to a specific BLE GATT server and communicating with it.

    A BleakClient can be used as an asynchronous context manager in which case it automatically
    connects and disconnects.
inside of the macOS
bleak/backends/corebluetooth/PeripheralDelegate.py
then services is populated by
peripheral.discoverServices_(None)
which calls into
CoreBluetooth
to https://developer.apple.com/documentation/corebluetooth/cbperipheral/1518706-discoverservices/
But when I call
discover_services
in Toit :
Copy code
remote_device := central.connect address
  print remote_device.address
  
  services := remote_device.discover_services 
  //0x0d    BLE_HS_ETIMEOUT    Operation timed out.
  //   EXCEPTION error. 
  // NimBLE error, Type: host, error code: 0x0d. See https://gist.github.com/mikkeldamsgaard/0857ce6a8b073a52...
  //   0: ble_get_error_            <sdk>/ble.toit:980:3
  //   1: Resource_.throw_error_    <sdk>/ble.toit:811:5
^^ I get the commented error, of a bluetooth timeout
or @mikkel.damsgaard did you mean to try to run the Toit code on macOS instead of on the ESP32 itself
f
I think that's what he meant, but it's probably helpful to have the python code as well
z
@floitsch the tutorials show how to run jag on Mac and then send the code to ESP32. Is there documentation of how to run toit code directly on Mac
f
I'm not sure if we have a 'brew' for Toit directly, but you can download the binaries from here: https://github.com/toitlang/toit/releases/tag/v2.0.0-alpha.47
Then use
toit.run
to execute your program.
Alternatively, you could also use the SDK that Jaguar downloads.
(It's actually the exactly same archive).
Jaguar stores its version of the SDK in
~/.cache/jaguar/sdk
You would have a
toit.run
there:
~/.cache/jaguar/sdk/bin/toit.run
.
Ooh. And I forgot. You can just ask Jaguar to run code locally with `-d host`:
jag run -d host foo.toit
.
That's clearly the easiest.
z
Copy code
jag run -d host  blescan.toit
cm started
B6DF069F-1B4D-A7B1-984A-FAFC71D6760A
that works without throwing the timeout exception
if I add a call to
print services
Copy code
jag run -d host  blescan.toit
cm started
B6DF069F-1B4D-A7B1-984A-FAFC71D6760A
[an instance with class-id 47, an instance with class-id 47, an instance with class-id 47]
f
That's good data. Looks like the ESP32 implementation seems to behave differently. Let's hope Mikkel (who wrote the BLE implementation) knows what to do next. Thanks for all the testing.
Fyi: the
an instance with class-id 47
is the generic way of printing an object that doesn't have a
stringify
method. We don't store the class names in the executable, to keep the executables smaller. If you wanted to get the name of the class, you would need to do the following steps: 1. compile the program to a snapshot:
~/.cache/jaguar/sdk/bin/toit.compile -w /tmp/out.snapshot your_program.toit
2. use toitp to print all class-ids:
~/.cache/jaguar/sdk/tools/toitp -c /tmp/out.snapshot
We already discussed that Jaguar (
jag run
) should automatically change these "`an instance with class-id XX`" to something more readable, but in theory this could be a text that the user printed and didn't want to be replaced.
z
Thanks for your help too! I guess this is interesting that I can try to get the rest of the program, e.g. the writing and receiving notifications from characteristics, using the mac bluetooth radio, until we get to the bottom of the ESP32 BLE stack
f
That should work. And makes development faster.
m
Ok. Thanks for that. I believe the python library does the service discovery for you, there is no way for you to communicate with a peripheral without having discovered the services and set up the shorthands for services and characteristics. It might seem that the NimBLE stack has a timeout that needs to be addressed.
z
@mikkel.damsgaard the timeout for the call on esp32 took about 20-30 sec. Running the toit code on macOS discovers the service right away
m
Yes, NimBLE has a hard timeout of 30 seconds. The timeout occurs because there was no perceived response by the NimBLE stack within that time frame. What causes the response to be lost is a bit more unclear. In our experience the default TX power of the bluetooth stack on ESP32 only allows for 3-4 meters of distance, whereas the mac can reach 30 meters or so. Is your ESP32 close to the peripheral?
z
Yep! 1-2m. It’s a pulse oximeter worn on a finger
Any logging or instrumentation that I can do, or increasing verbosity, to give you all better traceability?
m
I have been looking through the code and unfortunately there is not a lot I can do. I hand over the discovery request to NimBLE library that hands it over to the HW and no response comes back. I was wondering if it is an underlying incompatibility between ESP32 (4.2 BLE) and the ORing. What specs does the ORIng has on the BLE?
z
This is interesting to go on. I wonder if I can try making a toy example that works in C++ / Arduino that exhibits the same behavior using NimBLE directly.
From what I can find online it’s BLE 4.0 compatible
f
or related.
z
Maybe related. This issue says that esp-idf is working with the o2ring model. I think my next steps are to try a nimble and an esp-idf example to see if I can list the services without a timeout
m
Ok, I just oredered a O2ring, so I can debug that properly.
f
I hope you have other uses for it as well 🙂
z
😳whoa!
m
Not really. But I can find someone who has 🙂
z
they're useful for CPAP / sleep apnea patients... that's my use case
https://gist.github.com/z3ugma/b50fc6f2cd40729302e8a8c863e13729 Made a gist with a working Arduino BLE example that uses NimBLE. It connects to the O2Ring, discovers the service of interest, and lists the
write
characteristic of interest
m
Cool, we now know the HW is not at fault. What happens if you comment this out:
Copy code
#ifdef ESP_PLATFORM
  NimBLEDevice::setPower(ESP_PWR_LVL_P9); /** +9db */
#else
  NimBLEDevice::setPower(9); /** +9db */
#endif
z
@mikkel.damsgaard commented out those lines and flashed to ESP32, same result
m
Ok. Thanks.
It is a bit weird. I have checked the Arduino implementation against our implementation and they are quite identical. Could you turn on debug logging in the Arduino lib, so we get all the NIMBLE_LOGD's from the implementation?
#define CONFIG_NIMBLE_CPP_LOG_LEVEL 4
I think
z
By turning up the
Core
log level to
DEBUG
as well, I get this more verbose log stack https://gist.github.com/z3ugma/b50fc6f2cd40729302e8a8c863e13729#file-console_debug-log
m
Thanks. That is really helpful. Now we just have to wait until I get the device.
I think I found something 🙂
Unlike the Arduino lib, we were not waiting on the MTU exchange to finish before we declared the device connected. So that would result in the service discovery to possibly fail. You could try adding a
sleep --ms=500
before the call to
discover_services
z
!!! that works!
Copy code
[jaguar] INFO: program 73549571-68fd-594e-80c7-a2904b16810e started
#[0x01, 0xc6, 0xd1, 0x08, 0xef, 0x8a, 0x4d]
[an instance with class-id 48]
[an instance with class-id 45]
[jaguar] INFO: program 73549571-68fd-594e-80c7-a2904b16810e stopped
neat. now onto the nitty gritty of BLE, writing and receiving the data from the device. thanks both of you for your help so far
what's the difference between
wait_for_notification
and
subscribe
in the ble library ...and if you're subscribed, is there a place to give a callback function that gets called or run when the notification is sent by the device?
The way this O2Ring works is that you write a command string to '8b00ace7-eb0b-49b0-bbe9-9aee0a26e1a3', such as
b'\xaa\x17\xe8\x00\x00\x00\x00\x1b'
This gives the command to the ring to Notify/Indicate to us when the value of the ring next changes.
Copy code
b'aa17e8000000001b'
[O2Ring 0543] Sending aa17e8000000001b
Writing b'\xaa\x17\xe8\x00\x00\x00\x00\x1b' up to 20 b'\xaa\x17\xe8\x00\x00\x00\x00\x1b'
[O2Ring 0543] Characteristic 8b00ace7-eb0b-49b0-bbe9-9aee0a26e1a3 write value performed
[O2Ring 0543] Characteristic 0734594a-a8e7-4b1a-a6b1-cd5243059a57 updated: 5500ff00000d0061320000000000340000120100
o2pkt recv: 5500ff00000d0061320000000000340000120100
want 21 have 20
[O2Ring 0543] Need more data
[O2Ring 0543] Characteristic 0734594a-a8e7-4b1a-a6b1-cd5243059a57 updated: 07
o2pkt recv: 07
[O2Ring 0543] Final recv: 5500ff00000d006132000000000034000012010007
[O2Ring 0543] 61320000000000340000120100
[O2Ring 0543] SpO2  97%, HR  50 bpm, Perfusion Idx  18, motion   0, batt  52%
m
Copy code
characteristics.subscribe // This will subscribe to notifications on the characteristic. The call returns immediately
while true:
  notification := characteristics.wait_for_notification // This call blocks until a notification is received. The notification is a ByteArray
  print "I received a notification: $notification"
It is common to have the while loop in a task to have it run interleaved:
Copy code
characteristics.subscribe
task::
  while true:
    notification := characteristics.wait_for_notification
If you no longer wish to received notifications, call
unsubscribe
z
@mikkel.damsgaard did you receive your ring? XD
I’m going to spend some of today getting the write/notify loop working. I send a “command packet” to the write characteristic every second, and receive ~127 bytes back from the notify characteristic subscription
Got it working pretty well. Sometimes the ring does not play nicely, especially when you go out of range. Is there a way to
wait_for_notification
but cancel it after a timeout (some sort of nonblocking)
Copy code
while true: 
    write_characteristic.write #[0xaa, 0x1b, 0xe4, 0x00, 0x00, 0x01, 0x00, 0x00, 0x5e] // Realtime

    first_block := subscribe_characteristic.wait_for_notification //Get first packet
    msg_length := first_block[5]
    total_length := msg_length + 8
    print "$total_length @ $Time.now.utc"

    print (hex.encode first_block)
    
    //You'll get 20 bytes at a time
    (total_length / 20 ).to_float.floor.to_int.repeat:
      notificationline := subscribe_characteristic.wait_for_notification
      print (hex.encode notificationline)
    

    
    sleep --ms=800
Copy code
/**
  Waits until the remote device sends a notification or indication on the characteristics. Returns the
    notified/indicated value.
  See $subscribe.
  */
  wait_for_notification -> ByteArray?:
    if properties & (CHARACTERISTIC_PROPERTY_INDICATE | CHARACTERISTIC_PROPERTY_NOTIFY) == 0:
      throw "Characteristic does not support notifications or indications"

    while true:
      resource_state_.clear_state VALUE_DATA_READY_EVENT_
      buf := ble_get_value_ resource_
      if buf: return buf
      state := resource_state_.wait_for_state VALUE_DATA_READY_EVENT_ | VALUE_DATA_READ_FAILED_EVENT_ | DISCONNECTED_EVENT_
      if state & VALUE_DATA_READ_FAILED_EVENT_ != 0: throw_error_
      if state & DISCONNECTED_EVENT_ != 0: throw "Disconnected"
^^ here is how the method is implemented in https://github.com/toitlang/toit/blob/master/lib/ble.toit Maybe I could extend the class RemoteCharacteristic with another method, wait_for_notification_with_timeout that accepts a timeout parameter and only loops for that many ms
m
Hey. Yes I got the ring, but this week has been crazy. Will unpack it tomorrow evening and get the pr done
Normally timeout are done like this:
Copy code
with_timeout --ms=4000:
   data := char.wait_for_notification
That said, I think the BLE library needs to be more aware of disconnects. I am still thinking about how to implement that properly.
@z3ugma I have the o2ring working without the sleep now.
z
Great news
k
@z3ugma: Jaguar v1.8.6 is out with 3 fixes from @mikkel.damsgaard 🙂
4 Views