Hello, I noticed some varying Ota speeds on our toit devices, and decided to investigate the problem. The speeds would vary from 40 seconds to over 3 minutes. After i cleaned the SD card on the slow devices, it brought the time down to like 40 seconds. I made some tests with the SD-card, by making big files and many files to see the impact on the Ota speed. The files I made are in a folder being used by the device. I tried putting the files outside the working folder, but that did not change the Ota speed. In conclusion: The amount of files present in a working folder slows down the device. I have trouble finding out why this is the case. - We are appending to a single file 0-3 times a second - SPI clock is at 400 kHz - The

host

package is used to create a file stream to the 1 file - The Ota data is received with http client - Ota data is written with

firmware.FirmwareWriter

writer

That is very interesting. Is it possible to try to get some memory stats before the OTA process starts? I'm curious if it is slower because of memory pressure. Are you running everything from within the same container?

I can try and get some memory stats, and yes everything is from the same container

If it is from one container, then both

print_objects

and

serial_print_heap_report

can be useful.

I will try print_objects 👍 had to get some things ready

You may want to use

--gc

for the

process_stats

call.

--gc

is not in alpha 47 😦

My thoughts. The host package is synchroneous, so it will block during file io and also limit the http firmware.FirmwareWriter task. You can verify this by adding timestamps to the ota loop to see if it get called less frequently when there are many files on the sdcard.

Thank you for your input, I will test it tomorrow !

The host package was originally developed on Linux which does not have support for getting an event from the OS when file operations have completed. Linux will use huge disk caches that fill all available memory, so this is not normally a huge issue. It might be a bigger issue on file operations on ESP32.

250 files on the card is also unlikely to happen for us, but I was just curious and wanted to understand why 🙂 I guess I have to make sure we don't create too many files.

Will having subdirectories help?

I didnt try with subdirectories in the current folder, but that solution would cost more to implement, than reducing the amount of files generated 🙂

What would help you (here and now) is to use the microservice framework and then "spawn" the service responsible for either FW-update or SD-card writes. This makes it run in a separate FreeRTOS task on the ESP32 and the host package blocking will not affect the other process. I am doing something very similar to avoid TLS decryption to interfere with the audio data path.

Thats a great idea! I will try that after lunch 😄

You are welcome. The SD-card lib should probably not block though 🙂 It is now on my list.