Mount additional FAT partition when performing DFU

[Bogdan-Vasile]

I am trying to develop a datalogger based on ESP32S3, and it would be nice if, when booting for DFU, a data partition could also be mounted to retrieve some files from the FAT data partition. It would be even nicer if multiple additional partitions could be mounted. And maybe this could be integrated in the build system, to specify the optional partition names that should be mounted when performing DFU.

I think it should be something similar to https://github.com/hathach/tinyusb/tree/master/examples/device/msc_dual_lun
I'm not that proficient with tinyUSB, nor with tinyUF2, so I would appreciate some more or less detailed instructions.

[hathach]

it is relatively easy but I don't see why it is an bootloader job to mount and retrieve log file from application. I.e I don't think it is part of bootloader project.

[Bogdan-Vasile]

Thank you for your response @hathach !

I was thinking from the user's perspective. Updating the firmware or retrieving some files is a maintenance issue (for this application), and it would simplify the procedure a lot if they could put the device in "maintenance" mode and perform the needed actions.

As for the files... I'm not thinking of creating any log files, like it is done now for indexFile or uf2 file, but just mount a partition and get the existent files via standard operations.

I tried doing this in the application (ESP IDF), but there are some problems with using MSC and CDC (composite) devices at the same time. Moreover, while the user mounts the partition and copies the files, the system can't write data to that partition anyway. So at least for me it makes the code more complicated because of that, and I also have to implement code to make sure that the user doesn't forget the partition mounted.

[josuah]

I think this is more or less what was done here: #272

TinyUF2 got integrated into the firmware and runs side-by-side, as a library: no switching between maintenance mode and normal mode: the USB interface is always available on normal firmware operation.

Difference 1: This is for uploading, not downloading files.
Difference 2: This is for the RP2040, not sure about that issue with ESP IDF.
Difference 3: This runs side-by-side with the application: a single MSC interface along with other USB stuff from the application.

Here is how we have it configured (take note to also edit the number of CDC/MSC interfaces in the header file):
https://github.com/tinyvision-ai-inc/pico-ice-sdk/blob/4aea79e0e136b7fa08eb1c277a66755161207053/src/usb_descriptors.c#L107-L112

[Bogdan-Vasile]

Yes, I tried something like that, but in ESP IDF, and got hit by some problems along the way.

The only reason I started this thread was because of the "maintenance" mode described above, and because I was thinking it should not be that complicated to mount a second partition since other tinyUSB2 features are used. But I understand now a bit better the TinyUF2 project philosophy, and I can see why this would not fit into this story.

[josuah]

What you describe sounds like a generic use of TinyUSB rather than a special feature of TinyUF2?
Do you need to download files in the UF2 format? This is the main goal of TinyUF2 after all...
In hope you figure out the best compromise.

[josuah]

An analogy to understand UF2: instead of sending a text email, a scan of a handwritten message was sent as an attachment.
The recipient analyze the image content and convert it to text internally.

A bit like how in some forms, you have to write text well aligned with boxes so that a computer can read it from the scan:

But here, with a raw disk storage and the file content, instead of a piece of paper and the text onto it.

[josuah]

So I am not sure it would be the most suited tool to get plain file transfer over USB MSC.

[Bogdan-Vasile]

Thank you for your input @josuah!

The files that the user needs to download are some text files created a priori by the application, arranged in folders, on a FAT data partition. The file or folder names are not static, but depend on the measurement timestamp.

I was already investigating using TinyUSB in the main app, but the ESP IDF integration may have some bugs or I need to do some more advanced stuff, and it is not working ok (e.g. if I have the debug serial port connected, the data partition is not mounted and/or the board reeboots permanently) and I have to implement special handling for the case when the user wants to mount the data partition.

I am not proficient at all with TinyUF2. I read about the UF2 format, and I looked a bit at the TinyUF2 code. It seemed to me that this project uses TinyUSB under the hood, which already has the features I mentioned.

But if you think it is out of scope for this project, I totally understand it.

[josuah]

I think you might find some project that does most of what you want to do: for instance, the application would write to flash storage into a FAT filesystem. And then, a "maintenance bootloader" would be a project that turns an device into an USB key.

[josuah]

So if you find a project that permits to turn the microcontroller you use in to an USB key, you might be all done.
Although if you also need to offer firmware upgrade features through TinyUF2 along with that extra usb MSC interface (extra partition), then you would need to pick code from TinyUF2.

Another thing to consider is DFU (device firmware upgrade) USB protocol, which is handled natively by TinyUSB.