ProdESP32 #4: FatFS over SPIFFS

Storage is an important part of most embedded applications. The ESP32 supports several options for storing data such as NVS (non-volatile storage), a driver for SD card support, and filesystem support on flash partitions. This week we’re focusing on filesystem storage.

Filesystem storage can be used for things like:

• Static assets (webserver or audio files)
• Configuration files
• Certificates for authentication
• Logs

To utilize filesystem storage you simply have to define a partition and then choose one of the supported filesystem components; SPIFFS or FATFS. But if you come from the Arduino landscape this may look familiar.

Arduino IDE Partition Options

Based on this image you might assume that SPIFFS is the better choice. After all, it outnumbers FAT in the list of available defaults. You will also find many articles online that use SPIFFS as well.

For production applications I recommend you always choose FatFs over SPIFFS.

Why FatFs Instead of SPIFFS?

A fair question. There are other options such as LittleFS which has it’s own advantages. But for the purpose of this post we are only considering the two filesystem options natively supported by IDF. Let’s go through the reasons why we would use FatFs over SPIFFS.

It’s Deprecated…Unofficially

The SPIFFS repo is not really under active development. Some libraries that used SPIFFS declared it deprecated although there is no official deprecation notice on the repo itself. In any case, it has quite a few outstanding issues that will likely go unaddressed unless you want to fork the project and fix them yourself.

As such it should be considered as deprecated and therefore not suitable for use in a production application.

No Out-Of-Box Encryption

The Flash Encryption feature in IDF is not supported on SPIFFS partitions. If you want encryption you have to add it at the application level. Meaning, you would need to provide your own encryption key creation and storage mechanism, encryption algorithm, and encrypt data before saving it to the partition and decrypt it when reading from the partition.

All of this is supported out-of-the-box with FatFs in IDF and requires no special considerations in your firmware code itself. You simply have to mark the partition as encrypted in the partition table.

Without native encryption support SPIFFS presents a security vulnerability.

And a Few More

Besides these massive dealbreakers the notes for SPIFFS also give us a few more reasons why we wouldn’t want to use it in production. These are clearly documented in the Espressif docs.

• No directory support
• Only uses about 75% of the partition space. This is a huge issue with smaller partitions. You will run into very hard-to-track-down bugs when you start hitting max capacity.
• Performance degrades significantly as the partition fills.
• Deleting a file can leave unusable sections in the filesystem. This essentially fragments the filesystem and will reach an unusable point.

A Note on Naming

For those of you old enough to remember the good ol’ days of dialup internet and Windows 3.1 you will recall the 8.3 naming scheme. This is a limitation of the FAT filesystem. And even though we’re decades beyond those times, this limitation is the default configuration of the FatFs implementation. However, there is nothing at compile-time to stop you from trying to name a file with a .json extension. Just know it isn’t going to work.

You can change this in menuconfig using the CONFIG_FATFS_LONG_FILENAMES option. It will, however, have an impact on memory usage.

Action

• All new projects should use FatFs instead of SPIFFS
• Migrate existing projects from SPIFFS to FatFs where possible

Production Pointers

• Enable Wear Levelling. Without it you will wear out the flash memory significantly faster which will reduce the lifetime of your device.
• Use POSIX Functions. You can use native FatFs functions to read and write from the storage partition files but it is much better to use the portable POSIX API functions instead. This is supported out-of-box because of the IDF Virtual Filesystem component.
• Enable Encryption. Always make sure you encrypt the storage partition so the data can’t be easily read via a physical attack. To do this simply mark the partition with the encrypted flag in the partition table.
• Use Interfaces. This is an entirely separate topic but if you build your storage access behind an interface it is trivial to replace it later such as when FatFs is deprecated and a better option is available. All of your business logic will use the interface and you can change the implementation to anything you like later (assuming the partition supports it).

Conclusion

Using FatFs instead of SPIFFS ties into the Secure and Deterministic Pillars of Production. It’s secure because it supports encryption out of the box and it’s deterministic because it is a much more stable and reliable implementation of a filesystem than SPIFFS.

Join the community and get the weekly Production ESP32 newsletter. Concise, actionable content right in your inbox.