ProdESP32 #3: Understanding Partitions

If you’ve only ever used the Arduino IDE for your ESP32 development then this images sums up your entire exposure to the ESP32 partition table.

These are the the options for Partition Scheme in the Arduino IDE. This list doesn’t even begin to scratch the surface of what you can do with partitions on the ESP32.

What is a Partition?

A partition, in it’s simplest form, is a region of flash memory set aside to be used for a specific storage purpose. It may be for application code, or WiFi credentials, or a web server file system. Partitions are how you divide up the available flash. For example, if you have an ESP32 that is listed as having 4MB of flash then you can divide that flash up into different partitions to suit your application needs.

You can think of partitioning in your ESP32 in the same way your hard drive is partitioned. Here is a view of my hardrive partitions.

Notice each of these has a name, type and designated purpose. We can do the exact same thing with the available flash in our ESP32.

Partition Organization

Now, while you can customize the layout of the available flash there are some basic rules and limitations you need to understand. First and foremost is that the first 4KB (0x1000) of flash memory is reserved for Secure Boot purposes which will be covered in a future article. The second-stage bootloader is always flashed at offset address 0x1000. As you can see in menuconfig this is not configurable.

Beyond that, the choice is yours as to how you want to partition the remaining flash. However, you do have to follow a couple of basic rules.

• Application partitions must be aligned to offset multiples of 0x10000 (64K)
• All partitions must be sector-aligned which means they need to have an offset and size which is a multiple of 0x1000 (4KB)

So your clean slate of flash memory will look like this:

partitions.csv

The menuconfig utility in ESP-IDF offers the option, similar to the Arduino IDE, to select from a list of pre-defined partition layouts.

DON’T USE ONE. For your production application you should always select the option Custom partition table CSV

You can specify a file anywhere but I recommend you just name it partitions.csv and put it in the root of your project. This file is where you will define the layout of your partitions in flash. The syntax of this file is beyond the scope of this article but is thoroughly documented and not hard to learn.

You can create application partitions, NVS partitions, SPIFFS or FATFS partitions. You can specify encryption options. Using this approach you have access to the entire toolbox provided by Espressif to layout your flash memory. Even if you think you want a pre-defined partition layout use the CSV file so you can be explicit and see how your flash is arranged.

Quick Note On Optimization

Let’s take a look at the default OTA partition table from the Espressif docs.

# ESP-IDF Partition Table
# Name,   Type, SubType, Offset,   Size,     Flags
nvs,      data, nvs,     0x9000,   0x4000,
otadata,  data, ota,     0xd000,   0x2000,
phy_init, data, phy,     0xf000,   0x1000,
factory,  app,  factory, 0x10000,  1M,
ota_0,    app,  ota_0,   0x110000, 1M,
ota_1,    app,  ota_1,   0x210000, 1M,

You’ll notice a phy_init partition that is 0x1000 (4KB) in size. But from the docs we read:

In the default configuration, the phy partition is not used and PHY initialisation data is compiled into the app itself. As such, this partition can be removed from the partition table to save space.

There is also a factory partition. It’s 1MB in size! which is HUGE in a 4MB flash ESP32. Here the docs say the following:

The bootloader will execute the factory app unless there it sees a partition of type data/ota, in which case it reads this partition to determine which OTA image to boot…If you want to conserve flash usage in an OTA project, you can remove the factory partition and use ota_0 instead.

So if you are using OTA you don’t have to have a factory partition and, in fact, it will never be loaded as long as you are using OTA (there are exceptions to this but bear with me). So you can make an informed decision about whether you want to keep the factory partition or not and reclaim 1M of flash to redistribute among your OTA partitions.

There are lots of other ways to optimize your flash layout. Just remember you are in complete control and can optimize this usage to fit your application needs.

The Key Points

There are tons of things you can do with partitions and it gets even more complicated when you introduce encryption (a topic that will be covered in a future post). Here are the key points to get you started.

• Application partitions must be aligned to 0x10000 (64KB). This also means that it doesn’t make sense to specify an application partition size that is not a multiple of 64KB. If you do you will be wasting flash space as the IDF is going to round up to the nearest 64KB boundary for you.
• All other partitions must be sector aligned (0x1000 4KB). Same advice. Only specify sizes that are multiples of this. It doesn’t make sense to specify a partition size of 6KB.
• The bootloader always starts at offset 0x1000
• 0x0000 -> 0x1000 is reserved for Secure Boot. You cannot use this area of flash

Action

Here is what you can do today with this information to help your application be more production ready.

• Start using partitions.csv. Even if you want a standard layout, take the time to do it in partitions.csv.
• *Make sure your partitions are aligned and properly sized.** This ensures you aren’t wasting a byte of flash.

Production Pointers

• Always use partitions.csv. This codifies and explicity outlines how you are using every byte of your flash memory.
• Design your partition layout. For many projects this is an afterthought. “Just use a pre-defined one and move on.” This is a HUGE mistake. Once your product is out in the wild it is virtually impossible to change the partition layout. You need to be thoughtful here and try your best to anticipate future use. Maybe you don’t need a FATFS partition today but will you one day?
• Secure Boot affects bootloader size. The bootloader starts at 0x1000 and the partition table partition is typically located at 0x8000 with a size of 0x1000. That means you will almost always see partition table definitions start with an entry at 0x9000 and up. When you enable Secure Boot (which you must for production) the bootloader size will change and may no longer fit between 0x1000 and 0x8000. In that case you need to move the default location of the partition table and your partitions.csv entries will begin beyond 0x9000.
• You can reduce the size of the default NVS partition. The default NVS partition nvs is used for things like WiFi credential storage and other built-in IDF features. Depending on your usage of NVS the default size of 0x4000 (16KB) is probably overkill and you can dial it back to use the space for your other partitions.
• Know the important conversions. 0x400 = 1KB, 0x1000 = 4KB, 0x10000 = 64KB. Offsets and sizes are often listed in hex so it’s helpful to know some basic conversions to have an idea of how big or small a number is.

Conclusion

Understanding partitions and using partitions.csv ties into the Scalable and Deterministic Pillars of Production. It’s scalable because it allows you to plan ahead and reserve flash storage to implement future features. It’s deterministic because you are specifying, down to the last byte, exactly how you are utilizing the flash space on the ESP32. It’s exact, ensuring you aren’t wasting a bit of that memory.

I highly recommend you read through the Partition Table documentation. It is very good at describing all of the options available when defining your partition layout. Then start playing with some of the options and building firmware to see how it works. Once you are comfortable with it you will never go back to the training wheels of pre-defined partitions.

Join the community and get the weekly Production ESP32 newsletter. Concise, actionable content right in your inbox.