ProdESP32 #2: Use ESP-IDF

This edition could also be called “Stop Using the Arduino IDE!”. As I’ve consulted with many different founders on their projects this is a common challenge and concern.

“I should probably stop using the Arduino IDE, huh?”

The answer is a resounding YES! The sooner the better. The Arduino IDE is incredibly useful for what it is, a low barrier-to-entry IDE for beginner, student and weekend maker projects. If you want to move beyond that you will have to graduate. I recommend using ESP-IDF directly.

A Brief Note on PlatformIO

Before you fill the comments with “What about PlatformIO?” let me address it. I have only used PlatformIO for one production application that I consulted on because it’s what they already had in place. I know it works for some people and if it works for you, awesome, keep on keepin’ on.

My personal experience has been that it adds an additional layer to the learning curve of using ESP-IDF. PlatformIO is nothing more than a wrapper on ESP-IDF with some additional bells and whistles. The most common issue I’ve seen repeatedly is people trying to figure out how PlatformIO wraps a feature that is already very clearly documented and explained in ESP-IDF.

A recent example from Reddit of trying to figure out how to do something the PlatformIO way which is trivial in ESP-IDF

The second most common thing I’ve seen is waiting for PlatformIO to include a feature or fix that already exists in ESP-IDF. I recommend not treating the IDF as a black box. It’s not that difficult to learn and is extremely well documented. If you learn what’s in the black box you won’t need a fancy wrapper for it. Moving on.

Why You Need to Drop the Arduino IDE

Before going further we need to distinguish between using the Arduino coding syntax and the Arduino IDE. You can use the ESP-IDF and still write your Arduino-style code. I recommend not doing this but it is possible and quite easy to do with the arduino-esp32 component. The Arduino IDE, however, as IDEs go is massively underpowered and feature-poor. That said, the ESP-IDF is not an IDE, it’s a framework (that’s what the F stands for) that provides source code and tools for compiling and flashing devices. You have to bring your own IDE (Integrated Development Environment). I personally recommend VSCode and will cover this more in a future post.

Here is an abbreviated list of some of the things you either can’t do in the Arduino IDE or are non-trivial.

• Create a custom partition layout
• Access the full feature set available in menuconfig
• Create/Use custom ESP32 components that aren’t Arduino packages
• Unit testing
• CI/CD
• Step debugging
• Custom file layout (needs to follow the Arduino style)
• Secure Boot
• Flash Encryption

Action

• Stop using the Arduino IDE. You can do this in phases and even keep your Arduino syntax in the beginning although you’re definitely going to want to refactor that out later. Porting your Arduino code to ESP-IDF can usually be done in an evening. Throw a comment if you’d like me to create a How-To guide for doing so or make a video example.
• Try the ESP-IDF. Baby steps. Start by following the Espressif Getting Started guide to see how easy it is to get a simple program compiling and flashing to a dev board in the IDF. There is also a VSCode Docker version that works right out of the box.

Production Pointers

• Start simple. Get a basic “Hello World” program working with ESP-IDF and build up from there.
• Choose a fully featured IDE. I highly recommend VSCode.
• Drop the Arduino syntax. I know you’re used to setup() and loop() but those are training wheels that keep you from using the full power that is available in IDF and FreeRTOS.
• Read the docs. The IDF is very well documented and you’ll learn a ton by “reading the manual”.
• Use Linux and Docker. This is a pretty opinionated take but having developed firmware in all three major OSes (Mac, Windows and Linux) I promise Linux is the best for embedded development and it’s ability to run Docker natively pays huge dividends. I’ve also found it easiest to manage the IDF in Linux if you choose to do so outside of a containerized environment.

Conclusion

Using ESP-IDF ties into the Scalable and Maintainable Pillars of Production. It’s scalable because it allows you to use the entirety of the ESP32 feature set, most of which you simply can’t do with the Arduino IDE. You may not need all of those features today but when you do you’ll be glad they are immediately available. It also helps with maintenance because you are using the tool directly supported by the chip manufacturer. I can’t tell you how many issues I’ve seen people have with ESP32 projects because the Arduino IDE or PlatformIO hadn’t integrated upstream changes yet.

Start simple and work your way up and you will find the IDF is not a big, scary black box. It’s a powerful tool that will help you deliver a production-ready application.

Join the community and get the weekly Production ESP32 newsletter. Concise, actionable content right in your inbox.