ProdESP32 #5: Time Is Hard

Time is hard. Super hard. The concept seems simple enough until you realize there are endless edge cases and exceptions to rules you thought you knew.

This is especially true in IoT products where not only are you dealing with time across physical devices but also between devices and the cloud. Throw in a global customer base across all timezones and I promise, at some point, the “time problem” will break your brain.

The biggest problems I’ve experienced with time over the years has been not knowing what I didn’t know. Just when you think you have a grasp on all of the time edge cases you learn about leap seconds or clock drift.

This post is an attempt to give a very brief overview of some of the time handling concepts so you can better navigate time in your project. It’s not meant to be a comprehensive guide to time but is meant to give you just enough info to know what questions you might need to ask or what cases you may need to consider in your product.

Coordinated Universal Time (UTC)

UTC is your new best friend when it comes to time. It is the standard time used across the world and is “coordinated” as the world’s timing centers have agreed to keep this value closely synchronized.

This is the standard value of time you should try to use everywhere in your code. Many of the considerations listed below such as Daylight Savings Time, Timezones and others are additional layers of complexity added to UTC to help accommodate human understanding of time and local time policies.

You should store, compare, and transmit times in UTC. Make it the standard across your system.

Timezones

There are currently over 3 dozen timezones worldwide drawn in sometimes seemingly random ways. Your users don’t care about, and probably don’t even know about, UTC. They want to see and interact with dates and times local to them. Some things to consider:

• Do I need to support timezones?
• What happens if a user or device changes timezones?
• If you have scheduling, how will you handle timezones?
• How will you collect timezone data from your user?

To make matters worse, timezones are not static. They can change due to geopolitical reasons or government decisions. For instance, a country might decide to shift its timezone offset or change its DST rules. As a result, keeping your timezone database up-to-date is crucial to ensure your applications reflect the latest timezone information accurately. This is often achieved by using a third-party source of timezone data.

Daylight Saving Time (DST)

Daylight Saving Time is a practice where clocks are set forward by one hour during the summer months to make better use of daylight. This seemingly simple concept can wreak havoc in your application if not handled correctly. The rules for when DST starts and ends can vary by region and change over time, making it a moving target.

I recently saw a project where a scheduled, repeating event was an hour off after the DST switch because the application didn’t handle that adjustment properly.

Clock Drift

Even the most accurate clocks can drift over time due to various factors like temperature, humidity, or the precision of hardware. This means that an application that relies on system time can gradually become out of sync with the real world. You must consider strategies for synchronizing time or periodically recalibrating system time.

This gets especially tricky on devices like the ESP32 where clock drift can be highly volatile depending on your hardware and clock source. Deep sleep timing on an ESP32 using the built-in clock can drift by several minutes over a single 24 hour period. If timing is crucial in your application you must address this.

Leap Year and Leap Second

Leap years, where an extra day (February 29) is added to the calendar, and leap seconds, where a second is added to or subtracted from Coordinated Universal Time (UTC), are essential for keeping our timekeeping systems in sync with the Earth’s rotation. However, these events can pose challenges when calculating precise time intervals or performing date arithmetic and comparisons. Unless you require extreme accuracy it’s unlikely you will need to worry about leap seconds but it’s something to know about.

Time Synchronization

In most IoT products that involve time you will need to deal with synchronizing time between hardware devices and a hosted backend. Backend servers have much higher accuracy clocks (although not perfect) and easy access to synchronization methods. Edge devices may not have internet access and unstable, high-drift clocks. How do you ensure every part of your system agrees on time and stays in sync? How do you keep edge devices in sync with each other?

Transmitting Time

When transmitting time-related data between systems or storing it in databases, you need to consider how to serialize and deserialize timestamps properly. Different serialization formats (e.g., ISO 8601, Unix timestamps, or custom formats) can introduce compatibility and precision issues. Pick a standard format and stick with it.

I saw a system once that stored time in all of the following ways:

• Seconds since epoch
• Milliseconds since epoch
• String as date and time in UTC
• String as date and time in local timezone

You can imagine how difficult time operations became in this system. Everytime a date needed to be displayed or compared to another date you had to figure out how to modify them to be in the same format. The result was a stream of constant time-related bugs.

This also needs to be considered at the device firmware level. How do you get your time over a UART or other comm channel?

Timestamp Ambiguity

Timestamps are often used to record events, and in some cases, multiple events can occur simultaneously or within extremely tight windows. In such cases, it can be challenging to determine the order of events accurately, leading to potential data consistency issues.

For example, suppose you have an ESP32 that sends events to a main hub which then sends an event to your cloud backend. Where do you capture the timestamp of the event? The ESP32, by default, has a high-drift clock and maybe doesn’t have access to a WiFi connection to get time synchronization updates. Do you record a timestamp on the hub? On the cloud? If you do it on the cloud how do you account for the latency from when the event left the device and traveled over the network to get to the cloud?

User Interface Challenges

Displaying time to users in a human-readable and intuitive way can be complex, especially when dealing with multiple timezones or different date and time formats. Localizing time displays for international audiences adds another layer of complexity.

I previously recommended using UTC everywhere possible but if your device has to display date or time info you will need to transform it into the format the local user expects. Is the shorthand for November 17th, 2023 “11-17-23”(America) or “17-11-23” (basically everywhere else in the world 😂)? Does the firmware have to handle that or will you do it server side?

I worked on a project once where an ESP32 had to set the time on a BLE device using the Current Time Service. This had to be done in local time. Otherwise, historical data on the device would show UTC times which, to the end user, would be off by several hours due to timezone considerations.

Historical Time Data

All of the above complications combine into a violent, crushing perfect storm when you have to manipulate historical time data. If you have to do date and time calculations into the past how do you incorporate the fact that timezone borders have been altered and during that period the local legislature changed the rules for Daylight Savings Time?

This can get extremely complex especially as you go back further in time.

Compliance and Legal Requirements

Some industries, such as healthcare and finance, have strict regulatory requirements regarding the accuracy and security of timestamped data. Ensuring compliance with these regulations adds another layer of complexity to time handling.

Edge Cases

In many of the above-mentioned scenarios it’s not enough to simply know the concept but to understand what happens at the edges and handling it properly. What happens when an event falls right on the turn of a day? Or exactly at the moment Daylight Savings Ends? If you are doing simple comparisons what happens if an event starts one second before the end of the year and ends one second after the new year begins? If your device has to perform a task for a specified period of time, say 2 hours, and the switch to Daylight Savings happens in the middle, how do you properly account for that? Along the same lines, do you timestamp events when they begin, finish, or both? How do you compare a timestamp in a timezone that used to have DST but now doesn’t? There are a hundred other edge cases you may have to consider in your application to handle time properly.

Production Pointers

• Always store times in UTC. In almost all scenarios you should be using and transmitting times in UTC. This “normalizes” time data. Timezones should only be introduced in the UI or other places as needed and only when absolutely necessary. However, when it comes to storing time, always use UTC. You can choose the format that best suits your needs (seconds since epoch,string,etc) but the data itself should reflect a UTC date/time.
• Offload date/time handling. When possible, offload the handling of time to the cloud instead of processing it locally in firmware. This will reduce binary sizes and also reduces the number of locations where time bugs can occur. When bugs do occur, they are much easier to fix in a backend deploy versus a firmware update.
• Simplify. While there are many considerations when dealing with time it’s unlikely you will need to account for them all. If 5 minutes of clock drift over 24 hours is fine then don’t worry about figuring out how to keep your device clock perfectly synced. Only solve those time problems that actually affect your product.

Conclusion

Navigating time ties into the Deterministic Pillar of Production. It is a complex challenge that goes beyond basic timezone and daylight saving time considerations. You need to be aware of any additional concerns affecting your project and address them appropriately.

The good news, though, is that in many scenarios you will only have to deal with a small segment of time-keeping considerations. Simplify wherever you can and offload time-related operations to the systems that can handle them with the least amount of effort and design.

What other time-related concepts did I miss or that you’ve personally grappled with? Please share in the comments below.

Join the community and get the weekly Production ESP32 newsletter. Concise, actionable content right in your inbox.