# IMAP is indeed complicated
Sunday, 19. July 2026


I am building a soon-to-be released, self-hosting friendly email server that handles IMAP and SMTP.
The past years, I worked on infrastructure that handles millions of emails per day. My focus was on SMTP and related protocols. IMAP was not on the menu, which changed with this project.

When I chose to build the new email server, I started with IMAP. In software project management, we tend to prioritize and start with the unknowns, and I know SMTP is a smoth(er) ride for me than IMAP.

For the wire protocol, I built on top of [emersion/go-imap](https://github.com/emersion/go-imap). I know some of the other emersion go packages well and the maintainer even kindly merged a few pull requests I opened for the ecosystem. The package mainly handles parsing of IMAP requests and responses, leaving the implementation of the actual data management to me.

## IMAP

IMAP (Internet Message Access Protocol) is a protocol to give users access to messages (emails), which are organized into mailboxes (folders). Messages can carry flags. Most prominent is the `\Seen` flag. It does not matter how many IMAP clients you use simultaneously; If you viewed an email in one of the clients, you expect it to be marked "viewed" in all of your clients.

IMAP got overhauled a few times: I implemented [RFC 9051](https://www.ietf.org/rfc/rfc9051.html), which is _IMAP Version 4rev2_. To my knowledge, the first IMAP standard was [IMAP Version 2 in RFC 1064](https://www.rfc-editor.org/info/rfc1064/), published in 1988! IMAP "Version 1" was not publicly standardized per RFC.

## Long tails

Let's start simple with `LIST` mailboxes. `LIST` can request the `STATUS` of a mailbox. So we need to implement `STATUS` as well. `STATUS` returns the number of messages and requires to calculate the number of flagged `\Seen` messages and those that are flagged `\Deleted`. A simple list operation has a long tail.

My point here is that the protocol is quite circular. Messages affect mailboxes, flags affect messages. Flags and messages affect the information being returned for a mailbox.

A mailbox can be renamed the same that folders on your computer can be renamed. The always present INBOX folder behaves differently:
> Renaming INBOX is permitted and does not result in a tagged BAD response, and it has special behavior: It moves all messages in INBOX to a new mailbox with the given name, leaving INBOX empty.

Again, a command that seems simple to implement has a long tail. What would be a simple rename becomes creating a new mailbox and moving messages there.

## Data management challenges

Most mailboxes are `SUBSCRIBE`d. If a mailbox is not subscribed, the IMAP client will not list it. In Thunderbird, you can right-click on your Root mailbox and click "Subscribe..." to list potentially unsubscribed mailboxes. Once subscribed, they will show up in your sidebar. Subscriptions are based on the mailbox name. They do not rely on IDs.

IMAP allows subscribing to non-existing mailboxes. The reasoning behind it is that the server might create and remove mailboxes that are used for special purposes. When dealing with such a special mailbox that temporarily does not exist, it stays subscribed just in case it will exist in the future and should be listed in the IMAP client.

SQL database wisdom tells us to normalize data. That means a "subscriptions" table would point to a "mailboxes" table, with names only stored in the "mailboxes". But what if the mailbox does not exist? There are a few simple ways to solve this in SQL table layouts. Nonetheless, this is a surprising design choice in IMAP and I solved it by duplicating the mailbox name across both tables.

## Sequence numbers

Probably the biggest talking point for the complexity of IMAP implementations are sequence numbers. Let's start simple with message UIDs. That's a concept most (backend) developers grasp quite easily by thinking about UUIDs. In contrast to UUIDs, UIDs are strictly ascending but do not have to be contiguous. For simplicity, I skip `UIDVALIDITY` here, which is a compound ID for UIDs.

Besides UIDs, messages have sequence numbers. All messages in a mailbox are sorted by ascending UID. Then, a sequence number is assigned from 1...n. Sequence numbers are strictly contiguous. Whenever you remove a message from the sequence, the numbers will be immediately reassigned to form a contiguous 1...n sequence again. (For example, in the sequence `a:1,b:2,c:3`, removing message `b` from the list will result in a sequence of `a:1,c:2`)

While the server has a consistent view on the database and can correlate ascending UIDs to sequence numbers, IMAP is designed with parallel client sessions in mind. When a client moves or deletes a message, the server notifies all other clients about the change. Until this update has reached all clients, the server needs to remember each client's view, and interpret the sequence numbers accordingly. Therefore, each client command containing a sequence number has to be translated to the server view, and updates have to be translated back to all client session views. Luckily, `emersion/go-imap` handles translations of sequence numbers.

## JMAP

The future of mailbox management might be [JMAP Mail (The JSON Meta Application Protocol (JMAP) for Mail)](https://jmap.io), a new  protocol that tries to simplify the wire protocol relative to IMAP. The email server I am writing is designed to add JMAP one day if there's demand for it and a reasonable number of email clients support it.

If you are having fun learning about email, or you would like to self-host email, but find the gnarly parts intimidating, check out my service [p25.dev](https://p25.dev). I think it brings back the joy of self-hosting email.
Soon I will release the IMAP/SMTP p25.dev companion including the implementation of those interesting IMAP cases I explained above. Here is my initial announcement in [Modern email is CRM](https://raphting.dev/posts/mail-as-crm/).

## Tangent

This article as well as the new email server is written "by hand".
Writing an article of this kind would not be possible if the software that inspired it was generated by "agentic coding". For my own work, I concluded that the benefits of automated code generation do not outweigh the harm it causes on all layers of the field of software development. Leaving a quote from Edsger Dijkstra's 1972 Turing Award Lecture:

> The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.


By Raphael Sprenger