Case Study - Customer-held encryption keys on a shared search cluster
Enterprise buyers in regulated sectors wanted cryptographic ownership of their own data. Atlassian ran search on shared Elasticsearch clusters encrypted with platform-managed keys. A two-person team proved customer-held keys could work there without touching the services above.
- Client
- Atlassian
- Year
- Service
- Search infrastructure, security engineering
Overview
Enterprise buyers in regulated sectors had started asking Atlassian a question that is easy to state and hard to answer: can we hold our own encryption keys?
Search and analytics at Atlassian run on multi-tenant Elasticsearch clusters. Data was encrypted at rest, but with platform-managed keys — which is the ordinary, sensible design, and also exactly the thing a compliance officer wants changed. Moving to customer-supplied keys is not a configuration change. On a shared cluster it reaches into the storage path, and the risk is that it degrades performance for every tenant or forces changes on every service reading from the cluster.
This was work Atlassian's own team wanted done and did not have the bandwidth to prioritise. It came to us as a question rather than a specification: is this viable at all, and at what cost in latency?
The constraint
The answer had to be a real one — running code on a cluster shaped like production, not a design document. Three things bounded it:
Nothing above the cluster could change. A solution that required every consuming service to be modified would not have been adopted, however elegant. The toggle had to live at cluster level.
There was a latency budget. Encryption on the write path costs something. The question was whether it cost little enough to be worth having, measured rather than estimated.
The compliance posture had to survive. Any design that improved key ownership while weakening the existing controls would have been a net loss.
What we built
- Elasticsearch internals
- Envelope encryption
- AWS KMS external key store
- Key rotation
- Benchmark harness
The design is envelope encryption on the disk-write path. Data keys encrypt the segments; the customer's root key wraps the data keys and never leaves an external key store in an account they control. Revoking access to that root key renders the data unreadable without Atlassian having to do anything — which is the property the compliance conversation is actually about.
A benchmark harness was part of the build rather than an afterthought, because "does this cost too much latency" was the question we had been asked and the only credible answer is a measured one.
Results
The proof of concept encrypted indices and snapshots under customer-owned keys, with the toggle at cluster level and no changes required in the services consuming the cluster. Key rotation ran automatically without service impact.
Against the latency budget we had been given, the measured cost was well inside it — a median write penalty in the tens of milliseconds and a read penalty smaller still.
We brought in Rohit and his team to help us prototype a 'Bring Your Own Key' solution for Elasticsearch — an initiative our internal team didn't have bandwidth to prioritize at the time. Despite the complexity and sensitivity of the task, they ramped up quickly, worked closely with our security and infra stakeholders, and delivered a fully functional proof of concept that met all our requirements. The two-person team operated with clarity, autonomy, and rigor, matching the standards we expect internally at Atlassian. It was plug-and-play execution with zero overhead on our end — exactly what we needed.
Two people did this. That is the part worth sitting with: the work was security-critical, it touched the storage layer of a multi-tenant system, and it was delivered by a pair operating against a team that had already decided it could not spare the capacity itself.