Skip to content
MOITOITECH
EET--:--:--
SUN--:--

MoiToi.TECHArticles

Before a MySQL to TiDB cutover, let TiDB run your production queries first

· Andres Kepler · TiDB Engineering

Diagram of a MySQL to TiDB cutover test: ProxySQL serves the app from MySQL and mirrors reads to an idle, production-ready TiDB. Steps: capture and analyse queries (pt-query-digest, AI), replay with pt-upgrade, mirror real traffic, then a readiness gate. No live rollback; the way back is forward.

Rows copied, replication caught up, checksums green. None of that tells you whether TiDB can run your application's queries. I find out before the switch, not during it.

A MySQL to TiDB migration plan usually measures progress in data: rows copied, replication caught up, checksums matching. All of that can be green while the application is still one deploy away from an error nobody has seen.

Copying the data is a solved problem with well-documented tools. The data copy says nothing about the queries. TiDB speaks the MySQL protocol, but it is a different database underneath, and the question every migration has to answer is: what will break on compatibility, and what will be slower?

I do not answer that at the moment of the switch. I answer it before, with the application's real queries, and the switch only happens once TiDB has already run them.

Start from the queries production actually runs

The first step is evidence, not a checklist. Production queries are captured from the MySQL slow query log and grouped by fingerprint with pt-query-digest. That turns a large slow log into a list of query shapes, ranked by how much time they cost. It is the workload the migration has to carry, not the workload someone remembers.

The second step is static analysis. AI reviews the captured fingerprints and the schema against TiDB's documented differences from MySQL and flags the candidates. It is fast and reads every query. But it only produces suspects. Nothing it flags counts until it has been run against TiDB.

Prove it with Percona tools, then with real traffic

That run is done with pt-upgrade. It replays the captured queries against MySQL and TiDB side by side and compares results, errors, warnings and query time. Incompatibilities show up as differences in results or as errors. Slower queries show up as a time difference on the same statement. Both arrive as a list with the query attached, not as an incident after the switch.

Replay does not reproduce production concurrency. So the next step is mirroring. The TiDB cluster is already production-ready, but no application is connected to it. ProxySQL sends it a copy of production read traffic, so TiDB runs the real load while MySQL still answers the application. The mirrored results are discarded. What matters is what TiDB does with the load: errors and latency per query shape, measured on TiDB.

Writes are mirrored too, but never into the cluster that goes live: it already receives every write through replication, and a second copy would corrupt it. Reads and writes are mirrored to a separate, throwaway TiDB restored from a snapshot, where errors, latency and contention under real write load are measured. Mirroring cannot test transactions. A staging rehearsal with the real application covers them, and times the switch and the write freeze.

By the time traffic moves, every slow query and every incompatibility is either fixed, accepted with a reason, or a blocker. That list is one of the readiness gates.

Then the switch is the small part

With that done, the cutover itself is what it should be: a controlled step. Replication lag, data checks and readiness gates must pass before anything moves. Backends move between ProxySQL hostgroups as an explicit, recorded state change. Errors, latency and data are checked against the baseline taken before the switch. The transition is visible on dashboards while it happens.

There is no rollback in live. Once the application writes to TiDB, pointing it back at MySQL would lose every write made since the switch. The only way back is forward: MySQL to TiDB, then TiDB to MySQL, as a new migration through the same process. A quick flip back belongs to testing, where the switch is rehearsed. In production, the first write on TiDB is the point of no return. That is why everything before it is about evidence: if a gate fails, nothing moves.

The order matters. AI accelerates the analysis. The Percona tools and mirrored traffic are the test. I own the gates and the decision to switch. When a CTO asks whether production is safe on TiDB, the answer is not "we copied the data". It is "TiDB has already run our queries, here is what was different, and here is what we did about it."

The full description of how I approach cutover, from query checks to the switch, is at https://moitoi.tech/tidb#cutover.

The detail: moitoi.tech/tidb#cutover · Discuss it on LinkedIn


Next step

Working on this in production? TiDB Engineering.

Andres Kepler

Andres Kepler

Product engineer and infrastructure specialist

Discuss your TiDB problem