Cause
It's a race on shared state between threads. current_user is a class attribute, so every thread in a gunicorn worker process reads and writes the same slot. With --threads 4, two requests can interleave like this:
| Thread A (user Alice) | Thread B (user Bob) |
|---|---|
CartView.current_user = Alice |
|
_cart_cache.get(alice.id) → miss |
|
CartView.current_user = Bob |
|
load_cart(CartView.current_user) → loads Bob's cart |
|
_cart_cache[alice.id] = Bob's cart |
Alice is shown Bob's cart. Two details make it worse:
- The bad entry sticks. The wrong cart is stored in
_cart_cacheunder Alice's id. Her later requests to that worker hit the cache and keep showing Bob's cart until the process restarts. So one short race turns into a lasting leak for that user on that worker. - The key and the value come from different sources. The cache key uses the thread-local
request.user.id, but the value is loaded from the sharedCartView.current_user. Nothing checks that the two refer to the same user.
Why only production, and why rarely
-
Staging (
--threads 1): each worker process handles one request at a time, so nothing else can overwritecurrent_userbetween the write and the read. The bug can't happen there. Separate gunicorn workers are separate processes, so they don't share the class attribute. -
Production (
--threads 4): requests share the attribute. It only goes wrong when all three of these line up:- a cache miss (first visit to that worker, or after a restart)
- a second request writes
current_userin the short gap beforeload_cartreads it - the GIL switches threads at exactly that point
That combination fits "once in a few hundred requests."
How to confirm it before changing anything
These checks are cheap. I haven't run any of them; this diagnosis comes from reading the code.
- Log the mismatch. Temporarily log a warning in production or in a load test whenever
CartView.current_user.id != request.user.idjust beforeload_cart. Any hit proves this mechanism. - Reproduce in staging. Run staging with
--threads 4and send concurrent requests from many different users to the cart page. A fresh worker gives the most cache misses. Then check which user's items each response contains. Running the same load with--threads 1should show zero leaks. - Force it in a test. Make
load_cartsleep briefly or wait on athreading.Barrier, then callCartView().get()from two threads with different users. This reproduces it every time. - Check that it persists. Once a user has seen a wrong cart, their repeat requests to the same worker should keep showing that same wrong cart. Restarting the worker should clear it. If that's what happens, the cache is what turns the race into a lasting leak.
If check 1 or 2 fires, the cause is confirmed. Note that any fix will also need to clear the poisoned entries in _cart_cache, either by restarting the workers or by discarding the cache. Holding off on the fix as you asked.
Real run recorded with claude-code / claude-opus-5-5. Output is shown verbatim, unmodified.
できること
原因がはっきりしない不具合のための短い作業ルールです。証拠を集めて原因を特定してから、プロダクトのコードを変更します。
仕組み
- 観察された症状と、推測される原因を分けて考えます。
- 入力、状態の変化、責任の境界をたどり、仮説を証拠の強さと否定するコストの低さで並べます。
- 証拠をすべて説明できる仕組みが見つかるまでコードを変更せず、見つかったら原因と根拠を報告します。修正を頼まれていなければ直しません。
向いている場面
たまにしか起きないバグ、性能の低下、「ステージングでは動くのに本番で壊れる」問題。
知っておきたいこと
オープンソースプロジェクト Caveman に含まれる汎用的な作業パターンの一つです。指示文に Caveman のブランド色はなく、どのプロジェクトでも使えます。
指示だけのファイルです。スクリプトはなく、ネットワーク接続もファイルの書き込みも行いません。 パッケージには LICENSE、NOTICE、agents/openai.yaml(Codex 用の表示名と既定プロンプト)も含まれます。