Python Can't Weakref a ValueError. Sentry's Deduplication Found Out the Expensive Way: 1024 KB per Live Asyncio Task.

Why asyncio makes it hurt

`_last_seen` is a `ContextVar`. Under asyncio, every task gets its own copy of context. So this is not one retained exception process-wide. It is one retained exception per live task, each one dragging along its traceback and every frame local reachable from it. The reporter of issue #6094 was seeing this in an asyncio web crawler where the frames held fetched response bodies between 500 KB and 1 MB.

My first instinct was to call it an unbounded leak. I built a worker pool, ran it, and it stayed flat. That was worth finding out before I wrote it down anywhere. A fixed pool does not grow, because each worker's next error overwrites its previous one. Retention is bounded at (live tasks × payload). The growth story is not errors over time, it is tasks over time: one task per session, per subscription, per connection. So I measured against live task count instead.

The fix I got wrong first

The maintainers had already rejected the obvious fix. The reporter proposed skipping dedupe for builtins, and Sentry declined, saying they wanted 'a more robust fingerprinting approach that can also be used for built-in exceptions' instead. So I proposed a fingerprint: exception type, message, and the origin frame from the traceback. All immutable primitives, nothing retained. I posted it on the issue. Then the existing test suite told me I was wrong, in two different ways.

The attempt that came before mine

Late on, I went looking through the fork's branch list and found two abandoned maintainer branches from September 2025: `antonpirker/dedupe-integration-memory-usage` and `antonpirker/make-dedupe-integration-more-memory-efficient`. Neither was merged. The first one fingerprints on `(type_module, type_name, id(exc_value))`. That `id()` is the interesting part. Once you stop retaining the exception, which is the entire point of the change, its address becomes immediately reusable, and CPython reuses addresses aggressively.

The fix that shipped

Keep identity. Stop holding the object to express it. Attach a small weak-referenceable `_DedupeToken` to the exception (one per exception, alive precisely as long as the exception is), and weakly reference the token instead of the exception. `last_seen is token` is equivalent to the previous `last_seen is exc`, so deduplication behaviour is unchanged — which matters given the issue thread's explicit request not to change behaviour ahead of the fingerprinting rework. Attaching a private `_sentry_*` attribute to a user object to carry a weakref follows existing precedent in this SDK: `sentry_sdk/integrations/django/__init__.py` sets `_sentry_drf_request_backref = weakref.ref(...)` on the request.

Sentry's own Seer on it

Then I ran Sentry's own Seer on it. It found the root cause exactly, including that ContextVars are per-task under asyncio, which took me a debugging round to work out myself. It read the SDK source to do it. Then it proposed storing the type and the id of the exception. The same trap. Its fix would drop every distinct error as a duplicate.

One more twist worth naming, because it is the part that convinced me PRs from Seer need the same review any human PR gets. Seer's prose plan proposed the type + id fingerprint (which collapses distinct errors as shown above). Its actual generated patch did something different, and worse: it wrapped the failing `weakref.ref(exc)` in `except TypeError: pass`, silently dropping every builtin exception from dedupe entirely. In the identity-tuple test I ran the two variants against, that same-address collision pattern would have caused 1999 out of 2000 (100.0%) of the distinct errors in a session to be wrongly deduplicated. The prose it shows you for review is not necessarily the diff it writes.

The tests I added

Five new tests in `tests/integrations/dedupe/test_dedupe.py`, each written to catch a specific regression the two abandoned attempts would have introduced. `test_dedupe_does_not_retain_builtin_exception` weakrefs the token and asserts the exception itself is gc-collectable after the frame returns. `test_dedupe_still_dedupes_builtin_exception` asserts the fix does not break the case that motivates the integration in the first place — the same exception object caught and re-reported must still deduplicate. `test_dedupe_distinguishes_equal_builtin_exceptions` is the value-fingerprint regression: two `ValueError('same')` instances raised from the same line must not deduplicate as each other. `test_dedupe_leaves_unraised_exception_untouched` asserts the token attachment does not mutate exceptions the integration never sees. `test_dedupe_survives_exotic_exception_dict` covers the one Python quirk that could break the token approach: an exception subclass with `__dict__ = None`. In that case `_identity_token` returns None and the integration silently declines to dedupe rather than crash — a fail-safe path that Gemini's review flagged and I wrote a test for.

Gemini's second look on my patch

Before opening the PR I ran the diff through Gemini in adversarial-review mode: 'find every way this breaks.' It suggested six failure modes. Two were real crashes I had not thought about (the `__dict__ = None` case above, and one where the token attachment could race in a threaded context — I resolved that by making the attachment idempotent). One was refuted by reading the SDK's own precedent for `_sentry_*` private attributes on user objects. The remaining three (PII scrubbers stripping the token, exceptions that override `__setattr__`, garbage-collected weakref callbacks firing during shutdown) were all cases where the integration falls back to not-deduping, which is fail-safe: worst case is a duplicate event, not a crash, not a memory leak, not a wrong drop. I added tests for the two real ones and documented the fail-safe path for the other three. That is the pattern I want with AI review: it surfaces cases, I decide which are real, I write the test.