Python Can't Weakref a ValueError. Sentry's Deduplication Found Out the Expensive Way: 1024 KB per Live Asyncio Task.
sentry-python's DedupeIntegration stops the same error being reported twice by remembering the last exception. Remembering an exception is exactly what you must not do: an exception holds its traceback, a traceback holds its frames, a frame holds every local. The SDK knew this and stored a weakref.ref(exc). The three lines under the comment `# we can only weakref non builtin types` do the one thing the weakref existed to prevent: an `except TypeError:` catches Python's refusal to weakref a builtin and quietly holds the exception itself. Under asyncio each task keeps its own copy in a ContextVar, so 200 long-lived sessions pin 205 MB, all reachable after gc.collect(). Two attempts before the fix, a value fingerprint that fails the SDK's own tests, an id-based fingerprint that collapses 2000 distinct errors into 2 keys, then the identity-token approach that keeps behaviour unchanged and takes retained down to 0.5 MB.
DEV's own writeup naming the winners:
Why asyncio makes it hurt
`_last_seen` is a `ContextVar`. Under asyncio, every task gets its own copy of context. So this is not one retained exception process-wide. It is one retained exception per live task, each one dragging along its traceback and every frame local reachable from it. The reporter of issue #6094 was seeing this in an asyncio web crawler where the frames held fetched response bodies between 500 KB and 1 MB.
My first instinct was to call it an unbounded leak. I built a worker pool, ran it, and it stayed flat. That was worth finding out before I wrote it down anywhere. A fixed pool does not grow, because each worker's next error overwrites its previous one. Retention is bounded at (live tasks × payload). The growth story is not errors over time, it is tasks over time: one task per session, per subscription, per connection. So I measured against live task count instead.
The fix I got wrong first
The maintainers had already rejected the obvious fix. The reporter proposed skipping dedupe for builtins, and Sentry declined, saying they wanted 'a more robust fingerprinting approach that can also be used for built-in exceptions' instead. So I proposed a fingerprint: exception type, message, and the origin frame from the traceback. All immutable primitives, nothing retained. I posted it on the issue. Then the existing test suite told me I was wrong, in two different ways.
The attempt that came before mine
Late on, I went looking through the fork's branch list and found two abandoned maintainer branches from September 2025: `antonpirker/dedupe-integration-memory-usage` and `antonpirker/make-dedupe-integration-more-memory-efficient`. Neither was merged. The first one fingerprints on `(type_module, type_name, id(exc_value))`. That `id()` is the interesting part. Once you stop retaining the exception, which is the entire point of the change, its address becomes immediately reusable, and CPython reuses addresses aggressively.
The fix that shipped
Keep identity. Stop holding the object to express it. Attach a small weak-referenceable `_DedupeToken` to the exception (one per exception, alive precisely as long as the exception is), and weakly reference the token instead of the exception. `last_seen is token` is equivalent to the previous `last_seen is exc`, so deduplication behaviour is unchanged, which matters given the issue thread's explicit request not to change behaviour ahead of the fingerprinting rework. Attaching a private `_sentry_*` attribute to a user object to carry a weakref follows existing precedent in this SDK: `sentry_sdk/integrations/django/__init__.py` sets `_sentry_drf_request_backref = weakref.ref(...)` on the request.
Sentry's own Seer on it
Then I ran Sentry's own Seer on it. It found the root cause exactly, including that ContextVars are per-task under asyncio, which took me a debugging round to work out myself. It read the SDK source to do it. Then it proposed storing the type and the id of the exception. The same trap. Its fix would drop every distinct error as a duplicate.
One more twist worth naming, because it is the part that convinced me PRs from Seer need the same review any human PR gets. Seer's prose plan proposed the type + id fingerprint (which collapses distinct errors as shown above). Its actual generated patch did something different, and worse: it wrapped the failing `weakref.ref(exc)` in `except TypeError: pass`, silently dropping every builtin exception from dedupe entirely. In the identity-tuple test I ran the two variants against, that same-address collision pattern would have caused 1999 out of 2000 (100.0%) of the distinct errors in a session to be wrongly deduplicated. The prose it shows you for review is not necessarily the diff it writes.
The tests I added
Five new tests in `tests/integrations/dedupe/test_dedupe.py`, each written to catch a specific regression the two abandoned attempts would have introduced. `test_dedupe_does_not_retain_builtin_exception` weakrefs the token and asserts the exception itself is gc-collectable after the frame returns. `test_dedupe_still_dedupes_builtin_exception` asserts the fix does not break the case that motivates the integration in the first place, the same exception object caught and re-reported must still deduplicate. `test_dedupe_distinguishes_equal_builtin_exceptions` is the value-fingerprint regression: two `ValueError('same')` instances raised from the same line must not deduplicate as each other. `test_dedupe_leaves_unraised_exception_untouched` asserts the token attachment does not mutate exceptions the integration never sees. `test_dedupe_survives_exotic_exception_dict` covers the one Python quirk that could break the token approach: an exception subclass with `__dict__ = None`. In that case `_identity_token` returns None and the integration silently declines to dedupe rather than crash, a fail-safe path that Gemini's review flagged and I wrote a test for.
Gemini's second look on my patch
Before opening the PR I ran the diff through Gemini in adversarial-review mode: 'find every way this breaks.' It suggested six failure modes. Two were real crashes I had not thought about (the `__dict__ = None` case above, and one where the token attachment could race in a threaded context, I resolved that by making the attachment idempotent). One was refuted by reading the SDK's own precedent for `_sentry_*` private attributes on user objects. The remaining three (PII scrubbers stripping the token, exceptions that override `__setattr__`, garbage-collected weakref callbacks firing during shutdown) were all cases where the integration falls back to not-deduping, which is fail-safe: worst case is a duplicate event, not a crash, not a memory leak, not a wrong drop. I added tests for the two real ones and documented the fail-safe path for the other three. That is the pattern I want with AI review: it surfaces cases, I decide which are real, I write the test.
sentry-python DedupeIntegration Retained 1024 KB per Live Asyncio Task Because Python Cannot Weakref a ValueError. Root-Caused, Fixed with a Weak-Referenceable Identity Token, Repro'd, Measured, and Prepared as an Upstream PR to Fix Issue #6094.
View the project
Nightshift reads 2,000 patents in 4 minutes for $34 and finds the reference the examiner missed
Every existing prior-art tool is a retrieval system: rank a corpus, show a human the top few dozen. That has a measurable ceiling: on the strongest embedding available, a top-50 shortlist still misses 59.7% of the references a USPTO examiner actually applied. Nightshift is a judgment system: a vector pass narrows 171,695 patents, then Gemini reads two thousand of them, not fifty, deciding for each whether it discloses each limitation. Blinded against real USPTO office-action citations: 97.5% recall of examiner-applied anticipation refs (n=40), 92.5% on obviousness (n=40), 18.8% on a control set never cited. On the demo run, 4 minutes across 10 Cloud Run tasks, $34.57, it independently surfaced the examiner's own X-cite at depth 218, plus a 1998 reference the examiner missed that teaches six of seven limitations outright.
Still Here: Everyone Believes Black Dogs Get Left Behind. I Checked 99,916 Shelter Records. They Don't.
510 dogs have an intake record at the Austin animal shelter and no outcome record. Still Here is the board, ordered by how long they have waited. The one at the top is Pancho, 449 days. Then I went looking for the reason the dogs at the top are the ones at the top, and the answer surprised me: black dog syndrome does not appear in 99,916 completed stays, the colors that wait are the coats bully-type dogs come in. Hold breed constant and coat moves the median by at most 7.5 days; change the breed and it moves by 20.