Skip to content

Fixed thread local cleanup detected by IDEA. #9856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
+1 −1
Open

Fixed thread local cleanup detected by IDEA. #9856

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...trap/src/main/java/datadog/trace/bootstrap/instrumentation/java/concurrent/TPEHelper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ public static AgentScope getAndClearThreadLocalScope(Runnable task) {
return null;
}
AgentScope scope = threadLocalScope.get();
threadLocalScope.set(null);
threadLocalScope.remove();
Copy link
Contributor

@mcculls mcculls Nov 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC the reason for calling set(null) here rather than remove was for performance reasons.

If we assume that the thread will activate another scope later on, it is better to leave the entry in the thread-local's map rather than continually grow and shrink the underlying map. TBH it would be even better to use a holder pattern here, and avoid any mutating of the thread-local once setup.

So please don't merge this for now - I'll put together a quick benchmark to demonstrate the difference.

Copy link
Contributor Author

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mcculls! Your explanation is totally make sense. If benchmark will show that set(null) is better I will change my PR to put a comment why we have set(null).

Copy link
Contributor

@mcculls mcculls Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran a simple micro-benchmark that repeatedly calls the current TPEHelper.setThreadLocalScope and getAndClearThreadLocalScope methods. I then added alternatives that use remove() instead of set(null) and another implementation that uses the holder pattern (where you leave a one-element array in the ThreadLocal all the time and set and clear the element inside that array.)

Throughput results, higher is better:

Benchmark                                    Mode  Cnt   Score   Error   Units
TPEHelperBenchmark.threadLocalScopeHolder   thrpt    5  86.557 ± 1.260  ops/us
TPEHelperBenchmark.threadLocalScopeSetNull  thrpt    5  78.146 ± 0.342  ops/us
TPEHelperBenchmark.threadLocalScopeRemove   thrpt    5  58.394 ± 1.325  ops/us

This shows a clear drop in performance when using ThreadLocal.remove() vs ThreadLocal.set(null).

This is expected because remove() will mutate the underlying table in the thread, whereas set(null) just replaces the value stored in the existing entry.

Using the single-element array holder pattern is slightly better, because you can avoid the round-trip to the ThreadLocal when querying and updating in the same method (you just fetch the holder and use that to both query and update) - but it's marginal.

Note: one reason to use remove() is to avoid holding onto class-loaders that would otherwise be unloaded, but that doesn't apply here because the value managed by TPEHelper is the scope which is loaded by our class-loader which stays alive throughout the application's life.

The other reason is to save memory - this would mainly apply if a thread would never again have an active scope, but typically if a thread was involved in tracing once it would be involved again. Virtual threads might change this assumption - but the overhead of leaving a small null entry in place should be low vs. the performance cost of continually adding and removing the thread-local (as shown above.)

For now my recommendation is to leave this set(null) in place - with a comment to suppress the IDE message.

But I'd also recommend running an example app with a large virtual thread pool , where the work done by the pool is traced via @Trace, to calculate the exact overhead of leaving this small thread-local entry in place.

Copy link
Contributor Author

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow! Impressive explanation, thanks @mcculls!

return scope;
}

Expand Down
Loading