Skip to content

Add OpenTelemetry v2 plugin - #3089

Open
patbeqo wants to merge 1 commit into
patbeqo/otel-v2-workflow-prerequisites-2from
patbeqo/otel-v2
Open

patbeqo wants to merge 1 commit into
patbeqo/otel-v2-workflow-prerequisites-2from
patbeqo/otel-v2

Conversation

@patbeqo

@patbeqo patbeqo commented Sep 17, 2026 •

Copy link
Copy Markdown

What changed?

  • Added the experimental temporal-opentelemetry-v2 contrib module and registered it in settings.gradle and the BOM.
  • OpenTelemetryPlugin installs tracing interceptors on the workflow, schedule, activity, and Nexus clients plus the worker through the SimplePlugin builder helpers. Its builder exposes setHeaderKey (defaults to _tracer-data) and setAddTemporalSpans (defaults to false, so only context propagation happens unless enabled).
  • ReplaySafeOpenTelemetry is the OpenTelemetry to register with GlobalOpenTelemetry.set. Its tracer provider wraps spans so they are no-ops during replay and in read-only workflow contexts, and its id generator draws span and trace ids from named workflow random streams so ids are stable across replay. Propagators default to W3C trace context plus baggage and are configurable via setPropagators. OpenTelemetryPlugin.Builder.build() rejects a global that is not a ReplaySafeOpenTelemetry.
  • InterceptorTracer serializes the current context into Temporal headers with the global propagator and, when enabled, creates spans for Temporal operations (StartWorkflow, RunWorkflow, RunActivity, HandleSignal, HandleUpdate, RunStartNexusOperationHandler, and so on). It rethrows the Continue-As-New unwind without recording an error.

Why?

Part of #3046. This is the third slice, stacked on #3049 and #3082, which supplied the named random streams, read-only detection, and client plugin hooks the module depends on.

Breaking changes?

None. The module is new and marked experimental.

Server PR

None.

Test plan

Added tests covering all client, worker, and workflow paths the plugin instruments, with and without Temporal spans enabled.

  • mise exec -- ./gradlew :temporal-opentelemetry-v2:test -PtestServer=dev-server -x spotlessCheck -x spotlessJava (13 tests; the Reset and comprehensive tests require the dev server and are gated with assumeTrue)
  • mise exec -- ./gradlew :temporal-opentelemetry-v2:spotlessCheck
  • mise exec -- ./gradlew :temporal-opentelemetry-v2:javadoc

@patbeqo
patbeqo added this pull request to stack #3083 September 17, 2026 21:16
@patbeqo

patbeqo commented Sep 17, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T13:23:01.549334Z 97ec09f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eef033e5ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread contrib/temporal-opentelemetry-v2/build.gradle
@patbeqo

patbeqo commented Sep 18, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7d52f9ab1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@patbeqo

patbeqo commented Sep 18, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f239f732c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread contrib/temporal-opentelemetry-v2/build.gradle
@patbeqo

patbeqo commented Sep 18, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 14423f6442

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@patbeqo

patbeqo commented Sep 18, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 73678eb01d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@patbeqo

patbeqo commented Sep 18, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 632635bb43

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@patbeqo
patbeqo force-pushed the patbeqo/otel-v2 branch 2 times, most recently from c5b05aa to de76218 Compare September 18, 2026 19:37
@patbeqo

patbeqo commented Sep 18, 2026

Copy link
Copy Markdown
Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d3aa415401

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@patbeqo
patbeqo force-pushed the patbeqo/otel-v2 branch 2 times, most recently from b906ec6 to 9d77af7 Compare September 18, 2026 21:10
@patbeqo
patbeqo force-pushed the patbeqo/otel-v2 branch 2 times, most recently from ac9ffae to d4a61fb Compare September 18, 2026 21:43
@patbeqo
patbeqo marked this pull request as ready for review September 18, 2026 22:51
@patbeqo
patbeqo requested a review from a team as a code owner September 18, 2026 22:51
@patbeqo
patbeqo requested a review from a team September 18, 2026 22:51

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4a61fb775

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@DABH DABH left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultrareviewed the exact stacked delta and the completed #3090/#3091 follow-ups. I found one new interoperability issue inline and revalidated two previously resolved concerns: replay-only spans do leave arbitrary span processors with unmatched onStart calls, and the 48-bit ID-stream state remains a real collision limit (the foundational fix is noted on #3049). I am reopening those existing threads with concrete evidence rather than duplicating them.

The other prior comments are correctly fixed or handled later in the stack: child-thread propagation and factory-only installation are implemented in #3082, metrics/log suppression arrive in #3090/#3091, the README arrives in #3091, and the converter, schedule, attributes, Guava, explicit timestamp, and header-reuse fixes are present.

CI has one failed Edge job on the current SHA, but an identical duplicate job passed. The sole failure is the known MetricsTest.testStickyCacheSize race, reproduced on main and already tracked by #3069; it is unrelated to this module. Focused OpenTelemetry tests and isolated replay probes passed locally, and the worktree is clean.

@DABH DABH left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultrareviewed the exact stacked delta a second time, with an independent trace and scratch reproductions, and revalidated the open threads. One new P1 is inline: sticky-cache eviction ends and exports blocked inbound spans as complete, and the replayed run then re-exports the same deterministic trace and span IDs with different timings. Two new P2s are inline: OtelTestBase does not actually disable the sticky cache, so the module's tests never exercise a cache-miss replay, and span events and recorded exceptions during replay carry replay-time wall-clock timestamps.

The three open threads remain actionable as previously described: replay-only spans still reach SpanProcessor.onStart on every replay without a matching onEnd, the 48-bit stream state still bounds trace-ID uniqueness (fix on #3049), and the process-global ContextStorageProvider still loses to any second provider. On the Codex documentation thread I have recommended closing it in favor of the README in #3091, with a list of the gaps to fill there. Every previously resolved Codex item is genuinely fixed in code; the metrics suppression resolved via #3090 covers synchronous instruments only, and the observable-instrument gap is tracked on the open #3090 thread.

Smaller items, not blocking: ReplaySafeSpan does not override Span.addLink(...), so links added after start are silently dropped by the interface default; OpenTelemetryPlugin.Builder.build() checks the global through GlobalOpenTelemetry.getTracerProvider(), which initializes the global to no-op when unset, so the GlobalOpenTelemetry.set remedy the error message suggests then throws "already been called"; OpenTelemetryScheduleClientCallsInterceptor clears and writes the caller's live Header on ScheduleActionStartWorkflow rather than a copy; an undecodable _tracer-data header fails every workflow task permanently because SpanCodec.decode throws before any handler runs (parity with the OpenTracing module, but a fallback to the current context would be friendlier); the public interceptor classes are non-final, not @Experimental, and take the internal.InterceptorTracer type in their constructors; and temporalRunID is emitted as an empty string when a stub is unbound. CI has one failed Edge job on this SHA, again the known MetricsTest.testStickyCacheSize race tracked by #3069; the module's own tests pass in every job.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3fc75737fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d068dd0c61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b55061dd6b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9168629ff2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c1a4b3a85e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67f2601771

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3e7f66235b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread contrib/temporal-opentelemetry-v2/build.gradle

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants