Refactor OTel

[danschultzer]

Branched of #784

Hi @solnic,

Here's the draft with a few suggestions from using the OTel tracing with Sentry, namely:

• Make root span be the transaction itself (as per https://develop.sentry.dev/sdk/data-model/event-payloads/transaction/#anatomy)
• Include environment in transaction
• Rely on semantic conventions
• Handle all otel spans generic with minimal normalization
• Ensure all transactions and spans have names/descriptions (from otel)

From testing this I wanted the traces to be closer to the actual OTel traces rather than normalized to Sentry events. This also seems to be in line with how the other Sentry SDK's consumes OTel spans.

The big piece here is how I switched over to let OTel doing most of the work with minimal adjustments after. It doesn't tie itself to a specific instrumentation library, instead I just look at the semantic conventions attributes to see if we can infer additional context, e.g. is it a HTTP or DB type span. It defaults to just returning the name as op if we don't have any other info.

So if we want to set the source for oban tasks to task we can figure it out from the attributes by picking up on messaging.system and maybe oban.job.worker, same for view source were we may want to check if there's a phoenix.plug set. Right now I just set all sources to custom.

I've been pushing a few upstream fixes for otel erlang to make the attributes more rich which will help us here: https://github.com/open-telemetry/opentelemetry-erlang-contrib/pulls

Lmk what you think.

[danschultzer]

Also we can probably reduce a lot of the integration boilerplate, don't think we need the full fledged code gen app, especially if we go down this path.

[tsloughter]

I've been pushing a few upstream fixes

I'd say!

They just kept rolling in I eventually went searching for who you were and found this :). Thanks for the work!

[solnic]

Thanks @danschultzer - this is clearly a good direction and it fixes a bunch of things so I've merged this already and will follow up in my branch. Please let me know if I can help move things forward in the opentelemetry_* projects too. It would be great to make it a win-win situation :)