EDU-4084: Update observability.mdx

[joe-dakota]

Noting that workflow.UpsertTypedSearchAttributes creates NDE

What does this PR do?

This pull request includes a crucial update to the docs/develop/go/observability.mdx file, adding guidance on handling nondeterminism errors when upserting search attributes in existing workflows.

Notes to reviewers

• docs/develop/go/observability.mdx: Added a code example and explanation on using versioning to prevent failures caused by nondeterminism errors when upserting search attributes in existing workflows.

[CLAassistant]

All committers have signed the CLA.

[joe-dakota]

Ping @fairlydurable ... we hit this issue in production and would like to make sure others see this caveat before repeating the same mishap. 🙏

[fairlydurable]

Hi @joe-dakota! Unfortunately the personnel who can perform technical reviews this week are at Replay. Do you have access to any Temporal engineers who can vet this?

[fairlydurable]

Thank you for submitting this PR. This issue appears to apply to all SDKs, not just Golang.

My current wording:
Adding Search Attributes modifies your Workflow code.
Because of this, you need to use version markers to avoid non-determinism errors when upserting Search Attributes into existing Workflows.

[joe-dakota]

Thank you for submitting this PR. This issue appears to apply to all SDKs, not just Golang.

My current wording: Adding Search Attributes modifies your Workflow code. Because of this, you need to use version markers to avoid non-determinism errors when upserting Search Attributes into existing Workflows.

Alas, I don't have access to Temporal engineers.

I wasn't quite sure if it applied to golang, or all SDKs. Makes sense that it is applicable to all.

Thanks for the update!