mac-capture: Use RGBA64Half pixel format for screen capture

[jcm93]

Description

Changes ScreenCaptureKit content to use the RGBA64Half pixel format, improving both color and transparency fidelity for macOS Screen Capture.

Motivation and Context

Existing macOS Screen Captures have poor transparency fidelity because of the l10r pixel format, which uses only two bits of alpha, but which features high fidelity 10-bit color for representing content in the Display P3 color space.

Using RGBA64 enables us to both improve color fidelity with 16 bit floats per channel of color while granting the same fidelity to the alpha channel.

Higher fidelity transparency allows users to composite macOS Screen Capture window and application captures in front of arbitrary content in OBS with accurate transparency, enabling various creative usecases.

l10r (before) / RGhA (after):

Supersedes #12945.

Note

Support for this format inside ScreenCaptureKit is technically not documented. It is, however, used internally by macOS, and we have received guidance indicating that it should be acceptable to use for our purposes.

How Has This Been Tested?

Tested on Apple Silicon, macOS 15.7.1.

This change should be further verified to work on earlier macOS deployment targets, particularly macOS 13, before merging.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

• Tweak (non-breaking change to improve existing functionality)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[jcm93]

Per testing out-of-band, this format is unavailable on macOS 13.

This PR is now seeking a tester on macOS 14.

The functionality here will be gated by the availability of the earliest macOS version where the format is known to function without issues.

[PatTheMav]

What's the state of testing for older macOS versions?

[jcm93]

What's the state of testing for older macOS versions?

I haven't found a macOS 14 tester; I will likely just gate it behind macOS 15 today if one does not turn up.

[jcm93]

Amended the change to use 64RGBAHalf on macOS 15 and up, and fall back to 32BGRA on macOS 14 and below. If I am able to verify against macOS 14.0 I will update.

[jcm93]

Confirmed that 64RGBAHalf does not work on macOS 14.0 either. So the current state of this branch (gate behind macOS 15, fall back to 32BGRA otherwise) should be the way to go.