Charles Proxy not properly detected on MacOS

[dadarakt]

We used UnityWebRequest before but wanted to switch to BestHTTP for proper HTTP2 support.

With UnityWebRequest, we were able to proxy all requests through Charles' Proxy to inspect the outgoing request without any further setup.

After switching over to BestHTTP, it seems that it's bypassing the system's proxy settings.

We tried setting the global proxy manually, but then the TLS handshake fails.

The manual proxy setup:

HTTPManager.Proxy = new HTTPProxy(new Uri("http://localhost:8888"));

The stacktrace:

Can't resume session: StatusCode: 0, Message: user_canceled(90), Content: , StackTrace:  ...

Seen on the proxy:

user_canceled (90) - The handshake was canceled for some reason unrelated to a protocol failure

Is there any way to configure BestHTTP to use MacOS' system proxy setting?
Or at least understand why the manual proxy setup fails?

Thanks for your time!

[Benedicht]

Don't know why that error, but I use Charles with the plugin with this setting (on windows):

HTTPManager.Proxy = new HTTPProxy(new Uri("http://localhost:8888/"), null, true);

[dadarakt]

Thanks for the hint, I tried that with the following minimal reproduction code:

void OnRequestFinished(HTTPRequest request, HTTPResponse response)
{
    Debug.LogWarning(response.StatusCode);
}

void TestRequest()
{
    HTTPRequest r = new HTTPRequest(new Uri("https://www.google.com"), OnRequestFinished);
    r.Send();
}

This is the current callsite. In Charles, I enabled transparent proxy and added www.google.com to the SSL proxy domains.

TLSSecurity.Setup();
HTTPManager.Proxy = new HTTPProxy(new Uri("http://localhost:8888/"), null, true);
TestRequest();

The error is the same as shown above. The request never succeeds.

Here is a screenshot from Charles when running the test request:

[Benedicht]

Ahh, yes, the usage of TLSSecurity.Setup() call is very important information!

What happens is that because you checked the SSL proxying in Charles, it acts as a middle man between between the google server and the plugin. Because Charles want to intercept the request, it sends its own certificate to the plugin, however it's certificate isn't present in the plugin's store. Here's Charles help about its root certificates:

So, what options you have?

1. You can disable Charles' SSL proxying.
2. Or you can add Charles' root certificate to the plugin's Trusted Root CAs:

• Help\SSL Proxying\Save Charles Root Certificate... menuitem and save somewhere
• Use the TLS Security addon's Certificate Manager Window's Add Custom button to add the certificate you just saved in the previous step.
• SSL Proxying must work now.

[Benedicht]

Forgot to add the link to the Certificate Manager Window: https://benedicht.github.io/BestHTTP-Documentation/pages/best_http2/addons/tls_security/CertificationManagerWindow.html#trusted-root-cas

You can use the Add Custom (marked as 7) button to navigate and load the Charles certificate. In the search box you can search for Charles to make sure it's imported.

[dadarakt]

Thanks a lot for the detailed guide!
With these steps, it works if I manually set the proxy as described earlier, nice.

Out of interest: Would you expect it to automatically find and use Charles' proxy on any platform? As mentioned, UnityWebRequest worked out of the box (but failed in other ways).

I have a workaround, so feel free to close the issue in case that you find the automatic detection a non-issue.

[Benedicht]

I'm thinking about leaving it open, so i will remember to take a look on this part of proxy detection.