gh-144356: Make set iterator __length_hint__ and iternext race-safe under no-gil

[hyongtao-code]

Long log:

In free-threaded builds, setiter_len() and setiter_iternext() could race with concurrent set mutation and iterator exhaustion.

We now serialize operations on the iterator itself (and keep the set stable) using a critical section, and ensure the iterator’s exhaustion state is handled consistently.

The non-free-threaded build keeps the existing semantics. Add free-threading-style concurrency tests (similar to the itertools FT tests) to exercise __length_hint__ and iternext exhaustion under concurrent execution.

setiter_len() was reading so->used without atomic access while concurrent
mutations update it atomically under Py_GIL_DISABLED.

In free-threaded builds, setiter_len() could race with concurrent set
mutation and iterator exhaustion.

Use an atomic load for so->used to avoid a data race. This preserves the
existing semantics of __length_hint__ while making the access thread-safe.

Signed-off-by: Yongtao Huang yongtaoh2022@gmail.com

• Issue: Data race in set iterator length_hint under no-gil #144356

[hyongtao-code]

Thanks for the review. I’ve decided to address both issues in this PR. I also added a corresponding test case for the issue you pointed out.

[eendebakpt]

I left some more review comments. I think we can get this right, but a simpler approach here would be to put a critical section on the set iterator itself.

[hyongtao-code]

put a critical section on the set iterator

Thanks for the suggestion. I simplified the implementation by protecting both the iterator and the set with Py_BEGIN_CRITICAL_SECTION2(self, so), so concurrent use of the same iterator is properly serialized. This keeps the logic aligned with other iterators and avoids the races.

It looks like there is an unrelated flaky test case.

[eendebakpt]

@hyongtao-code Thanks for your work on this! The core idea of the PR has shifted a bit, which is why I have some more review comments, but we are getting closer I believe.

[eendebakpt]

The test_iternext_concurrent_exhaust_race and test_length_hint_exhaust_race test slightly difference things, but I think they can be combined (to reduce pressure on the CI)

[hyongtao-code]

something like this work?

def worker():
    n = 0
    while True:
        if n & 1:
            it.__length_hint__()
        try:
            next(it)
        except StopIteration:
            break
        n += 1