[SPARK-54617][PYTHON][SQL] Enable Arrow Grouped Iter Aggregate UDF registration for SQL

[Yicong-Huang]

What changes were proposed in this pull request?

This PR enables Arrow grouped iter aggregate UDFs to be registered and used in SQL queries. Previously, Arrow iter aggregate UDFs could only be used via DataFrame API, but not in SQL.

The main change is adding SQL_GROUPED_AGG_ARROW_ITER_UDF to the allowed eval types in UDFRegistration.register() method, along with comprehensive test cases.

Why are the changes needed?

Arrow iter aggregate UDFs provide a memory-efficient way to perform grouped aggregations by processing data in batches iteratively. However, they could only be used via DataFrame API, not in SQL queries. This limitation prevented users from using these UDFs in SQL-based workflows.

Does this PR introduce any user-facing change?

Yes. Users can now register Arrow grouped iter aggregate UDFs and use them in SQL queries.

Example:

from typing import Iterator
from pyspark.sql.functions import arrow_udf
import pyarrow as pa

@arrow_udf("double")
def arrow_mean_iter(it: Iterator[pa.Array]) -> float:
    sum_val = 0.0
    cnt = 0
    for v in it:
        sum_val += pa.compute.sum(v).as_py()
        cnt += len(v)
    return sum_val / cnt if cnt > 0 else 0.0

# Now this works:
spark.udf.register("arrow_mean_iter", arrow_mean_iter)
spark.sql("SELECT id, arrow_mean_iter(v) as mean FROM test_table GROUP BY id").show()

How was this patch tested?

Added comprehensive test cases covering:

• Single column Arrow iter aggregate UDF in SQL
• Multiple columns Arrow iter aggregate UDF in SQL

Was this patch authored or co-authored using generative AI tooling?

No.