I want to know if using all zeros instead of 1,0 as DroPE will yield the same result. It is hoped that such an experiment can be supplemented.

[xiezhipeng-git]

Currently, DroPE uses position encoding of 1,0, right? What will happen to the result if the following code is replaced with position encoding of all zeros?And what if it's half -1 and half 1, or the average of -1, 0, and 1?Is it the same result, or which one is the best?

# Null positional embeddings
    original_cos, original_sin = position_embeddings
    cos, sin = (
        torch.ones_like(original_cos),
        torch.zeros_like(original_sin),
    )