Skip to content

Commit e315848

Browse files
committed
Add more comparison methods for Rating objects and update documentation.
1 parent d21f236 commit e315848

File tree

10 files changed

+282
-2
lines changed

10 files changed

+282
-2
lines changed

.bumpversion.cfg

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[bumpversion]
2-
current_version = 2.3.0
2+
current_version = 2.4.0
33
commit = False
44
tag = False
55
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)\.(?P<build>\d+))?

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,9 @@ The default model is `PlackettLuce`. You can import alternate models from `opens
135135
- Full pairing should have more accurate ratings over partial pairing, however in high k games (like a 100+ person marathon race), Bradley-Terry and Thurstone-Mosteller models need to do a calculation of joint probability which involves is a k-1 dimensional integration, which is computationally expensive. Use partial pairing in this case, where players only change based on their neighbors.
136136
- Plackett-Luce (**default**) is a generalized Bradley-Terry model for k ≥ 3 teams. It scales best.
137137

138+
## Advanced Usage
139+
You can learn more about how to configure this library to suit your custom needs in the [project documentation](https://openskillpy.readthedocs.io/en/stable/advanced.html).
140+
138141

139142
## Implementations in other Languages
140143
- [Javascript](https://github.com/philihp/openskill.js)

docs/_static/ordinals.png

80 KB
Loading

docs/_static/tau.png

508 KB
Loading

docs/advanced.rst

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
==============
2+
Advanced Guide
3+
==============
4+
5+
.. warning::
6+
7+
These features have not been tested against real world data to check for their efficacy. You should test the parameters against your own data to ensure they suit your needs.
8+
9+
10+
Additive Dynamics Factor
11+
========================
12+
If ``sigma`` gets too small, eventually the rating change volatility will decrease. To prevent this you can pass the parameter ``tau`` to :class:`~openskill.rate.rate`. ``tau`` should preferably be a small decimal of two significant digits.
13+
14+
Here are some visuals of how ordinals change with different ``tau`` values:
15+
16+
.. image:: _static/tau.png
17+
18+
You can combine ``tau`` with another parameter ``prevent_sigma_increase`` to ensure the win probability always remains congruent with the actual win rate.
19+
20+
Here is how this looks:
21+
22+
.. image:: _static/ordinals.png
23+
24+
Time Decay
25+
==========
26+
27+
You can represent the decay of a player's skill over time by increasing ``sigma`` by a little per match not played (or every time interval). First collect data on by how much sigma goes up or down by resetting sigma to it's default value per interval. The value by which ``sigma`` should change must be an average of the change of ``sigma`` over all matches when ``sigma`` is reset per interval.
28+
29+
Here is an example of how to do this:
30+
31+
.. literalinclude:: time_decay.py
32+
33+
This will produce the following output:
34+
35+
.. code-block:: console
36+
37+
Before Large Decay:
38+
Player X: mu=26.986479759996925, sigma=1.879261533806081
39+
Player Y: mu=22.87672143851661, sigma=1.879261533806081
40+
41+
Predict Winner Before Decay:
42+
X has a 70.27% chance of winning over Y
43+
44+
Player X's Rating After Decay:
45+
Player X: mu=26.983781247317594, sigma=5.101382249884723
46+
47+
After Large Decay (1 Year):
48+
Player X: mu=28.199913286886318, sigma=4.958583411621401
49+
Player Y: mu=22.711677880164803, sigma=1.881565104224607
50+
51+
Predict Winner After Decay:
52+
X has a 58.51% chance of winning over Y
53+

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ Contents
1818
Contribution Guidelines <contributing>
1919
Installation <installation>
2020
User Manual <manual>
21+
Advanced Usage <advanced>
2122
Module Reference <api/modules>
2223

2324

docs/time_decay.py

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
from openskill import Rating, rate, predict_win
2+
3+
x, y = Rating(), Rating()
4+
5+
mu_precision = 1 + 0.0001 / 365
6+
sigma_precision = 1 + 1 / 365
7+
8+
# Let player X win 66% of the games.
9+
for match in range(10000):
10+
if match % 3:
11+
[[x], [y]] = rate([[x], [y]])
12+
else:
13+
[[y], [x]] = rate([[y], [x]])
14+
15+
# Decay Rating - Assume 1 Match Per Day
16+
x.mu /= mu_precision
17+
y.mu /= mu_precision
18+
19+
x.sigma *= sigma_precision
20+
y.sigma *= sigma_precision
21+
22+
print("Before Large Decay: ")
23+
print(f"Player X: mu={x.mu}, sigma={x.sigma}")
24+
print(f"Player Y: mu={y.mu}, sigma={y.sigma}\n")
25+
26+
print("Predict Winner Before Decay:")
27+
x_percent, y_percent = predict_win([[x], [y]])
28+
print(f"X has a {x_percent * 100: 0.2f}% chance of winning over Y\n")
29+
30+
# Decay Rating - Assume 365 Days Passed
31+
for match in range(365):
32+
33+
# Only player X's rating has decayed.
34+
if (x.mu < 25 + 3 * 25 / 3) or (x.mu > 25 - 3 * 25 / 3):
35+
x.mu /= mu_precision
36+
37+
if x.sigma < 25 / 3:
38+
x.sigma *= sigma_precision
39+
40+
print("Player X's Rating After Decay: ")
41+
print(f"Player X: mu={x.mu}, sigma={x.sigma}\n")
42+
43+
# One Match b/w X and Y
44+
[[x], [y]] = rate([[x], [y]])
45+
x.mu /= mu_precision
46+
y.mu /= mu_precision
47+
x.sigma *= sigma_precision
48+
y.sigma *= sigma_precision
49+
50+
51+
print("After Large Decay (1 Year): ")
52+
print(f"Player X: mu={x.mu}, sigma={x.sigma}")
53+
print(f"Player Y: mu={y.mu}, sigma={y.sigma}\n")
54+
55+
print("Predict Winner After Decay:")
56+
x_percent, y_percent = predict_win([[x], [y]])
57+
print(f"X has a {x_percent * 100: 0.2f}% chance of winning over Y")

openskill/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,4 @@
99
)
1010

1111

12-
__version__ = "2.3.0"
12+
__version__ = "2.4.0"

openskill/rate.py

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,114 @@ def __eq__(self, other):
5858
"You can only compare Rating objects with each other or a list of two floats."
5959
)
6060

61+
def __lt__(self, other):
62+
if isinstance(other, Rating):
63+
if ordinal(self) < ordinal(other):
64+
return True
65+
else:
66+
return False
67+
elif isinstance(other, (list, tuple)):
68+
if len(other) == 2:
69+
for value in other:
70+
if not isinstance(value, (int, float)):
71+
raise ValueError(
72+
f"The {other.__class__.__name__} contains an "
73+
f"element '{value}' of type '{value.__class__.__name__}'"
74+
)
75+
if ordinal(self) < ordinal([other[0], other[1]]):
76+
return True
77+
else:
78+
return False
79+
else:
80+
raise ValueError(
81+
f"The '{other.__class__.__name__}' object has more than two floats."
82+
)
83+
else:
84+
raise ValueError(
85+
"You can only compare Rating objects with each other or a list of two floats."
86+
)
87+
88+
def __gt__(self, other):
89+
if isinstance(other, Rating):
90+
if ordinal(self) > ordinal(other):
91+
return True
92+
else:
93+
return False
94+
elif isinstance(other, (list, tuple)):
95+
if len(other) == 2:
96+
for value in other:
97+
if not isinstance(value, (int, float)):
98+
raise ValueError(
99+
f"The {other.__class__.__name__} contains an "
100+
f"element '{value}' of type '{value.__class__.__name__}'"
101+
)
102+
if ordinal(self) > ordinal([other[0], other[1]]):
103+
return True
104+
else:
105+
return False
106+
else:
107+
raise ValueError(
108+
f"The '{other.__class__.__name__}' object has more than two floats."
109+
)
110+
else:
111+
raise ValueError(
112+
"You can only compare Rating objects with each other or a list of two floats."
113+
)
114+
115+
def __le__(self, other):
116+
if isinstance(other, Rating):
117+
if ordinal(self) <= ordinal(other):
118+
return True
119+
else:
120+
return False
121+
elif isinstance(other, (list, tuple)):
122+
if len(other) == 2:
123+
for value in other:
124+
if not isinstance(value, (int, float)):
125+
raise ValueError(
126+
f"The {other.__class__.__name__} contains an "
127+
f"element '{value}' of type '{value.__class__.__name__}'"
128+
)
129+
if ordinal(self) <= ordinal([other[0], other[1]]):
130+
return True
131+
else:
132+
return False
133+
else:
134+
raise ValueError(
135+
f"The '{other.__class__.__name__}' object has more than two floats."
136+
)
137+
else:
138+
raise ValueError(
139+
"You can only compare Rating objects with each other or a list of two floats."
140+
)
141+
142+
def __ge__(self, other):
143+
if isinstance(other, Rating):
144+
if ordinal(self) >= ordinal(other):
145+
return True
146+
else:
147+
return False
148+
elif isinstance(other, (list, tuple)):
149+
if len(other) == 2:
150+
for value in other:
151+
if not isinstance(value, (int, float)):
152+
raise ValueError(
153+
f"The {other.__class__.__name__} contains an "
154+
f"element '{value}' of type '{value.__class__.__name__}'"
155+
)
156+
if ordinal(self) >= ordinal([other[0], other[1]]):
157+
return True
158+
else:
159+
return False
160+
else:
161+
raise ValueError(
162+
f"The '{other.__class__.__name__}' object has more than two floats."
163+
)
164+
else:
165+
raise ValueError(
166+
"You can only compare Rating objects with each other or a list of two floats."
167+
)
168+
61169

62170
def ordinal(agent: Union[Rating, list, tuple], **options) -> float:
63171
"""
@@ -147,6 +255,9 @@ def rate(teams: List[List[Rating]], **options) -> List[List[Rating]]:
147255
:param prevent_sigma_increase: A :class:`~bool` that prevents sigma from ever increasing.
148256
:param options: Pass in a set of custom values for constants defined in the Weng-Lin paper.
149257
:return: Returns a list of :class:`~openskill.rate.Rating` objects.
258+
259+
.. note::
260+
``tau`` will default to ``25/300`` in the next major version update.
150261
"""
151262
original_teams = copy.deepcopy(teams)
152263
if "tau" in options:

tests/test_rating.py

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,25 @@ def test_rating():
99
assert Rating(mu=32.124, sigma=1.421) != Rating()
1010
assert Rating(mu=32.124, sigma=1.421) != [23.84, 3.443]
1111

12+
assert Rating() <= Rating()
13+
assert Rating() >= Rating()
14+
assert Rating(mu=32.124, sigma=1.421) > [23.84, 3.443]
15+
assert Rating(mu=32.124, sigma=1.421) > Rating(mu=23.84, sigma=3.443)
16+
assert Rating(mu=23.84, sigma=3.443) < [32.124, 1.421]
17+
assert [23.84, 3.443] < Rating(mu=32.124, sigma=1.421)
18+
assert Rating(mu=23.84, sigma=3.443) < Rating(mu=32.124, sigma=1.421)
19+
assert [23.84, 3.443] <= Rating(mu=32.124, sigma=1.421)
20+
assert Rating(mu=23.84, sigma=3.443) <= [32.124, 1.421]
21+
assert not (Rating(mu=32.124, sigma=1.421) < [23.84, 3.443])
22+
assert not (Rating(mu=32.124, sigma=1.421) < Rating(mu=23.84, sigma=3.443))
23+
assert not ([23.84, 3.443] > Rating(mu=32.124, sigma=1.421))
24+
assert not (Rating(mu=23.84, sigma=3.443) > Rating(mu=32.124, sigma=1.421))
25+
assert not (Rating(mu=23.84, sigma=3.443) > [32.124, 1.421])
26+
assert not (Rating(mu=32.124, sigma=1.421) <= Rating(mu=23.84, sigma=3.443))
27+
assert not (Rating(mu=32.124, sigma=1.421) <= [23.84, 3.443])
28+
assert not (Rating(mu=23.84, sigma=3.443) >= Rating(mu=32.124, sigma=1.421))
29+
assert not (Rating(mu=23.84, sigma=3.443) >= [32.124, 1.421])
30+
1231
with pytest.raises(ValueError):
1332
assert Rating(mu=32.124, sigma=1.421) == ["random_string", 5.6]
1433

@@ -17,3 +36,39 @@ def test_rating():
1736

1837
with pytest.raises(ValueError):
1938
assert Rating(mu=32.124, sigma=1.421) == "random_string"
39+
40+
with pytest.raises(ValueError):
41+
assert Rating(mu=32.124, sigma=1.421) < ["random_string", 5.6]
42+
43+
with pytest.raises(ValueError):
44+
assert Rating(mu=32.124, sigma=1.421) < [23.84, 3.443, 5.6]
45+
46+
with pytest.raises(ValueError):
47+
assert Rating(mu=32.124, sigma=1.421) < "random_string"
48+
49+
with pytest.raises(ValueError):
50+
assert Rating(mu=32.124, sigma=1.421) > ["random_string", 5.6]
51+
52+
with pytest.raises(ValueError):
53+
assert Rating(mu=32.124, sigma=1.421) > [23.84, 3.443, 5.6]
54+
55+
with pytest.raises(ValueError):
56+
assert Rating(mu=32.124, sigma=1.421) > "random_string"
57+
58+
with pytest.raises(ValueError):
59+
assert Rating(mu=32.124, sigma=1.421) <= ["random_string", 5.6]
60+
61+
with pytest.raises(ValueError):
62+
assert Rating(mu=32.124, sigma=1.421) <= [23.84, 3.443, 5.6]
63+
64+
with pytest.raises(ValueError):
65+
assert Rating(mu=32.124, sigma=1.421) <= "random_string"
66+
67+
with pytest.raises(ValueError):
68+
assert Rating(mu=32.124, sigma=1.421) >= ["random_string", 5.6]
69+
70+
with pytest.raises(ValueError):
71+
assert Rating(mu=32.124, sigma=1.421) >= [23.84, 3.443, 5.6]
72+
73+
with pytest.raises(ValueError):
74+
assert Rating(mu=32.124, sigma=1.421) >= "random_string"

0 commit comments

Comments
 (0)