-
Notifications
You must be signed in to change notification settings - Fork 549
robot-name: Redesign for testability #2114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
This PR touches files which potentially affect the outcome of the tests of an exercise. This will cause all students' solutions to affected exercises to be re-tested. If this PR does not affect the result of the test (or, for example, adds an edge case that is not worth rerunning all tests for), please add the following to the merge-commit message which will stops student's tests from re-running. Please copy-paste to avoid typos. For more information, refer to the documentation. If you are unsure whether to add the message or not, please ping |
5dc78ed to
eef4709
Compare
The previous exercise design had two fundamental flaws wrt. testability: - The source of randomness was not injected by the tests. It was expected that users would initialize an RNG on their own. This lead to users simply not using any randomness at all. - The fact that names were required to be _globally_ unique meant that there was no way to test for deterministic behavior, given a PRNG with a stable seed. These two problems are solved by injecting an RNG from the tests and by narrowing the requirement of unique names to a single "robot factory". Motivated by this discussion on the forum: https://forum.exercism.org/t/rust-track-check-predictability-of-names-in-robot-name-exercise/20077 The function `RobotFactory::new_robot` takes a `&mut self`, even though it could take a `&self` as well. `&self` would work, because the robot factory needs shared ownership & interior mutability anyway to track robot names generated with `Robot::reset`. However, that would slightly interfere with the TTD flow of this exercise. If `RobotFactory::new_robot` takes a `&self`, users will be forced to use interior mutability right away, even though it's unclear why that is even necessary until they get around to implementing `Robot::reset`.
eef4709 to
e8ab978
Compare
|
I mentioned this on the forum, but I'll repeat it here for reference: This redesign deliberately breaks all existing solutions, because at least the top two community solutions don't use any randomness. Therefore, making a clean slate of the community solutions is actually benefitial. |
mk-mxp
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have no idea, if that is good Rust...anyways, I think I understand enough to see the randomness issue solved.
ellnix
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not that this is the only metric, but this was very fun to solve 😄
I think this redesign is excellent, there is nothing I can think of that you are not accounting for.
The one change I could see being interesting is to ensure that names are released to be used again when a robot's name is reset. With the current name pattern there are so many possible names that it's easy to find a non-conflicting name in every test, having a rotating set of names could be an interest twist.
Although of course it would also make sense for the problem to ensure that once a name is assigned once it is never assigned again, since that is what you would typically want if this was a real-life system.
This is all an aside though, not directly relevant to the goal of this PR, so this looks very good to me.
That's a great idea. The instructions state:
(ephasis mine.) I would say your idea matches these instructions from problem-specifications perfectly. And in combination with the randomness injection, this was quite easy to test for!
|
|
I'm wonder if we should get rid of the probabilistic tests entirely. I'm not sure what value the add at this point. |
|
Okay, I tweaked the tests a little bit more... I deleted I changed the test I also moved the test |
I agree, the student's solution is actually entirely deterministic, if you consider This is ready to be shipped from my perspective, but I understand if you want to wait for a review from someone from exercism's maintainers. |
Great, thanks. I'll let the PR stew for a while longer, if only in case one of us suddenly comes up with a new edge case in the next couple of days. |
The previous exercise design had two fundamental flaws wrt. testability:
The source of randomness was not injected by the tests. It was expected that users would initialize an RNG on their own. This lead to users simply not using any randomness at all.
The fact that names were required to be globally unique meant that there was no way to test for deterministic behavior, given a PRNG with a stable seed.
These two problems are solved by injecting an RNG from the tests and by narrowing the requirement of unique names to a single "robot factory".
Motivated by this discussion on the forum:
https://forum.exercism.org/t/rust-track-check-predictability-of-names-in-robot-name-exercise/20077
The function
RobotFactory::new_robottakes a&mut self, even though it could take a&selfas well.&selfwould work, because the robot factory needs shared ownership & interior mutability anyway to track robot names generated withRobot::reset. However, that would slightly interfere with the TTD flow of this exercise. IfRobotFactory::new_robottakes a&self, users will be forced to use interior mutability right away, even though it's unclear why that is even necessary until they get around to implementingRobot::reset.