feature: Added support for evaluation relative to a single date. Useful for production usecases. #806
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces support for reference-date-based windowed evaluation in the OpenSTEF Beam evaluation pipeline. The main goal is to allow users to compute evaluation metrics focused on a specific time window around a given reference date, rather than always evaluating over all possible windows. This is achieved by propagating a new
reference_dateparameter through the pipeline and updating window iteration logic accordingly. Unit tests are also added to ensure correct behavior.Enhancements to windowed evaluation:
reference_dateparameter to the evaluation pipeline (evaluation_pipeline.py), enabling evaluation focused on a specific time period around the provided date. This parameter is propagated throughrun,run_for_subset, and_evaluate_subsetmethods, and is documented in their docstrings. [1] [2] [3] [4] [5] [6] [7] [8]window_iterators.py) to accept and handle thereference_dateparameter. When provided, only a single window corresponding to the reference date is yielded, rather than all possible windows. [1] [2] [3] [4] [5]Window model improvements:
get_timerangeandis_futuremethods to theWindowclass, which calculate the time range for a window based on a reference date and check if a window ends after the reference date, respectively.Testing:
test_window_iterators.pyto verify that window iteration returns either all windows or a single window depending on whetherreference_dateis set.Minor updates:
datetimewhere needed for the new functionality. [1] [2]