Docs/mkdoc #1379

PGijsbers · 2024-10-21T18:43:26Z

Work converting the Python docs to mkdocs. Left to do before it's "functional":

Make sure notebooks run
Add the test server warning back, preferably in a reusable way as was before.

Otherwise, most of the files have been converted.

That said, the documentation really needs a rewrite. I refrained from changing any content as much as I could, with rare exceptions. But the main problems, to me, are:

Information is all over the place. A few cohesive examples with a proper ToC would be better (I think).
The formatting is bad.
There is a mix of styles. Some examples even talk about "exercises" (e.g., here).
Some of the content is outdated.

How should we move forward with this PR? I don't mind rewriting (some of) this documentation, but I think it would be better to merge this at one point so it's easier to collaborate. WDYT? @eddiebergman @LennartPurucker

order plugin is important: first gen_files needs to generate the virtual files for the jupyter extension to render them

I also tried `no_input: true` but that removed the whole cell.

LennartPurucker · 2024-10-21T19:56:10Z

IMO: merge first and focus on user stories/use cases afterward to refactor the documentation. That is, likely move all the old documentation to an "extended example" list and only keep a set of user stories / use cases in the new main documentation.

SubhadityaMukherjee

Merged all the changes, they seem correct and run correctly. Next I will test the notebooks/scripts to verify if everything runs.

…e if everything still runs and is up to date

codecov-commenter · 2025-03-25T13:35:43Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.71%. Comparing base (c66d22a) to head (2a16c03).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1379   +/-   ##
========================================
  Coverage    53.71%   53.71%           
========================================
  Files           38       38           
  Lines         5229     5229           
========================================
  Hits          2809     2809           
  Misses        2420     2420

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

SubhadityaMukherjee · 2025-04-29T16:38:50Z

Seems I forgot to update this. These examples dont work. Im not entirely sure how to fix them. All the other examples seem to run. I modified some slightly to make some of them work. They are from examples/

20_basic/
simple_flows_and_runs_tutorial : the flow seems to be wrong. expected flow parameters: [] model parameters: ['bootstrap', 'ccp_alpha', 'class_weight', 'criterion', 'max_depth', 'max_features', 'max_leaf_nodes', 'max_samples', 'min_impurity_decrease', 'min_samples_leaf', 'min_samples_split', 'min_weight_fraction_leaf', 'monotonic_cst', 'n_estimators', 'n_jobs', 'oob_score', 'random_state', 'verbose', 'warm_start’]

30_extended/
run_setup_tutorial : IndexError: positional indexers are out-of-bounds

flows_and_runs_tutorial : No reg-eye-state, or dataset 68 seems to be something else.

study_tutorial: the flow seems to be wrong. ValueError: Parameters of the model do not match the parameters expected by the flow: expected flow parameters: [] model parameters: ['bootstrap', 'ccp_alpha', 'class_weight', 'criterion', 'max_depth', 'max_features', 'max_leaf_nodes', 'max_samples', 'min_impurity_decrease', 'min_samples_leaf', 'min_samples_split', 'min_weight_fraction_leaf', 'monotonic_cst', 'n_estimators', 'n_jobs', 'oob_score', 'random_state', 'verbose', 'warm_start’]

40_paper/
2018_kdd_rijn_example.py: This example is deprecated, remove the if False in this code to use it manually.

PGijsbers · 2025-05-05T13:12:29Z

Thanks! I can have another look once AIoD calms down a little (i.e., after May 22nd).

… with new deps

… with new deps- fixed typo

… with new deps-removed incorrect path

SubhadityaMukherjee · 2025-06-16T13:59:12Z

Removed linkcheck for now since the one I picked apparently is not maintained.
Also an option for versioning using "mike" which requires some extra work. - https://squidfunk.github.io/mkdocs-material/setup/setting-up-versioning/

SubhadityaMukherjee · 2025-06-16T14:00:55Z

Ah @PGijsbers I think we will also need to point the gh-pages option in settings to a different page. I dont have access to it so I cant check if its the same as sphinx.
It should be Pages -> Deploy from branch : gh-pages -> root

PGijsbers · 2025-06-18T08:04:29Z

Ah @PGijsbers I think we will also need to point the gh-pages option in settings to a different page. I dont have access to it so I cant check if its the same as sphinx. It should be Pages -> Deploy from branch : gh-pages -> root

That's already set. Sorry for the slow reply. Next time just ask me directly if I don't respond :)

PGijsbers · 2025-06-18T13:43:55Z

.github/workflows/docs.yaml

+        if: (contains(github.ref, 'develop') || contains(github.ref, 'main')) && github.event_name == 'push'
+        run: |
+          mkdocs gh-deploy --force
+        env:


Does this take into account the branch? For example, if we publish version 0.16 and its documentation, to me it reads like our develop documentation will overwrite it as soon as we push to develop.

I know I initially said we didn't need it but.. maybe it might be a good idea to look into mike? https://squidfunk.github.io/mkdocs-material/setup/setting-up-versioning/

it seems to handle the thing we'd want later and it would also solve this problem.
Then we can simply build docs when:

push to develop (as dev)

a release is made, i.e., a tag of format v*

Some reference for what you might need from mike if you want versioned docs.
https://github.com/automl/amltk/blob/5221d80fb4fec92f2d6f2fc73fce4eb16139c09f/.github/workflows/release.yml#L90-L110

It's also good to be aware that all mike does for you is work on the gh-pages branch and so you can always go nuke it or manually edit it if testing goes wrong. Took me a while before realising that, but it means you can really test out the workflow locally before doing the annoying dance with GH actions to get it to work.

The root of this branch is pretty instructive for what it's doing, i.e. with the symlink, the index.html pointig to latest and the versions.json it's using to store metadata
https://github.com/automl/amltk/tree/gh-pages

A wild Eddie appears!

You can just use regular pokeball, no need for an ultra

throws a master ball

struggle, struggle, pokémon Eddie breaks free

SubhadityaMukherjee · 2025-06-19T09:00:04Z

all the tests pass in reality but the added files check fails, maybe mike adds them. Im looking into it

LennartPurucker

Minor comments

LennartPurucker · 2025-06-19T09:56:46Z

examples/30_extended/datasets_tutorial.py

+<<<<<<< docs/mkdoc -- Incoming Change
+# %%
+import openml
+=======
+>>>>>>> develop -- Current Change
 import pandas as pd

 import openml
 from openml.datasets import edit_dataset, fork_dataset, get_dataset

-############################################################################
-# Exercise 0
-# **********
+# %% [markdown]
+# ## Exercise 0
 #
+<<<<<<< docs/mkdoc -- Incoming Change
+# * List datasets
+
+# %%
+datalist = openml.datasets.list_datasets(output_format="dataframe")
+=======
 # * List datasets and return a dataframe
 datalist = openml.datasets.list_datasets()
+>>>>>>> develop -- Current Change


LennartPurucker · 2025-06-19T09:59:57Z

tests/files/misc/features_with_whitespaces.xml.pkl

What is this file?

LennartPurucker

LGTM!

LennartPurucker · 2025-06-19T10:26:42Z

uv.lock

this file is not needed

PGijsbers added 25 commits July 29, 2024 11:58

Start migration to mkdocs

ed970ee

Fix reference, avoids duplicate installation headers

ff29c99

Improve citation section, fix last broken link

81a0a7b

Remove files which are to be generated

5cb04dd

remove orphan statements

94d3024

Add plugin for apidoc generation

6dcb538

Generate API documentation pages

0ef96ee

Add API pages to nav, use section-index for init

f9dd346

Remove examples doc as they are generated

12a5066

Render notebooks from example folder

2cc23af

make clearer the symmetry

a4efa61

Put notebooks as virtual files in doc, reorder plugins

2bc16b8

order plugin is important: first gen_files needs to generate the virtual files for the jupyter extension to render them

Start converting to jupytext

62c40d9

Add other files to toc

7d29016

Fix formatting and links

978839d

Fix formatting

1428872

fix broken links

f38af1f

Add more notebooks

177faf1

Remove notebooks

ce24b44

Convert more examples to jupytext

e5f60f5

Convert remaining files to jupytext

957c0bd

Add titles

792d759

Make codeblock have light background

d148f5d

Hide the "In [#]" area

df5099a

I also tried `no_input: true` but that removed the whole cell.

Fix (link) formatting errors

cb49496

SubhadityaMukherjee reviewed Mar 25, 2025

View reviewed changes

SubhadityaMukherjee previously approved these changes Mar 25, 2025

View reviewed changes

merged pending changes, next commit will test notebooks/scripts to se…

3896a5a

…e if everything still runs and is up to date

modified some examples to make them work

96e3fce

PGijsbers self-assigned this May 5, 2025

PGijsbers and others added 5 commits June 16, 2025 14:30

Merge branch 'develop' into docs/mkdoc

3ec1801

converted docs workflow from sphinx to mkdocs, updated pyproject.toml…

6e3b059

… with new deps

converted docs workflow from sphinx to mkdocs, updated pyproject.toml…

25ca8d6

… with new deps- fixed typo

converted docs workflow from sphinx to mkdocs, updated pyproject.toml…

133c016

… with new deps-removed incorrect path

removed linkcheck for now

937ede8

SubhadityaMukherjee marked this pull request as ready for review June 16, 2025 14:01

fix for #1347

866fbe5

This was referenced Jun 16, 2025

Cache directory typo in docs #1347

Closed

Invalid estimation procedure ids #1249

Open

Create an example with datasets that have different types of features #1062

Open

Example request : "Create openml dataset from an image dataset" in documentation #1008

Open

Merge branch 'develop' into docs/mkdoc

dc25a71

PGijsbers commented Jun 18, 2025

View reviewed changes

SubhadityaMukherjee and others added 2 commits June 19, 2025 10:12

trying out mike

f0a233e

Merge branch 'develop' into docs/mkdoc

6ae4125

trying to fix files being created

091d2c5

LennartPurucker requested changes Jun 19, 2025

View reviewed changes

SubhadityaMukherjee added 2 commits June 19, 2025 12:07

resolved some comments

868ca77

uv lock to gitignore

2a16c03

LennartPurucker approved these changes Jun 19, 2025

View reviewed changes

uv.lock Outdated

Copy link

Contributor

LennartPurucker Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is not needed

LennartPurucker merged commit d0f31b9 into develop Jun 19, 2025
12 of 13 checks passed

LennartPurucker deleted the docs/mkdoc branch June 19, 2025 10:37

Uh oh!

Docs/mkdoc #1379

Docs/mkdoc #1379

Uh oh!

Conversation

PGijsbers commented Oct 21, 2024

Uh oh!

LennartPurucker commented Oct 21, 2024

Uh oh!

SubhadityaMukherjee left a comment

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

SubhadityaMukherjee commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PGijsbers commented May 5, 2025

Uh oh!

SubhadityaMukherjee commented Jun 16, 2025

Uh oh!

SubhadityaMukherjee commented Jun 16, 2025

Uh oh!

PGijsbers commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eddiebergman Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SubhadityaMukherjee commented Jun 19, 2025

Uh oh!

LennartPurucker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LennartPurucker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-commenter commented Mar 25, 2025 •

edited

Loading

SubhadityaMukherjee commented Apr 29, 2025 •

edited

Loading

PGijsbers commented Jun 18, 2025 •

edited

Loading

eddiebergman Jun 18, 2025 •

edited

Loading