gpyrobotstxt, a Python Native port of this repo

[CoconSe]

Thanks for releasing this reference code!

We have ported a Python version of this library, exe and test suite.
The Python version keeps the same behaviour as the original library and passed 100% C++ testing cases.

Here is the repository: https://github.com/Cocon-Se/gpyrobotstxt

[fili]

Nice! Thanks ;)

[nzrsky]

Btw, I made a performance comparison between this port and the C++ version (including my own C++ fork) https://github.com/nzrsky/robotstxt/blob/master/COMPARISON.md

[LeMoussel]

@nzrsky Your performance tests read the robots_all.bin file which contains robots.txt files in simple binary format. How do you generate robots_all.bin file ?

[nzrsky]

@LeMoussel The robots_all.bin file is a simple concatenation of robots.txt files with length prefixes (I collected robots.txt files from my bookmarks list). The benchmark data is available at: https://github.com/nzrsky/robotstxt-benchmark-data

Format:
[4 bytes: uint32_le length][N bytes: content]
...

[LeMoussel]

@nzrsky It is not possible to create an issue on repository https://github.com/nzrsky/robotstxt