Note: The project has moved to https://github.com/rhyru9/
This tool helps you filter URL lists by removing duplicates, trimming to root or subfolder, removing parameters, and cleaning URLs.
Clone the repository and run the tool:
git clone https://github.com/RHYru9/Url-Cleaner.git
cd Url-Cleaner
python CleanUrl.py -L yourlist.txtAlternatively, you can use the web-based version at https://rhyru9.github.io/tools/