Pygtrie fails to install in linux with latest pip

[nsorros]

System

Python 3.7.5
pip 21.3.1
Ubuntu 18.04.3

To replicate

# no cache dir in case it finds the wheel in cache
# 2.4.2 version as otherwise it backtracks to another version
pip install --no-cache-dir pygtrie==2.4.2

This throws

Collecting pygtrie==2.4.2
  Downloading pygtrie-2.4.2.tar.gz (35 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /data/code/test/travis_bug/venv/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0wq_8vbs/pygtrie_62d541023f2d4328bfb177$
3cd0adcda/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0wq_8vbs/pygtrie_62d541023f2d4328bfb17793cd0adcda/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if $
s.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __fi$
e__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-_u9evd_6
       cwd: /tmp/pip-install-0wq_8vbs/pygtrie_62d541023f2d4328bfb17793cd0adcda/
  Complete output (33 lines):
  running egg_info
  creating /tmp/pip-pip-egg-info-_u9evd_6/pygtrie.egg-info
  writing /tmp/pip-pip-egg-info-_u9evd_6/pygtrie.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-_u9evd_6/pygtrie.egg-info/dependency_links.txt
  writing top-level names to /tmp/pip-pip-egg-info-_u9evd_6/pygtrie.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-_u9evd_6/pygtrie.egg-info/SOURCES.txt'
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-0wq_8vbs/pygtrie_62d541023f2d4328bfb17793cd0adcda/setup.py", line 179, in <module>
      distutils.core.setup(**kwargs)
    File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/data/code/test/travis_bug/venv/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 299, in run
      self.find_sources()
    File "/data/code/test/travis_bug/venv/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
      mm.run()
    File "/data/code/test/travis_bug/venv/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 541, in run
      self.add_defaults()
    File "/data/code/test/travis_bug/venv/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults
      sdist.add_defaults(self)
    File "/usr/lib/python3.7/distutils/command/sdist.py", line 226, in add_defaults
      self._add_defaults_python()
    File "/data/code/test/travis_bug/venv/lib/python3.7/site-packages/setuptools/command/sdist.py", line 113, in _add_defaults_python
      self._add_data_files(self._safe_data_files(build_py))
    File "/data/code/test/travis_bug/venv/lib/python3.7/site-packages/setuptools/command/egg_info.py", line 621, in _safe_data_files
      return build_py.get_data_files_without_manifest()
    File "/usr/lib/python3.7/distutils/cmd.py", line 103, in __getattr__
      raise AttributeError(attr)
  AttributeError: get_data_files_without_manifest
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/a5/8b/90d0f21a27a354e808a73eb0ffb94db990ab11ad1d8b3db3e5196c882cad/pygtrie-2.4.2.tar.gz#sha256=43205559d28863358dbbf25
045029f58e2ab357317a59b11f11ade278ac64692 (from https://pypi.org/simple/pygtrie/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full comman
d output.
ERROR: Could not find a version that satisfies the requirement pygtrie==2.4.2 (from versions: 0.9.1, 0.9.2, 0.9.3, 1.0, 1.1, 1.2, 2.0, 2.1, 2.2, 2.3, 2.3.2, 2.3.3, 2.4.0, 2.4.1,
2.4.2)
ERROR: No matching distribution found for pygtrie==2.4.2

[mina86]

Hmm, it certainly works for me:

$ python3 --version
Python 3.9.7
$ python3 -m pip --version
pip 21.3.1 from /home/mpn/.local/lib/python3.9/site-packages/pip (python 3.9)
$ pip install --no-cache-dir pygtrie==2.4.2
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pygtrie==2.4.2 in ./.local/lib/python3.9/site-packages (2.4.2)
$ pip install --no-cache-dir pygtrie==2.4.2
Defaulting to user installation because normal site-packages is not writeable
Collecting pygtrie==2.4.2
  Downloading pygtrie-2.4.2.tar.gz (35 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pygtrie
  Building wheel for pygtrie (setup.py) ... done
  Created wheel for pygtrie: filename=pygtrie-2.4.2-py3-none-any.whl size=19062 sha256=6f74abeb626b7b1c1500e79b8186ead9ed18edb538ac8903b839093f2f372a6a
  Stored in directory: /tmp/pip-ephem-wheel-cache-u3_fwr3d/wheels/2e/5d/81/d6e52fec193180ffffef19b185ecf1d0a85d418808c20c09c9
Successfully built pygtrie
Installing collected packages: pygtrie
Successfully installed pygtrie-2.4.2

The failure being in /usr/lib/python3.7/distutils/cmd.py makes me
wonder whether this is some issue with pip 21 not being compatible
with Python 3.7.

PS. Note that pygtrie is now at https://github.com/mina86/pygtrie.

[nsorros]

Hmm, it certainly works for me:

$ python3 --version
Python 3.9.7
$ python3 -m pip --version
pip 21.3.1 from /home/mpn/.local/lib/python3.9/site-packages/pip (python 3.9)
$ pip install --no-cache-dir pygtrie==2.4.2
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pygtrie==2.4.2 in ./.local/lib/python3.9/site-packages (2.4.2)
$ pip install --no-cache-dir pygtrie==2.4.2
Defaulting to user installation because normal site-packages is not writeable
Collecting pygtrie==2.4.2
  Downloading pygtrie-2.4.2.tar.gz (35 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pygtrie
  Building wheel for pygtrie (setup.py) ... done
  Created wheel for pygtrie: filename=pygtrie-2.4.2-py3-none-any.whl size=19062 sha256=6f74abeb626b7b1c1500e79b8186ead9ed18edb538ac8903b839093f2f372a6a
  Stored in directory: /tmp/pip-ephem-wheel-cache-u3_fwr3d/wheels/2e/5d/81/d6e52fec193180ffffef19b185ecf1d0a85d418808c20c09c9
Successfully built pygtrie
Installing collected packages: pygtrie
Successfully installed pygtrie-2.4.2

The failure being in /usr/lib/python3.7/distutils/cmd.py makes me wonder whether this is some issue with pip 21 not being compatible with Python 3.7.

PS. Note that pygtrie is now at https://github.com/mina86/pygtrie.

Let me try whether that resolves this. There is definitely an incompatibility that is introduced recently as this was discovered from Travis faling to create an environment it could create a few days ago.

[mina86]

It looks like it may be pypa/setuptools#2849 so it is possible that the issue is with pygtrie’s setup.py. I don’t know if I’ll have time to look into this today though.

[mina86]

Yeah, pulling latest setuptools with pip install -U setuptools breaks the install.

[nsorros]

Yeah, pulling latest setuptools with pip install -U setuptools breaks the install.

Any recommendation of how this can be fixed in the interim? I still have not found a combination of python, pip, pygtrie that works

[nsorros]

Actually I think pygtrie-2.3.3 works so maybe that is an interim solution

[mina86]

Yes, that’ll work (provided that you’re not using bool(children) in the node factory callback of traverse method). Looks like setuptools is also preparing a backwards compatibility patch: pypa/setuptools#2855 so hopefully it’ll soon work. I’ll also try to update pygtrie’s setup.py but like I’ve said I most likely won’t be able to do it today.

[mina86]

This is now resolved on setuptools side with 58.5.3 release.

[nsorros]

Super, thanks 🙏