Skip to content

Faster queries#4

Open
jclymo wants to merge 8 commits intolrcfmd:mainfrom
jclymo:faster-queries
Open

Faster queries#4
jclymo wants to merge 8 commits intolrcfmd:mainfrom
jclymo:faster-queries

Conversation

@jclymo
Copy link

@jclymo jclymo commented Feb 2, 2023

Hi Cameron,

when I was querying ICSD I wanted to speed it up a bit and also to generate a zip file of the downloaded cifs and this escalated into quite a lot of changes.

  • separating the logic of making a query to the ICSD and managing the overall query (i.e. the difference between I want to get 3000 cifs and needing to make multiple calls within the cif limit).
  • context manager for login / logout
  • collect cifs into zip file
  • multithreaded calls to ICSD
  • managing the multiple logins required for the separate calls
  • allowing generic search strings to be passed in rather than dict of search_term: value because one of the queries I wanted to make couldn't be naturally expressed using only 'and' or 'or'. Using a dict of search terms to create the search string is still there but as a separate method now.

I have tried to maintain the existing functionality where possible but the interface has changed a bit. What do you think?

Jude

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant