-
Notifications
You must be signed in to change notification settings - Fork 10
Updates spk's FileMatcher to lazily create its internal gitignore object
#1290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
| pub struct FileMatcher { | ||
| rules: Vec<String>, | ||
| gitignore: ignore::gitignore::Gitignore, | ||
| gitignore: Arc<Mutex<Option<ignore::gitignore::Gitignore>>>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| gitignore: Arc<Mutex<Option<ignore::gitignore::Gitignore>>>, | |
| gitignore: Arc<Mutex<Option<Result<ignore::gitignore::Gitignore>>>>, |
I suggest making it distinguishable between an uninitialized value and a value that failed to initialize, to avoid repeatedly trying to initialize something that will fail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to include this and store the fail error.
| pub struct FileMatcher { | ||
| rules: Vec<String>, | ||
| gitignore: ignore::gitignore::Gitignore, | ||
| gitignore: Arc<Mutex<Option<ignore::gitignore::Gitignore>>>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Profile first, but ArcSwap may be a cheaper way to manage this field. Instead of creating numerous Mutex instances. I'm more concerned about this significantly slowing down builds for packages with a lot of files and calling matches on each file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe a OnceCell to leverage get_or_try_init? Maybe that's been tried, but leaving the note anyway
| /// The first call to this will construct an internal gitignore | ||
| /// matching object and cache it for subsequent calls. | ||
| pub fn matches<P: AsRef<std::path::Path>>(&self, path: P, is_dir: bool) -> Result<bool> { | ||
| let mut matcher = self |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before acquiring the lock you could check if self.rules is empty, if so then this could early exit. I'm assuming an empty Gitignore returns false for every input. This may alleviate some of the concerns about the Mutex overhead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a check for a FileMatcher with no rules.
… first call to matches() method. Signed-off-by: David Gilligan-Cook <dcook@imageworks.com>
…hing lazy initialisation. Signed-off-by: David Gilligan-Cook <dcook@imageworks.com>
…dly trying to initialise it when that will fail Signed-off-by: David Gilligan-Cook <dcook@imageworks.com>
1c26c2e to
f01b410
Compare
This updates spk's
FileMatcherobjects to lazily create its internalgitignoreobject on first call to thematches()method.The
FileMatcherand itsgitignorefield was accounting for about a quarter of the load/read time for a package build spec. The matching is only carried out when building a package, so setting it up lazily saves time in other parts of spk (e.g. a solve that reads 15k package build specs).Todo: