extractChanges is misleading. It returns changes that's already in db as new changes.

[adityathebe]

extractChanges returns

• a list of new changes to insert and
• a list of existing changes that needs to be updated.

In practice, almost all changes (event those that already exist in the db) are returned as newOnes.

extractChanges never checks whether a change with the same (config_id, external_change_id) already exists. That means any repeated change goes into newOnes even if it’s already in the config_changes table.

GithubAction scraper is a good example for this as it produces the same runs as changes again and again.

Side effects:

• The change is counted as new (metrics and summaries) even though we already had it.
• The count field is reset to 1 because the new struct has Count: 1 and the upsert does a full update.

[adityathebe]

A lookup on external_change_id was most likely avoided due to performance reasons.

however, this is creating incorrect metrics and scrape summary.