suggested rewording, based on my understanding of how this works. but please correct me if this isn't accurate.

I'm not sure I understand what this the affected range is. Is there a pair commits you could link to that would roughly correspond to it?

Any chance mitxonline could be in this situation?

I'll want to make a forum post and blast it out to #dev so that devs can recognize if they're in this state and know how to get out of it.

after some modulestore_migrations referencing the old app mdoels were run

oh, I think the overloaded use of the term "migration" is tripping me up here. @ormsbee , so this is only a problem for someone who ran Legacy Library Migrations before the Ulmo cutoff? is a whole dev database wipe really necessary, or could they just drop/clear the affected tables?

but before the most recent modulestore_migrations creating foreign keys to those models

I don't think I quite understand this

Any chance mitxonline could be in this situation?

No. Anyone on the latest master is good. Anyone on Ulmo is good. Anyone who makes a stop at Ulmo and migrates before moving on to something more recent is good.

I struggle to communicate this without a whole lot of detail that I thought might distract/confuse further. FWIW, when I say "migration", I mean Django migration, and not v1 -> v2 lib migration. I should probably pick a different app to illustrate this, come to think on it. Or maybe choose my squash ranges more carefully than I'm doing now... anyhow....

Let's take the modulestore_migrator migrations on master today:

https://github.com/openedx/openedx-platform/tree/master/cms/djangoapps/modulestore_migrator/migrations

Migration 0001 creates foreign keys to a handful of old authoring app models, like Collection, DraftChangeLog, etc.

Migration 0004 modifies a foreign key to PublishableEntity.

Keep in mind that all our openedx-learning migrations will potentially run before any of these openedx-platform migrations. This means that the tables underlying the openedx_content migrations have potentially been renamed with their new prefix before modulestore_migrator migrations run.

If no modulestore_migrator migrations have run yet, the new squashed migration will run, and we're golden. The new squashed migration jumps straight to the new openedx_content models, so old authoring model state is irrelevant.

If we're in a state where migration 0004 has already run, we're likewise fine. We can't run the squashed migration at this point, but all the actual database-altering, authoring-app-referencing migrations for the modulestore_migrator app have already run. Our skeleton backcompat model app state is good enough to satisfy the framework.

The Problem: Say the database is in a state where migration 0001-0003 have run, but 0004 has not yet run. The modulestore_migrator app doesn't exist in LMS at all, so someone runs the LMS migrations and we get to a state where all the openedx_content migrations have run and the tables have been renamed. Now 0004 wants to run and alter its foreign key reference to oel_publishing.publishableentitty. It thinks it can do that because the phantom backcompat model state says that it's still there. But the underlying table has actually been renamed. So when 0004 tries to run, it blows up with a SQL error about a missing table.

...

Okay, as I write that out, I realize that I might be able to fold together in a more robust way by properly by reflecting the rename in the backcompat model states and migrations. My first attempt at that didn't work out because there were conflicts in index naming, but that was before I realized that I could hollow out the backcompat models to just their primary keys.

I'll look into this, but I've lost track of how many times I've iterated on the database migration approach, and I really want to just push it at this point. 😩

Actually, nvm, I can't do it. The migrations framework will error if multiple models try to point to the same database table.

Thanks for the explanation. That makes sense.

Okay, as I write that out, I realize that I might be able to fold together in a more robust way by properly by reflecting the rename in the backcompat model states and migrations. My first attempt at that didn't work out because there were conflicts in index naming, but that was before I realized that I could hollow out the backcompat models to just their primary keys.
I'll look into this, but I've lost track of how many times I've iterated on the database migration approach, and I really want to just push it at this point.

Yeah, from my perspective, no need to make it any more leakproof than you already have it. I just want to know what to tell devs. We can tell them run migrations before pulling down this commit and rebuilding your openedx image but I'm sure many will miss that memo. So we can also tell them: when you see error $X during migrations, then either (a) drop your whole DB and re-run tutor-dev-init, or (b) run the following SQL and then try running migrations again:

DELETE FROM django_migrations WHERE app = 'modulestore_migrator';
DROP TABLE modulestore_migrator_modulestoreblockmigration;
DROP TABLE modulestore_migrator_modulestoreblocksource;
DROP TABLE modulestore_migrator_modulestoremigration;
DROP TABLE modulestore_migrator_modulestoresource;

# and the samea idea for the affected contentstore tables
DELETE FROM django_migrations WHERE app = 'contentstore' and name IN (
     '000x_blah', '000y_blah'
)
DROP TABLE contentstore_blah1;
DROP TABLE contentstore_blah2;

does that look right?