Inside of Entity Framework Migrations or How to View Generated Xml-Schema

Entity Framework migrations are very clever. But cleverness can come out from the wrong end sometimes. Over last 2 days I’ve spent at least a day debugging my migration scripts and not getting anywhere.

The trouble was in EF thinking that there is a migration pending, but nothing given to me to migrate: when I run Add-Migration, all I got was empty migration:

public void Up()
{
}
public void Down()
{
}

But still, on every attempt to use domain context, I was thrown an exception:

Unable to update database to match the current model because there are pending changes and automatic migration is disabled. Either write the pending model changes to a code-based migration or enable automatic migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to true to enable automatic migration.

That was soul destroying. I’ve spent hours banging my head against the keyboard with no results.

When I had to fix other problems I kinda put out the fire temporarily by saying

Database.SetInitializer<DomainContext>(null);

This is a hack, it makes the exception to go away, but does not fix the problem with migration. So I only put that in place as a quick fix to move forward a bit.

Today I added source code from Entity Framework into my project and stepped through EF code. That was fun! I say EF is very complex and very-well written. I was impressed with over 13K tests!

I eventually found the issue I’ve had with migration. I think this is a bug in EF 6.1 and I’ll try re-create it and will file a bug-report.

But I’ve learned a lot by looking on EF code. And I’ll share some of my findings with you.

How Entity Framework Migrations work on the inside

We’ve all run commands Add-Migration and Update-Database. And when it works – it is a fantastic tool (did I say that before?). But when it breaks, you are pretty much screwed. And it helps to understand what is happening behind the scenes.

EF tracks all executed migrations on database in table __MigrationHistory that looks like this:

migrations_table

If you run EF before version 6, you would not have ContextKey column – this is a new feature in EF6+.

Please look on Model column – this is some binary data stored there. All the time I worked with EF, I always thought this is a hash of model state. Turns out this is not a hash. This is just GZip-compressed XML string. And you can read it. Sometimes it is useful to read through the model definition to understand what happens and why you get an incorrect result in your migration script.

To read it you need to GZip-decompress this string:

[TestCase("MyMigration")]
public void DecompressDatabaseMigration(String migrationName)
{
    const string ConnectionString = // connection string to DB with migrations
    var sqlToExecute = String.Format("select model from __MigrationHistory where migrationId like '%{0}'", migrationName);

    using (var connection = new SqlConnection(ConnectionString))
    {
        connection.Open();

        var command = new SqlCommand(sqlToExecute, connection);

        var reader = command.ExecuteReader();
        if (!reader.HasRows)
        {
            throw new Exception("Now Rows to display. Probably migration name is incorrect");
        }

        while (reader.Read())
        {
            var model = (byte[])reader["model"];
            var decompressed = Decompress(model);
            Console.WriteLine(decompressed);
        }
    }
}

/// <summary>
/// Stealing decomposer from EF itself:
/// http://entityframework.codeplex.com/SourceControl/latest#src/EntityFramework/Migrations/Edm/ModelCompressor.cs
/// </summary>
public virtual XDocument Decompress(byte[] bytes)
{
    using (var memoryStream = new MemoryStream(bytes))
    {
        using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Decompress))
        {
            return XDocument.Load(gzipStream);
        }
    }
}

This code connects to database, reads migration information from the database, decompresses it and prints out into console. Alternatively you can review your generated migrations without database, only from generated migration classes:

[Test]
public void DecompressMigrationEncoding()
{
    var migrationClass = (IMigrationMetadata)new MyMigration();
    var target = migrationClass.Target;
    var xmlDoc = Decompress(Convert.FromBase64String(target));
    Console.WriteLine(xmlDoc);
}

You can view the whole class in this gist.

Looking on plain-text schema is useful, how it sees EF is useful for debugging. That’s how I fixed my problem.

Schema Comparing

Before I thought that EF looks on database and compares it’s schema with what it has in the model. And then generates migration script based on differences. Oh, boy, how wrong was I!

EF does not actually cares about your database state. It only cares about __MigrationHistory table and records in there. If you update your database to the latest migration, and drop all the tables apart from __MigrationHistory, EF will be convinced that your database is in a good shape.

When you run Add-Migration command from nuget console, EF reaches for __MigrationHistory tables, grabs the latest record from there, decompresses the xml with model state and compares xml-schema with what it has in the model. If the schemas do not match, it creates the script to update your database to the latest state.

So if you like to manually update generated migration scripts, stop it! This is a path to madness, you’ll eventually hit the same problem I had: schema in xml records got disconnected from the actual database schema and it took a lot of time to figure out where the differences are.

Sample Solution

To make all the above more understandable, I’ve created a tiny sample solution with command line project. Look through the code for more details of how you can view the xml-schema.

You can execute the sample. It’ll create a database in your local .sqlexpress instance of SqlServer, execute one migration on it and then will print out xml schema extracted from source code than from __MigrationHistory table in the freshly created database.

I certainly hope this write-up will help somebody to debug their EF Migration problem.