Deploy of incorrect production configs

Incident Report for Sellbrite

Resolved

All times PDT
At 1004 we started a deploy with code changes that included a bad production-only config
At 1020 we saw our first full system down error
By 1023 we were in a meeting and deploying a good rollback commit
By 1028 we had most systems up
By 1043 we were totally back up

Posted Nov 30, 2021 - 10:04 PST