Site performance issue
Incident Report for Sellbrite
Postmortem

We have identified a couple of troubling queries that span all shards of our distributed Citus database, causing our database to max out at 100 MB/s EBS I/O cap. Optimization of these queries will take time and will be launched after the Holiday Season code freeze. In the meantime we have put in a few patches such as reshuffling job queue prioritization, chunking out feed item to smaller bites, and dedupe-ing identical jobs, to prevent the proliferation of database I/O churn.

Posted Dec 20, 2017 - 16:10 PST

Resolved
System has self-stabilized. Application should be normal now. Team is working to diagnose root cause.
Posted Dec 20, 2017 - 10:45 PST
Investigating
We notice a database spike at 10am PT, then 10:08 and 10:15.
Posted Dec 20, 2017 - 10:44 PST