October 6, 2021
Graph updates are being delayed. Post engagements are continuing without issues.
A review has determined that post statistics updates are being delayed. Additional servers have been launched to cover the delay.
Graph updates have returned to normal.
This incident has not affected post engagements. Post engagements are occurring normally
The post-incident review has been completed. Surprisingly, the root cause is something else entirely: A recent large uptick in the number of new users in Lempod has led to more user statistics updates than we had capacity for. The capacity limit is due to a performance bottleneck in user security and identification matching. The improvement for this is scheduled for approximately late October / early November.
Until then, the workaround solution is to update user statistics once every 36-48 hours instead of every 12-24 hours.
We are in the process of deploying the fix to production. Along the way there has been a need for additional code fixes. This is an emergency fix and all hands are working hard. We apologize for the delay.