Design Twitter / Social Media Feed System (Fan-Out on Write vs Fan-Out on Read).

In Simple Words: Design an interactive timeline where users can post tweets (tweets/sec: 10K writes) and view a personalized chronological feed of tweets from accounts they follow (feeds/sec: 500K reads).

The Architectural Dilemma: Fan-Out on Write vs Fan-Out on Read:

• Fan-Out on Write (Push Model):
  • When user posts a tweet, system looks up all their followers and writes the tweet ID directly into each follower's pre-computed Redis feed cache.
  • Read Path: Blazing fast $O(1)$ read from follower's feed cache.
  • Failure Mode: Celebrity accounts with 100M followers take minutes to push to 100M caches (wasting immense CPU and RAM).
• Fan-Out on Read (Pull Model):
  • Tweets are stored in a single table. When a user opens their feed, system queries latest tweets from all followed users and merges them dynamically.
  • Write Path: Blazing fast $O(1)$ insert.
  • Read Path: Extremely expensive multi-way merge ($O(N log K)$).
• The Hybrid Industry Standard:
  • Normal users ($< 25,000$ followers): Use Fan-Out on Write.
  • Celebrities ($> 25,000$ followers): Do not fan-out. Merge celebrity tweets dynamically into user's feed upon read request.

// Hybrid Feed Generation Logic
export async function generateUserFeed(userId: string, redis: any, db: any): Promise<string[]> {
  // 1. Fetch pre-computed timeline containing normal user tweets (Fast O(1))
  const cachedFeed: string[] = await redis.zrevrange(`timeline:${userId}`, 0, 50);

  // 2. Fetch IDs of followed celebrities
  const celebrityFollowees: string[] = await db.getFollowedCelebrities(userId);

  // 3. Fetch latest tweets from celebrities and merge with timeline in-memory
  const celebrityTweets = await db.getLatestCelebrityTweets(celebrityFollowees, 20);

  return mergeChronologically(cachedFeed, celebrityTweets).slice(0, 50);
}