#Design Tinder Feed (Tinder Scale)
Target Scale: 75M DAU, 2B swipes/day, 50M active profiles, global deployment.
#1. Problem Statement & Clarifications
#Functional Requirements
- Serve a Discovery Feed: Show a user a ranked stack of candidate profiles within their set distance and age preferences.
- Record Swipe Events: Capture LEFT (pass), RIGHT (like), and SUPER LIKE with sub-50ms acknowledgement.
- Detect Mutual Matches: When two users both swipe right, immediately notify both and create a match.
- Update Location: Periodically update user geolocation to refresh their candidate pool.
- Respect Filters: Honor user-configured filters β max distance, age range, gender preference.
- Exhaust & Refill: When a user exhausts their local candidate stack, asynchronously refill it.
#Non-Functional Requirements
| Requirement | Target |
|---|---|
| Feed load latency (P99) | < 200 ms |
| Swipe acknowledgement (P99) | < 50 ms |
| Match notification latency | < 2 s |
| Availability | 99.99% |
| Candidate freshness | < 5 min stale |
| Swipe event durability | No loss (at-least-once) |
| Scale | 75M DAU, 2B swipes/day |
#Out of Scope
- Chat/messaging after a match is formed.
- Payment / Tinder Gold subscription billing.
- Photo upload pipeline and moderation.
- Boost and Super Boost ad features.
- Cross-device session sync.
#Assumptions
- A user's location is updated every time the app is foregrounded (not continuous GPS polling).
- Candidates are pre-fetched into a per-user stack; the client holds ~20 profiles client-side at a time.
- The ranking model is ML-based but its training pipeline is out of scope; we design the serving layer.
- 80% of swipes are LEFT; 20% are RIGHT or SUPER LIKE.
#2. Back-of-Envelope Estimation
#Traffic Estimates
DAU = 75M
Swipes/user/day = 2B / 75M β 27 swipes/user/day
Swipe write QPS = 2B / 86400 β 23K QPS avg; Γ 3 peak = 69K QPS
Feed fetch QPS = assume 1 fetch per 20 swipes β 23K / 20 β 1.2K QPS
Location update QPS = 1 update per app open β 75M / 86400 β 870 QPS
Match detection QPS = 20% right-swipes β 4.6K right-swipes/s
mutual match rate β 2% β ~90 matches/s#Storage Estimates
User profile row = ~2 KB (metadata, preferences)
Total profiles = 50M Γ 2 KB = 100 GB (fits in PostgreSQL)
Swipe record = ~100 B (user_id, target_id, direction, ts)
Swipes/day = 2B Γ 100 B = 200 GB/day
1-year swipe log = 200 GB Γ 365 β 73 TB β Cassandra, tiered storage
Candidate stack cache = 50 profiles Γ 2 KB = 100 KB/user
Active users cached = 10M concurrent Γ 100 KB = 1 TB Redis cluster#Bandwidth
Feed response = 20 profiles Γ (2 KB metadata + 5 thumbnail URLs) β 50 KB
Feed bandwidth = 1.2K QPS Γ 50 KB = 60 MB/s (metadata only)
Photo delivery = via CDN (not counted in app server bandwidth)
Swipe write = 23K QPS Γ 200 B = 4.6 MB/s#Cache Estimates
Hot candidate stacks = 10M active users Γ 100 KB = 1 TB
Seen-set per user = 50K seen profiles Γ 8B (user_id) = 400 KB per user
β store as Bloom filter: 50K entries, 1% FPR β 72 KB per user
β 10M users Γ 72 KB β 720 GB (Redis cluster, Bloom filter per key)
Match cache (recent) = 90 matches/s Γ 24h Γ 1 KB = ~8 GB (fully fits in RAM)#3. API Design
#GET /v1/feed β Fetch Candidate Stack
GET /v1/feed?limit=20&lat=37.77&lon=-122.41
Authorization: Bearer <jwt>
Response 200:
{
"candidates": [
{
"user_id": "u_abc123",
"display_name": "Alex",
"age": 28,
"bio": "Hiker, coffee addict.",
"distance_km": 3.2,
"photo_urls": [
"https://cdn.tinder.com/photos/u_abc123/p1_thumb.webp",
"https://cdn.tinder.com/photos/u_abc123/p2_thumb.webp"
],
"score": 0.87
}
],
"stack_ttl_s": 300,
"next_refill_hint": "2026-06-02T00:05:00Z"
}#POST /v1/swipe β Record a Swipe
POST /v1/swipe
Authorization: Bearer <jwt>
Content-Type: application/json
{
"target_user_id": "u_abc123",
"direction": "RIGHT", // LEFT | RIGHT | SUPER_LIKE
"swipe_duration_ms": 1240,
"feed_position": 3
}
Response 200:
{
"swipe_id": "sw_xyz789",
"match": {
"matched": true,
"match_id": "m_def456",
"conversation_id": "conv_ghi012"
}
}
// match is null if no mutual match#POST /v1/location β Update User Location (Internal / Client)
POST /v1/location
Authorization: Bearer <jwt>
{ "lat": 37.774, "lon": -122.419, "accuracy_m": 15 }
Response 204 No Content#POST /v1/match/notify β Internal Match Notification (Match Service β Notification Service)
POST /v1/match/notify [INTERNAL]
{
"match_id": "m_def456",
"user_a": "u_xyz",
"user_b": "u_abc123",
"matched_at": "2026-06-02T00:00:05Z"
}#4. Data Model
#User Profiles (PostgreSQL β sharded by user_id)
CREATE TABLE users (
user_id UUID PRIMARY KEY,
display_name VARCHAR(50) NOT NULL,
birth_date DATE NOT NULL,
gender VARCHAR(20),
seeking_gender VARCHAR(20)[],
bio TEXT,
geohash6 CHAR(6), -- current 6-char geohash cell
lat DOUBLE PRECISION,
lon DOUBLE PRECISION,
location_updated_at TIMESTAMPTZ,
min_age_pref SMALLINT DEFAULT 18,
max_age_pref SMALLINT DEFAULT 99,
max_dist_km SMALLINT DEFAULT 50,
elo_score FLOAT DEFAULT 1500, -- desirability score
is_active BOOLEAN DEFAULT TRUE,
last_active_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Indexes
CREATE INDEX idx_users_geohash ON users (geohash6) WHERE is_active = TRUE;
CREATE INDEX idx_users_last_active ON users (last_active_at DESC);#Swipe Events (Cassandra β partitioned by swiper_id, clustered by ts DESC)
CREATE TABLE swipes (
swiper_id UUID,
ts TIMESTAMP,
target_id UUID,
direction TEXT, -- LEFT | RIGHT | SUPER_LIKE
PRIMARY KEY (swiper_id, ts)
) WITH CLUSTERING ORDER BY (ts DESC)
AND default_time_to_live = 7776000; -- 90 days TTL
-- Separate table for fast "did A already swipe B?" lookups
CREATE TABLE swipe_lookup (
swiper_id UUID,
target_id UUID,
direction TEXT,
swiped_at TIMESTAMP,
PRIMARY KEY (swiper_id, target_id)
);#Matches (PostgreSQL β append-only)
CREATE TABLE matches (
match_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_a UUID NOT NULL REFERENCES users(user_id),
user_b UUID NOT NULL REFERENCES users(user_id),
conversation_id UUID,
matched_at TIMESTAMPTZ DEFAULT NOW(),
CONSTRAINT ordered_pair CHECK (user_a < user_b)
);
CREATE INDEX idx_matches_user_a ON matches (user_a, matched_at DESC);
CREATE INDEX idx_matches_user_b ON matches (user_b, matched_at DESC);#Candidate Stack Cache (Redis β pre-computed per user)
Key: stack:{user_id}
Type: Redis List (RPUSH / LPOP)
Value: JSON-encoded candidate profile (2 KB each)
TTL: 300 seconds (5 min); refill triggered when len < 5
Key: seen:{user_id}
Type: Redis Bloom Filter (RedisBloom module)
Cap: 50,000 entries, 1% FPR
TTL: 30 days (sliding)
Key: loc:{user_id}
Type: Redis GEOADD (sorted set under hood)
Usage: GEORADIUS queries for nearby users
TTL: 10 minutes (refreshed on each location update)#Access Patterns
| Query | Store | Key / Index |
|---|---|---|
| Fetch candidate stack | Redis | stack:{user_id} LPOP |
| Check if user seen | Redis | seen:{user_id} BF.EXISTS |
| Find nearby users by geohash | PostgreSQL | idx_users_geohash on geohash6 |
| Record swipe | Cassandra | partition swiper_id |
| Check mutual swipe | Cassandra | swipe_lookup (swiper_a, swiper_b) |
| User profile lookup | PostgreSQL | PRIMARY KEY user_id |
| List user's matches | PostgreSQL | idx_matches_user_a/b |
#5. High-Level Design (HLD) & Scale Evolution
#Stage 1: MVP / Startup Scale (100K DAU, 1 region)
- Single PostgreSQL for users + swipes + matches.
- On feed load: query PostgreSQL for nearby users (lat/lon bounding box), filter by preferences, rank by recency, return top 20.
- Bottleneck: PostgreSQL handles all geo queries; no pre-computation; slow at 100K+ users.
#Stage 2: Growth Scale (10M DAU, 2 regions)
- Introduce Redis for candidate stacks (
stack:{user_id}list). - Introduce Kafka for async swipe processing and match detection.
- Move swipe history to Cassandra.
- Add a background Stack Builder service to pre-compute and cache stacks.
- Bottleneck: Stack Builder is single-threaded; geohash cell granularity must be tuned; no ML ranking.
#Stage 3: Tinder Scale (75M DAU, global, ML-ranked)
#Architecture Diagram
ββββββββββββββββ
Mobile App β API Gateway β (rate limiting, auth, routing)
β β + Load Balancerβ
β ββββββββ¬ββββββββ
ββββββ β
β βββββββββββββββββΌβββββββββββββββββ
β βΌ βΌ βΌ
β βββββββββββββββ βββββββββββββ ββββββββββββββββ
β β Feed Service β βSwipe Svc β βLocation Svc β
β β (read heavy) β β(write hvy)β β(geo updates) β
β ββββββββ¬βββββββ βββββββ¬ββββββ ββββββββ¬ββββββββ
β β β β
β ββββββββΌβββββββ β βββββββΌβββββββ
β β Redis β β β Redis GEO β
β β stack:{uid}β β β loc:{uid} β
β β seen:{uid} β β β GEORADIUS β
β βββββββββββββββ β βββββββ¬βββββββ
β β β
β βββββββββββββΌβββββββ β
β β Kafka β β
β β topics: β β
β β - swipe-events β β
β β - location-upd β β
β ββββ¬βββββββββββ¬βββββ β
β β β β
β ββββββββββββΌβββ βββββΌβββββββββββ β
β β Match Detectβ β Stack Builder ββββ
β β Service β β (Flink job) β
β ββββββββ¬βββββββ ββββββββ¬ββββββββ
β β β
β ββββββββΌβββββββ ββββββββΌβββββββββββ
β β PostgreSQL β β PostgreSQL β
β β matches β β users / geohash β
β βββββββββββββββ ββββββββββββββββββββ
β
β ββββββββββββββββββββββββββββββββββββ
βββββββΊβ Cassandra (swipe_events, lookup) β
ββββββββββββββββββββββββββββββββββββ
Photos: S3 β CloudFront CDN β Mobile App (not in app-server path)#Component Breakdown
| Component | Responsibility | Tech Choice |
|---|---|---|
| API Gateway | Auth, rate-limit, TLS termination | Envoy / Kong |
| Feed Service | Serve pre-computed stack from cache | Go (stateless, horizontally scaled) |
| Swipe Service | Write swipe events, trigger match check | Go + Kafka producer |
| Location Service | Update Redis GEO, publish location events | Go |
| Stack Builder | Geo-query β filter β ML rank β push to Redis | Apache Flink job |
| Match Detect Service | Kafka consumer, checks mutual swipe in Cassandra | Go |
| Notification Service | Push FCM/APNs on match | Go + FCM/APNs |
| User DB | Profile storage, geohash index | PostgreSQL (sharded) |
| Swipe Store | High-write swipe log + lookup | Cassandra |
| Candidate Cache | Pre-computed stacks + seen-set | Redis Cluster |
| Photo Storage | Profile photos | S3 + CloudFront CDN |
#Data Flow
Feed Read Path:
Client β Feed Service
β Redis LPOP stack:{user_id} (hit: return 20 profiles, ~1ms)
β If stack empty:
β Stack Builder triggered (async refill)
β Fallback: query PostgreSQL geohash cells directly (slower, ~50ms)
β Return candidates with CDN photo URLsSwipe Write Path:
Client β Swipe Service
β Redis BF.ADD seen:{user_id} {target_id} (mark as seen)
β Cassandra INSERT swipe_lookup (swiper, target) (durable record)
β Kafka PRODUCE swipe-events topic (async processing)
β Return ack to client immediately
Kafka Consumer (Match Detect):
β On RIGHT/SUPER_LIKE: Cassandra SELECT swipe_lookup (target, swiper)
β If mutual match found:
β PostgreSQL INSERT matches
β Kafka PRODUCE match-events
β Notification Service β FCM/APNs push to both usersLocation Update Path:
Client β Location Service
β Redis GEOADD loc-index {lat} {lon} {user_id}
β Kafka PRODUCE location-updates
β Stack Builder (Flink) consumes β triggers async stack rebuild#6. Deep Dive β Core Components
#Candidate Stack Builder β Detailed Design
This is the heart of the system. It runs as an Apache Flink streaming job.
Input triggers:
- Location update event (user moved > 500m)
- Stack exhaustion signal (Feed Service detects < 5 profiles left)
- Periodic refresh (every 5 minutes for active users)
Algorithm:
1. GEOGRAPHIC CANDIDATE FETCH
- Decode user's geohash6 cell (β1.2km Γ 0.6km)
- Query target cells = home_cell + 8 neighbors
- Expand to geohash5 neighbors if count < 50 candidates
- SQL: SELECT user_id FROM users WHERE geohash6 IN (cells)
AND is_active = TRUE AND last_active_at > NOW() - INTERVAL '7 days'
2. PREFERENCE FILTER
- Filter by age range (birth_date between min/max pref)
- Filter by gender preference
- Filter by distance (Haversine check for precise km)
3. SEEN-SET FILTER
- BF.MEXISTS seen:{user_id} candidate_ids[]
- Drop candidates already in seen-set
4. ML RANKING
- Batch call to Ranking Service (gRPC)
- Features: ELO scores, photo engagement rate, activity recency, filter match %
- Returns scored list, sort DESC by score
5. PUSH TO CACHE
- Redis DEL stack:{user_id}
- Redis RPUSH stack:{user_id} ranked_candidates (top 50)
- Redis EXPIRE stack:{user_id} 300#Match Detection Service β Detailed Design
Kafka consumer group. Scales by partition count on swipe-events topic.
- Partition key: swiper_id (ensures same-user swipes ordered)
- On consuming RIGHT swipe (user_a β user_b):
1. Cassandra SELECT direction FROM swipe_lookup
WHERE swiper_id=user_b AND target_id=user_a
2. If direction = RIGHT or SUPER_LIKE:
β Dedup check: Redis SET NX match:{min(a,b)}:{max(a,b)} TTL=60s
β If acquired lock: PostgreSQL INSERT matches
β Kafka PRODUCE match-events β Notification ServiceWhy Cassandra for swipe_lookup?
2B swipes/day = 23K writes/s. Cassandra's LSM-tree handles this trivially with tunable consistency (QUORUM for match reads).
#Scaling Strategy
| Component | Strategy |
|---|---|
| Feed Service | Stateless; horizontal scale; read from Redis |
| Swipe Service | Stateless; Kafka producer; horizontal scale |
| Stack Builder (Flink) | Increase Flink task slots; partition by user_id |
| Redis Candidate Cache | Redis Cluster (16-shard); evict LRU inactive users |
| Cassandra Swipe Store | 12-node ring; RF=3; QUORUM reads for match check |
| PostgreSQL Users | Read replicas per region; shard by user_id hash |
| Kafka | 200 partitions on swipe-events; 50 on location-updates |
#Caching Strategy
Layer 1 β Client cache
20 profiles prefetched on device; swiped from local cache
β Eliminates per-swipe network round-trips
Layer 2 β Redis Candidate Stack (stack:{user_id})
50 profiles per user; LPOP on each swipe; TTL 5 min
β Serves 99%+ of feed requests at ~1ms
Layer 3 β Redis GEO Index (loc:{user_id})
GEORADIUS queries for Stack Builder candidate fetch
β TTL 10 min per location key
Layer 4 β Redis Bloom Filter (seen:{user_id})
~72KB per user; 1% FPR; avoids redundant DB seen-set queries
β Prevents showing already-swiped profiles
Layer 5 β PostgreSQL read replicas
Cache-aside for profile metadata on cache miss
β P99 < 20ms with connection pooling (PgBouncer)#Consistency Models
| Data | Model | Rationale |
|---|---|---|
| Swipe recording | Eventual (Cassandra QUORUM write) | Swipe loss rare; at-least-once via Kafka DLQ |
| Match detection | Strong (Cassandra QUORUM read + Redis dedup lock) | Must not miss or double-notify a match |
| Candidate stack | Eventual (Redis cache, 5-min TTL) | Stale profile data acceptable |
| User profile | Strong (PostgreSQL primary) | Profile changes must be immediately visible |
| Location index | Eventual (Redis GEO, 10-min TTL) | Slight geo staleness acceptable |
#7. Low-Level Design β Core Functionality
#Key Algorithms
Algorithm 1: Geohash Candidate Fetch with Expansion
import geohash2
from math import radians, cos, sin, asin, sqrt
def fetch_candidates(user: User, db, min_count: int = 50) -> list[str]:
"""
Fetch candidate user_ids using geohash cell expansion.
Starts with geohash precision 6 (~1.2km), expands if too few candidates.
"""
precision = 6
while precision >= 4:
home_cell = geohash2.encode(user.lat, user.lon, precision)
neighbors = geohash2.neighbors(home_cell)
cells = [home_cell] + list(neighbors.values()) # 9 cells total
candidates = db.query(
"SELECT user_id, lat, lon FROM users "
"WHERE geohash{p} IN %s AND is_active=TRUE "
"AND last_active_at > NOW() - INTERVAL '7 days' "
"AND user_id != %s".format(p=precision),
params=(tuple(cells), user.user_id)
)
if len(candidates) >= min_count:
break
precision -= 1 # expand search to larger cells
# Precise Haversine filter to user's max_dist_km
return [c for c in candidates if haversine(user.lat, user.lon, c.lat, c.lon) <= user.max_dist_km]
def haversine(lat1, lon1, lat2, lon2) -> float:
"""Returns distance in km between two lat/lon points."""
R = 6371
dlat, dlon = radians(lat2 - lat1), radians(lon2 - lon1)
a = sin(dlat/2)**2 + cos(radians(lat1)) * cos(radians(lat2)) * sin(dlon/2)**2
return 2 * R * asin(sqrt(a))Algorithm 2: ELO-Style Desirability Score Update
K_FACTOR = 32 # how fast scores change
def update_elo_scores(swiper: User, target: User, direction: str, db) -> None:
"""
After a swipe, update ELO scores for both users.
RIGHT swipe = swiper "chose" target β target's score increases.
LEFT swipe = target "lost" β minor negative for target.
"""
expected_right = 1 / (1 + 10 ** ((swiper.elo - target.elo) / 400))
if direction == "RIGHT":
actual = 1.0 # target won this encounter
elif direction == "SUPER_LIKE":
actual = 1.2 # weighted win
else: # LEFT
actual = 0.0
# Target gains/loses rating; swiper rating unchanged on pass
target_delta = K_FACTOR * (actual - expected_right)
new_target_elo = target.elo + target_delta
db.execute(
"UPDATE users SET elo_score = %s WHERE user_id = %s",
(max(100, min(3000, new_target_elo)), target.user_id)
)
# Publish to Kafka for async ML feature store update
publish_elo_event(target.user_id, new_target_elo)Algorithm 3: Stack Refill Trigger with Debounce
import redis
import time
def trigger_stack_refill(user_id: str, r: redis.Redis, flink_producer) -> None:
"""
Debounced refill trigger β prevents thundering herd when
many swipes drain the stack simultaneously.
"""
lock_key = f"refill_lock:{user_id}"
# Only one refill per 30s per user
acquired = r.set(lock_key, "1", nx=True, ex=30)
if not acquired:
return # Refill already in-flight
stack_size = r.llen(f"stack:{user_id}")
if stack_size < 5:
# Signal Flink job via Kafka
flink_producer.produce(
topic="stack-refill-requests",
key=user_id,
value={"user_id": user_id, "ts": time.time()}
)#Design Patterns Used
| Pattern | Where | Why |
|---|---|---|
| CQRS | Feed (read) vs Swipe (write) services | Independent scaling; read path is Redis-only |
| Fan-Out | Stack Builder pre-computes stacks proactively | O(1) feed read at cost of O(N) write-time computation |
| Circuit Breaker | Feed Service β PostgreSQL fallback | If Redis is down, fall back to direct DB with degraded latency |
| Bloom Filter | seen:{user_id} in Redis |
Probabilistic seen-set; 72 KB vs 400 KB for exact set |
| Event Sourcing | Kafka swipe-events as source of truth | Replay for ML training, audit, and match recomputation |
| Geo-Hashing | Candidate fetch + geohash6 DB index | O(1) geohash lookup vs O(N) lat/lon scan |
#8. Failure Handling & Edge Cases
#What Happens When X Fails?
| Failure | Impact | Mitigation |
|---|---|---|
| Redis cluster node down | Stack cache miss for subset of users | Redis Cluster auto-failover to replica; Feed Service falls back to PostgreSQL geo-query (slower but correct) |
| Kafka broker down | Swipe events buffered in producer | Kafka replication RF=3; producer retries with exponential backoff; Swipe Service returns 202 Accepted to client |
| Match Detect consumer lag | Match notification delayed, not lost | Consumer group offset-tracked; DLQ for poison messages; SLA: match notify < 2s P95 |
| Stack Builder Flink crash | Stacks go stale after 5-min TTL | Flink checkpointing every 30s; job restarts from last checkpoint; staleness bounded by TTL |
| PostgreSQL primary failover | Write unavailability for ~30s | Patroni-managed automatic failover; Swipe Service buffers to Kafka; writes replayed on recovery |
| Cassandra node failure | Reduced replication factor | RF=3 tolerates one node loss per rack; QUORUM reads/writes still succeed with 2/3 nodes |
| Location Service crash | Stale geolocation | Redis GEO TTL-10min; on expiry, Stack Builder uses last known geohash from PostgreSQL |
#Domain-Specific Edge Cases
- Celebrity Profiles (10M+ impressions): If a celebrity user has massive right-swipe volume, Cassandra
swipe_lookuphot partition risk. β Mitigation: Shardswipe_lookupby(swiper_id % 32, target_id)for high-volume targets. - Exhausted Candidate Pool: User in rural area has < 5 nearby profiles. β Progressively loosen filters: distance Γ2, age range +5yr, then show "expand search" prompt.
- Duplicate Swipe: Client retries a swipe (network flaky). β Cassandra
swipe_lookupINSERT is idempotent (same PK = overwrite); Kafka produces dedup'd onswipe_idUUID. - Simultaneous Mutual Swipe: Both users swipe RIGHT within milliseconds. β Redis
SET NX match:{min}:{max}dedup lock prevents double-insert into PostgreSQL matches. - Inactive User in Candidate Pool: Profile cached in another user's stack; user deactivates. β Feed Service checks
is_activeflag before returning profile to client; skips if deactivated. - Seen-Set Bloom Filter False Positive (1% FPR): Valid candidate incorrectly marked as seen. β Acceptable trade-off; 1% miss rate vs 82% storage savings over exact set.
#9. Monitoring & Observability
#Key Metrics
| Metric | Target | Alert Threshold |
|---|---|---|
| Feed P99 latency | < 200 ms | > 350 ms for 5 min |
| Swipe ack P99 latency | < 50 ms | > 100 ms for 2 min |
| Redis stack cache hit rate | > 95% | < 90% for 5 min |
| Match notification P95 latency | < 2 s | > 5 s for 3 min |
| Kafka consumer lag (match-detect) | < 10K msgs | > 100K msgs |
| Cassandra write latency P99 | < 10 ms | > 25 ms |
| Stack Builder job uptime | 99.9% | Any outage > 60s |
| Seen-set Bloom FPR (sampled) | < 1% | > 3% |
#Alerting Strategy
P0 β Page immediately (any time)
- Feed Service P99 > 500ms for 3+ minutes
- Match notification completely down (Kafka consumer group dead)
- Redis cluster loss of > 1 shard (stack cache unavailable for users)
- Cassandra cluster losing QUORUM
P1 β Page during business hours
- Stack cache hit rate drops below 90%
- Kafka consumer lag > 100K messages for 10+ min
- Stack Builder Flink job restarting repeatedly
- PostgreSQL replication lag > 60s
P2 β Ticket, no page
- Feed P99 > 300ms but < 500ms
- Bloom filter FPR creeping above 2%
- Individual Cassandra node latency elevated
- Location update throughput anomaly
#SLAs / SLOs
Feed Load Time:
SLO: P50 < 80ms, P95 < 150ms, P99 < 200ms
SLA: 99.9% of requests within 200ms over 30-day window
Swipe Acknowledgement:
SLO: P99 < 50ms
SLA: Swipe events durable (no loss) β at-least-once delivery
Match Notification:
SLO: P95 < 2s from mutual swipe to push notification
SLA: 99.99% of matches notified within 30s
System Availability:
SLA: 99.99% uptime = < 52 min downtime/year#10. Trade-off Summary
| Decision | Chose | Over | Because |
|---|---|---|---|
| Pre-computed candidate stacks (push) | Fan-out-on-write to Redis | Fan-out-on-read at query time | O(1) feed reads at Tinder's 1.2K QPS; acceptable 5-min staleness |
| Cassandra for swipe store | Cassandra | PostgreSQL | 23K writes/s peak; LSM-tree handles write-heavy workload; no ACID needed for swipes |
| Geohash6 for geo-partitioning | Geohash6 (~1.2km cells) | PostGIS ST_DWithin | Geohash enables O(1) Redis lookup and simple string-prefix DB index; avoids full geo scan |
| Redis Bloom Filter for seen-set | Probabilistic (1% FPR) | Exact Redis Set | 82% memory savings (72 KB vs 400 KB per user Γ 10M active = 720 GB vs 4 TB) |
| Kafka for swipe pipeline | Async Kafka | Sync dual-write | Decouples swipe ack latency from Cassandra write + match check; enables event replay |
| Match dedup via Redis NX lock | Redis atomic SET NX | DB unique constraint alone | Sub-millisecond check; prevents double-notification race without DB round-trip |
| ELO-style scoring | ELO (lightweight) | Full ML model in sync path | ELO updates async; ML ranking runs only in Stack Builder (offline from swipe path) |
| PostgreSQL for user profiles | PostgreSQL + geohash index | Cassandra or DynamoDB | Profile data is read-heavy, structured, low write rate; ACID for preference updates matters |
#11. Extensions & Follow-ups
#What Would You Add With More Time?
- Real-time ML Ranking: Replace ELO with a two-tower neural network (user embedding Γ candidate embedding). Run inference in Stack Builder as a gRPC call to a TensorFlow Serving cluster.
- A/B Testing Framework: Shadow-rank candidates with multiple models; assign users to experiment arms; measure right-swipe rate and match rate as primary metrics.
- Boost Feature: During "Boost," temporarily elevate user's ELO for candidate ranking, and prioritize them in more stacks. Implement as a TTL flag in Redis.
- Cross-Region Passport Mode: When user sets travel mode, replicate geohash lookup to target region's user shard. Stack Builder switches to target geohash cells.
- Undo Swipe: Keep a per-user LRU ring buffer (last 3 swipes) in Redis; Cassandra UPDATE on undo; remove from partner's seen-set (Bloom false-negative acceptable).
- Candidate Diversity Enforcement: After ML ranking, post-process with MMR (Maximal Marginal Relevance) to prevent showing 20 near-identical profiles.
#How Would This Change at 100x Scale?
- 7.5B DAU equivalent: Geohash6 cells in dense cities (Manhattan) would have millions of users per cell β move to geohash8 for inner-city users, geohash5 for rural.
- Stack Builder bottleneck: Flink parallelism alone insufficient β shard users across multiple Flink clusters by geography; use separate Kafka consumer groups per region.
- Redis memory (100 TB): Move cold stacks to a tiered cache (Redis + Memcached for overflow); evict stacks for users inactive > 30 min to RAM; keep bloom filters on NVMe-backed Redis.
- Cassandra swipe store (2PB/year): Introduce time-based compaction and S3 archival for swipes older than 90 days; only recent 90 days in hot Cassandra nodes.
- Multi-region active-active: Each region owns its geo-sharded user subset; cross-region match detection via global Kafka MirrorMaker with conflict resolution on match_id.
#12. Cross-References
#Related Topics in This Repo
| Topic | Connection |
|---|---|
| Recommendation Engine (#23) | ML ranking, two-tower models, feature store patterns |
| Fraud Detection (#24) | Fake profile detection, bot swiping anomaly detection |
| E-Commerce (#11) | Fan-out for personalized feeds, CQRS read/write separation |
| Web Crawler (#12) | Bloom filters for deduplication, consistent hashing for partitioning |
| URL Shortener (#1) | Redis caching patterns, consistent hashing |
#Building Blocks Used
- Redis β Candidate stack cache (List), seen-set (Bloom Filter), geo index (GEOADD), match dedup lock (SET NX)
- Cassandra β High-throughput swipe event log and lookup table
- PostgreSQL β User profiles with geohash index; matches table
- Kafka β Swipe event pipeline, location updates, match events
- Apache Flink β Stack Builder streaming job (geo query β filter β rank β cache)
- S3 β Profile photo storage
- CDN β Low-latency global photo delivery to mobile clients
- Load Balancer β Traffic distribution across Feed, Swipe, Location services
- Geospatial Index β Geohash-based proximity candidate lookup
#Concepts Used
- Geo-Hashing β 6-char geohash cells for geographic bucketing and DB indexing
- Fan-Out β Push-model candidate stack pre-computation (fan-out-on-write)
- CQRS β Separate Feed (read-only Redis) and Swipe (write-heavy Cassandra) services
- Bloom Filters β Probabilistic seen-set per user in Redis (72 KB, 1% FPR)
- Sharding β PostgreSQL users sharded by user_id; Cassandra partitioned by swiper_id
- Consistency Models β Eventual for feed/location, strong for match detection
- Event Sourcing β Kafka swipe-events as immutable source of truth for ML training and audit
- Circuit Breaker β Feed Service degrades to direct DB query if Redis is unavailable