Polyglot Persistence: NoSQL & RDBMS

Connecting Redis to a Persistent Database like MySQL

While Redis can persist its data using RDB or AOF, it’s often used as a caching or intermediate layer in conjunction with a relational (e.g., PostgreSQL, MySQL) or NoSQL (e.g., MongoDB) database. Here’s how:

Two-Layer Design: Redis + Database

1. Redis as the Cache Layer
   • Leaderboard data (e.g., photo scores, votes, views) is kept in Redis for fast access.
   • Votes and views are immediately updated in Redis and displayed to users in real time.
2. Database as the Source of Truth
   • Periodically sync Redis data back to your persistent database (e.g., once every hour or on demand).
   • If Redis crashes or you need to rebuild the leaderboard, you can repopulate it from the database.

Workflow Example

• User Votes for a Photo:
  1. The vote is recorded in Redis (e.g., incrementing the score in a ZSET).
  2. Optionally, the vote is asynchronously logged to the database for long-term storage.
• Leaderboard is Displayed:
  • Data is fetched directly from Redis, ensuring low-latency performance.
• Sync Back to Database:
  • A background job runs periodically to write Redis scores back to the database, ensuring persistence.

Advantages of Connecting Redis to a Database

• Prevents data loss in case Redis crashes, even if persistence mechanisms fail.
• You can perform complex, historical queries in the database (e.g., tracking trends over time).
• Redis stays lightweight, focusing on real-time performance.

Redis Persistence Mechanisms

1. RDB (Redis Database Snapshotting)
   • Redis periodically takes snapshots of the data and writes them to disk.
   • Use Case: Best for scenarios where occasional data loss is acceptable, like a leaderboard that could be rebuilt from the database in case of failure.
   • Pros: Low overhead on write operations, as snapshots are infrequent.
   • Cons: Potential data loss if Redis crashes between snapshots.
2. AOF (Append-Only File Logging)
   • Redis logs every write operation (like adding a vote or view) to a file, ensuring durability.
   • Use Case: Best for scenarios requiring high durability (e.g., real-time voting systems).
   • Pros: Less data loss risk (you can replay logs to recover).
   • Cons: Slightly slower than RDB due to continuous logging.
3. Hybrid (AOF + RDB)
   • Combine both RDB snapshots and AOF logging for a balance of performance and durability.
   • Default in Redis 4.0+

Use Cases

1. Social Networking Sites: Social platforms like Twitter use Redis Lists to populate their timelines or homepage feeds, and can customize the top of their feeds with trending tweets or stories.
   
2. RSS Feeds: We can create news feeds from custom sources where you can pull the latest updates and allow interested followers to subscribe to their favorite RSS feed.
   
3. Leaderboards: This is more of high use cases where Forums like Reddit and other voting platforms utilize Redis Lists to add articles to the leaderboard and even sort them by most viewed and voted entries.

Conclusion

• If high durability is critical (e.g., you can’t lose votes or views even in rare cases), combine Redis with a database. Redis will handle the real-time ranking, while the database will act as the source of truth.
• If simplicity and performance are more important, rely on Redis with AOF and RDB for persistence and durability.