Table of contents
Requirements
Design a system where the user
can upload video
can watch videos
will see a live feed on the landing page
Load calculations
Assumptions
Total number of users watching videos per day 200M
Total number of users uploading videos per day 2M
Average file size 200MB
Max file life 50 years
Average file meta 10MB ( Thumbnails, Title, and other analytical data )
Storage
File size per day = 2M x (200MB + 10MB)
\= 2M x ( 210MB )
\= 0.42k x M x MB
\= 0.42 x B X MB
\= 0.42 x 10^9 x MB
\= 0.42 PB
Total Storage = 0.42 PB x File Life
\= 0.42PB/day x 50y
\= 0.42PB x 50 x 365
~ 8,000PB
\= 8 EB
Network calls
Read
Read request per sec = 200M/86400
\= 0.2M/86.4
\= 2.4k req/sec
Write
Write request per sec = 2M/86400
\= 24 req/sec
Bandwidth
Read = 2.4k x 200M
~ 500 GB / sec
Write = 24 x 200M
0.5 GB / sec
System design
Our system will handle tons of read-and-write requests every second so to optimize the experience and cost of infrastructure we can reduce consistency. This means if someone uploads a video we need to instantly publish it to 100% of our users.
We will divide our infra into multiple microservices
Video upload process
App server - It handles users requests and then forwards metadata to "Meta data" service and pushes video to the queue
Meta data - It is a service where data like user profile, video title, and all get saved. We can use SQL or Non-SQL DB to store metadata. SQL DB will be cheap and consistent but will be slower in speed as compared to No SQL DB. We can use NoSQL dbs where sharding is easy to handle scale or we can just use open source solutions like Vitess that make sharding on SQL dbs easy.
Queue - The queue will pass a specific number of videos to the "video encoder". We will design our "Video encoder" service to handle specific numbers of requests only. All the pending requests will be kept inside queue and when ever "video encoder" service is free it will fetch the next video from the queue
Video encoder - We will have some specific algorithm that will encode videos so that it will take less space and give optimal quality. Some video formats are good for mobile and some are optimized for desktop. We can also store videos in multiple resolutions like 1020p, and 720p the one which is most used. For example, youtube will convert uploaded files to MP4 format. ( we can also ask users to upload in some specific format only if our encoding algorithm is public )
Video chunks - We will break 1 video into multiple chunks. Like 10 mins videos can be divided into 10 1min videos. So when the user requests for video we will load the first 1-minute video which will be faster as compared to loading 10 10-minute videos. When a user is playing the first 1-minute video we will load the next chunk. Also in some cases where the user will skip to a specific time in video, we will only be loading the target chunk instead of loading the full video.
Object DB - We can store video files in the object DB database which is specifically built for storing static media files. Where data is stored once and read many times. It can handle scale and is fast.
Other options
File storage but for this you will need to know exact path of file where its stored and can become complex and slow when you scale.
Block storage it will break the file into multiple chunks and will store ID to search and retrieve the full file. It is fast and reliable and can be used for mission-critical cases. But on the other hand, is expensive and is mainly use where we need consistency and its not required in this project.
Video watch process
Protocol - We will use TCP over UDP as UDP is good for streaming data like live cricket matches. UDP can miss chunks and we can not go back and load them again whereas in TCP it's possible. It will come as a loading time cost as TCP needs more time to load.
We will use CDN to cache the Latest videos or the most played videos to load videos faster and reduce the load on DB. CDNs are a great way to reduce loading time as they are geographically available closer to the user. We can use LFU ( Least Frequently Used ) technique to store cache.
We can also publish videos inclemently without pushing it to all users. So in case of viral videos or videos of creators with more subscribers, our server resources won't spike up. Let's say we will publish that app to 10% of subscribers every 15 mins till 100% of users are covered.
Video feeds [WIP]