Read API
server the get requests .our focus is to make the query fast and data accurate .
Cache
usually here we put redis or memcached cluster . key difference :
redis support more data structures (hash, list, array etc);memcached is simple key-string storage
redis value size per key is 512M;memcached is 1MB
redis single threaded with IO multiplexing (build socket connection to reuse I/O channel);memcached is multi-threaded
Both of them got cluster solution
Write API
#1 simple Api (async all the way to database)
#2 Write request to queue(consider message broker)-> worker process(service)
#3 parallel processing
Parallel processing (map reduce/partition aggregate)
kafka :streaming/general partitioning purpose
big data: hadoop spark/storm
Message broker
RabbitMQ/Kafka
key difference : kafka is partition framework, it’s fast;log everything (use zero copy so it is fast), support at least once and exact once,come with zookeeper ,more complex compare to rabbitmq ,should keep in mind make zookeeper as cluster also;rabbit is good for general purpose MQ, message will be done once consumed ,simple to setup .
General design
