Software system design — final

3 min readAug 29, 2020

in previous articles we talked some design tricks by different layers —




Now lets piece them together


First thing come first . this is the most important thing to keep in mind before every ‘perfect’ picture paint .

want database go cluster from single node ? need to maintain multiple data nodes start/stop ;data replication , think about consistency ,data migration .if master-master model even more to think .

move from SQL to no-SQL Sharding? think about which column to shard, hash key consistency ,data migration ,data consistency .

drop monolith go micro-service ? think about how service talk, should introduce RPC framework for internal api call or define topic in message broker? debug for each service ,how to see each service log ,deployment for all service or group of service ,use what database for which service , which services can be containerized ?which on cloud and which on premise..this can keep growing . so usually we do not even think to go micro-service.

Software design always should grow with business model .premature or early optimization introduce unnecessary headache going to be faced everyday .

there is no ‘one for all’ solution in software design .and we should grow our system as we need-partially and slowly . e.g. Database read slow ,simply add cache node see how it helps and instead of directly go database cluster or Sharding .

Below points from different concern ,could use as a ‘checklist’ when design your system .


Load Balancer

hardware and software balancer


not only resolve host ,also could balance the load before request hits IP server

Geo-based CDN

static hosting: html/js/css/font/image/video

Rate Limiting/Throttling

limit the number of request hits (use bucket token algorithm )

Request filtering

Front-end Cache

query/api result

Read API

(distributed) Cache

Write API

Message Broker


Parallel processing(map reduce framework)


apache hadoop

Database Sharding

distributed id : mongoDB objectId/base62



offload it in Api gateway for performance .internal api call can choose one RPC framework only if necessary

data encryption

encrypt sensitive or personal data in data store .

Audit (traffic) log

Blacklist/whitelist host or ip

Circuit breaker

# failed request > threshold ,break



For every availability design first thing come in mind should be replication . server node replication,data node replication, etc .


we have to trade off more data consistency . e.g. mariadb galera


1 master and N slave nodes ;one way data sync ;master for write, slave for read .

Fail over

multiple data center setup


Active-stand by

Check pointing

take snapshot on system state

Dead Letter queue

store the failed request and retry later

Retry Mechanism

Exponential back off

attempt < max; wait longer time than previous.


add randomness second/minute to each retry interval .



Read may not see update. e.g. memcached .


Read will see correct value in last (step) commit ;data replicated asynchronously .e.g. DNS/email

we pick eventual consistency as usual


Read always see correct value, in every step(commit);data replicated synchronously .


data base lock : for data write consistency

cache lock : for data read consistency

AI — make your system smart

after system running years ,there will be data generated in your system .could be used to train models building prediction/classification/automation features to make business workflow smoother.

and this is the last part .




A channel which focusing on developer growth and self improvement