in previous articles we talked some design tricks by different layers —
Now lets piece them together
First thing come first . this is the most important thing to keep in mind before every ‘perfect’ picture paint .
want database go cluster from single node ? need to maintain multiple data nodes start/stop ;data replication , think about consistency ,data migration .if master-master model even more to think .
move from SQL to no-SQL Sharding? think about which column to shard, hash key consistency ,data migration ,data consistency .
drop monolith go micro-service ? think about how service talk, should introduce RPC framework for internal api call or define topic in message broker? debug for each service ,how to see each service log ,deployment for all service or group of service ,use what database for which service , which services can be containerized ?which on cloud and which on premise..this can keep growing . so usually we do not even think to go micro-service.
Software design always should grow with business model .premature or early optimization introduce unnecessary headache going to be faced everyday .
there is no ‘one for all’ solution in software design .and we should grow our system as we need-partially and slowly . e.g. Database read slow ,simply add cache node see how it helps and instead of directly go database cluster or Sharding .
Below points from different concern ,could use as a ‘checklist’ when design your system .
hardware and software balancer
not only resolve host ,also could balance the load before request hits IP server
static hosting: html/js/css/font/image/video
limit the number of request hits (use bucket token algorithm )
Parallel processing(map reduce framework)
distributed id : mongoDB objectId/base62
offload it in Api gateway for performance .internal api call can choose one RPC framework only if necessary
encrypt sensitive or personal data in data store .
Audit (traffic) log
Blacklist/whitelist host or ip
# failed request > threshold ,break
For every availability design first thing come in mind should be replication . server node replication,data node replication, etc .
we have to trade off more data consistency . e.g. mariadb galera
1 master and N slave nodes ;one way data sync ;master for write, slave for read .
multiple data center setup
take snapshot on system state
Dead Letter queue
store the failed request and retry later
Exponential back off
attempt < max; wait longer time than previous.
add randomness second/minute to each retry interval .
Read may not see update. e.g. memcached .
Read will see correct value in last (step) commit ;data replicated asynchronously .e.g. DNS/email
we pick eventual consistency as usual
Read always see correct value, in every step(commit);data replicated synchronously .
data base lock : for data write consistency
cache lock : for data read consistency
AI — make your system smart
after system running years ,there will be data generated in your system .could be used to train models building prediction/classification/automation features to make business workflow smoother.
and this is the last part .