Scaling

Scaling MySQL in the cloud with Vitess and Kubernetes – Percona Live 2015

moderndba MySQL , , , ,

“Scaling MySQL in the cloud with Vitess and Kubernetes” by Anthony Yeh from Google.  He works on the YouTube team at Google.  The talk was basically how they use Vitess and their container platform (Kubernetes).  Vitess provides

  • Performance
  • Protection
  • Monitoring/Analysis
  • Mangement
  • Sharding

It sits between the application and database layer. It leverages a vtgate which abstracts the data layer and what shard to go get the information from.  Can separate live traffic from analytic traffic.  Vitess can cache based on primary key identified rows.  This is different from the block caching performed by MySQL.  Vitess will kill poorly performing queries automatically to avoid replicating bad queries.  It performs transparent rerouting.  It also performs live resharding.

Vitess is a container cluster.  Kubernates is a Container Cluster OS built on top of Docker.  For more info see vitess.io/getting-started.  Looks very promising and will need to see how this fits with Trove.

Sharding, horizontal and vertical scaling – Percona Live 2015

moderndba MySQL , , ,

In “Sharding, horizontal and vertical scaling“, Tom Christ from Tumblr discussed sharding for scale.  He discussed scale dimensions of read, write, and size.  While challenges are often specific to data sets and/or applications, there are ways to categorize and reason about them.  Read challenges include query complexity, query volume, and buffer pool effectiveness.  To fix read issues you can scale horizontally with replica fanout, or you can add caching.  You can also scale vertical with bigger servers, or leverage smaller concurrent queries.

He discussed the options for write and size scaling and this is where sharding comes into help.  He discussed the trade off between costs, complexity, write amplification, and concurrency when deciding how to scale and if sharding should be part of your solution.  Tom discussed the types of sharding lookup, range, and  mod sharding.  He also went over forms of caching: key/value (memcache) and structured (redis).

All in all a good overview of issues with scale and how sharding and cache can help and the pitfalls.