OpenWorld 2017: The State of the Dolphin & Customer Experiences

Session on where MySQL is at and what is coming up.

  • MySQL is #1 database for the web
  • 2nd most popular behind Oracle (DB Engines is the source for this)
  • Most popular according to stack overflow
  • MySQL 5.7
    • 3x better performance
    • replication enhancements
    • json
    • improved security
  • MySQL InnoDB Cluster
    • mysql group replication
    • mysql router
    • mysql shell
  • MySQL 8.0
    • Data Dictionary
    • Roles
    • Unicode 9
    • CTEs
    • Window Functions
  • Cloud information about Oracles cloud
    • Freudian slip, as the speaker said cloud would make us more expensive (he meant responsive and quickly correct it)
    • Oracle cloud machine info
      • Has MySQL Cloud service on it
  • Facebook
  • Booking
  • Engineering Updates
    • MySQL 8.0
      • JSON Document Store
      • Improved GIS support
      • utf8mb4 is now default
      • New collation based on DUCET, Japanese and Russian added
      • Improved performance
      • MySQL Shell
        • Customizable prompt – includes context and session info
        • Custom font support
        • Persistent command line
        • DBA Admin API
          • used to set up InnoDB cluster
      • Showed how to use syntax to embed json inside SQL Statements
      • Common Table Expressions
      • Window Functions
      • InnoDB Cluster
        • Router – Routes client to a server online with the correct role
        • Group Replication – “virtually synchronous”
      • Better Handling of Hot Row Contention – NOWAIT and SKIP LOCKED
      • Invisible Indexes
        • Hidden to Optimizer
        • For soft delete of index (recycle bin) and staged rollout
      • Performance Schema
        • Histograms
        • Indexes
        • Data Lock Instrumentation
        • SQL Errors instrumentation
        • Variables instrumentation
        • Table plugin
        • Improved defaults
        • over 30x faster
      • Transactional Data Dictionary
        • Common data dictionary for db and inoodb
        • crash-safe & atomic DDL
        • meta-data locking for FK constraints
        • Scalable Information schema
      • Faster Table/Range Scans
      • Histograms
      • Parallel Replication
      • Improved Cost Model
      • Resource Groups
        • bind operations to cores – i.e updated to a # of cores
    • Most application today are mobile first

OpenWorld 2017: What’s New in MySQL NDB Cluster?

This presentation was given by Bernd Ocklin, Senior Director MySQL Cluster Development at Oracle.  It was a good overview of what NDB Cluster is, where it came from and some recent changes.  My notes are:

  • Trade-Offs: Cellular Network
  • Trade-Offs: Online Games – Massive Parallel
  • 99.999% uptime
  • Distributed real-time in-memory network database
  • Runs standalone or as a storage engine for MySQL
  • Auto-sharding based on key
    • transparent between node groups
    • each fragment active in one Data Node with synchronous replication to 2nd data node
  • DBA chooses what part of Primary key to shared on
  • parallelization works even on single queries
  • batches automatically
  • event-driven and asynchronous
  • Active-Active with Geo-Replication in NDB Cluster
    • Update anywhere
    • conflict detection
    • auto-conflict-resolution
  • Async-Replication is for read scale out
  • MySQL Group Replication – innoDB Cluster
    • scale out writes as well as reads (limited write scale out)
  • Moving towards mixing NDB cluster as a node in InnoDB cluster (not there yet)
  • Synchronous locally and use replication for geo-replication (asynchronous)
  • Updates scale pretty much linearly
  • version 7.6 new features
    • makes joins much faster
    • added ndb import to read in csv files
    • MEM integration
    • MySQL 5.7
    • dynamic resource allocation
    • Partial Local Checkpoint – for performance improvements and scalability (now support much larger databases)
    • Multi-treaded backup
  • in MySQL 8.0 they are moving NDB to base MySQL to avoid forking and placing it as a first class citizen
  • MySQL 8.0 new feature
    • CTEs
    • Window Functions
    • Data Dictionary
    • utf8mb4
    • Security Roles
    • Improved Information Schema Performance

OpenWorld 2017: MySQL Document Store

This was a great session that was centered on the updates to JSON support in MySQL 8.  But to paint the full picture of how MySQL 8 is making JSON a first class data citizen in the platform, it also covered some updates to InnoDB Cluster and the new MySQL Shell.  The session was present by Alfredo Kojima, Miguel Araujo, and Mike Zinner all from Oracle.  Mike is responsible for client tools – shell, workbench, router, connectors, etc.  Alfredo was there to talk about the changes in InnoDB Cluster, and Miguel for the Document Store itself.

These changes make it more possible for you to use MySQL as a NoSQL document store.  With MySQL however you can retain ACID compliance while doing most (if not all) that NoSQL data platforms like MongoDB provide.  It will be interesting to get my hands on this to see what tradeoffs they make to retain the ACID model.  Especially compared to other NoSQL data engines that take the BASE model.

My notes from the session where.

  • MySQL Document Store Intro
    • What is a document?
      • structured data
      • They are using JSON (=JavaScript Object Notation)
      • Others are XML, YAML, etc.
    • Relational Model vs. Document Model
      • Higher Upfront effort during design vs. lower effort upfront
      • Database ensures integrity vs. Application responsible for integrity
      • Schema updates can be problematic vs. Simple schema change
    • Now MySQL supports both relational and document models
    • Exposes new CRUD interface, while keeping SQL
    • Standard plugin for MySQL Server (since MysqL 5.7.12)
    • Store JSON documents in MySQL inside tables (JSON type column)
    • Store JSON outside of relational tables (collection not table)
      • Collections belong to a database/schema just like a table, so you can have collections of the same name in different schemas on the same instance
      • Collections are implemented as tables(with an id and a doc column of type json)
    • Key takeaways
      • INNODB
      • transactions and locking
      • ect…
  • Components of the MySQL Document Store
    • MySQL Shell – new command line interface
    • Works with InnoDB Cluster
    • X Protocol – new protocol to deal with documents (more efficient)
    • Connectors for
      • .NEt
      • Node.JS
      • Python
      • Java
      • etc..
    • Document Store is tightly connected to InnoDB Cluster
      • for scale out
  • InnoDB Cluster
    • basically turning MySQL into a scale out architecture (looks like MongoDB or other NoSQL system architectures to me)
  • The X DevAPI – A modern CRUD App Programming Interface
    • To create a collection – “var products = db.createCollection(‘products’);”
    • Add document – “products.add({“_id”: 123, “name”: “Test1″}).execute();”
    • retrieve/select – “products.find(“price > 10 and price < 100”).fields([“_id”, “name”]).sort([“name”]).execute();
    • change – “products.modify(“_id”: 123  <missed it>
    • delete – products.remove(” <missed it>
  • Node Js example – Async call to MySQL and function (doc) as loopback

var schema = session.getSchema(‘mySchema’);

var coll = schema.getCollection(‘myColl’);

var query = “$.name == :name”;

coll.find(query).bind(‘name’,  ‘Alfredo’).execute(function (doc) {

console.log(doc);

})

.catch(function (err) {

console.log(err.message);

console.log(err.stack);

});

  • Demo of shell interface
    • port 33060 for XDev API port
    • new shell defaults to java script mode
    • can switch to sql with \sql
    • \js to switch back to javascript mode
    • dba.  used to manage the instance with javascript
    • session.sql(“SELECT * FROM TEST1;”).execute(); – for sql in javascript shell
    • session.sql(“select ?”).bind(“‘Hello World'”).execute(); – by using bind it escapes everything for you
    • much more powerful interface even for DBAs
    • scripts require the .execute(), but not needed while using the shell
    • new shell documentation – https://dev.mysql.com/doc/x-devapi-userguide/en/
    • new uuid with timestamp at the end to be more sequential for performance (within a session)
  • Documents via SQL
  • Behind the Scenes

Oracle OpenWorld 2017: Day 1 Wrap Up

I wasn’t sure what to expect from my first OpenWorld conference.  If the rest is anything like day 1 then I think it will be a great week.  There was plenty of marketing (as to be expected at a vendor sponsored conference), but a lot of useful content as well.  The highlights for me were the sessions on Kafka, Docker, and Data Guard.  All contained information I can take back to the team and put in use immediately.  I was impressed with the session on CloudDBA (and did not miss the parallel to Oracles announcement for their autonomous database), but would love to have seen more detail on the architecture.  What do they use for monitoring?  What platform are they using for log streaming?  etc.

I hope that the rest of the week provides similar information and ideas that I can take back.

Open World 2017: Mobile- and Bot-Enable Your Oracle Enterprise Apps

I have been a big proponent of chat bots ever since attending Percona Live 2015 when I first heard about their use at other companies.  It has not caught on where I work, but I thought I would attend a session to see what options existed from Oracle.  They presented a pretty interesting set of platforms that can help you quickly stand up mobile and chat based systems.  My notes are:

Oracle Mobile Cloud Enterprise

  • Mobile Core (MCS)
    • Built-in Mobile Services
    • Mobile API Catalog
    • MAX with Mobile Services
  • Intelligent Bots (IB)
    • AI/NLP Intent Detection
    • Multi Chanel Integration
    • Rich Conversational Flow
    • Entity Detection & Slot Filling
  • Customer Experience Analytics (CxA)
  • Challenge is in integrating back-end systems
    • OMCe – Mobile Services
      • Open source mobile client development tools (JET)
      • Tools for service devs to shape Mobile optimized APIs
  • New pieces are Bots and Customer Experience
  • AI first will replace “cloud first, mobile first” in next 10 years
  • App sprawl getting out of control
  • Integrates with a number of chat clients, voice, alexa, etc…
  • Value of chatbot is being able to integrate into a number of back-end systems
  • CxA – API to instrument your app

Case Study: Brocade

  • Looking to integrate Oracle EBS, Salesforce, Taleo, Microsoft Exchange, etc.
  • Used Mobile Iron to push app
  • Have mobile now and working on bot
  • Works with 11i and R12
  • Worked with Rapid Value for this

Demo

  • 50 – 60% of developers time is consumed in backend development
  • App fatigue
    • Too many apps – consumer and enterprise
    • Complexity in managing and upgrading enterprise apps
    • No common place to see all notifications for enterprise apps
  • Started off with pre-build mobile app offered by Rapid Value
  • MCS brought the following to the table
    • Enterprise Security
    • MDM Integration
    • Analytics and Reporting
    • Instance Life Cycle MAnagement
    • Mobile API Catalog
    • Offline Sync, Push Notifications
  • No infrastructure at Brocade
  • Strategic Advantages aimed at using Chatbots
    • Conversational mode Convenience
    • Reduce App store approval challenges
    • No App hosting and distribution
    • Zero User Training
  • Chatted with Oracle bot using Facebook Messenger

 

Ability to test the bot as well

YAML used for dialogue flow builder

OpenWorld 2017: Modernize your IT-Landscape with API Driven Development

Sven Bernhardt and Danilo Schmiedel gave a session that mostly turned into a sales pitch for a product Oracle purchased named Apiary.  It was a pretty cool tool, but not something that closes any holes where I work.  My notes are:

  • API Mangement
    • more than 12,000 public APIs offered by companies (2015)
    • Salesforce makes 50% revenue via API
      • Expedia 90%
      • ebay 60%
    • APIs are the door to the digital tomorrow
      • used to discover new business models and to evolve digital economies
    • Can support key business goals
      • Revenue Growth
      • Customer Satisfaction & Engagement
      • Operational Effeciencies
      • Partner Contribution & Ecosystem
    • What does it provide
      • Approved APIs for app developers
      • Provides design guidance for backend API developers
  • Taming the Monolith – Challenges + Demo
    • see pic for sample of Monolith Application
    • Task: Improve customer satisfaction with new innovative apps and modernized what we have today to increase our flexibility
    • see pic for decoupling and API breakdown
  • Architectural Considerations

 

OpenWorld 2017: Docker 101 for Oracle DBAs

My team is looking at changing the way we manage Oracle.  First, we are looking at migrating off of Exadata and onto a different architecture.  But more fundamentally, we are questioning how we deliver database services and how we can drive innovation within the company by changing our processes and/or technologies.

Docker has come up as one such technology.  We have a lot of developers moving toward microcservice architectures and they need a good way to quickly spin up databases for development.  We also need to better standardize our processes across data platforms and improve our ability to scale environments based on a number of factors.  Adeesh Fulay did a great job of showing the state of containers as it pertains to the Oracle database.  The short is that Oracle and LXC are very mature, while Docker is quickly closing.  I am still very interested in seeing how we can leverage Docker containers and orchestration to improve our efficiency with Oracle as well as our other platforms.  My notes are:

Why Containers?

  • Comparison to Shipping Goods
  • Intermodal Shipping Contianer
  • Issue similar with IT
    • App with list of steps to deploy
    • Prone to mistakes
    • Need to solve dependency challenges
    • Also need to make good on promise of virtualization to provide mobility
  • Docker
    • Developer: Build once, Run anywhere
    • Operator: Configure once, Run Anything

Linux Container Overview

  • VM contains app and: Guest OS, libraries, etc.
  • Containers: No OS or Hypervisor
    • Own network interface
    • file system
    • Isolation (security) – linux kernel’s namespaces
    • Isolation(resource usage) – linux kernel’s cgroups
  • Why
    • OS-Based Lightweight Virtualization technology
    • Content & Resource Isolation
    • Better Performance
    • Better Efficiency and smaller footprint
    • Build, Ship, Deploy anywhere
    • Separation of Duties
  • Types
    • Application Containers
      • Docker
        • Requires application to be repackaged and reconfigured to work with Docker image format
      • One container, one app (primary app)
      • patch by replacing containers
      • good for modern applications
    • System Containers
      • Each container runs an entire service stack
      • LXC, OpenVZ, Solaris Zones
        • No need to reconfigure applications
      • Patch in place
      • good for legacy apps
    • Oracle and Docker
    • No RAC on Docker
    • LXC Support http://ww.oracle.com/technetwork/database

Look Under the Hood

  • LXC is more mature with Oracle
  • Docker Host
    • daemon/engine – manages networking, images, containers, (data) volumes, plugins, orchestration, etc.
    • images
    • Dockerfile
      • Instructions for turning base image into a new docker image
    • containers
      • Like a directory. It is created from an image, and contains everything required to run an app.  Containers like VMs have state and are portable.
      • Containerd: Simple daemon that uses run times to manage
    • Volumes
      • Supports local or shared/distributed
      • Choose between mobility & performance
      • Docker outsourced issue of storage to other vendors
    • Networking
      • Types:
        • None: no external network
        • Host: shard host n/w stack
        • Bridge: NATing, exposing ports, poor performce
        • macvlan: uses host subnet, VLAN, swarm mode
        • Overlay: multi-host VXLAN, swarm mode
        • 3rd Party: OpenvSwitch, Weave, etc..
    • Orchestration
      • kubernetes
      • Mesos
      • Docker Swarm
      • Rancher
      • Nomad
      • ROBIN
  • Docker client
    • docker build
    • docker pull
    • docker run
  • Registry – Public or Private contains pre-build images

Getting Hands Dirty

Containerized Databases

Oracle late to the party but quickly catching up.  Has joined CNCF.  Working with kubernetes.  Using in Oracle cloud.

Advantages

  • Advantages of VMs without the performance penalty