When the industry started its big data journey, we were willing and required to jettison many of the niceties of the relational database system (RDBMS) to achieve scale. Features like transactions, the relational data model and standard SQL access were dropped and in their place grew solutions like core Hadoop (with Hive and HDFS), HBase, Elasticsearch and MongoDB. While these solutions offered scale, their adoption and usefulness has been limited by their lack of deep and compliant SQL access and other familiar features. Based on new advances, the opportunity exists to build high function, high scale and easily consumed data platforms for our clients. The systems will serve the skilled data scientist, the casual SQL practitioner in the business unit and everyone in between. The audience will learn about a key advances high scale SQL solutions including:
- Query Engines / Data Virtualization – How Presto or Dremio can be used as a SQL front end to a variety of data sources, with a practical example in Dremio
- Highly Available, High Scale Operational Data Stores – How AWS Aurora or Cockroach DB can be used to build high scale (and with Cockroach cross-Cloud) operational data stores, with a practical example using Cockroach DB and StreamSets
- High Scale Time-Series Analysis – How the specialized real of time series data storage and analysis is being made more accessible and SQL ready with Timescale DB and MemSQL, with a practical example in Timescale DB
Last the audience will learn about the practical limits and lessons learned of the SQL first approach, with specific focuses on transactions, search data stores and graph data stores.