This is the second part of my interview with Microsoft Technology Evangelist, Vinod Kumar. You can find the first part of this interview at here.
Jacob: Could you briefly explain "SQL Server Data Services"?
Vinod: Microsoft SQL Data Services (SDS), formerly known as SQL Server Data Services (SSDS), is a highly scalable, on-demand data storage and query processing utility service. SDS will be offering true relational capabilities through SQL Server’s existing network protocol, Tabular Data Stream (TDS) and existing query language Transact-SQL (T-SQL). This will provide customers direct access to the familiar relational model, T-SQL programming language and the existing development and management tools, while continuing to deliver on our key value props of fault tolerance, high availability, friction free provisioning and pay as you grow scaling. Fundamentally, in a nutshell it is SQL Server on the Cloud beyond imagination the scalability that you can ever imagine in a seamless manner.
Jacob: What according to you, is the future of "SQL Server Data Services"?
Vinod: Well, I am not a astrologer nor can forecast what the pipeline of features which will get included with SDS. Interestingly, if you look at the current feature set of Tables, Stored Procedures, Triggers, Indexes, Views, VS Compatibility, ODBC Compatibility etc makes one feel at home already with SDS. So when people start getting a handle of storing data outside of their premise on the cloud and use the rich capabilities of consuming data this can become really a cheap viable option. Having said that, this opens doors for richer API or Apps to build Data Mining or analysis of data on these data too. So I surely see an opportunity for a lot of scope and usage that can blow the mind away for small departmental applications to enterprise ready applications. I would like to also take a moment on the TDS addition as a great asset because we will start looking at applications build for desktop and then moving to Web and not to cloud entirely giving a very seamless pay-as-you-go model becoming the mantra of the future.
Jacob: Could you give a few real life scenarios where XML data storage is beneficial over relational model?
Vinod: There are tons and tons of customers I have interacted with have used XML in innovative ways. I have seen XML as a powerful addition to SQL Server arsenal for ages and the power is underestimated. For me some of the classic places where XML can be used are where the schema is fluid. I have seen a lot of product companies in specific domains state their product needs to be flexible enough to accommodate extensions of customer requests and in such cases the schema design cannot be locked. I have also seen a lot of web applications use the XML streams very effectively in their products like mad. They take the XML output and directly apply a XLST at the frontend as visual elements. To give you one more dimension, I was recently working with a partially-connected product which uses Smartphone to run their apps and they used XML to store the offline data and an elaborate method to send the XML directly to SQL Server to get it into their database appropriately updating the data.
I personally feel there is enough potential in using XML services in the current generation of products. I also have seen sometimes these get abused because of excessive use. So one needs to be very cognizant of when, where and why one needs to move away from the conventional Relational world into the XML world. Finally, XML is like a swiss-knife and one needs to be cautious enough to know when and how to use it.
Jacob: What is your comment if you hear someone says "XML is evil"?
Vinod: First, I recognize the sentiments when someone tells me this, I next ask – Why, explain? As I told you in the previous question if we don’t understand the genesis of when and where to use we are surely digging our very own grave. One cannot dismiss XML as evil as a blanket statement. Though XML can represent almost everything we represent in Relational, it is not advisable doing so. I recollect vividly at TechEd 2005, someone from the audience asked me about storing the complete Northwind database as a single table, single column structure. One needs to know there are something’s best being relational like hierarchy data, referential integrity etc. The maturity levels in the audiences are better now and they understand these nuances clearly.
I can give real life instances where people have had wrong XML structure or ignorance of using XML indexes or wrong query construct that have degraded performance. But once the user education happens people have had more confidence in using XML. I would highly recommend two whitepapers – XML Best practices for SQL Server 2005 and Performance Optimizations for XML Data Type. These are a good starter for developers working with XML inside SQL Server.
Jacob: Where can we catch you up on the web?
Vinod: This is the easiest question you have asked me today. Well, you can Bing me with SQL Server and you will get to me J. On a serious note, my online coordinates are:
Pinalkumar Dave is one of my favorite SQL Server bloggers. What I most appreciate is the simplicity of his posts and the relevance of the content. Most of the times he writes about simple things, but very useful for the SQL Server developers. And when he writes about complex stuff, he makes sure that he present them in a very simple manner so that both experts and beginners can understand them alike.
Pinal is a Microsoft SQL Server MVP and a prominent authority on SQL servers. He has written over 1000 articles on the subject on his blog at http://blog.sqlauthority.com . He is a dynamic and proficient Principal Database Architect, Corporate Trainer and Project Manager, who specializes in SQL Server Programming and has 7 years of hands-on experience. He holds a Masters of Science degree and a number of certifications, including MCDBA and MCAD (.NET). He was awarded Regional Mentor for PASS Asia and is also mentor for Solid Quality India (http://www.solidq.in).
Jacob: You are well known for your capability to explain complex things in a simple manner. Could you explain what "beyond relational (BR)” means?
Pinal: In simple terms, this means that it is beyond relations to traditional RDBMS. There are so many things to talk about when we stop thinking in terms of relationals. When we say “beyond relationals”, this does not mean that we move away from the relational database. In fact, for me words suggest quite the opposite; they imply strong ties to relational databases. It would not be termed “beyond” if it had nothing to do with relationals. The science of “beyond relational” picks up relationals where traditional RDBMS stop.
Jacob: Are you more of a “relational” or a “beyond relational” guy, or both?
Pinal: I am “relational” guy, who has recently learnt a lot about “beyond relational”. If you ask me the same question in 3 months, my answer will probably be that I am both.
Jacob: What is the relevance of BR in SQL Server and what do you think is future of BR?
Pinal: SQL Server and BR are comparable to non-identical twins. Although the two concepts don’t look alike, they cannot be separated. As an end user, it is very difficult to find separation between them.
There are many components in the BR feature set and it is very difficult to identity all of them here. The important BR components are XML, CLR Integration, Filestream Storage, Hierarchical Data and Spatial Data. Describing the features of all the BR components could take years. Let us simply say that without BR, we would be in a relational prison, with very limited relational potential, whereas with the exploration of the features of BR, the possibilities are unlimited.
Jacob: What is your favorite BR feature available in SQL Server and why?
Pinal: This is a difficult question to answer. It is like asking a parent who their favorite child is. While I like most of the features, I will select one as my favorite. I am particularly impressed with the Filestream storage type. As you are probably aware, varbinary and image are very difficult data types to handle and it is a pain to manage them. Filestream storage overcomes all the limitations and has emerged as a true winner in terms of storage. I have analyzed this subject on one of my recent blogs: http://blog.sqlauthority.com/2009/07/08/sql-server-blob-pointer-to-image-image-in-database-filestream-storage/. BTW: I read your FILESTREAM article http://www.simple-talk.com/sql/sql-training/an-introduction-to-sql-server-filestream/. I think it is a must-read for all SQL Server developers.
Jacob: Where do you see XML in the BR world?
Pinal: To my mind XML, has an important place in both the “Before” and “Beyond” relational world. It is a link or bridge between these worlds. Instead of justifying this statement by way of example, let me rather ask you this: “Can you relate a PDF file, a movie file and their analysis database to a relational database, so that their attributes are searchable and remain meaningful, in any other way than by XML format?”
Jacob: Could you give a real life scenario where XML data storage is beneficial over the relational model?
Pinal: This question belongs in the “‘it depends”’ category. One example where I use XML is with online listings of auction site items. I am not always sure what will be important to each client. For some colors matter most and some find size more important. Instead of creating new columns for everything, I simply store the data in XML format and its schema in another column. I use the techniques demonstrated in the book “The art of XSD” to validate the schema and give different search conditions to different clients based on their preference.
Jacob: What is your comment if you hear some one says "XML is evil"?
Pinal: Without the existence of evil, divine does not exist. Everything is relative. I would not correct a person who thinks XML is evil because it is their opinion. In fact, I simply avoid discussions on the subject. XML is useful; all things have their uses. Closing our eyes does not make world blind, but perhaps I should stop philosophizing here.
Jacob: Where can we catch you up on the web?
Pinal: I am at http://blog.sqlauthority.com. Please feel free to drop me a line at pinal ‘at’ sqlauthority.com
During Tech-ED India 2009, I had an opportunity to meet SQL Server Legend Vinod Kumar and he agreed to give an interview. Vinod Kumar is my favorite SQL Server speaker and mentor. I take speaking lessons from him, every time I meet him. Thank you Vinod for your continuous support and encouragement. Here is a summary of the discussion.
Jacob: Please give a brief description of your self
Vinod: A lot of folks in the community know me because of my webspace – www.ExtremeExperts.com and the passionate sessions I have delivered for them in the community on SQL Server from time-to-time. I currently work with Microsoft as a Technology Evangelist specializing in Databases and BI workloads. These are my core areas of expertise and I love them like anything. If you are keen, you can also catch me over my blogs and Twitter where I try to share my learning’s from time-to-time.
Jacob: could you explain what is meant by "beyond relational"
Vinod: For me “Beyond Relational” is more of a fact and reality. I think in this internet era, we talk of relational data but applications have pressures of storing and getting data that defy the fundamentals of relational world. Images, chat scripts, medical data, location aware data, documents in .doc or .pdf, our mails, podcasts, videos everything have become seamless part of the internet world and people don’t want to store or write applications differently to just integrate these data. Finally, even the world of XML is beyond relational and it seems to have got into places where we never imagined. Nowadays, documents are stored as XML, languages are around XML schemas, definition templates are around XML notations and more. These are some incarnations of the world – “Beyond Relational”.
Jacob: Are you more of a relational guy or beyond relational guy? or both?
Vinod: This is a tough question – I used to be purely a relational guy about 5yrs back. I saw it was impossible to be a pure-relational person looking at the application needs which drive our very existence. If you need to survive in the database world and be really successful in understanding the spectrum of the databases requirements, one needs to have a blend of both. At the same time, I want to caution the thinking for both these worlds are quite different and we need to be very careful not to mix the fundamentals between these worlds.
Jacob: what is the relevance of BR in SQL Server
Vinod: SQL Server also has a life “Beyond Relational” without any questions. In fact, in the early days of SQL Server 2000, we exposed XML streams before it became a first-class citizen in SQL Server 2005. So our relationship is more than a decade old if people can recollect correctly. But since SQL Server 2005, this new avatar of SQL Server gave data beyond relational a new life. Obviously with the last release of SQL Server 2008 this association has just increased and we have made some great enhancements to target these new and complex requirements of real-world problems.
Jacob: What are components and features that are part of the BR feature set?
Vinod: Though I will not be getting an exhaustive list for a reference but I want to just give you some interesting stuff as starting pointers -
· XML – This feature has been with us for close to a decade as I mentioned before. FOR XML, XML Indexes, FLOWR statement, .NET XML API via CLR, XML Schema support, XQuery support, DML on XML etc are something that will really get the XML nerds really happy. Now being a first-class data type inside SQL this is a boon to all developers.
· CLR Integration – This is also a feature we introduced with SQL Server 2005 and there was a lot of synergy in utilizing the .NET CLR’s BCL (Base class libraries) inside the SQL Server environment to get the max out of SQL Server logics. A lot of them have been defensive on using this feature. As much as I also recommend certain design decisions must be in place before using this feature, it is surely a safe feature to have and very powerful too.
The next three features were introduced with SQL Server 2008 and hence wanted to call them out specifically.
· Filestream Storage – Fundamentally storing large binary objects inside of database is very sub-optimal and such large objects database memory buffers and cause database fragmentation too. Hence Filestream is a special datatype that stores the files on the filesystem and yet gives you all the ACID rule properties of a databases to these objects. Moreover, it also enhances the experience of backup, recovery, transactional consistency, no limit on file size etc. These are a bonus to any database engine.
· Hierarchical Data – Most data schemas involve some form of hierarchy. Manager -> employee, parent company -> child company, Region -> County -> Town etc. Storing these in a relational model is difficult and full of compromises. Hence giving power back to the user especially via a programming paradigm is critical. I know this is a underplayed feature and it will take some time for people to understand its power. Unlike LDAP’s ADSI model, there is nothing new one needs to learn but use the same TSQL constructs for using these special data.
· Spatial Data – This I consider as a wishist item that got into SQL Server 2008. Now SQL Server supports both Geometry and Geography data - Both types support all of the OGC types. Apart from this SQL Server also supports two dimensional data like X &Y or Lat + Long data, Z-member, M-members etc.
It is important to understand that SQL Server or for that matter any upcoming databases will need to store and support both relational and non-relational data. And SQL Server 2008 has enough firepower under its arsenal to keep you busy.
Jacob: What is your favorite BR feature available in SQL Server and why?
Vinod: This is the real tough question – I personally loved SQLCLR when SQL Server 2005 and with SQL Server 2008 I have started loving the Filestream feature more than the others. Well, I know there are quite a lot of unique requirements which can capitalize this feature beyond imagination. Workloads of DAM (Digital Asset Management) or even Sharepoint in the future can get the real value out of this feature.
Jacob: What according to you is the future of BR?
Vinod: For me, this sort of data storage / usage are going to get complex and the application requirements will also get interesting. I think we are just getting started with this type of data manipulation strategies. As a database developer, administrator or architect if we fail to unearth these hidden riches - then we are utilizing maybe just 5-10% of what the technology is capable off. I also understand not all applications will need to embrace this, but if you close your eyes to this - your competition is surely making the best use of this opportunity you gave them.
Jacob: Where do you see XML stand in the BR world?
Vinod: XML for me is no more a separate entity inside the database world. It is very much an integral part of data and there is no way we can separate these entities. I see more and more support of XML into the relational epitome and as you see XML seamlessly co-existing with SQL Server - this bonding is going to get stronger and stronger moving forward. If you had asked me this question about a 5yrs back, I would have given you theories of XML as B2B scenario’s, integration, interoperability etc – though all these still exist we have surely moved beyond this myopic view for XML usages. Look at how we package RDL (Report Definition Languages) as XML with Reports Server, how we store word documents as XML, Call the Analysis processing via XML constructs etc – so the usages and life of XML is surely expanding even without us knowing it.