When we think of big databases, we probably think in the 20GB+ range – that’s just our experience. So getting to 100GB would be pretty serious.

Move that up to more than a TB in a database and you’re working with a seriously large database.

OK, so some database administrators reading this would be laughing, since they’ve likely worked with databases in this range a number of times.

But then when we hear about petabyte databases we just lose perspective. How do you put that in context? It’s bloody big right?

As announced this week, SQL Server 2008 is now preparing to crunch petabyte databases:

Perhaps the most impressive application of SQL Server so far – and one of the most dramatic – is the Panoramic Survey Telescope and Rapid Response System, or Pan-STARRS for short, a wide-field celestial imaging facility being built at the University of Hawaii’s Institute for Astronomy. Its architects plan to photograph the entire available sky several times each month, trying to discover asteroids and comets that could pose a danger to Earth. The huge volume of images produced by this system will no doubt also prove valuable for many other scientific programs.

When Pan-STARRS is fully operational, it will have four telescopes, each with a digital camera capable of 1.4-gigapixel resolution. With just one telescope in operation so far, the facility already generates 1.4 terabytes of image data per night. For the longer term, its architects are installing 1.1 petabytes (quadrillion bytes) of disk storage. Although Pan-STARRS won’t use up all of that storage right away, it will still rank as one of the world’s largest databases.

Compressing, storing and crunching that data is the job of SQL Server.

Source: Microsoft PressPass

That’s seriously huge. It compares with Yahoo’s 2 petabyte database and the likes of eBay, Amazon and the National Energy Research Scientific Computing Center. The Top 10 largest databases are listed here.

There’s been tons of blog posts, articles, video interviews and general opinion on Azure this week. And understandably so, since this will likely be looked back on as one of the biggest turning points for Microsoft in their entire history.

An interesting point to note in the posts has been how so many have written about their understanding of what Azure is, without taking the time to actually read up on it. So, we’ve seen videos with people being interviewed about their reactions and thoughts on what it will include, how it might be applied, and how to use it. This is very strange. Why? Because everything you need to know is clearly articulated here on the Azure site. There’s even a whitepaper covering all the minute details.

We’re going to dive into it a little in this post, but as a general community service here’s the links you need to read:

• What is Azure?
• Why use Azure?
• Azure and Software plus Services
• Pricing and Licensing

You’ll notice that all four of these links come from Microsoft’s own Azure site. Amazing! Microsoft actually explaining what they are are doing. Who’d have thought?

Well, us actually. But apparently not most of the journalists and prominent commentators. But enough of our complaining.

Digging into Azure

If you are too lazy or time poor to read through the site (even the FAQ) then here’s the essentials you need to know.

Here’s the pretty diagram everyone has been using (straight from the Azure site)

Obviously the first thing you notice is that there is an operating system component – called Windows Azure – and a bunch of services. The combination of the services and the operating system are collectively referred to as the Azure Services Platform or more simply just: Azure.

Azure is all hosted in Microsoft’s data centers and is their cloud offering. It’s basically a cloud services platform. It’s reasonably open in the protocols it supports, and whilst initially targeting .NET and Microsoft developers, is designed to support other platforms in the future (eg Ruby and PHP).

Windows Azure – the operating system part – is what was referred to last month as Microsoft’s Cloud OS (and even Windows Strata for a time). It is all the glue that manages resources in the cloud.

The rest are the services. Live Services is a growing area, and one that will become prevalent on all devices. The .NET services and SQL services are no real surprise to watchers of the space. Even SharePoint services is an understandable offering since Microsoft is pushing SharePoint big time these days.

The one that caught us by surprise was the inclusion of Microsoft CRM in Azure collection.

Yes, hosted CRM is here and logical, but adding a complete service component for CRM in the Azure space is interesting. It shows that Microsoft is placing a big bet on CRM – a component that has had numerous adoption problems in the corporate space. Perhaps this will be its break out opportunity. We know many companies have changed their strategy on CRM in the last week, from having it as a minor piece of their consulting work, to now being an important strategic practice.

Our opinion: Microsoft has changed the game completely. This is a revolution. Amazon and others have been providing excellent token offerings (eg with EC2) leading up to this point in time. But now Microsoft has come in and demonstrated they have the vision, capability and passion to completely dominate the cloud. They’ve been at it for a while, and now its time to get on board. We are in awe.

Well it looks like the rumour page was correct. SQL Server 2005 SP3 is almost upon us. The CTP of SQL Server 2005 SP3 has been announced, and you can download it here. Doesn’t look like there’s too much to write about just yet though.

The SQL Server 2008 Experience site is up. As mentioned here in the Press Release, it features more than 500 videos. This is all part of the ‘worldwide readiness outreach’ that is about to kick off. Brace yourself!

Here’s the site. That red outline on the middle left is highlighting the styling issues the site has in IE8 Beta 2.