Monthly Archives: February 2017

SQL 2016 – Partitioning An Existing Table With A Columnstore Index

— by Lori Brown @SQLSupahStah

I recently ran across a data warehouse that did have the largest table partitioned. We were migrating up to SQL 2016 and we wanted to make sure that the partitions were still working and in place. Since I felt I needed to refresh my table partitioning skills, I decided to conduct a small scale test of partitioning up an existing table by year and to make it more fun, I wanted to have a columnstore index present to see how interesting things could be.

I am using the Fact.Purchase table in the new WideWorldImportersDW database. WideWorldImporters replaces AdventureWorks and can be researched and downloaded from here: https://blogs.technet.microsoft.com/dataplatforminsider/2016/06/09/wideworldimporters-the-new-sql-server-sample-database/

I did a little querying in the Fact.Purchase table and found that the Date Key column is great to use to partition it by year. And, I found that the dates in the table are generally from 2013 through 2016.

First you have to add the filegroups that will be used for the table once it is partitioned. I have added 4 filegroups for years 2013 – 2016.

blog_20170224_1

Once that is done, I have to add an .NDF file for each filegroup and map the file to the correct filegroup.

blog_20170224_2

 

Next I need a partition function and a partition scheme.

USE [WideWorldImportersDW]

GO

CREATE PARTITION FUNCTION [PF_Purch_Year](date) AS RANGE RIGHT FOR VALUES (‘2014-01-01’, ‘2015-01-01’, ‘2016-01-01’)

GO

CREATE PARTITION SCHEME [PS_Purch_Year] AS PARTITION [PF_Purch_Year] TO (Purch2013, Purch2014, Purch2015, Purch2016)

GO

What I am basically doing is setting parameters or boundaries for objects that use the PF_PURCH_YEAR function to place date values older than 01/01/2014 into the Purch2013 filegroup, values between 01/01/2014 and 01/01/2015 into the Purch2014 filegroup, values between 01/01/2015 and 01/01/2016 into the Purch2015 filegroup and finally everything newer than or equal to 01/01/2016 into the Purch2016 filegroup.

blog_20170224_3

Now that I have everything set up, I have to move the data to it. However, I cannot do that with an existing clustered index. I have to drop the clustered index and rebuild it with the partition scheme. This does get a bit confusing if you are trying to move existing things as you will see.

I first dropped all my indexes. Of course I had scripted them all out for rebuild but ran into trouble when I tried to build them again.

USE [WideWorldImportersDW]

GO

DROP INDEX [CCX_Fact_Purchase] ON [Fact].[Purchase]

GO

ALTER TABLE [Fact].[Purchase] DROP CONSTRAINT [PK_Fact_Purchase]

GO

DROP INDEX [FK_Fact_Purchase_Date_Key] ON [Fact].[Purchase]

GO

DROP INDEX [FK_Fact_Purchase_Stock_Item_Key] ON [Fact].[Purchase]

GO

DROP INDEX [FK_Fact_Purchase_Supplier_Key] ON [Fact].[Purchase]

GO

 

Here we go trying to rebuild….this is where it gets interesting…

USE [WideWorldImportersDW]

GO

CREATE CLUSTERED COLUMNSTORE INDEX [CCX_Fact_Purchase] ON [Fact].[Purchase]

WITH (DROP_EXISTING = OFF)

ON PS_Purch_Year ([Date Key])

GO

I received an error:

Msg 35316, Level 16, State 1, Line 3

The statement failed because a columnstore index must be partition-aligned with the base table. Create the columnstore index using the same partition function and same (or equivalent) partition scheme as the base table. If the base table is not partitioned, create a nonpartitioned columnstore index.

 

Uh oh! This meant that the table had already been part of a different partition scheme. I did not think to look for that before I started. Ugh! I thought that it must be easy to change the base table partition alignment and it is but it is not super intuitive. Since I had already dropped all my indexes, I figured that I needed to recreate the original clustered index on the old partition since that would be how the base table is associated with anything. I rebuilt it using the old partition. So far so good. When I tried rebuilding it with DROP_EXISTING = ON I received the same error as before. After thinking about it for a bit, I dropped everything again but this time created a regular clustered index on the new partition to align the base table. This worked!

ALTER TABLE [Fact].[Purchase] ADD CONSTRAINT [PK_Fact_Purchase] PRIMARY KEY CLUSTERED

(

[Purchase Key] ASC,

[Date Key] ASC

)ON PS_Purch_Year ([Date Key])

GO

 

Now the base table is aligned with the partition scheme that I wanted it on but I need the clustered index to be the columnstore index. We have to drop and create again and everything falls into place.

ALTER TABLE [Fact].[Purchase] DROP CONSTRAINT [PK_Fact_Purchase]

GO

USE [WideWorldImportersDW]

GO

CREATE CLUSTERED COLUMNSTORE INDEX [CCX_Fact_Purchase] ON [Fact].[Purchase]

WITH (DROP_EXISTING = OFF)

ON PS_Purch_Year ([Date Key])

GO

ALTER TABLE [Fact].[Purchase] ADD CONSTRAINT [PK_Fact_Purchase] PRIMARY KEY NONCLUSTERED

(

[Purchase Key] ASC,

[Date Key] ASC

)ON PS_Purch_Year ([Date Key])

GO

CREATE NONCLUSTERED INDEX [FK_Fact_Purchase_Date_Key] ON [Fact].[Purchase]

(

[Date Key] ASC

)ON PS_Purch_Year ([Date Key])

GO

CREATE NONCLUSTERED INDEX [FK_Fact_Purchase_Stock_Item_Key] ON [Fact].[Purchase]

(

[Stock Item Key] ASC

)ON PS_Purch_Year ([Date Key])

GO

CREATE NONCLUSTERED INDEX [FK_Fact_Purchase_Supplier_Key] ON [Fact].[Purchase]

(

[Supplier Key] ASC

)ON PS_Purch_Year ([Date Key])

GO

 

Success!!!! Now I just needed to make sure that things are where I expect them to be. I pilfered the following queries from MSDN…

SELECT *

FROM sys.tables AS t

JOIN sys.indexes AS i

ON t.[object_id] = i.[object_id]

JOIN sys.partition_schemes ps

ON i.data_space_id = ps.data_space_id

WHERE t.name = ‘Purchase’;

GO

blog_20170224_4

blog_20170224_5

As you scroll across the above results I can see that all my indexes are in the PS_Purch_Year partition scheme.

SELECT t.name AS TableName, i.name AS IndexName, p.partition_number, p.partition_id, i.data_space_id,

f.function_id, f.type_desc, r.boundary_id, r.value AS BoundaryValue

FROM sys.tables AS t

JOIN sys.indexes AS i

ON t.object_id = i.object_id

JOIN sys.partitions AS p

ON i.object_id = p.object_id AND i.index_id = p.index_id

JOIN sys.partition_schemes AS s

ON i.data_space_id = s.data_space_id

JOIN sys.partition_functions AS f

ON s.function_id = f.function_id

LEFT JOIN sys.partition_range_values AS r

ON f.function_id = r.function_id and r.boundary_id = p.partition_number

WHERE t.name = ‘Purchase’

ORDER BY p.partition_number;

 

This query shows the boundaries.

blog_20170224_6

I also found a really handy query created by David Peter Hansen that gives you info on partitions on a table much more concisely. https://davidpeterhansen.com/view-partitions-in-sql-server/  As they say….go read the whole thing. J

blog_20170224_7

I hope this helps someone who wants to partition an existing table. I did this on my own system with a small table and not a production system so please be sure to test your process against a test system before you accidentally cause an issue in production.

For more information about blog posts, concepts and definitions, further explanations, or questions you may have…please contact us at SQLRx@sqlrx.com. We will be happy to help! Leave a comment and feel free to track back to us. Visit us at www.sqlrx.com!

 

Shrinking Database Files – Let’s Not And Say We Did

— by Lori Brown  @SQLSupahStah

I recently ran into someone who swears that shrinking database files on a production server is a good thing and proudly said they did it on all systems they work on.  OMG!!  Let me post a quote directly from MSDN regarding weather or not shrinking database files causes fragmentation (https://msdn.microsoft.com/en-us/library/ms189493.aspx) :

blog_20170216_1

Please notice I am talking about production SQL Servers and not development, test, QA, staging or anything that is not production.  Of course if you shrink database files in any environment you are still going to cause more fragmentation but since the use of those systems is often vastly different than a production system, you as the DBA have to judge the ramifications on those non-production systems.

blog_20170216_2

Have I ever had to shrink database files on the production systems I monitor…Uhhh, yes.  But, I do so very infrequently and with reason.  I even have one system where the guy who is in charge of it refuses to allow me to disable the nightly job that shrinks some of the databases.  I can only do so much but he can’t say I have not duly warned him.

 

There are tons of good articles and even rants of why you should not shrink your database files.  Here are some links in case you think I am the only one telling you to NOT shrink your database files.

 

https://www.brentozar.com/archive/2009/08/stop-shrinking-your-database-files-seriously-now/  – Gotta love some Ozar!

 

http://www.sqlskills.com/blogs/paul/why-you-should-not-shrink-your-data-files/ – From THE man!

 

http://www.sqlservercentral.com/blogs/simple-sql-server/2016/01/19/shrinking-database-data-files/ – Good post from a new blogger with the appropriate warnings.

 

https://www.am2.co/2016/04/shrink-database-4-easy-steps/ – Lots of warnings throughout the post.

 

https://www.mssqltips.com/sqlservertip/3008/solving-sql-server-database-physical-file-fragmentation/ – Shows a tool from sysinternals that allows you to see the physical fragmentation and states that once db files are physically fragmented the only way to undo the damage is to take your databases offline and defrag them at the disk level.

 

If you go to the links, you may be thinking that some of those posts are old.  However, look through the comments…there are new ones up to the present and they are still getting the same answers. 

If after all that you are still not convinced that shrinking database files causes physical fragmentation, well… I tried.  But go ahead and keep at it.  If you are lucky you may not ever have an issue or experience things getting slow because of fragmentation.  Roll those dice!

blog_20170216_3

 

For more information about blog posts, concepts and definitions, further explanations, or questions you may have…please contact us at SQLRx@sqlrx.com. We will be happy to help! Leave a comment and feel free to track back to us. Visit us at www.sqlrx.com!

 

 

 

Moving Non-clustered Indexes To New Data File

-by Ginger Keys

It is not uncommon for database files to grow to the point you need more disk space. But what if it’s not possible or not in the budget to add more space? There are several creative options you have to move or shrink files, or delete unneeded items from the disk. One solution we recently implemented with a client was to move the indexes of a large database to another drive.

Create File/Filegroup

Only non-clustered indexes can be moved, and they must reside in a different filegroup from the primary filegroup. If you don’t already have a separate file and filegroup created, you must do this first. Make sure you have created a folder path on your system to the drive that will be holding your index files.

blog_20170209_1

Get Space Requirement for Indexes

Determine the non-clustered indexes you have in your database and their size by running this script to ensure you have enough space allocated on the target drive to house the index files.

USE AdventureWorks2016

GO

SELECT

OBJECT_NAME(i.OBJECT_ID) AS TableName,

i.name AS IndexName,

8 * SUM(a.used_pages) AS ‘Indexsize(KB)’

FROM sys.indexes AS i

JOIN sys.partitions AS p ON p.OBJECT_ID = i.OBJECT_ID AND p.index_id = i.index_id

JOIN sys.allocation_units AS a ON a.container_id = p.partition_id

JOIN sys.tables AS t ON i.object_id = t.object_id

WHERE i.type_desc = ‘NONCLUSTERED’

GROUP BY i.OBJECT_ID,i.index_id,i.name

ORDER BY OBJECT_NAME(i.OBJECT_ID),i.index_id

blog_20170209_2

Create and Execute Stored Procedure

The following script will create a stored procedure called MoveIndexToFileGroup that will move an index from one file group to another, and even supports the INCLUDE columns. Run this script against the master database to create the stored procedure: https://gist.github.com/dalenewman/6377911

In order to get a list of tables for your database, run the following script.

Use AdventureWorks2016 –use your database name

GO

SELECT * FROM INFORMATION_SCHEMA.TABLES

blog_20170209_3

After the above stored procedure is created and you have gotten a list of schemas/tables/indexes you want to move, execute the following

EXEC MoveIndexToFileGroup

@DBName = ‘AdventureWorks2016’, –your database name

@SchemaName = ‘HumanResources’, –schema name that defaults to dbo

@ObjectNameList = ‘Shift,  

Department,

                    EmployeeDepartmentHistory,

                    EmployeePayHistory,

                    Employee,

                    JobCandidate’, –a table or list of tables

@IndexName = NULL, –an index or NULL for all of them

@FileGroupName = ‘INDEX’; –the target file group

 

Your indexes will appear in the target destination you created.

blog_20170209_4

Conclusion

There are many useful reasons for moving your non-clustered indexes to a separate drive…in this case ours was simply to create more space for the drive holding the data files. Whatever your reason, the steps above should give you a simplistic way to achieve this.

For more information about blog posts, concepts and definitions, further explanations, or questions you may have…please contact us at SQLRx@sqlrx.com. We will be happy to help! Leave a comment and feel free to track back to us. Visit us at www.sqlrx.com!