Showing posts with label case. Show all posts
Showing posts with label case. Show all posts

Thursday, March 22, 2012

best practices on maxinsertcommitsize

Hi,

I wonder if anyone knows what would be the best case scenario for the property 'maxinsertcommitsize' for the sql destination task if I want to load 6m records into a target. Is the best setting 0 (try loading all in one batch) or should I choose a different value for example 1000000 per batch?

Thanks,

Marc

This setting is up to you. How big of a batch do you want to insert? If you encounter an error in the load, do you want to roll back ALL records, or just the batch?

There is no "best practice" because this is user dependent. Everyone's situation is different.|||

Ok, but does it affect performance? For our situation the following applies:

- We don't care how big the batch will be as long as it is optimized for maximum performance
- If we encounter an error in the load it doesn't matter if ALL records roll back or just the batch.

|||I don't think there will be much of a performance difference. The one thing to watch out for in a big batch is the potential for filling up the transaction log.

Try different settings and report your results back to us.|||Actually there may be a performance difference between using batches and not. When SQL Server commits a batch it has to update any affected indexes. 6M records could be a good deal of work. I've seen it take over an hour to commit a batch that size when indexes are involved. By using smaller batches, SQL Server can get started on this work while SSIS is still sending it rows. But that advantage depends on how long your SSIS process takes. If it can generate those 6M in 30 seconds, then giving SQL a head start isn't going to do much good.

Sunday, March 11, 2012

Best Practice for Counts/Distinct Count Measures for Filtered Data

Let's say the Count/Distinct Count measure is based on filtered data.

When the situation allows, is it better to do a CASE statement with 1s and 0s from the table in the DSV and then do a SUM on this field? Or is it better to create a new Named Query in the DSV filtering out the data through the WHERE clause based off of the original table and then do a DISTINCT COUNT measure on the newly created Named Query?

If there's no clear cut answer as to what is the "better" approach, please offer up the advantages and disadvantages for both. My preference is best performance.

An Example:

Measure: Active SKU Count
Products dimension has product_key, SKU and Status

In the DSV for the Products dimension, should I add a CASE statement:

CASE WHEN [Status] = 'Active' THEN 1 ELSE 0 END AS [Active Status]

Then create a new SUM measure based on the Active Status column.

OR

Create a new Named Query in the DSV:

SELECT * from Products WHERE Status = 'Active'

Then create a new DISTINCT COUNT measure based on the SKU column from the new Named Query?


These 2 alternatives do not produce the same results. If a given SKU has fact records in Jan, Feb, and Mar, the first approach using SUM would return a value of 3 for Calendar Quarter 1. Then second approach based on a distinct count measure would return 1 (which is probably what you want)

Distinct count measures should be implemented in their own measure group anyway, so if you only ever need the active count then filtering out the non-active records is a good option.

If you need to get distinct counts of both active and inactive SKUs then you would be better off adding the Active flag as an attribute to the product dimension and implementing it as a slowly changing dimension.

|||Darren,

I had a feeling that these 2 alternatives would not produce the same results when I was typing up the question. Perhaps my example scenario wasn't a good one, because the limited testing with my applicable situation yielded the same results.

In any event, I will just go ahead and go forward with the separate measure group with the new Named Query approach.

Thanks!

Best Practice Database Owner 'sa' SQL Server 2005

I known in SQL Server 2000 it's MS best practice to create database owner as
'sa'.
Is this still the case using SQL Server 2005?I prefer to have sa as owner of the databases. That way, I have a known SID for the owner, and that
sid exists on any SQL Server I chose to restore that database.
--
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"Joe P." <Joe P.@.discussions.microsoft.com> wrote in message
news:C564C628-DA50-41D7-9C5B-2982058BD226@.microsoft.com...
> I known in SQL Server 2000 it's MS best practice to create database owner as
> 'sa'.
> Is this still the case using SQL Server 2005?
>|||"Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote in
message news:uZ3Qkzz4GHA.4976@.TK2MSFTNGP06.phx.gbl...
>I prefer to have sa as owner of the databases. That way, I have a known SID
>for the owner, and that sid exists on any SQL Server I chose to restore
>that database.
>
Ditto. And the SA login should be disabled, even if you aren't running in
Windows Authentication mode.
David|||And especially even if you ARE running in Windows Authentication mode.
;-)
--
Arnie Rowland, Ph.D.
Westwood Consulting, Inc
Most good judgment comes from experience.
Most experience comes from bad judgment.
- Anonymous
"David Browne" <davidbaxterbrowne no potted meat@.hotmail.com> wrote in
message news:u1A4d9z4GHA.1256@.TK2MSFTNGP04.phx.gbl...
>
> "Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote
> in message news:uZ3Qkzz4GHA.4976@.TK2MSFTNGP06.phx.gbl...
>>I prefer to have sa as owner of the databases. That way, I have a known
>>SID for the owner, and that sid exists on any SQL Server I chose to
>>restore that database.
> Ditto. And the SA login should be disabled, even if you aren't running in
> Windows Authentication mode.
>
> David

Best Practice Database Owner 'sa' SQL Server 2005

I known in SQL Server 2000 it's MS best practice to create database owner as
'sa'.
Is this still the case using SQL Server 2005?I prefer to have sa as owner of the databases. That way, I have a known SID
for the owner, and that
sid exists on any SQL Server I chose to restore that database.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"Joe P." <Joe P.@.discussions.microsoft.com> wrote in message
news:C564C628-DA50-41D7-9C5B-2982058BD226@.microsoft.com...
> I known in SQL Server 2000 it's MS best practice to create database owner
as
> 'sa'.
> Is this still the case using SQL Server 2005?
>|||"Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote in
message news:uZ3Qkzz4GHA.4976@.TK2MSFTNGP06.phx.gbl...
>I prefer to have sa as owner of the databases. That way, I have a known SID
>for the owner, and that sid exists on any SQL Server I chose to restore
>that database.
>
Ditto. And the SA login should be disabled, even if you aren't running in
Windows Authentication mode.
David|||And especially even if you ARE running in Windows Authentication mode.
;-)
Arnie Rowland, Ph.D.
Westwood Consulting, Inc
Most good judgment comes from experience.
Most experience comes from bad judgment.
- Anonymous
"David Browne" <davidbaxterbrowne no potted meat@.hotmail.com> wrote in
message news:u1A4d9z4GHA.1256@.TK2MSFTNGP04.phx.gbl...
>
> "Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote
> in message news:uZ3Qkzz4GHA.4976@.TK2MSFTNGP06.phx.gbl...
> Ditto. And the SA login should be disabled, even if you aren't running in
> Windows Authentication mode.
>
> David

Thursday, March 8, 2012

Best Practice Database Owner 'sa' SQL Server 2005

I known in SQL Server 2000 it's MS best practice to create database owner as
'sa'.
Is this still the case using SQL Server 2005?
I prefer to have sa as owner of the databases. That way, I have a known SID for the owner, and that
sid exists on any SQL Server I chose to restore that database.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
"Joe P." <Joe P.@.discussions.microsoft.com> wrote in message
news:C564C628-DA50-41D7-9C5B-2982058BD226@.microsoft.com...
> I known in SQL Server 2000 it's MS best practice to create database owner as
> 'sa'.
> Is this still the case using SQL Server 2005?
>
|||"Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote in
message news:uZ3Qkzz4GHA.4976@.TK2MSFTNGP06.phx.gbl...
>I prefer to have sa as owner of the databases. That way, I have a known SID
>for the owner, and that sid exists on any SQL Server I chose to restore
>that database.
>
Ditto. And the SA login should be disabled, even if you aren't running in
Windows Authentication mode.
David
|||And especially even if you ARE running in Windows Authentication mode.
;-)
Arnie Rowland, Ph.D.
Westwood Consulting, Inc
Most good judgment comes from experience.
Most experience comes from bad judgment.
- Anonymous
"David Browne" <davidbaxterbrowne no potted meat@.hotmail.com> wrote in
message news:u1A4d9z4GHA.1256@.TK2MSFTNGP04.phx.gbl...
>
> "Tibor Karaszi" <tibor_please.no.email_karaszi@.hotmail.nomail.com> wrote
> in message news:uZ3Qkzz4GHA.4976@.TK2MSFTNGP06.phx.gbl...
> Ditto. And the SA login should be disabled, even if you aren't running in
> Windows Authentication mode.
>
> David

Sunday, February 12, 2012

beginner - case sensitive query

Hi

We are taking case sensitive data, and loading it into an Access 2000 database using an OBDC. We then use VB 6 to produce an SQL .exe file which manipulates the data into new tables for the CR.

We have a a select statement where we have
SELECT ....
INTO A_POS
FROM POS LEFT JOIN CHTAB ON POS.STATUS = CHTAB.STATUS

The trouble is the client has the codes "F" and "f" in the CHTAB.STATUS, so I am producing duplicate A_POS records for any POS record with a status of "F".

Any suggestions as to how I can check the case sensitivity in Access so I create clean tables?

Thanks.
_Do you want to make all "f" to "F"?
Post some sample data and the result you want|||Unfortuneately, we are a software house, so we can't go and demand our clients change the codes. One client had all their company structure codes in upper case, and when they changed the company structure, they kept all the same codes, but changed them to lower case. Thus on the database "MELB" and "melb" could appear in one historic view, and have totally different properties.
I have an Organisation chart file ORG, which is keyed by the field OFFICE. I also have an employee positions file, POS. One of its attributes is OFFICE. Both "MELB" and "melb" are valid, but distinct, office codes.
My select statment is:-

select ... from POS LEFT JOIN ORG WHERE POS.OFFICE = ORG.OFFICE

And I need to somehow tell the script that I want the matching to be case sensitive, so I only pick up the correct upper (or lower) case office, not both.

Thanks

Friday, February 10, 2012

Bday dates revisited

Hi,
I'm trying to create a query that will tell me when someone's birth day is
approaching from my employee table. I tried Case When
Datediff(dd,b_day,getdate()) < 4 Then 'Remind' Else '' End As B_dayCk
Datepart(dd,b_day) - Datepart(dd,b_day) < 4 (This checks for the day but not
for the month). If I And or Or with Datepart(mm,b_day) - Datepart(mm,b_day)
< 0 doesn't work either.
I'm sure there's a simple way out of this. Can someone help pls.
TIA
MikeThe problem might be with you storing "birthday" and not "date of birth".
This code takes the date of birth, uses the current year an calculates the
difference between now and the birthday for this year:
declare
@.dob datetime
set @.dob = '1960-04-20'
select
datediff (dd, getdate(), str (year (getdate()), 4) +
right (convert (char (8), @.dob, 112), 4))
Tom
----
Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
SQL Server MVP
Toronto, ON Canada
.
"Mike_B" <nospam@.yahoo.com> wrote in message
news:ervuHLZYGHA.3848@.TK2MSFTNGP05.phx.gbl...
Hi,
I'm trying to create a query that will tell me when someone's birth day is
approaching from my employee table. I tried Case When
Datediff(dd,b_day,getdate()) < 4 Then 'Remind' Else '' End As B_dayCk
Datepart(dd,b_day) - Datepart(dd,b_day) < 4 (This checks for the day but not
for the month). If I And or Or with Datepart(mm,b_day) - Datepart(mm,b_day)
< 0 doesn't work either.
I'm sure there's a simple way out of this. Can someone help pls.
TIA
Mike|||Thank you for your quick response, Tom.
I tried the following query:
Select
Case When Datediff (dd, Getdate(), str (year (Getdate()), 4) +
Right (convert (char (8), b_day, 112), 4)) < 4 then dbo.employees
.employee_name Else '' End As TEST
From dbo.employees
And as a result I get:
1975-04-19 Bob
1934-03-03 Mary
1956-03-26 David
1958-04-06 Sonia
I expected Bob to be the only results since he has an upcoming bday which is
within 4 days.
Mike
"Tom Moreau" <tom@.dont.spam.me.cips.ca> escribi en el mensaje
news:u%23wOOSZYGHA.3868@.TK2MSFTNGP04.phx.gbl...
> The problem might be with you storing "birthday" and not "date of birth".
> This code takes the date of birth, uses the current year an calculates the
> difference between now and the birthday for this year:
> declare
> @.dob datetime
> set @.dob = '1960-04-20'
> select
> datediff (dd, getdate(), str (year (getdate()), 4) +
> right (convert (char (8), @.dob, 112), 4))
> --
> Tom
> ----
> Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
> SQL Server MVP
> Toronto, ON Canada
> .
> "Mike_B" <nospam@.yahoo.com> wrote in message
> news:ervuHLZYGHA.3848@.TK2MSFTNGP05.phx.gbl...
> Hi,
> I'm trying to create a query that will tell me when someone's birth day is
> approaching from my employee table. I tried Case When
> Datediff(dd,b_day,getdate()) < 4 Then 'Remind' Else '' End As B_dayCk
> Datepart(dd,b_day) - Datepart(dd,b_day) < 4 (This checks for the day but
> not
> for the month). If I And or Or with Datepart(mm,b_day) -
> Datepart(mm,b_day)
> < 0 doesn't work either.
> I'm sure there's a simple way out of this. Can someone help pls.
> TIA
> Mike
>|||Try:
declare @.t table
(
Employee varchar (20) primary key
, DOB datetime not null
)
insert @.t (DOB, Employee) values ('1975-04-19', 'Bob')
insert @.t (DOB, Employee) values ('1934-03-03', 'Mary')
insert @.t (DOB, Employee) values ('1956-03-26', 'David')
insert @.t (DOB, Employee) values ('1958-04-06', 'Sonia')
select
Employee
from
@.t
where
datediff (dd, getdate(), str (year (getdate()), 4) +
right (convert (char (8), DOB, 112), 4)) between 0 and 4
Part of the problem is the filtering. Some of these people have birthdays
that have already passed.
Tom
----
Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
SQL Server MVP
Toronto, ON Canada
.
"Mike_B" <nospam@.yahoo.com> wrote in message
news:uj%23yBjZYGHA.4916@.TK2MSFTNGP04.phx.gbl...
Thank you for your quick response, Tom.
I tried the following query:
Select
Case When Datediff (dd, Getdate(), str (year (Getdate()), 4) +
Right (convert (char (8), b_day, 112), 4)) < 4 then dbo.employees
.employee_name Else '' End As TEST
From dbo.employees
And as a result I get:
1975-04-19 Bob
1934-03-03 Mary
1956-03-26 David
1958-04-06 Sonia
I expected Bob to be the only results since he has an upcoming bday which is
within 4 days.
Mike
"Tom Moreau" <tom@.dont.spam.me.cips.ca> escribi en el mensaje
news:u%23wOOSZYGHA.3868@.TK2MSFTNGP04.phx.gbl...
> The problem might be with you storing "birthday" and not "date of birth".
> This code takes the date of birth, uses the current year an calculates the
> difference between now and the birthday for this year:
> declare
> @.dob datetime
> set @.dob = '1960-04-20'
> select
> datediff (dd, getdate(), str (year (getdate()), 4) +
> right (convert (char (8), @.dob, 112), 4))
> --
> Tom
> ----
> Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
> SQL Server MVP
> Toronto, ON Canada
> .
> "Mike_B" <nospam@.yahoo.com> wrote in message
> news:ervuHLZYGHA.3848@.TK2MSFTNGP05.phx.gbl...
> Hi,
> I'm trying to create a query that will tell me when someone's birth day is
> approaching from my employee table. I tried Case When
> Datediff(dd,b_day,getdate()) < 4 Then 'Remind' Else '' End As B_dayCk
> Datepart(dd,b_day) - Datepart(dd,b_day) < 4 (This checks for the day but
> not
> for the month). If I And or Or with Datepart(mm,b_day) -
> Datepart(mm,b_day)
> < 0 doesn't work either.
> I'm sure there's a simple way out of this. Can someone help pls.
> TIA
> Mike
>|||That's it. Thank you much.
Mike.
"Tom Moreau" <tom@.dont.spam.me.cips.ca> escribi en el mensaje
news:O$0JFtZYGHA.128@.TK2MSFTNGP05.phx.gbl...
> Try:
> declare @.t table
> (
> Employee varchar (20) primary key
> , DOB datetime not null
> )
> insert @.t (DOB, Employee) values ('1975-04-19', 'Bob')
> insert @.t (DOB, Employee) values ('1934-03-03', 'Mary')
> insert @.t (DOB, Employee) values ('1956-03-26', 'David')
> insert @.t (DOB, Employee) values ('1958-04-06', 'Sonia')
> select
> Employee
> from
> @.t
> where
> datediff (dd, getdate(), str (year (getdate()), 4) +
> right (convert (char (8), DOB, 112), 4)) between 0 and 4
> Part of the problem is the filtering. Some of these people have birthdays
> that have already passed.
>
> --
> Tom
> ----
> Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
> SQL Server MVP
> Toronto, ON Canada
> .
> "Mike_B" <nospam@.yahoo.com> wrote in message
> news:uj%23yBjZYGHA.4916@.TK2MSFTNGP04.phx.gbl...
> Thank you for your quick response, Tom.
> I tried the following query:
>
> Select
> Case When Datediff (dd, Getdate(), str (year (Getdate()), 4) +
> Right (convert (char (8), b_day, 112), 4)) < 4 then dbo.employees
> .employee_name Else '' End As TEST
> From dbo.employees
>
>
> And as a result I get:
>
> 1975-04-19 Bob
> 1934-03-03 Mary
> 1956-03-26 David
> 1958-04-06 Sonia
>
> I expected Bob to be the only results since he has an upcoming bday which
> is
> within 4 days.
> Mike
>
> "Tom Moreau" <tom@.dont.spam.me.cips.ca> escribi en el mensaje
> news:u%23wOOSZYGHA.3868@.TK2MSFTNGP04.phx.gbl...
>|||Hey,
Just be aware that leap year birthdays will break this code. Run the
following and see (note that I used a variable to change the comparison
date):
DECLARE @.CompDate datetime
--SET @.CompDate = GETDATE()
SET @.CompDate = '20060225'
declare @.t table
(
Employee varchar (20) primary key
, DOB datetime not null
)
insert @.t (DOB, Employee) values ('1975-04-19', 'Bob')
insert @.t (DOB, Employee) values ('1934-03-03', 'Mary')
insert @.t (DOB, Employee) values ('1956-03-26', 'David')
insert @.t (DOB, Employee) values ('1958-04-06', 'Sonia')
insert @.t (DOB, Employee) values ('2000-02-29', 'Leap')
select
Employee
from
@.t
where
datediff (dd, @.CompDate, str (year (@.CompDate), 4) +
right (convert (char (8), DOB, 112), 4)) between 0 and 4
There must be a better way, but here's my quick patch for Tom's code:
select
Employee
from
@.t
where
datediff (dd, @.CompDate, str (year (@.CompDate), 4) +
REPLACE(right (convert (char (8), DOB, 112), 4), '0229', '0228'))
between 0 and 4
HTH,
Stu|||You already have a Calendar table, right? Of course you do; it is one
of the first things you put in a schema.
So add a column for "(n) business days in the future". This kind of
column is used to get due dates for business purposes, where n = {30,
60, 90, 120}; you just want to have (n=4) from your example.|||Joe,
I prefer to do it slightly differently: I have a business_day_number
column in my Calendar table. That done, 17 business days after 20060420
is selected as
select later.business_day
from Calendar c1 join Calendar later
on later.business_day_number = c1.business_day_number + 17
where c1.business_day = '20060420'
For a non business day, such as Saturday, business_day is set to next
work day, otherwise business_day coincides with the date.
Advantages: I don't need to add another column for each and every value
of n, and I have less maintenance when all of a sudden they say January
2 is a work day - I need to modify only 2 columns, business_day and
business_day_number.
Divantage: lookups are just a tad slower.
Makes sence?|||Good point Stu.
So you're replacing the leap year to a non leap year?
"Stu" <stuart.ainsworth@.gmail.com> escribi en el mensaje
news:1145230290.066560.260910@.j33g2000cwa.googlegroups.com...
> Hey,
> Just be aware that leap year birthdays will break this code. Run the
> following and see (note that I used a variable to change the comparison
> date):
> DECLARE @.CompDate datetime
> --SET @.CompDate = GETDATE()
> SET @.CompDate = '20060225'
> declare @.t table
> (
> Employee varchar (20) primary key
> , DOB datetime not null
> )
> insert @.t (DOB, Employee) values ('1975-04-19', 'Bob')
> insert @.t (DOB, Employee) values ('1934-03-03', 'Mary')
> insert @.t (DOB, Employee) values ('1956-03-26', 'David')
> insert @.t (DOB, Employee) values ('1958-04-06', 'Sonia')
> insert @.t (DOB, Employee) values ('2000-02-29', 'Leap')
>
> select
> Employee
> from
> @.t
> where
> datediff (dd, @.CompDate, str (year (@.CompDate), 4) +
> right (convert (char (8), DOB, 112), 4)) between 0 and 4
> There must be a better way, but here's my quick patch for Tom's code:
>
> select
> Employee
> from
> @.t
> where
> datediff (dd, @.CompDate, str (year (@.CompDate), 4) +
> REPLACE(right (convert (char (8), DOB, 112), 4), '0229', '0228'))
> between 0 and 4
> HTH,
> Stu
>|||Yeah, Tom's code would create a comparison value of '20060229', which
is a non-existant date; mine simply ignores all dates of '0229', making
them '0228' instead. Note that even in a leap year, the comparison
will still go against '0228' rather than the correct date.
However, the idea of a calendar table is probably the best way to go; I
especially like Alexander's join concept.
Stu