Showing posts with label contains. Show all posts
Showing posts with label contains. Show all posts

Wednesday, March 7, 2012

Best method for query against large database(zip codes)

I am a little unsure of the best way of doing things. I am creating a dealer locator module. I have the database which contains all necessary info such as zipcode, latitude, longitude called ZipCode. I obviously have a Dealers table which has dealer info, including zipcode. My question for the community is if there is a better way to do this process, because we in the usa alone there is over 65k zipcodes, and large chains can have thousands of locations.

Currently, i grab the zipcode and the radius the want results on from a form. I then take the zipcode and query the database, returning the lat and long of the zip the user entered on the form.

I pass this lat and long to the next procedure which incorporates the radius taken in on the form. This calculates all the lats and longs i will use to "box" off my area to perform a query on areas located within the box.

I then query Zipcodes (select * zipcodes where...) using <= and >= for all sides of the "box"(high and low latitude, and high and low longitude) at this point an inner join is done by zipcode with the zipcode on the dealers table.

Lastly, I do a distance calculation on the returned data dropping all those not within the radius.

Besides anyone have a better way to do this, does anyone have a suggestion about this:
Do I use a inner join for the dealer locator, or should I just add the lat and long to the dealers table automatically upon dealer registration of a cite?


CREATE PROCEDURE dbo.sp_GetDistanceByZip
(
@.StartZip char(5),
@.EndZip char(5)
)
AS
SET NOCOUNT ON
--DECLARE @.StartZip char(5)
--DECLARE @.EndZip char(5)

--SET @.StartZip = '92833'
--SET @.EndZip = '90005'

DECLARE @.LatA float
DECLARE @.LongA float
DECLARE @.LatB float
DECLARE @.LongB float
DECLARE @.Distance float

SET @.LatA = pi() * (Select Top 1 Latitude From zips Where ZipCode = @.startZip) / 180
SET @.LongA = pi() * (Select Top 1 Longitude From zips Where ZipCode = @.startZip) / 180
SET @.LatB = pi() * (Select Top 1 Latitude From zips Where ZipCode = @.endZip) / 180
SET @.LongB = pi() * (Select Top 1 Longitude From zips Where ZipCode = @.endZip) / 180

SET @.Distance = ACOS(
SIN
(
convert(Float,(@.LatA))
)
*SIN
(
convert(Float,(@.LATB))
)
+COS
(
convert(Float,(@.LatA))
)
*COS
(
convert(Float,(@.latB))
)
*COS
(
convert(Float,((@.longa) - (@.LongB)))
)
) * 3963.1

SELECT @.Distance, @.lata, @.longa, @.latb, @.longb
GO

My suggestion for you is to use many short selects rather than a huge join. It'll run much faster, and index it properly.|||Thank you for the reply.
Question:
If I have narrowed down the amount of possible zip codes by my "box"(only the zips, lats, and longs are going to be returned that fall within this "box"), and I then do the join, is this more taxing on the system than your suggestion?|||It is less. you should try to stay away from processing in the SQL part as much as possible.

I would also have gone with a "box" solution.|||Thanks for the input.

I would strongly suggest for anyone interested in this, to purchase the solution along with a subscription for the database. For me, this is more of a matter of being able to do it rather than saving money. If it were a matter of dollars and cents, this solution would cost me roughly forty dollars for a site, and I spent an entire day on it. I may not be the best programmer, but I am worth more than 40 a day.|||Honestly, I have the entire US/Canada, and I've run that procedure with 10 threads. Each returned in less than 2 seconds. I've condensed it to specific regions and such, but for the most part, I'm very much ok with it's performance.

Explain your "boxing" method. That interests me|||Draw a cricle, now draw a box around it, the square should touch the box at four points, all other points contained within the circle are also contained within the box. Why should we query the entire continent or world if we can calculate that box from the radius using latitude and longitude. This drastically limits the number of records that we have to due our distance calculation on. We just get the zipcode and radius from the user, pull the latitude and longitude from the database by doing a select statement of the row containing that zipcode input by the user, next we do some calculations that determine our box from the radius where the centerpoint is the zipcode. Then we select only those records whose latitude and longitude fall within our box. Then we calculate distance based on only those in the box instead of the entire country, continent, etc. Everything outside of our radius distance, gets dropped(these would have still fallen within the box). The results are returned to the user.

I hope I am clear and this explains the "box" technique. Any questions, just post back. There is a little more than what I said which is involved in this, i just tried to explain it as simple as I could. Much easier to draw something like this than use words.|||Hi, I was just wondering...did you ever happen to find a fast solution to your problem?

I am currently working on a project that involved US AND Canada zip codes.

Seeing as Canada has over 700,000 post codes I really need to find the fastest way to accomplish returning results.

The way my app will need to work is pass 1 postal code...and return all records within a 50 mile radius.

Can you please let me know how your project went? Can you please share with me how you were able to achieve this?|||This really sholud not be an issue. We have databases of 300 million records that can be parsed quickly. It is down to your indexing strategies.

For retrieving areas within a circle, use a bit of pythagoras to work out the distances. Job's done pretty easy :)|||i am also about to take on this type of project with the exact same concept...I need to retrieve a business located within a 50 mile radius from a zip code. So any input on this would be great. I am assuming I need to buy a zip code database so if anyone knows of one that will accomplish this please let me know.

if anyone is interested in building it or has the code let me know what you charge to do it.
contact me at rpanek90@.hotmail.com

Saturday, February 25, 2012

Best design for edit tracking?

Hey all,
This is a general question. Here's my scenario: We have a legacy
database. The core table within contains almost 4 million records in
SQL Server. Currently we have a web front end which allows uses to
search through the database for the information they need.
What the client wants is a web front end which allows some users to
edit the core table above. No problem. What the client also wants is
for the non-editable web front end (used for research) to display (via
colored text) which records have been edited. Example:
1.) Jimmy edits records x, y, and z in coretable1 using the editing
web app.
2.) Jane comes along and uses the research web app to hunt down some
records. She views records s through z. In her view she notices that
certain cells in rows x, y, and z are colored red. This tells her that
Jimmy has edited those specific fields in those specific rows.
My question is: what is the most efficient way to track these column
specific edits so that my web app can display them? This may seem like
a web dev question, but the reality is that my web apps have to
interact with SQL Server so if anyone has any input, I'd love to hear
it! Thanks.OK, the quickest solution to this goest something like this...
Add two rows to your core table, the first is a DateTime with a Default
constraint of GetDate(). The second column is used to identify who made the
change.
Next when update one of the rows, you need to simply include the identifier
of the person who made the change.
Next when you read the table you need to read and compare two rows. The
most recent row, and the second most recent row. This provides the
information about what changed - as a freebie you'll gain access to the old
version of the row.
Another approach would be to have two tables. The first is the core table
as it is now. The second table contains a flag for each column in your core
table, the DateTime column (as above) and the user identifier column (as
above). This time when you edit the row, you also place a new row into this
table settings the flags for the columns which were altered. Don't forget
to include a foreign key back to your core table. When you select the row
from the core table you can also join on your change map, selecting the
max(datetime) and this will tell what columns were changed in the last edit.
But it won't tell you the values that they were before. There is a big
advantage in this method as your core table doesn't need to be altered.
Regards
Colin Dawson
www.cjdawson.com
"roy.@.nderson@.gm@.il.com" <roy.anderson@.gmail.com> wrote in message
news:1147534237.719954.218790@.j33g2000cwa.googlegroups.com...
> Hey all,
> This is a general question. Here's my scenario: We have a legacy
> database. The core table within contains almost 4 million records in
> SQL Server. Currently we have a web front end which allows uses to
> search through the database for the information they need.
> What the client wants is a web front end which allows some users to
> edit the core table above. No problem. What the client also wants is
> for the non-editable web front end (used for research) to display (via
> colored text) which records have been edited. Example:
> 1.) Jimmy edits records x, y, and z in coretable1 using the editing
> web app.
> 2.) Jane comes along and uses the research web app to hunt down some
> records. She views records s through z. In her view she notices that
> certain cells in rows x, y, and z are colored red. This tells her that
> Jimmy has edited those specific fields in those specific rows.
> My question is: what is the most efficient way to track these column
> specific edits so that my web app can display them? This may seem like
> a web dev question, but the reality is that my web apps have to
> interact with SQL Server so if anyone has any input, I'd love to hear
> it! Thanks.
>|||Colin
> Add two rows to your core table, the first is a DateTime with a Default
> constraint of GetDate(). The second column is used to identify who made
> the change.
I think you menat "add to columns", and it is worth mentioning that with
this solutuin you will have to write a trigget on that table in order to
track changes
"Colin Dawson" <newsgroups@.cjdawson.com> wrote in message
news:yRn9g.68620$wl.14113@.text.news.blueyonder.co.uk...
> OK, the quickest solution to this goest something like this...
> Add two rows to your core table, the first is a DateTime with a Default
> constraint of GetDate(). The second column is used to identify who made
> the change.
> Next when update one of the rows, you need to simply include the
> identifier of the person who made the change.
> Next when you read the table you need to read and compare two rows. The
> most recent row, and the second most recent row. This provides the
> information about what changed - as a freebie you'll gain access to the
> old version of the row.
> Another approach would be to have two tables. The first is the core table
> as it is now. The second table contains a flag for each column in your
> core table, the DateTime column (as above) and the user identifier column
> (as above). This time when you edit the row, you also place a new row
> into this table settings the flags for the columns which were altered.
> Don't forget to include a foreign key back to your core table. When you
> select the row from the core table you can also join on your change map,
> selecting the max(datetime) and this will tell what columns were changed
> in the last edit. But it won't tell you the values that they were before.
> There is a big advantage in this method as your core table doesn't need to
> be altered.
> Regards
> Colin Dawson
> www.cjdawson.com
>
> "roy.@.nderson@.gm@.il.com" <roy.anderson@.gmail.com> wrote in message
> news:1147534237.719954.218790@.j33g2000cwa.googlegroups.com...
>|||oops, I did mean columns yes.
A trigger won't really be able to help as you'll need to supply extra
information than what is stored in the original table. It would be better
to use a Stored procedure and directly enter the data into the new columns.
Of course, the exception to this is that if the application connects to SQL
using seperate usernames, it is possible to use the @.@.User in a trigger to
accomplish the same result. With the applications that my company creates,
this is not possible as they alway connect with the same user (connection
pooling)
Regards
Colin Dawson
www.cjdawson.com
"Uri Dimant" <urid@.iscar.co.il> wrote in message
news:uSloXSxdGHA.4108@.TK2MSFTNGP03.phx.gbl...
> Colin
> I think you menat "add to columns", and it is worth mentioning that with
> this solutuin you will have to write a trigget on that table in order to
> track changes
>
> "Colin Dawson" <newsgroups@.cjdawson.com> wrote in message
> news:yRn9g.68620$wl.14113@.text.news.blueyonder.co.uk...
>|||On 13 May 2006 08:30:37 -0700, roy.@.nderson@.gm@.il.com wrote:

>Hey all,
>This is a general question. Here's my scenario: We have a legacy
>database. The core table within contains almost 4 million records in
>SQL Server. Currently we have a web front end which allows uses to
>search through the database for the information they need.
>What the client wants is a web front end which allows some users to
>edit the core table above. No problem. What the client also wants is
>for the non-editable web front end (used for research) to display (via
>colored text) which records have been edited. Example:
>1.) Jimmy edits records x, y, and z in coretable1 using the editing
>web app.
>2.) Jane comes along and uses the research web app to hunt down some
>records. She views records s through z. In her view she notices that
>certain cells in rows x, y, and z are colored red. This tells her that
>Jimmy has edited those specific fields in those specific rows.
>My question is: what is the most efficient way to track these column
>specific edits so that my web app can display them? This may seem like
>a web dev question, but the reality is that my web apps have to
>interact with SQL Server so if anyone has any input, I'd love to hear
>it! Thanks.
Hi Roy,
This is impossible to answer, because the requirements are incomplete.
For instance, what happens if Jimmy edits rows x, y, and z (as in your
example), then Joan edits rows w and x, then Jimmy edits z another time
and then Jane views rows s through z. What columns in what rows have to
be marked as "changed"?
Another question - what if Jimmy edits rows x, y, and z; then nothin
happens for a long time. After a year, Jane looks at rows s through z.
Should Jimmy's changes still be marked?
Hugo Kornelis, SQL Server MVP|||>
> My question is: what is the most efficient way to track these column
> specific edits so that my web app can display them? This may seem like
> a web dev question, but the reality is that my web apps have to
> interact with SQL Server so if anyone has any input, I'd love to hear
> it! Thanks.
There are several ways to accomplish that. Do you want your system to
be optimized for retrieval of the current version but support
occasional drill down into editing history. Or do you want to optimize
retrieving history of edits and are ready to pay the price of slowing
down retrieval of current version?

Friday, February 24, 2012

Best Advice

I have a table containing 65 fields. One of the fields is a varchar(6) field
which is unique and contains a client number . witin the 64 other fields
there are 10 'PIN number' fields ( PIN1, PIN2 etc ) and i need to check if a
given pin exists
ie check if '123456' exists in PIN1, PIN2 - PIN10
any suggestionsDo all the pin columns have different values? Or is one of them populated
and the rest of them NULL? And is '123456' the client number in this case?
On 3/12/05 3:37 PM, in article
A13152B1-6807-4282-8D2A-1035B7A4DAB7@.microsoft.com, "Peter Newman"
<PeterNewman@.discussions.microsoft.com> wrote:

> I have a table containing 65 fields. One of the fields is a varchar(6) fie
ld
> which is unique and contains a client number . witin the 64 other fields
> there are 10 'PIN number' fields ( PIN1, PIN2 etc ) and i need to check if
a
> given pin exists
> ie check if '123456' exists in PIN1, PIN2 - PIN10
> any suggestions|||Peter,
If there is no business-related distinction beween what is in PIn1, from
what is in Pin2 - Pin 10, i.e., if it does not matter where ValueA is in Pin
1
and ValueB in Pin2, or the other way around, then what you have is a simple
list of Pins associated with the parent record... (which s a client, I take
it)
If this is true, then you might want to read up on database normalization
concepts somewhere. The *Right* way to store these data items would be in
another table. If you had the data that way, this query, (as well as man
y
others) would be much simpler... not to mention a whole hos of other
issues...
"Peter Newman" wrote:

> I have a table containing 65 fields. One of the fields is a varchar(6) fie
ld
> which is unique and contains a client number . witin the 64 other fields
> there are 10 'PIN number' fields ( PIN1, PIN2 etc ) and i need to check if
a
> given pin exists
> ie check if '123456' exists in PIN1, PIN2 - PIN10
> any suggestions|||Sorry Aaron, The PIN numbers will be different or Null. There will always b
e
a PIN1 value but from PIN2 - PIN10 may be Null's
'123456' is the pin number i am sing in this case ane you '111111' the
client number
Thanks
"Aaron [SQL Server MVP]" wrote:

> Do all the pin columns have different values? Or is one of them populated
> and the rest of them NULL? And is '123456' the client number in this case
?
>
>
> On 3/12/05 3:37 PM, in article
> A13152B1-6807-4282-8D2A-1035B7A4DAB7@.microsoft.com, "Peter Newman"
> <PeterNewman@.discussions.microsoft.com> wrote:
>
>|||Hi
You may want to try something like:
CREATE FUNCTION dbo.fn_IsNew( @.ClientName VARCHAR(20), @.PIN CHAR(6))
RETURNS INT
AS
BEGIN
IF EXISTS ( SELECT 1 FROM Cushion WHERE ClientName = @.ClientName
AND ( PIN1 = @.PIN OR PIN2 = @.PIN OR PIN3 = @.PIN OR PIN4 = @.PIN OR PIN5 =
@.PIN OR PIN6 = @.PIN OR PIN7 = @.PIN OR PIN8 = @.PIN OR PIN9 = @.PIN OR PIN10 =
@.PIN ) )
RETURN 1
RETURN 0
END
CREATE TABLE Cushion ( ClientName VARCHAR(20) NOT NULL,
PIN char(6) NOT NULL ,
PIN1 char(6),
PIN2 char(6),
PIN3 char(6),
PIN4 char(6),
PIN5 char(6),
PIN6 char(6),
PIN7 char(6),
PIN8 char(6),
PIN9 char(6),
PIN10 char(6),
CONSTRAINT IsNew CHECK ( dbo.fn_IsNew(ClientName ,PIN) = 0 )
)
INSERT INTO Cushion ( ClientName, PIN, PIN1, PIN2 )
VALUES ( 'ABC', '123456', '234567', '345678' )
INSERT INTO Cushion ( ClientName, PIN, PIN1, PIN2 )
VALUES ( 'DEF', '123456', '123456', '345678' )
/*
Server: Msg 547, Level 16, State 1, Line 1
INSERT statement conflicted with TABLE CHECK constraint 'IsNew'. The
conflict occurred in database 'Needlepoint', table 'Cushion'.
The statement has been terminated.
*/
Normalising the structure would make your queries easier, but you may want
to try something like:
CREATE VIEW NormalisedCushion AS
SELECT ClientName, PIN1 AS OldPin FROM Cushion
UNION ALL SELECT ClientName, PIN2 FROM Cushion
UNION ALL SELECT ClientName, PIN3 FROM Cushion
UNION ALL SELECT ClientName, PIN4 FROM Cushion
UNION ALL SELECT ClientName, PIN5 FROM Cushion
UNION ALL SELECT ClientName, PIN6 FROM Cushion
UNION ALL SELECT ClientName, PIN7 FROM Cushion
UNION ALL SELECT ClientName, PIN8 FROM Cushion
UNION ALL SELECT ClientName, PIN9 FROM Cushion
UNION ALL SELECT ClientName, PIN10 FROM Cushion
SELECT * FROM NormalisedCushion
WHERE OldPin = '234567'
John
"Peter Newman" <PeterNewman@.discussions.microsoft.com> wrote in message
news:A13152B1-6807-4282-8D2A-1035B7A4DAB7@.microsoft.com...
>I have a table containing 65 fields. One of the fields is a varchar(6)
>field
> which is unique and contains a client number . witin the 64 other fields
> there are 10 'PIN number' fields ( PIN1, PIN2 etc ) and i need to check if
> a
> given pin exists
> ie check if '123456' exists in PIN1, PIN2 - PIN10
> any suggestions|||Okay, can you provide DDL, sample data, and desired results. See
http://www.aspfaq.com/5006
http://www.aspfaq.com/
(Reverse address to reply.)
"Peter Newman" <PeterNewman@.discussions.microsoft.com> wrote in message
news:0A679D44-51D8-45D5-8737-C4791812C84B@.microsoft.com...
> Sorry Aaron, The PIN numbers will be different or Null. There will always
be
> a PIN1 value but from PIN2 - PIN10 may be Null's
> '123456' is the pin number i am sing in this case ane you '111111' the
> client number
> Thanks
> "Aaron [SQL Server MVP]" wrote:
>
populated
case?
field
fields
check if a

Monday, February 13, 2012

Beginner Question: How to return list of right-most entries in a cross tab tabl

I currently have a report that contains a crosstab. Each row is a different property and each column is a different time. Several of the entries in the table are blank i.e. not all properties are recorded at all the times.

I would like to be able to turn this report into a list of the most recent values for each property along with the time it was recorded. I would be very grateful if someone could provide an example of a formula that would approximate this. Please let me know if I have not provided enough information.

Many thanks

JonI think you must ckick the time field in crosstab expert then select ascending.