INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Insert DISTINCT records into an existing table

Insert DISTINCT records into an existing table

(OP)
Hi all,
I've been tasked with creating tables used solely for displaying lists, for example: All currently available transaction types. The tables will be used in a report tool.
These tables will need to be updated on a recurring basis, so this process needs to be as efficient and robust as possible.
The database I'll be working with is HUGE! shocked So distinct queries are pretty slow.
Currently I'm testing with a db that has approx 23M records (as of a few days ago), so it gives me an idea of how fast/slow it might be.

So I started creating a table with the one column. Temp_List_Test
Then did an insert into the table from a select distinct statement for all transaction types for a partial timeframe of the last full month 12/1 - 12/17 (5217 records).
I did a partial month so that I could validate the number of records when I get the next insert statement to work. (For the entire month of Dec, the same select distinct statement returns 6367 Records)

Then because MySQL does not allow concurrent inserts, I throw a select field from table limit 1 in between.
(Normally, the inserts won't be concurrent, so this is just something to placate machinegun MySQL)

First I tried Insert...Select Distinct...From BaseTable where the date >= '2012-12-18' and the transaction type not in (Select....From Temp_List_Test)
0 rows affected Records 0.

Then I tried Insert...Select Distinct...From BaseTable where the date >= '2012-12-18' and not exists (Select....From Temp_List_Test)
0 rows affected Records 0.

Next I tried Insert...Select Distinct...From BaseTable t1 Left Join Temp_List_Test t2 on t1.TransType = t2.TransType and t2.TransType is NULL and date >= '2012-12-18'
Bummer, lost the connection during the query...either way, it was over 11 minutes and that's not good. banghead

Any help would be appreciated.
Thanks in advance.
Jacque

RE: Insert DISTINCT records into an existing table

Can you use a view instead, create it as a select statement with a rolling date?

eg :

create view V_TRANSACTION_TYPES as select distinct transaction_type from table where yourdate >= date_sub(curdate(), interval 30 day);

______________________________________________________________________
There's no present like the time, they say. - Henry's Cat.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close