COPY DIRECT vs AUTO with ROLLBACK

Moderator: NorbertKrupa

Post Reply
User avatar
Jbaskin
Intermediate
Intermediate
Posts: 61
Joined: Sat Jan 28, 2012 1:21 pm

COPY DIRECT vs AUTO with ROLLBACK

Post by Jbaskin » Wed Sep 25, 2013 7:54 pm

Hi guys,

Can I get your opinion on this? I am loading a lot of data using the DIRECT and NO COMMIT options of the COPY command. Then I run the ANALYZE_CONSTRAINTS() function on the table. If I find duplicates I rollback. Do you think it'd be faster if I did not use the AUTO option instead of DIRECT in this situation? I would think rolling back data changes on disk (ROS) would be slower than memory (WOS), or does it not matter?

What do y'all think?

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: COPY DIRECT vs AUTO with ROLLBACK

Post by id10t » Wed Sep 25, 2013 8:59 pm

Hi!

Probably it will be faster (and I believe it will be faster, because as you pointed it will be in memory).

1. But on the other hand you will need to manage Tuple Mover (for intensive loads big chance to get "too much ROS containers")
2. It's memory overhead (data in memory). What about other users?

If you have enough memory and you succeed to configure TM so go on it!

User avatar
Jbaskin
Intermediate
Intermediate
Posts: 61
Joined: Sat Jan 28, 2012 1:21 pm

Re: COPY DIRECT vs AUTO with ROLLBACK

Post by Jbaskin » Thu Sep 26, 2013 12:58 pm

Hi sKwa,

The load is happening at night when there aren't too many users online. I am just trying to get make sure I get the fastest rollback method so that I can fix what ever is wrong and reload the data as quickly as possible.

Thanks for your input!

Post Reply

Return to “Vertica Data Load”