Vertica SCD Component Performance Issues

Moderator: NorbertKrupa

Post Reply
User avatar
fsalvelt
Intermediate
Intermediate
Posts: 54
Joined: Sun Mar 18, 2012 1:34 am

Vertica SCD Component Performance Issues

Post by fsalvelt » Fri Feb 01, 2013 1:06 am

Hi all,

We're trying to use Talend 5.1 to load data into Vertica. Unfortunately we're getting terrible performance with the Vertica SCD component. In fact, we're seeing only about 4 rows per second being inserted/updated into our dimension table! And what's really bad is that we're copying data on the same database (different schemas). Shouldn't this be super fast?

Anyone know if there are any settings in Talend I need to fix/alter? We have a 500,000 row dimension table and we can't afford a daily 34 hour maintenance window!!!
Attachments
Talend-SCD-Super_Slower.png
Talend Screen Shot
Talend-SCD-Super_Slower.png (6.89 KiB) Viewed 2339 times
Thank, Fred

zvika
Beginner
Beginner
Posts: 25
Joined: Thu Apr 19, 2012 7:55 am

Re: Vertica SCD Component Performance Issues

Post by zvika » Sat Feb 02, 2013 10:43 am

If you are using updates a lot then I can tell you that we are suffering from performance issue on our updates ( part of it is our fault as we took Oracle code and use it in Vertica) and we are getting 4 updates/sec on a simple dimension .
From my experiences Updates are not performing well in Vertica.

User avatar
fsalvelt
Intermediate
Intermediate
Posts: 54
Joined: Sun Mar 18, 2012 1:34 am

Re: Vertica SCD Component Performance Issues

Post by fsalvelt » Tue Feb 05, 2013 7:21 pm

Thanks. I've given up on trying to do updates and inserts. We are truncating most tables and just reloading all the data every day.
Thank, Fred

Post Reply

Return to “Talend”