Page 1 of 1

Vertica SCD Component Performance Issues

Posted: Fri Feb 01, 2013 1:06 am
by fsalvelt
Hi all,

We're trying to use Talend 5.1 to load data into Vertica. Unfortunately we're getting terrible performance with the Vertica SCD component. In fact, we're seeing only about 4 rows per second being inserted/updated into our dimension table! And what's really bad is that we're copying data on the same database (different schemas). Shouldn't this be super fast?

Anyone know if there are any settings in Talend I need to fix/alter? We have a 500,000 row dimension table and we can't afford a daily 34 hour maintenance window!!!

Re: Vertica SCD Component Performance Issues

Posted: Sat Feb 02, 2013 10:43 am
by zvika
If you are using updates a lot then I can tell you that we are suffering from performance issue on our updates ( part of it is our fault as we took Oracle code and use it in Vertica) and we are getting 4 updates/sec on a simple dimension .
From my experiences Updates are not performing well in Vertica.

Re: Vertica SCD Component Performance Issues

Posted: Tue Feb 05, 2013 7:21 pm
by fsalvelt
Thanks. I've given up on trying to do updates and inserts. We are truncating most tables and just reloading all the data every day.