Vertica SCD Component Performance Issues
Posted: Fri Feb 01, 2013 1:06 am
Hi all,
We're trying to use Talend 5.1 to load data into Vertica. Unfortunately we're getting terrible performance with the Vertica SCD component. In fact, we're seeing only about 4 rows per second being inserted/updated into our dimension table! And what's really bad is that we're copying data on the same database (different schemas). Shouldn't this be super fast?
Anyone know if there are any settings in Talend I need to fix/alter? We have a 500,000 row dimension table and we can't afford a daily 34 hour maintenance window!!!
We're trying to use Talend 5.1 to load data into Vertica. Unfortunately we're getting terrible performance with the Vertica SCD component. In fact, we're seeing only about 4 rows per second being inserted/updated into our dimension table! And what's really bad is that we're copying data on the same database (different schemas). Shouldn't this be super fast?
Anyone know if there are any settings in Talend I need to fix/alter? We have a 500,000 row dimension table and we can't afford a daily 34 hour maintenance window!!!