Hi all,
We're trying to use Talend 5.1 to load data into Vertica. Unfortunately we're getting terrible performance with the Vertica SCD component. In fact, we're seeing only about 4 rows per second being inserted/updated into our dimension table! And what's really bad is that we're copying data on the same database (different schemas). Shouldn't this be super fast?
Anyone know if there are any settings in Talend I need to fix/alter? We have a 500,000 row dimension table and we can't afford a daily 34 hour maintenance window!!!
Vertica SCD Component Performance Issues
Moderator: NorbertKrupa
Vertica SCD Component Performance Issues
- Attachments
-
- Talend Screen Shot
- Talend-SCD-Super_Slower.png (6.89 KiB) Viewed 18844 times
Thank, Fred
Re: Vertica SCD Component Performance Issues
If you are using updates a lot then I can tell you that we are suffering from performance issue on our updates ( part of it is our fault as we took Oracle code and use it in Vertica) and we are getting 4 updates/sec on a simple dimension .
From my experiences Updates are not performing well in Vertica.
From my experiences Updates are not performing well in Vertica.
Re: Vertica SCD Component Performance Issues
Thanks. I've given up on trying to do updates and inserts. We are truncating most tables and just reloading all the data every day.
Thank, Fred