Page 1 of 1

Store into vertica using pig

Posted: Thu Jul 19, 2012 2:58 pm
by herberth
Hi,
we are having a look into the vertica hadoop connector using pig.

Use case is, that we load data from hdfs and then store it into vertica. Simple tests worked fine so far.
But now... we would like to insert only changed data and new data.

To be honest - I have currently no idea how to do that. Is there a way to store the hdfs data in in tmp table and then merge this somehow with the 'Master'-table, meaning insert the new data and update the changed data?

I would be happy about any suggestions.

Thanks
Herberth

Re: Store into vertica using pig

Posted: Thu Oct 11, 2012 4:10 pm
by JimKnicely
Hi herberth,

Did you find a solution to your issue?

If not, check out the MERGE command here:

viewtopic.php?f=7&t=153

You should be able to execute this command via Pig...

Thanks!