Store into vertica using pig

Moderator: NorbertKrupa

Post Reply
herberth
Newbie
Newbie
Posts: 1
Joined: Thu Jul 19, 2012 2:50 pm

Store into vertica using pig

Post by herberth » Thu Jul 19, 2012 2:58 pm

Hi,
we are having a look into the vertica hadoop connector using pig.

Use case is, that we load data from hdfs and then store it into vertica. Simple tests worked fine so far.
But now... we would like to insert only changed data and new data.

To be honest - I have currently no idea how to do that. Is there a way to store the hdfs data in in tmp table and then merge this somehow with the 'Master'-table, meaning insert the new data and update the changed data?

I would be happy about any suggestions.

Thanks
Herberth

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Store into vertica using pig

Post by JimKnicely » Thu Oct 11, 2012 4:10 pm

Hi herberth,

Did you find a solution to your issue?

If not, check out the MERGE command here:

viewtopic.php?f=7&t=153

You should be able to execute this command via Pig...

Thanks!
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

Post Reply

Return to “Hadoop Connector”