Oracle to Vertica process !

Moderator: NorbertKrupa

Post Reply
adrian.oprea
Intermediate
Intermediate
Posts: 163
Joined: Tue Jun 19, 2012 2:44 pm
Location: Rio de Janeiro
Contact:

Oracle to Vertica process !

Post by adrian.oprea » Wed Oct 17, 2012 4:37 pm

I am in the process of creating an Oracle to Vertica process !!
We are looking to create a Vertica DB that will run heavy reports. For now is all cool Vertica is fast space use is great and all well and nice until we get to the main part getting the data from Oracle to Vertica.
OK , initial load is ok, dump to csv from Oracle to Vertica , load times are a joke no problem so far everybody things is bad joke or there's some magic stuff going on !! well is Simply Fast.
Bad Part Now -> Databases are up and going ORACLE/VERTICA - and i have data getting altered in ORACLE so i need to replicate my data in VERTICA. What now:
From my tests and from what i can understand about Vertica insert,updates are not to used unless maybe max 20 per sec - so real time replication is out of question.
So i was thinking to read the arch log from oracle and ETL -it to create CSV data with the new data, altered data, deleted values-changed data and then applied it into VERTICA but i can not get a list like this :
insert into ....
update xx where ....
delete from where ...
Because explicit data change in VERTICA leads to slow performance.

So i am looking for some ideas about how i can solve this issue, knowing i cannot:
1- alter my ORACLE production structure.
2- use ORACLE env resources for filtering the data.
3- cannot use insert, update or delete statements in my VERTICA load process.

Things i depend on :
1- the use of copy command
2- data consistency
3- a max of 60 min window(every 60 min - new/altered data need to go to VERTICA).

I have seen the Continuent data replication, but it seems that nowbody wants to sell their prod , i cannot get in touch with them.

Thank you all !!! Hope there are some big heads out there !!
trying so hard !!!

jpcavanaugh
Intermediate
Intermediate
Posts: 149
Joined: Mon Apr 30, 2012 10:04 pm
Location: New York
Contact:

Re: Oracle to Vertica process !

Post by jpcavanaugh » Wed Oct 17, 2012 4:40 pm

If you are looking for a product to do it, I hear good things about Wisdomforce but I don't know if its a good fit for you. If you are looking for other ideas, can you batch the changes ... say every X minutes? Do you have PK definitions on your tables?

adrian.oprea
Intermediate
Intermediate
Posts: 163
Joined: Tue Jun 19, 2012 2:44 pm
Location: Rio de Janeiro
Contact:

Re: Oracle to Vertica process !

Post by adrian.oprea » Wed Oct 17, 2012 5:02 pm

For now is just on paper, looking to remove the critical point first !!
How do you use Vertica ??
trying so hard !!!

Post Reply

Return to “Vertica Migration”