identity columns
Posted: Mon Jun 05, 2017 2:56 pm
Ive been experimenting with the identity settings and had some questions.
I have a table with 1.5 billion records in IQ that I need to move to Vertica. It makes use of an identity column.
In my testing so-far, when I specify "identity(1,1,1)" I get the correct identity values migrated from IQ to Vertica, however, it takes approx 24 hours to load 6M records. That would translate into an approx 8-month load time for the full 1.5B records, which is obviously not going to work.
When I increase the cache value, I get much better load times, but I would need to go to "identity(1,1,1500000000) in order to get the correct values to load.
Is there any downside to using such a large cache value for this column? Does it consume un-necessary resources or cause other issues that I may not be aware of?
Alternativly, is there a way to change the "cache" value after I load the table and go back to a "identity(1,1,1)" setting after the back-fill is completed?
I have a table with 1.5 billion records in IQ that I need to move to Vertica. It makes use of an identity column.
In my testing so-far, when I specify "identity(1,1,1)" I get the correct identity values migrated from IQ to Vertica, however, it takes approx 24 hours to load 6M records. That would translate into an approx 8-month load time for the full 1.5B records, which is obviously not going to work.
When I increase the cache value, I get much better load times, but I would need to go to "identity(1,1,1500000000) in order to get the correct values to load.
Is there any downside to using such a large cache value for this column? Does it consume un-necessary resources or cause other issues that I may not be aware of?
Alternativly, is there a way to change the "cache" value after I load the table and go back to a "identity(1,1,1)" setting after the back-fill is completed?