Hi all,
We have two environments of Vertica 7.1.1-10, 10 node each, database of 40TB, K-safety 1.
One as the production and the other is DR.
We want to close the DR site and merge the two clusters to one 20 nodes production cluster.
Several questions:
1. Do I need to uninstall Vertica from the DR site before I'm adding the nodes to the production site? or I need only to drop the existing database on the DR site?
2. How it's work with the downtime when I'm adding the nodes? There will be downtime? Can I still work with the production environment while I'm adding the nodes (at the re-balancing time)?
3. Can I add 10 nodes together? or I need to add them in batches, 5 nodes each time for example?
4. What is the overhead on the system?
5. How can I determent the re-balancing time?
If you have any more insights and things that I need to check or to give more attention... I'd love to hear
Thanks
Chen
Marge two Vertica clusters
Moderator: NorbertKrupa
-
- GURU
- Posts: 527
- Joined: Tue Oct 22, 2013 9:36 pm
- Location: Chicago, IL
- Contact:
Re: Marge two Vertica clusters
While we can certainly guide you with your questions, this is something that should be done under guidance of support.
Checkout vertica.tips for more Vertica resources.
- JimKnicely
- Site Admin
- Posts: 1825
- Joined: Sat Jan 21, 2012 4:58 am
- Contact:
Re: Marge two Vertica clusters
I totally agree with #NorbertKrupa. You are clearly dealing with a lot of data.
However ... First, I would upgrade to Vertica 7.2. It's cool.
Anyway, my fun 2 cents:
1. Do I need to uninstall Vertica from the DR site before I'm adding the nodes to the production site? or I need only to drop the existing database on the DR site? JIMK: On the DR site, I would suggest shutting down Vertica and deleting the files from the catalog and data directories (make sure you won't need this data) that Vertica will use when adding the nodes to the production cluster. Actually, I think you'll get an error from the installer if they are not empty.
2. How it's work with the downtime when I'm adding the nodes? There will be downtime? Can I still work with the production environment while I'm adding the nodes (at the re-balancing time)? JIMK: There is no down time. You will of course see a performance hit as the hardware (i.e. CPU, network, disks) have to move the data! But your database is still up and query-able.
3. Can I add 10 nodes together? or I need to add them in batches, 5 nodes each time for example? [JIMK]: Moving massive amounts of data between servers is of course slow. Just add all of the nodes. Take the hit! But in the end you will be really happy with the new cluster size.
4. What is the overhead on the system? JIMK: A lot see #2, do it in the off hours
5. How can I determent the re-balancing time? JIMK How many projections do you have? What is your network and disk read/write speeds? That'll determine the time.
If you have any more insights and things that I need to check or to give more attention... I'd love to hear
JIMK: Why are you getting rid of your DR site? What happens in the event of a disaster?
However ... First, I would upgrade to Vertica 7.2. It's cool.
Anyway, my fun 2 cents:
1. Do I need to uninstall Vertica from the DR site before I'm adding the nodes to the production site? or I need only to drop the existing database on the DR site? JIMK: On the DR site, I would suggest shutting down Vertica and deleting the files from the catalog and data directories (make sure you won't need this data) that Vertica will use when adding the nodes to the production cluster. Actually, I think you'll get an error from the installer if they are not empty.
2. How it's work with the downtime when I'm adding the nodes? There will be downtime? Can I still work with the production environment while I'm adding the nodes (at the re-balancing time)? JIMK: There is no down time. You will of course see a performance hit as the hardware (i.e. CPU, network, disks) have to move the data! But your database is still up and query-able.
3. Can I add 10 nodes together? or I need to add them in batches, 5 nodes each time for example? [JIMK]: Moving massive amounts of data between servers is of course slow. Just add all of the nodes. Take the hit! But in the end you will be really happy with the new cluster size.
4. What is the overhead on the system? JIMK: A lot see #2, do it in the off hours
5. How can I determent the re-balancing time? JIMK How many projections do you have? What is your network and disk read/write speeds? That'll determine the time.
If you have any more insights and things that I need to check or to give more attention... I'd love to hear
JIMK: Why are you getting rid of your DR site? What happens in the event of a disaster?
Jim Knicely
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
-
- Intermediate
- Posts: 51
- Joined: Wed Sep 09, 2015 9:34 am
Re: Marge two Vertica clusters
Hi JimKnicely, thank you for your answer.
We create a new DR site at AWS.
We create a new DR site at AWS.
- JimKnicely
- Site Admin
- Posts: 1825
- Joined: Sat Jan 21, 2012 4:58 am
- Contact:
Re: Marge two Vertica clusters
AWS as a DR? That is smart
Jim Knicely
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
-
- Intermediate
- Posts: 51
- Joined: Wed Sep 09, 2015 9:34 am
Re: Marge two Vertica clusters
Not a good idea? why?
- JimKnicely
- Site Admin
- Posts: 1825
- Joined: Sat Jan 21, 2012 4:58 am
- Contact:
Re: Marge two Vertica clusters
No, I think it is a good idea! I like the concept of AWS as a back up site. Make sure your data storage policies comply. I lot of my clients have issues with storing data in the cloud (i.e. HIPPA laws).
Jim Knicely
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.
Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.