Hi,
We are building a Vertica cluster consisting of 20+ nodes. This will be hosting multiple critical applications along with moderate severity applications. The goal is to make sure load on one application do not impact other application. Consider each application has its own schema.
Some applications are small and some are big , some have high concurrent users and some have moderate load.
I need opinion here on what is the best way to achieve this.
Using resource pool ( no matter how you configure, one bad application can impact all other applicatons as underlying hardware is shared) or
keep usage of applications on its exclusive set of nodes (HW level segregation)
Let's assume we have 4 application and they are classified -->
App1 -- 20 queries / sec , 100 users , DB size 100GB - Critical
App2 -- 5 queries / sec , 300 users , DB Size 50GB - Critical
App3 -- 10 queries / sec , 10 users , DB Size 10GB - Non Critical
App4 -- 10 queries / sec , 5 users , DB Size 5 GB - Non Critical
One way to achieve is this by creating 3 databases
DB1 is active on Node1-Node9 , App1 is active
DB2 is active on Node10 - Node15, App2 is active
DB3 is active on Node 15 - Node20 , App3 and App4 is active on this set of nodes.
Keep some spare nodes for each setup , incase , of failure.
Any thoughts on this set up. Are there any drawbacks ?
Isolated Databases in Vertica cluster
Moderator: NorbertKrupa
-
- GURU
- Posts: 527
- Joined: Tue Oct 22, 2013 9:36 pm
- Location: Chicago, IL
- Contact:
Re: Isolated Databases in Vertica cluster
What you're looking to implement is a multitenancy architecture.
Application 1 projections segmented on Nodes 1 - 9
Application 2 projections segmented on Nodes 10 - 15
Application 3 projections segmented on Nodes 16 - 20
The issue here is with cluster integrity, meaning a single "application" could shutdown the entire database. This would limit your resource activity to the nodes on which the projections reside on.
If it's an option, you could split these into physical clusters. I would recommend using the spare nodes instead of leaving them as standby.
However, the most sensible architecture would be to use schemas as the logical organization layer (in lieu of the database). You then create resource pools and allocate them to certain applications based on the needs of the application.
That's not necessarily true. Properly configured, the resource pools are there to ensure that this doesn't happen.aas1974 wrote:Using resource pool ( no matter how you configure, one bad application can impact all other applicatons as underlying hardware is shared)
You can't have more than one database running at once. What you can do, however, under the guidance of support, is break up projection segmentation by nodes.aas1974 wrote:One way to achieve is this by creating 3 databases
DB1 is active on Node1-Node9 , App1 is active
DB2 is active on Node10 - Node15, App2 is active
DB3 is active on Node 15 - Node20 , App3 and App4 is active on this set of nodes.
Application 1 projections segmented on Nodes 1 - 9
Application 2 projections segmented on Nodes 10 - 15
Application 3 projections segmented on Nodes 16 - 20
The issue here is with cluster integrity, meaning a single "application" could shutdown the entire database. This would limit your resource activity to the nodes on which the projections reside on.
If it's an option, you could split these into physical clusters. I would recommend using the spare nodes instead of leaving them as standby.
However, the most sensible architecture would be to use schemas as the logical organization layer (in lieu of the database). You then create resource pools and allocate them to certain applications based on the needs of the application.
Checkout vertica.tips for more Vertica resources.
Re: Isolated Databases in Vertica cluster
Hi Nobert,
Thanks for the input. We are exploring resource pool option as well.
On projections, it can be configured to be created on certain nodes, only issue is ,with ALL NODES options, it can spread across nodes. I have
not seen any setting which can always restrict to be on certain node. It might cause maintenance issue down the line.
If DB itself is created on certain nodes, even if you put ALL NODES options, projection will be created only on nodes DB is created with.
Thanks for the input. We are exploring resource pool option as well.
You can have multiple databases running at the same time on the same cluster as long as ports are different.norbertk wrote:You can't have more than one database running at once. What you can do, however, under the guidance of support, is break up projection segmentation by nodes.
On projections, it can be configured to be created on certain nodes, only issue is ,with ALL NODES options, it can spread across nodes. I have
not seen any setting which can always restrict to be on certain node. It might cause maintenance issue down the line.
If DB itself is created on certain nodes, even if you put ALL NODES options, projection will be created only on nodes DB is created with.
-
- GURU
- Posts: 527
- Joined: Tue Oct 22, 2013 9:36 pm
- Location: Chicago, IL
- Contact:
Re: Isolated Databases in Vertica cluster
A brilliant man once said, "I would caution against confusing 'possible' with 'deployable'."aas1974 wrote:You can have multiple databases running at the same time on the same cluster as long as ports are different.
Checkout vertica.tips for more Vertica resources.
Re: Isolated Databases in Vertica cluster
good one
Need to let Vertica know to make possible and deployable synonymous
Need to let Vertica know to make possible and deployable synonymous