Compression Ratio varies with number of nodes

Moderator: NorbertKrupa

banurajadurai
Newbie
Newbie
Posts: 21
Joined: Tue Aug 27, 2013 10:31 am

Compression Ratio varies with number of nodes

Post by banurajadurai » Tue Nov 12, 2013 7:12 am

HI

How Exactly data is compressed in Multinode Vertica set up ? Is there any difference in Compression ratio with number of nodes ?

banurajadurai
Newbie
Newbie
Posts: 21
Joined: Tue Aug 27, 2013 10:31 am

Re: Compression Ratio varies with number of nodes

Post by banurajadurai » Tue Nov 12, 2013 10:07 am

Reply soon !!!!!!

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1778
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Compression Ratio varies with number of nodes

Post by JimKnicely » Tue Nov 12, 2013 1:52 pm

Hi,

I don't think the compression ratio would change significantly based on the number of nodes. That is, the sum of the compressed data on all nodes should have a similar size to the compressed data on a single node. But that's assuming you have optimized projections. Default super projections might not optimize database performance, resulting in slow query performance and low data compression.
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

banurajadurai
Newbie
Newbie
Posts: 21
Joined: Tue Aug 27, 2013 10:31 am

Re: Compression Ratio varies with number of nodes

Post by banurajadurai » Wed Nov 13, 2013 6:23 am

i have loaded 39 GB of raw data
In single node it is compressed to 5.1 GB and in 3 node it is compressed to 12.01 GB with K safety 1 (Original data + 1 backup copy)


1 GB differece .....


All Table structure , Optimized Query , Db designs are same in both the single and 3 node Vertica ..

why this large difference ?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1778
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Compression Ratio varies with number of nodes

Post by JimKnicely » Wed Nov 13, 2013 3:19 pm

Are you sure the physical designs are the same?

What are the projection counts from PROJECTIONS table in each solution? What are the projection counts where the is_super_projection = t?

On the single node I bet all projections are super projections that can be poorly compressed as expressed by the documentation.
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

banurajadurai
Newbie
Newbie
Posts: 21
Joined: Tue Aug 27, 2013 10:31 am

Re: Compression Ratio varies with number of nodes

Post by banurajadurai » Thu Nov 14, 2013 5:52 am

IN single node vertica , For super projection Compression ratio was 12.71 GB .

User avatar
nnani
Master
Master
Posts: 302
Joined: Fri Apr 13, 2012 6:28 am
Contact:

Re: Compression Ratio varies with number of nodes

Post by nnani » Thu Nov 14, 2013 10:41 am

So now the compression size is matching in your single node and your 3 node cluster.
nnani........
Long way to go

You can check out my blogs at vertica-howto

Post Reply

Return to “Vertica Analytics”