Partitions and ROS Containers

Moderator: NorbertKrupa

Post Reply
beth
Intermediate
Intermediate
Posts: 58
Joined: Tue Oct 01, 2013 12:42 pm

Partitions and ROS Containers

Post by beth » Fri Nov 13, 2015 4:47 pm

Hi all,

From the admin guide I read this:

Partitions and ROS Containers
1. Data is automatically split into partitions during load / refresh / recovery operations.
2. The Tuple Mover maintains physical separation of partitions.
3. Each ROS container contains data for a single partition, though there can be multiple ROS containers for a single partition.

So for #3, under what circumstances would there be "multiple ROS containers for a single partition"?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1775
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Partitions and ROS Containers

Post by JimKnicely » Fri Nov 13, 2015 5:08 pm

Prior to a MERGEOUT operation, you could see extra ROS containers for partitioned data.

Simple example:

Code: Select all

dbadmin=> drop table test;
DROP TABLE

dbadmin=> create table test (c1 int not null, c2 int) partition by c1;
CREATE TABLE

dbadmin=> insert /*+ direct */ into test values (1, 1);
OUTPUT
--------
      1
(1 row)

dbadmin=> insert /*+ direct */ into test values (2, 1);
OUTPUT
--------
      1
(1 row)
After the inserts, we have two ROS containers (see ros_count column):

Code: Select all

dbadmin=> select projection_name, ros_row_count, ros_count from projection_storage where anchor_table_name = 'test';
projection_name | ros_row_count | ros_count
-----------------+---------------+-----------
test_super      |             2 |         2
(1 row)

Code: Select all

dbadmin=> insert /*+ direct */ into test values (1, 1);
OUTPUT
--------
      1
(1 row)

dbadmin=> insert /*+ direct */ into test values (2, 1);
OUTPUT
--------
      1
(1 row)
After these inserts, we now have four ROS containers. Vertica did not at this point combine the partition data into a single container for each partition:

Code: Select all

dbadmin=> select projection_name, ros_row_count, ros_count from projection_storage where anchor_table_name = 'test';
projection_name | ros_row_count | ros_count
-----------------+---------------+-----------
test_super      |             4 |         4
(1 row)

dbadmin=> select do_tm_task('mergeout', 'test');
                              do_tm_task
----------------------------------------------------------------------
Task: mergeout
(Table: public.test) (Projection: public.test_super)

(1 row)
But after a MERGEOUT, Vertica did combine (i.e. merge) the partitioned data into a single container for each partition:

Code: Select all

dbadmin=> select projection_name, ros_row_count, ros_count from projection_storage where anchor_table_name = 'test';
projection_name | ros_row_count | ros_count
-----------------+---------------+-----------
test_super      |             4 |         2
(1 row)
Make sense?
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

scutter
Master
Master
Posts: 301
Joined: Tue Aug 07, 2012 2:15 am

Re: Partitions and ROS Containers

Post by scutter » Fri Nov 13, 2015 8:08 pm

A few additional comments beyond what Jim wrote.

- The “active” partitions (1 or more) will have more than one ROS container, until they become “past partitions”

- Past partitions may have more than one ROS container. Originally the TM would aggressively merge past partitions into a single ROS container, but in current versions it won’t always merge multiple ROS containers in past partitions to a single ROS. If you have a ROS container in a past partition it’s not necessary worthwhile to merge it with a large ROS, so the TM won’t do that.

- DVROSes (for delete vectors) may also co-exist and count towards the number of ROS containers.

- I’ve observed that recent versions have a higher limit for ROS containers than the technical limit. For example if the limit in the configuration parameter is 1024, it lets you go to 2048. I haven’t spotted anything in the documentation for this, so there’s no commitment that this would remain true, but it does give you more head room.

—Sharon
Sharon Cutter
Vertica Consultant, Zazz Technologies LLC

beth
Intermediate
Intermediate
Posts: 58
Joined: Tue Oct 01, 2013 12:42 pm

Re: Partitions and ROS Containers

Post by beth » Sat Nov 14, 2015 4:34 pm

Thank you Jim and Sharon! Your posts are very, very helpful!!!

Post Reply

Return to “New to Vertica Database Administration”