negative partition_key in partitions table?

Moderator: NorbertKrupa

Post Reply
kalensk
Newbie
Newbie
Posts: 1
Joined: Wed Dec 23, 2015 2:58 am

negative partition_key in partitions table?

Post by kalensk » Wed Dec 23, 2015 3:01 am

Does it make sense for the partition_key to be negative? I am seeing the following error in my application and then the following error on vertica.


011org.springframework.dao.DataAccessResourceFailureException: StatementCallback; SQL [SELECT DROP_PARTITION('<table name>', -17875912)]; [Vertica][VJDBC](5060) ERROR: Too many data partitions; nested exception is java.sql.SQLNonTransientException: [Vertica][VJDBC](5060) ERROR: Too many data partitions


vertica: Event Posted: Event Code:14 Event Id:261 Event Severity: Warning [4] PostedTimestamp: 2015-12-22 16:49:11.907154 ExpirationTimestamp: 2015-12-22 16:49:26.907154 EventCodeDescription: Timer Service Task Error ProblemDescription: threadShim: Too many data partitions DatabaseName: <db name> Hostname: <node hostname>



The above occurred with following settings in place:
ALTER DATABASE <db name> SET MergeOutInterval = 60;
ALTER RESOURCE POOL tm MEMORYSIZE '2G' MAXCONCURRENCY 6;
ALTER DATABASE <db name> SET MergeoutCache = 1;


What should I be investigating to figure out the "too many data partitions" errors?

NorbertKrupa
GURU
GURU
Posts: 527
Joined: Tue Oct 22, 2013 9:36 pm
Location: Chicago, IL
Contact:

Re: negative partition_key in partitions table?

Post by NorbertKrupa » Wed Dec 23, 2015 4:30 am

I would check how many partitions exist and how partitioning is defined. Vertica recommends between 10-20 partitions per table.

The system tables to look at would include v_monitor.partitions, v_monitor.partition_status, v_monitor.partition_reorganize_errors.

The more partitions per table, the more files that will potentially be opened. For example, a table with 20 columns and a year's worth of data partitioned by day, will have 365 partitions. If a query requests all 365 days and all 20 columns, potentially 7300 files may be opened. With a ulimit of 65536, this can become easily filled up.
Checkout vertica.tips for more Vertica resources.

Post Reply

Return to “Vertica Data Load”