Parallel Execution of UDFs

Moderator: NorbertKrupa

Post Reply
Yonatan
Newbie
Newbie
Posts: 1
Joined: Thu Aug 22, 2013 12:39 pm

Parallel Execution of UDFs

Post by Yonatan » Thu Aug 22, 2013 4:15 pm

Hi,
I have implemented a UDF written in C++ and I am trying to understand the performance I am experiencing.
I am running on a 24 cores 16GB Server (single node) and using Vertica 6.1.
My function is running in fenced mode.

My query is relatively simple and looks like:
select
key1,
key2,
key3,
AnalyzeMetric(
timestamp,
value)
OVER
(PARTITION BY key1,
key2,
key3
ORDER BY timestamp)
from DB.yonatan.application_performance where application like 'p%'

My function named AnalyzeMetric returns a single row for each invocation, and the partition clause I am using should break the input into exactly 1000 invocations that potentially may be executed in parallel but for some reason it seems like Vertica runs my query using a single thread and does not take advantage of other available cores.

During execution I see that UDX process nlwp goes from 2 to 3.

ps -eo pid,comm,lstart,etime,time,nlwp | grep vertica-udx-C++
10639 vertica-udx-C++ Thu Aug 22 14:03:43 2013 41:55 00:22:23 /opt/vertica/bin/vertica-ud 3

Should I be expecting parallel execution only on multi node cluster where data is segmented?
What am I missing?

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: Parallel Execution of UDFs

Post by id10t » Fri Aug 23, 2013 8:43 am

Hi!

Bug in Vertica. Follow to this thread : https://community.vertica.com/vertica/t ... y_utilized

Regards Daniel

scutter
Master
Master
Posts: 302
Joined: Tue Aug 07, 2012 2:15 am

Re: Parallel Execution of UDFs

Post by scutter » Fri Aug 23, 2013 1:06 pm

I think that the pointed to Community discussion on fully utilizing the system is related, but not necessarily the same root issue. That discussion is about Joins and the fact that they use a subset of the cores.

The documentation states: "Analytic functions using the partition by clause automatically run in parallel when possible to improve query performance. " It sounds like this isn't true for UDFs - that they don't try to run in parallel. I'd ask this question over in the Community or open a Support Ticket for clarification.

--Sharon
Sharon Cutter
Vertica Consultant, Zazz Technologies LLC

Post Reply

Return to “Vertica User Defined Functions (UDFs)”