Page 1 of 1

INSERT speed highly depends on server load

Posted: Sat Nov 14, 2015 1:43 pm
by andylens
Hi!
I faced a problem with very slow INSERTs.
I made such test: create empty table with no projections and start inserting in it in batch, using

Code: Select all

INSERT INTO table
SELECT 111,222,333 UNION ALL
SELECT 444,555,666 UNION ALL
SELECT 777,888,999;
1000 rows in one query
On idle server one query runs about 2sec and it's alright, but then I start generating some load making SELECTs (not very big load though, about 10 LA on 32 core machine) and INSERTs start to performs 20x slower ( 40-60 sec each 1000 rows INSERT ). It's too slow for my conditions. What can I do besides copying files to Vertica server and make COPY which seems to perform good, but too complicated?

Re: INSERT speed highly depends on server load

Posted: Sat Nov 14, 2015 3:25 pm
by scutter
One question is whether the INSERTs slowed down because they were queueing on the resource pools, or because system resources were truly pegged.

But you definitely don’t want to load large amounts of data using individual INSERT statements. Use COPY - the files don’t need to be local to the server. The client drivers can also automatically batch INSERT statements into COPY statements.

—Sharon

Re: INSERT speed highly depends on server load

Posted: Sat Nov 14, 2015 7:50 pm
by andylens
scutter wrote:One question is whether the INSERTs slowed down because they were queueing on the resource pools, or because system resources were truly pegged.

But you definitely don’t want to load large amounts of data using individual INSERT statements. Use COPY - the files don’t need to be local to the server. The client drivers can also automatically batch INSERT statements into COPY statements.

—Sharon
Thanks a lot for answer!

Re: INSERT speed highly depends on server load

Posted: Sat Nov 14, 2015 10:47 pm
by NorbertKrupa
You also posted this on StackOverflow.