COPY running, but no tuples in projection_storage

[dbmsuser7]

We have a long-running COPY operation going. I can see that rows are coming in via LOAD_STREAMS, but when i look in projection_storage, I don't see anything in ROS or WOS (no rows, no bytes). (LOAD_STREAMS sees billions of rows.)

This operation has been running for tens of hours, and I'm a little worried that bytes are being dumped into the ether, but I'm not sure how to check that.

The last operation I see in vertica.log with the same transaction ID as the copy statement is

Code: Select all

 <INFO> finalizeThreadsToUseForLoad(): max_threads_allowed = 24, numSources = 1

But that is dated 2014-10-26 11:36:46...

Here's some of the info from LOAD_STREAMS:

Code: Select all

load_start             | 2014-10-26 11:36:46.198257-04
load_duration_ms       | 
is_executing           | t
accepted_row_count     | 15137058816
rejected_row_count     | 0
read_bytes             | 0
input_file_size_bytes  | 0
parse_complete_percent | 0
unsorted_row_count     | 30273982976
sorted_row_count       | 0
sort_complete_percent  | 0

[scutter]

LOAD_STREAMS suggests that it’s sorting data. The efficiency of the sort will be a function of how large the “query budget” (resource_pool_status table) is for the pool that you’re using to load — keeping more data in memory and spilling less to disk will be faster, and create larger/fewer chunks of data on disk for sorting. If you want postive confirmation of activity, query the EXECUTION_ENGINE_PROFILES table for that transaction_id and statement_id, for the Sort operator_name, and look at the various “merge phase” operators. There’s one for total number of merge phases, and one for merge phases completed.

—Sharon

[dbmsuser7]

There's no rows at all in EE Profiles for operator Sort (for any counter).

[scutter]

Sorry - try the DataTarget operator.

[dbmsuser7]

Thanks.

Just for the record, there were "merge phase" counters with nonzero values for the DataTarget operator, and the COPY did end up functioning correctly (i.e., the data did finally appear in the correct table).