Copy Rejected Data

Moderator: NorbertKrupa

ClevelandSteve
Newbie
Newbie
Posts: 16
Joined: Fri Aug 09, 2013 7:10 pm

Re: Copy Rejected Data

Post by ClevelandSteve » Thu Jun 26, 2014 4:07 pm

But I need to specify a directory when copying many files using ON ANY NODE. I'm not sure what you are getting at. In an attempt to give Vertica what it wants I have tried specifying a file just to see what happens. When I specify a file it gave me the opposite error, "is not a directory". I'm stuck in a damned if you do, damned if you don't scenario.
Last edited by ClevelandSteve on Thu Jun 26, 2014 7:41 pm, edited 1 time in total.

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Copy Rejected Data

Post by JimKnicely » Thu Jun 26, 2014 7:06 pm

Is that "/myDirectory/" directory a shared directory between the nodes? Or a local directory on each node?
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

ClevelandSteve
Newbie
Newbie
Posts: 16
Joined: Fri Aug 09, 2013 7:10 pm

Re: Copy Rejected Data

Post by ClevelandSteve » Thu Jun 26, 2014 7:40 pm

Yes, it is a shared directory using a common mount point on each node. I do it that way so that I don't need to ssh to each node looking for the reject files. Each of the data files being used by the Copy statement have a unique name to prevent two nodes trying to write the same reject file.

ClevelandSteve
Newbie
Newbie
Posts: 16
Joined: Fri Aug 09, 2013 7:10 pm

Re: Copy Rejected Data

Post by ClevelandSteve » Tue Sep 23, 2014 3:16 pm

I'm still having this problem, and now it happening on a big important load. Has anyone else encountered this since I started the thread?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Copy Rejected Data

Post by JimKnicely » Tue Sep 23, 2014 3:38 pm

Since you are using a shared directory maybe you need to have unique file names for each node?

Example:

Copy myTable From '/myDirectory/*' On Any Node
Rejected Data '/rejects_directory/node1.rej' On node_1, '/rejects_directory/node2.rej' on node_2
Exceptions '/exceptions_directory/node1.exp' On node_1, '/exceptions_directory/node2.exp' on node_2
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

NorbertKrupa
GURU
GURU
Posts: 527
Joined: Tue Oct 22, 2013 9:36 pm
Location: Chicago, IL
Contact:

Re: Copy Rejected Data

Post by NorbertKrupa » Tue Sep 23, 2014 3:39 pm

You may want to start a conversation on the Vertica Community.
Checkout vertica.tips for more Vertica resources.

scutter
Master
Master
Posts: 302
Joined: Tue Aug 07, 2012 2:15 am

Re: Copy Rejected Data

Post by scutter » Tue Sep 23, 2014 4:23 pm

I didn’t see any information in this thread on the version being used - or maybe I missed it. If you’re using 7.0 or later you can use REJECTED DATA AS TABLE and avoid this issue. But I imagine you would have tried that already so you must be on 6.1.3? :-)
Sharon Cutter
Vertica Consultant, Zazz Technologies LLC

Post Reply

Return to “Vertica Data Load”