Delimiters other than Pipe and Tab

Moderator: NorbertKrupa

rajasekhart
Intermediate
Intermediate
Posts: 87
Joined: Wed Apr 25, 2012 8:10 am

Delimiters other than Pipe and Tab

Post by rajasekhart » Mon May 07, 2012 7:41 am

Hi,

Do we have delimiters other than tab and pipe?

Actually, i want to load data using copy command in which pipe delimiter is used.

But if the data itself containing pipe symbol(|) as a character in a column (say col1), it is rejecting the rows , throwing the error as
Less number of columns found in the destination table.

The same case when i used tab delimiter.

So , can anyone help me how to solve this problem.

Thanks in advance ,
Raj
Rajasekhar.T|HP ATP Vertica Big Data Solutions V1

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: Delimiters other than Pipe and Tab

Post by id10t » Mon May 07, 2012 12:03 pm

Hi Raj!

Does it helps Loading UTF-8 Format Data?

johnr
Newbie
Newbie
Posts: 15
Joined: Fri Jul 27, 2012 6:30 am

Re: Delimiters other than Pipe and Tab

Post by johnr » Fri Jul 27, 2012 6:52 am

Hi Raj,

Were you successful in getting through this? I too have exactly the same problem. The delimiter is "|" and it is part of the data as well.
Would be good if you can share how did you get around the problem. That would be much appreciated.

Thanks,
-John

rajasekhart
Intermediate
Intermediate
Posts: 87
Joined: Wed Apr 25, 2012 8:10 am

Re: Delimiters other than Pipe and Tab

Post by rajasekhart » Fri Jul 27, 2012 10:37 am

Hi Johnr,

I followed a simple method to avoid this problem.

While generating the file i replaced the pipe symbol in the data with another unused symbol(say ~).
Then once file is generated with the pipe as delimiter, load that file into the table u wanted.
And then again replace the symbol(~) in the table data with pipe..

Correct me if i followed a lenghty process.

Thanks,
Raj.
Rajasekhar.T|HP ATP Vertica Big Data Solutions V1

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Delimiters other than Pipe and Tab

Post by JimKnicely » Fri Jul 27, 2012 1:07 pm

Hi,

Can you enclose the text containing the pipe symbol in your load file with double quotes and then add the ENCLOSED BY '"' option to the COPY command?

Following is an example...

The data file (notice all the | symbols in column one):

Code: Select all

dbadmin=> \! cat /usr/home/dbadmin/test.txt
"This|is|a|test|of|col1"|"This is a test of col2"
The COPY command:

Code: Select all

dbadmin=> copy test from '/usr/home/dbadmin/test.txt' enclosed by '"' exceptions '/usr/home/dbadmin/test.err';
 Rows Loaded
-------------
           1
(1 row)
The data loaded:

Code: Select all

dbadmin=> select * from test;
          col1          |          col2
------------------------+------------------------
 This|is|a|test|of|col1 | This is a test of col2
(1 row)
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

rajasekhart
Intermediate
Intermediate
Posts: 87
Joined: Wed Apr 25, 2012 8:10 am

Re: Delimiters other than Pipe and Tab

Post by rajasekhart » Mon Jul 30, 2012 8:06 am

Hi Knicely,

I too got the same idea, but the problem is "While generating the file , how can you enclose the columns with the double quote?".

If we are able to generate the file enclosed by double quote , then thats a good solution..

Kindly help me ...

Thanks,
Raj.
Rajasekhar.T|HP ATP Vertica Big Data Solutions V1

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Delimiters other than Pipe and Tab

Post by JimKnicely » Mon Jul 30, 2012 1:03 pm

Hi Raj,

How are you generating your data files? Does the tool you are using include an an option to enclose the text with quotes? Or do you already have data file and want to add the double quotes?

Thanks.
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

Post Reply

Return to “Vertica Database Development”