Delimiters other than Pipe and Tab

Moderator: NorbertKrupa

id10t
GURU
GURU
Posts: 732
Joined: Mon Apr 16, 2012 2:44 pm

Re: Delimiters other than Pipe and Tab

Post by id10t » Mon Jul 30, 2012 3:40 pm

Hi!

Vertica support any ASCII character in COPY statement except `\000` - end of string. For example char PERCENT `%` is `\045` in OCT BASE.
  • Create data with PERCENT as delimiter

    Code: Select all

    %> cat out.dat 
    foo%bar
    baz%egg
    sux%sex
    
  • Create table

    Code: Select all

    %> vsql -c "create table PercentDelimiter (col1 varchar(3), col2 varchar(3))"
    CREATE TABLE
    
  • Load data with PERCENT as delimiter

    Code: Select all

    %> vsql -c "copy PercentDelimiter from '/home/dbadmin/out.dat' delimiter e'\045' direct" 
    Rows Loaded 
    -------------
               3
    (1 row)
    
  • Show data

    Code: Select all

    %> vsql -c "select * from PercentDelimiter"
     col1 | col2 
    ------+------
     baz  | egg
     foo  | bar
     sux  | sex
    (3 rows)
    
---
PS In such way you can use ESCAPE char as delimiter or any non-printable char as delimiter
Image

rajasekhart
Intermediate
Intermediate
Posts: 87
Joined: Wed Apr 25, 2012 8:10 am

Re: Delimiters other than Pipe and Tab

Post by rajasekhart » Tue Jul 31, 2012 8:11 am

Hi sKwa,

Thats solved my problem.

Thanks a lot,
Raj.
Rajasekhar.T|HP ATP Vertica Big Data Solutions V1

rajasekhart
Intermediate
Intermediate
Posts: 87
Joined: Wed Apr 25, 2012 8:10 am

Re: Delimiters other than Pipe and Tab

Post by rajasekhart » Tue Jul 31, 2012 8:29 am

Hi Knicely,

I am generating the files in the following way.

please find the attachment.

Thanks,
Raj.
Attachments
genfile1.JPG
genfile1.JPG (24.7 KiB) Viewed 14429 times
Rajasekhar.T|HP ATP Vertica Big Data Solutions V1

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Delimiters other than Pipe and Tab

Post by JimKnicely » Tue Jul 31, 2012 5:07 pm

Raj,

Are you copying data from Vertica to Vertica? I'm asking because it looks like you're generating your data files from a Vertica database and this post thread began as a question about how to load data into Vertica...

Thanks!
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

johnr
Newbie
Newbie
Posts: 15
Joined: Fri Jul 27, 2012 6:30 am

Re: Delimiters other than Pipe and Tab

Post by johnr » Wed Aug 01, 2012 2:30 am

Hi Knicely and all,

Thanks for the replies.

My prob is the data is generated from a different environment and we don't have control over it.
So, we can't explicitly go and change the format of the data after it has arrived since it is multiple files with millions of rows.
More than that, "|" is just one of them. There are a number of other characters as well which have been included in the data.

I did try as suggested in one of the replies different delimiters and went with what was allowed to be loaded. Thanks for the suggestions and recommendations. Much appreciated.

-John

johnr
Newbie
Newbie
Posts: 15
Joined: Fri Jul 27, 2012 6:30 am

Re: Delimiters other than Pipe and Tab

Post by johnr » Wed Aug 01, 2012 2:36 am

Hi Raj,

Thanks for your reply. I did try with different delimiters and the one that is not part of the data obviously works.

-John

jpcavanaugh
Intermediate
Intermediate
Posts: 149
Joined: Mon Apr 30, 2012 10:04 pm
Location: New York
Contact:

Re: Delimiters other than Pipe and Tab

Post by jpcavanaugh » Wed Aug 01, 2012 2:50 am

Are you exporting from another Vertica environment?

Post Reply

Return to “Vertica Database Development”