Exporting Vertica tables to S3

Forum to discuss topics related to loading data into Vertica (i.e. COPY command)

Moderator: NorbertKrupa

Post Reply
Beg1nner
Newbie
Newbie
Posts: 20
Joined: Fri Jul 24, 2015 8:46 pm

Exporting Vertica tables to S3

Post by Beg1nner » Fri Feb 24, 2017 3:24 pm

Hi all,

We are using vertica 7.2.1 and we have 50 odd tables in a schema (total data size ~5 TB). What is the best approach you would suggest to move this data into S3?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1736
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Exporting Vertica tables to S3

Post by JimKnicely » Mon Feb 27, 2017 2:49 pm

Have you check out "Exporting to Amazon S3 using the Vertica Library for Amazon Web Services"?

https://my.vertica.com/docs/8.0.x/HTML/ ... ibrary.htm
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

Beg1nner
Newbie
Newbie
Posts: 20
Joined: Fri Jul 24, 2015 8:46 pm

Re: Exporting Vertica tables to S3

Post by Beg1nner » Tue Feb 28, 2017 4:57 pm

Yes I did check them out. But we are on Vertica 7.2.1. Other than upgrading and using awslib, is there any better way?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1736
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Exporting Vertica tables to S3

Post by JimKnicely » Sat Mar 04, 2017 4:05 am

Hi,

Maybe you can mount an S3 Bucket per the the instructions in the article "How to Mount S3 Bucket on CentOS/RHEL and Ubuntu using S3FS"?

https://tecadmin.net/mount-s3-bucket-ce ... sing-s3fs/

Once you have the S3 Bucket mounted, you should be able to export data out of Vertica onto it.

vsql -c "select * from some_table;" -o /s3mnt/some_table.txt
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

Post Reply

Return to “Vertica Data Load”

Who is online

Users browsing this forum: No registered users and 1 guest