Exporting Vertica tables to S3

Moderator: NorbertKrupa

Post Reply
Beg1nner
Newbie
Newbie
Posts: 20
Joined: Fri Jul 24, 2015 8:46 pm

Exporting Vertica tables to S3

Post by Beg1nner » Fri Feb 24, 2017 3:24 pm

Hi all,

We are using vertica 7.2.1 and we have 50 odd tables in a schema (total data size ~5 TB). What is the best approach you would suggest to move this data into S3?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1758
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Exporting Vertica tables to S3

Post by JimKnicely » Mon Feb 27, 2017 2:49 pm

Have you check out "Exporting to Amazon S3 using the Vertica Library for Amazon Web Services"?

https://my.vertica.com/docs/8.0.x/HTML/ ... ibrary.htm
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

Beg1nner
Newbie
Newbie
Posts: 20
Joined: Fri Jul 24, 2015 8:46 pm

Re: Exporting Vertica tables to S3

Post by Beg1nner » Tue Feb 28, 2017 4:57 pm

Yes I did check them out. But we are on Vertica 7.2.1. Other than upgrading and using awslib, is there any better way?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1758
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Exporting Vertica tables to S3

Post by JimKnicely » Sat Mar 04, 2017 4:05 am

Hi,

Maybe you can mount an S3 Bucket per the the instructions in the article "How to Mount S3 Bucket on CentOS/RHEL and Ubuntu using S3FS"?

https://tecadmin.net/mount-s3-bucket-ce ... sing-s3fs/

Once you have the S3 Bucket mounted, you should be able to export data out of Vertica onto it.

vsql -c "select * from some_table;" -o /s3mnt/some_table.txt
Jim Knicely

Image

Note: I work for HPE. My views, opinions, and thoughts expressed here do not represent those of my employer.

Post Reply

Return to “Vertica Data Load”