Page 1 of 1

Exporting Vertica tables to S3

Posted: Fri Feb 24, 2017 3:24 pm
by Beg1nner
Hi all,

We are using vertica 7.2.1 and we have 50 odd tables in a schema (total data size ~5 TB). What is the best approach you would suggest to move this data into S3?

Re: Exporting Vertica tables to S3

Posted: Mon Feb 27, 2017 2:49 pm
by JimKnicely
Have you check out "Exporting to Amazon S3 using the Vertica Library for Amazon Web Services"?

https://my.vertica.com/docs/8.0.x/HTML/ ... ibrary.htm

Re: Exporting Vertica tables to S3

Posted: Tue Feb 28, 2017 4:57 pm
by Beg1nner
Yes I did check them out. But we are on Vertica 7.2.1. Other than upgrading and using awslib, is there any better way?

Re: Exporting Vertica tables to S3

Posted: Sat Mar 04, 2017 4:05 am
by JimKnicely
Hi,

Maybe you can mount an S3 Bucket per the the instructions in the article "How to Mount S3 Bucket on CentOS/RHEL and Ubuntu using S3FS"?

https://tecadmin.net/mount-s3-bucket-ce ... sing-s3fs/

Once you have the S3 Bucket mounted, you should be able to export data out of Vertica onto it.

vsql -c "select * from some_table;" -o /s3mnt/some_table.txt