We at Spilgames are using a lot of python with Vertica.
Over time we developed a Python library which currently features a high speed loader and an extendable base class to write batch importers. Some helper scripts are shipped with as well for instance to easily import csv-like formatted files into Vertica in one command. Depending on used hardware and extra processing, it is easily possible to import 100.000 records / second.
We are now making this library open, under a BSD license. You can find the source on github at https://github.com/spilgames/pyvertica and python package at http://pypi.python.org/pypi/pyvertica with its documentation at http://packages.python.org/pyvertica/ on the pypi repository.
Using it is very easy, for instance:
Code: Select all
from pyvertica.batch import VerticaBatch batch = VerticaBatch( dsn='VerticaDWH', table_name='schema.my_table', truncate=True, column_list=['column_1', 'column_2'], ) row_list = [ ['row_1_val_1', 'row_1_val_2'], ['row_2_val_1', 'row_2_val_2'], ... ] for column_data_list in row_list: batch.insert_list(column_data_list) error_bool, error_file_obj = batch.get_errors() if error_bool: print error_file_obj.read() batch.commit()