"ERROR 5127" on Backup

Moderator: NorbertKrupa

Post Reply
otw
Newbie
Newbie
Posts: 3
Joined: Mon Oct 14, 2013 11:16 am

"ERROR 5127" on Backup

Post by otw » Mon Oct 14, 2013 1:57 pm

Hi all,

we just recently updated our Vertica from 5.0 to 6.1
After that we added two new nodes to the, until then, single instance (so three alltogether now).

But now the backups fails with a rather strange eror:

Code: Select all

[dbadmin@n131 ~]$ /opt/vertica/bin/vbr.py -t backup --debug 2
Preparing...
Found Database port:  5433
[...]
v_x_node0001 10.x.x.131 /data/vertica_db/x/v_x_node0001_catalog
v_x_node0002 10.x.x.132 /data/vertica_db/x/v_x_node0002_catalog
v_x_node0003 10.x.x.133 /data/vertica_db/x/v_x_node0003_catalog
set([('10.x.x.133', True), ('10.x.x.193', False), ('10.x.x.131', True), ('10.x.x.132', True)])
/opt/vertica/bin/vsql  -dx -p5433 -Udbadmin -h 10.x.x.133 -X -q -c "select database_snapshot('test', true);
"
ERROR 5127:  Unable to create snapshot Could not link file [/data/vertica_db/x/v_x_node0003_catalog/Snapshots/test/data/vertica_db/x/procedures/dumpquerytotmp.sh] to [/data/vertica_db/x/procedures/dumpquerytotmp.sh]: File exists

When communicating with vertica, the process failed with code 1
backup failed!
I'm starting the backup from the original single instance host (131) and backup should go to a separate backup host (193)

I already tried deleting one or the other of the files, but no change.
With one, I get same error, with the other that the file is missing.

Any idea what else I can do?

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: "ERROR 5127" on Backup

Post by JimKnicely » Mon Oct 14, 2013 4:05 pm

Hi,

Not sure if this is relevant, but these are listed as assumptions for the Python script vbt.py:
  • # 1. Python 2.7+ and rsync 3.0.5+ packaged by vertica exist on all hosts.
    # 2. It's ok to use backup hosts outside the cluster, as long as python and rsync with same version/path are installed.
    # 3. Current user can ssh on all hosts influenced without password prompt.
    # 4. Current user has write access on all target directories, and must be a Vertica super user.
    # 5. Only one vbr script instance is running on the same cluster & all backup nodes. Cancellation can be done by ^C.
    # 6. When doing backup, the cluster must be UP. Vbr script must be run on one of the cluster node.
    # 7. When restoring full cluster, cluster must exist and the nodes to be restored must be in DOWN state. When restoring objects, cluster must be UP. Vbr script must be run on one of the cluster node.
    # 8. When using copycluster, source cluster and all nodes must be UP, target cluster must be DOWN. Vbr script must be run on one of the source cluster node.
Do you meet all of these requirements?
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

otw
Newbie
Newbie
Posts: 3
Joined: Mon Oct 14, 2013 11:16 am

Re: "ERROR 5127" on Backup

Post by otw » Mon Oct 14, 2013 5:13 pm

Hi,

yeah, I have seen those infos in the script and check them.

- Package versions etc. are identical as all hosts are running Centos 5.9 with latest updates.
- passwordless access for 'dbadmin' also works fine in any direction and 'dbadmin' has all filesystem permissions where needed

I also tried running it from the other cluster nodes (even node3 where the error message originates) - no difference, same error :(

Now also tried changing snapshot name (new one, never used before): same error

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: "ERROR 5127" on Backup

Post by JimKnicely » Mon Oct 14, 2013 5:58 pm

Hmm. I would try backing up a single table to see if at least that works.
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

scutter
Master
Master
Posts: 302
Joined: Tue Aug 07, 2012 2:15 am

Re: "ERROR 5127" on Backup

Post by scutter » Mon Oct 14, 2013 8:55 pm

Since you've already tried a new snapshot name with the same results, I wonder if there is a bug related to to backing up external procedures scripts. If you have a test environment where you can create a new database for testing, try backing up a multi-node database that contains an external procedure just to make sure that this piece is working correctly.

--Sharon
Sharon Cutter
Vertica Consultant, Zazz Technologies LLC

otw
Newbie
Newbie
Posts: 3
Joined: Mon Oct 14, 2013 11:16 am

Re: "ERROR 5127" on Backup

Post by otw » Tue Oct 15, 2013 11:48 am

scutter wrote:Since you've already tried a new snapshot name with the same results, I wonder if there is a bug related to to backing up external procedures scripts. If you have a test environment where you can create a new database for testing, try backing up a multi-node database that contains an external procedure just to make sure that this piece is working correctly.

--Sharon
Hello Sharon,

thanks, that was just the right keyword :)
I thought that file in the "procedures" directory belongs to Vertica, but it is just a custom script my colleagues are using (sorry, I'm not the Vertica specialist, just trying to back it up ;))

So that external procedure in our case is doing nothing very important so we dropped it for testing.
After that, the backup-script is running like a charm!!!

We're letting the backup run for now, so we have an up-to-date backup again.

Afterwards we'll try to add the external procedure again and see what happens then.

Post Reply

Return to “Vertica Backup & Recovery”