Starting database at AWS stuck in INITIALIZING

Moderator: NorbertKrupa

Post Reply
chenshaulian
Intermediate
Intermediate
Posts: 51
Joined: Wed Sep 09, 2015 9:34 am

Starting database at AWS stuck in INITIALIZING

Post by chenshaulian » Mon Mar 07, 2016 3:29 pm

Hi,

I have created Vertica 10 nodes Cluster at AWS (7.1.1-10) and I did CopyCluster from production environment to AWS.
42TB … finished after 10 days.

Now after the CopyCluster finished, I'm trying to start the database from the admintools:
*** Starting database: vertica_polonium_mtc ***
Starting nodes:
v_vertica_polonium_mtc_node0001
v_vertica_polonium_mtc_node0002
v_vertica_polonium_mtc_node0003
v_vertica_polonium_mtc_node0004
v_vertica_polonium_mtc_node0005
v_vertica_polonium_mtc_node0006
v_vertica_polonium_mtc_node0007
v_vertica_polonium_mtc_node0008
v_vertica_polonium_mtc_node0009
v_vertica_polonium_mtc_node0010

Starting Vertica on all nodes. Please wait, databases with large catalogs may take a while to initialize.

Node Status: v_vertica_polonium_mtc_node0001: (INITIALIZING) v_vertica_polonium_mtc_node0002: (INITIALIZING)
v_vertica_polonium_mtc_node0003: (INITIALIZING) v_vertica_polonium_mtc_node0004: (INITIALIZING)
v_vertica_polonium_mtc_node0005: (INITIALIZING) v_vertica_polonium_mtc_node0006: (INITIALIZING)
v_vertica_polonium_mtc_node0007: (INITIALIZING) v_vertica_polonium_mtc_node0008: (INITIALIZING)
v_vertica_polonium_mtc_node0009: (INITIALIZING) v_vertica_polonium_mtc_node0010: (INITIALIZING)
...
It is suggested that you continue waiting.
Do you want to continue waiting? (yes/no) [yes] no
Database startup successful, but it may be incomplete. Some nodes
remain in a transitional state. Waiting can sometimes resolve this
problem. See Database Cluster State in Main Menu for updated
cluster state. If this state persists, try using the Advanced menu
to Stop Vertica on Host, then Restart Database.
Press RETURN to continue


The INITIALIZING running for several hours and didn't end.
I didn't find anything in the vertica.log or in the adminTools-dbadmin.log (attached) or in any log ... everything seems ok :(
I checked the admintools.conf (all right and the same on all nodes), the spread is running on all nodes (when the start database command is running), restart the cluster ... and again, everything seems ok, so what is the problem? why the nodes don't come UP ?

Anyone can help ?
Attachments
admintools.txt
(2.25 KiB) Downloaded 545 times
vertica.log
(118.56 KiB) Downloaded 594 times
adminTools-dbadmin.log
(174.08 KiB) Downloaded 573 times

chenshaulian
Intermediate
Intermediate
Posts: 51
Joined: Wed Sep 09, 2015 9:34 am

Re: Starting database at AWS stuck in INITIALIZING

Post by chenshaulian » Mon Mar 21, 2016 9:40 am

Fixed :)

The source cluster catalog was at Broadcast and target cluster (AWS) catalog was at pt2pt (point to piont).
With help from HP support, we edit the catalog of the target cluster to pt2pt

(the AWS created with pt2pt parameter, but the copycluster command overwrite the configuration and update it to Broadcast)

User avatar
JimKnicely
Site Admin
Site Admin
Posts: 1825
Joined: Sat Jan 21, 2012 4:58 am
Contact:

Re: Starting database at AWS stuck in INITIALIZING

Post by JimKnicely » Wed Mar 23, 2016 3:08 am

Thanks for posting the solution!
Jim Knicely

Image

Note: I work for Vertica. My views, opinions, and thoughts expressed here do not represent those of my employer.

Post Reply

Return to “Vertica Database Administration”