Subject: agentStdMsg.log full of errors from inotify.py from trying to open DataCollector/StorageInfoByMinute_*.log files that do not exist.
While running vertica-6.1.1-0.x86_64 on RedHat EL6.3 2.6.32-279.19.1.el6.x86_64 I noticed the gentStdMsg.log is full of errors from inotify.py, for example:
Traceback (most recent call last):
File "/opt/vertica/agent/services/inotify.py", line 221, in process_IN_MODIFY
fptr = open(fname, 'r')
IOError: [Errno 2] No such file or directory: '/data/1/catalog/vmartdb/v_vmartdb_node0001_catalog/DataCollector/StorageInfoByMinute_425738709892126.log'
Traceback (most recent call last):
File "/opt/vertica/agent/services/inotify.py", line 176, in process_IN_CREATE
fptr = open(fname, 'r')
IOError: [Errno 2] No such file or directory: '/data/1/catalog/vmartdb/v_vmartdb_node0001_catalog/DataCollector/StorageInfoByMinute_425739243639302.log'
wc -l for agentStdMsg.log for today reports 720,600 lines and is 1.8G, max line length according to wc -L is 152, the start of the file is full of nulls:
hexdump -v -n 64 /opt/vertica/log/agentStdMsg.log
0000000 0000 0000 0000 0000 0000 0000 0000 0000
0000010 0000 0000 0000 0000 0000 0000 0000 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 0000 0000 0000 0000 0000 0000
0000040
tail -n 700000 /opt/vertica/log/agentStdMsg.log | grep 'File' | sort | uniq -c
4198 File "/opt/vertica/agent/services/inotify.py", line 176, in process_IN_CREATE
170802 File "/opt/vertica/agent/services/inotify.py", line 221, in process_IN_MODIFY
The vertica process was likely consuming all the CPUs (32 x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz) on the host (a hp proliant dl380p gen8) at the time.
I suspect I may need to increase the retention policy for StorageInfoByMinute to avoid this.
https://my.vertica.com/docs/6.1.x/HTML/ ... #16155.htm
Any thoughts on this much appreciated, thanks.
SELECT GET_DATA_COLLECTOR_POLICY('StorageInfoByMinute');
-[ RECORD 1 ]-------------+-----------------------------------------
GET_DATA_COLLECTOR_POLICY | 64KB kept in memory, 256KB kept on disk.
SELECT * from data_collector where component='StorageInfoByMinute' and node_name='v_vmartdb_node0001';
-[ RECORD 1 ]----------+------------------------------------------------------------------
node_name | v_vmartdb_node0001
component | StorageInfoByMinute
table_name | dc_storage_info_by_minute
description | Storage information (Used and Free space) (historical, by minute)
access_restricted | t
in_db_log | f
in_vertica_log | f
memory_buffer_size_kb | 64
disk_size_kb | 256
record_too_big_errors | 0
lost_buffers | 2701
lost_records | 142072
retired_files | 243693
retired_records | 26309816
current_memory_records | 0
current_disk_records | 406
current_memory_bytes | 0
current_disk_bytes | 245688
first_time | 2013-06-28 10:18:00.852526-04
last_time | 2013-06-28 10:18:00.869702-04
kb_per_day | 2311603.10757301
agentStdMsg.log errors from inotify.py open StorageInfoByMin
Moderator: NorbertKrupa
-
- Newbie
- Posts: 4
- Joined: Fri Jun 28, 2013 10:29 am
agentStdMsg.log errors from inotify.py open StorageInfoByMin
Post by thatsafunnyname » Fri Jun 28, 2013 3:32 pm
Return to “New to Vertica Database Administration”
Jump to
- General
- ↳ Welcome to vertica-forums.com!
- ↳ Board Announcements
- ↳ Request for New Forums Categories
- ↳ Announcements, Events and Activities
- ↳ Vertica Links
- ↳ General
- ↳ Employment
- ↳ Vertica Certification
- Newbie
- ↳ New to Vertica
- ↳ New to Vertica Database Administration
- ↳ New to Vertica Database Development
- ↳ New to Vertica SQL
- Using Vertica
- ↳ Vertica Database Administration
- ↳ Vertica Database Development
- ↳ Vertica SQL
- ↳ Vertica SQL Functions
- ↳ Vertica Database Designer (DBD)
- ↳ Vertica User Defined Functions (UDFs)
- ↳ Vertica External Procedures
- ↳ Vertica Analytics
- ↳ Vertica Management Console
- ↳ Vertica Error Codes
- ↳ Vertica Backup & Recovery
- ↳ Vertica Installation
- ↳ Vertica Security
- ↳ Vertica Performance Tuning
- ↳ Vertica Administration Tools
- ↳ Vertica Upgrade
- ↳ Vertica Migration
- ↳ Vertica and the Operating System
- ↳ Vertica Data Load
- ↳ Vertica Tips, Lessons and Examples
- ↳ Vertica "How to..."
- Connecting to Vertica
- ↳ ADO.NET
- ↳ Cognos
- ↳ DBeaver
- ↳ dbVisualizer
- ↳ HDFS Connector
- ↳ Hadoop Connector
- ↳ HCatalog Connector
- ↳ Informatica PowerCenter
- ↳ JDBC
- ↳ Microsoft SQL Server Analysis Services (SSAS)
- ↳ MicroStrategy
- ↳ Microsoft SQL Server Integration Services (SSIS)
- ↳ ODBC
- ↳ Perl
- ↳ PHP
- ↳ Python
- ↳ R Language Integration
- ↳ Rhapsody Interface Engine
- ↳ SQL Workbench/J
- ↳ SQuirreL SQL Client
- ↳ Talend
- ↳ Tableau
- ↳ Toad for Data Analysts
- ↳ Vertica SDK API
- ↳ VoltDB
- ↳ vSQL
- Vertica Portfolio
- ↳ Vertica Premium Edition
- ↳ Vertica Express Edition
- ↳ Vertica Community Edition
- ↳ Vertica SQL on Hadoop
- ↳ Vertica in the AWS Cloud
- ↳ Vertica on Microsoft Azure
- All times are UTC+01:00
- Top
- Delete all board cookies
Powered by phpBB ® | phpBB3 theme by KomiDesign