Page 1 of 1

Trouble to load Big data set

Posted: Thu May 12, 2016 3:26 pm
by Nsingh45
Hi all,I have a dataset with 10000 features (terms) with tf_idf values.
when I try to load the data (which is in C45 format) I get the following error:
data = orange.ExampleTable(file)
SystemError: C45ExampleGenerator: line 1 of file '../features/' too long
So I reduced the feature space to 5000 by use of term frequency treshold (>2).
But it doesn't work neather.
Does Orange have a problem with 5000 features and if so, where is its limit?
best regards,

Re: Trouble to load Big data set

Posted: Sat May 14, 2016 4:42 am
by sonamjain4715
you can have much more than 10000 features. The problem is that the code for parsing the files in C4.5 is old and ugly, and limits the length of lines to 10000 characters. This shouldn't be difficult to fix, so we'll do it some day soon. Till then, you can just convert your data to tab-delimited format