I also tried to replace the CONFIGURATION 'EXTRACTION_CORE' with 'indexConfigNER.xml' but it didn't help.
The table SYS.M_FULLTEXT_QUEUES shows that all the documents are in queue except a few hundred errors.
The trace shows the following errors:
[4905]{-1}[-1/-1] 2013-08-28 12:33:01.608447 e preprocessor PreprocessorImpl.cpp(00407) : process(): Error in Document, theProcessResult was set to 6604
[4905]{-1}[-1/-1] 2013-08-28 12:33:01.608465 e preprocessor Preprocessor.cpp(00651) : HANDLE: DISPATCH - Processing Document with key 'U815FA4F5FC8BFB:TWEETS3en.TXT:36790' failed, returning PREPROCESSOR_ACTIVITY_ERROR (Code 6604)
[4564]{-1}[-1/-1] 2013-08-28 12:33:02.613224 e preprocessor PreprocessorImpl.cpp(01233) : Text analysis error 'docmodel/analyzer/src/AggregateAnalyzer.cpp: Line 104: Error 0x01000005: Request to create analyzer of unknown type:
SAP.TextAnalysis.DocumentAnalysis.FormatConversion.FormatConversionAnalyzer', number of errors: 1
[4564]{-1}[-1/-1] 2013-08-28 12:33:02.613250 e preprocessor PreprocessorImpl.cpp(00407) : process(): Error in Document, theProcessResult was set to 6604
[4564]{-1}[-1/-1] 2013-08-28 12:33:02.613268 e preprocessor Preprocessor.cpp(00651) : HANDLE: DISPATCH - Processing Document with key 'U815FA4F5FC8BFB:TWEETS3en.TXT:36791' failed, returning PREPROCESSOR_ACTIVITY_ERROR (Code 6604)
[9129]{-1}[-1/-1] 2013-08-28 12:33:03.617294 e preprocessor PreprocessorImpl.cpp(01233) : Text analysis error 'docmodel/analyzer/src/AggregateAnalyzer.cpp: Line 104: Error 0x01000005: Request to create analyzer of unknown type:
SAP.TextAnalysis.DocumentAnalysis.FormatConversion.FormatConversionAnalyzer', number of errors: 1