Thursday, February 19, 2015

Slow Solr Startup - Disabling the Suggester solved it

My Solr has 130M documents over 8 shards takes 20min of heavy CPU to start up.
The log showed big time gaps around the suggester being called:
[2/17/15 10:20:33:657 GMT] 000000ca SolrSuggester I org.apache.solr.spelling.suggest.SolrSuggester reload reload()
[2/17/15 10:20:33:657 GMT] 000000ca SolrSuggester I org.apache.solr.spelling.suggest.SolrSuggester build build() 
Suggester is configured by default in solrconfig.xml over non existing fields, despite that, and despite its request handler marked as Lazy startup it still does a huge amount of work.

Disabling the suggest searchComponent and requestHandler solved the issue.
I assume that the suggester is building a huge FST in memory.
 
<!--
<searchComponent name="suggest" class="solr.SuggestComponent">
   <lst name="suggester">
      <str name="name">mySuggester</str>
      <str name="lookupImpl">FuzzyLookupFactory</str>      <!-- org.apache.solr.spelling.suggest.fst -->
      <str name="dictionaryImpl">DocumentDictionaryFactory</str>     <!-- org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory --> 
      <str name="field">cat</str>
      <str name="weightField">price</str>
      <str name="suggestAnalyzerFieldType">string</str>
    </lst>
  </searchComponent>

  <requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
    <lst name="defaults">
      <str name="suggest">true</str>
      <str name="suggest.count">10</str>
    </lst>
    <arr name="components">
      <str>suggest</str>
    </arr>
  </requestHandler>
--> 
 
Later on I saw that someone went through the same thing before me.
Downloaded Solr 4.10 and saw it was disabled in stock solrconfig via to see it's fixed via SOLR-6679.

No comments:

Post a Comment