Archive for the ‘FAST ESP’ Category

Reference: http://technet.microsoft.com/en-us/library/ff191228(v=office.14).aspx

Use the following procedures to reset the content index.

Warning:
The procedures will erase the content index. After a reset, search results will not be available until new crawls have been run.

Reset the content index in SharePoint Server 2010

1. Verify that the user account that is performing this procedure is a service application administrator for your FAST Search Content Search Service Application.

2. In Central Administration, in the Application Management section, click Manage service applications.

3. On the Service Applications page, in the list of service applications, click your FAST Search Content SSA.

4. On the Search Administration page, under Crawling, click Index Reset.

5. On the Search Service Application: Index Reset page, click Reset Now.

6. A confirmation dialog box appears; click OK to confirm the content index reset. The Search Service Application: Search Administration page opens and the System Status is displayed.

Before you start any new crawls, you must manually clear the content from the content collection your FAST Search Content Search Service Application was feeding to. The default collection name is sp. Note that clearing the content collection is irreversible.

To clear a FAST Search Server 2010 for SharePoint content collection, log onto the server that hosts FAST Search Server 2010 for SharePoint and follow this procedure.

Clear the content collection

1. Verify that you meet the following minimum requirements: You are a member of the FASTSearchAdministrators local group on the computer where FAST Search Server 2010 for SharePoint is installed.

2. On the Start menu, click All Programs.

3. Click Microsoft FAST Search Server 2010 for SharePoint.

4. Click the Microsoft FAST Search Server 2010 for SharePoint shell.

5. At the Microsoft FAST Search Server 2010 for SharePoint shell command prompt, type the following command:

Clear-FASTSearchContentCollection -Name <collection name>

Where:

o <collection name> is the content collection you are about to clear.

6. Wait for the command to finish. This may take some time.

Removing documents from FAST Index

Posted: December 3, 2012 in FAST ESP

There are two simple approaches to removing specific documents from a given collection. The difference between the two methods depends on whether the user has contentids of the documents to be deleted or internalids of the documents to be deleted.

1. If the user has contentids of the documents to be deleted, the filetraverser can be used with a text file containing the contentids of the documents. In this case, the contentids should appear in the text file as one per line. For example:
http://contoso.com/mydoc1.html
http://contoso.com/mydoc2.pdf
http://contoso.com/mydoc3.html

The command for the filetraverser must include the collection name and the path to the text file containing the contentids of the documents to be deleted:
filetraverser -c <collection> -t <delete_file>

For example (collection = mycollection, text file = delete.txt):
filetraverser -c mycollection -t C:\temp\delete.txt

2. If the user has internalids of the documents to be deleted, the indexeradmin command can be used with a text file containing the internalids of the documents. In this case, the internalids should appear in the text file as one per line. For example:
975d02ee431a7a022a5bc3abc53fb8ed_mycollection
bd9c3e19343f4d18c8d5d37b1160b0cf_mycollection
9beb827015e00ffd709667693566fc76_mycollection

The indexeradmin command must include the rdocs flag, the path to the text file containing the internalids of the documents to be deleted, the collection name and a sessionid:
indexeradmin rdocs <delete_file> <collection> <sessionid>

For example (text file = delete.txt, collection = mycollection, sessionid = 99):
indexeradmin rdocs delete.txt mycollection 99

Note:The sessionid is an integer, usually 99 or 100. The number used is not important to the functionality. A warning message may be displayed indicating the sessionid is in use. In this case, the command should be run again with the sessionid increased by one.

The FAST ESP error, Document summary too short, couldn’t unpack, occasionally occurs after performing an index profile update. In the past, my solution to this problem involved executing a manual cold update. While this works every time, it does require that you refeed all of your content. Definitely not the ideal solution when dealing with a production system.

I discovered a TechNet posting this morning that offers another solution that’s not destructive. The following are the steps described in the post:

1. Stop the QR Server (nctrl stop qrserver).

2. Delete the %FASTSEARCH%\var\qrserver\webcluster\15100\cache_cs directory.

3. Start the QR Server (nctrl start qrserver).

4. Stop Search (nctrl stop search-1).

5. Delete the %FASTSEARCH%\var\searchctrl directory.

6. Start Search (nctrl start search-1)

7. Repeat the above steps on any systems in the cluster running a QR Server or Search component.

A big thanks to Rob Va for the solution!

Issue: Getting error “The lql job had problems running” in FAST ESP logs.

Resolution : This issue could be due to a corrupted datasources file. Try the following steps to resolve:

1) Backup the file datasources.xml from %fastsearch/components/logtransformer/data/datastore.
2) Stop the logtransformer process:
nctrl stop logtransformer
3) Delete the datasources.xml file from the above location (make sure you took a backup)
4) Start the logtransformer process
nctrl start logtransformer

Symptom:

One may notice the following error in the Indexer log while trying to feed content into ESP:

[2011-03-11 01:50:36] WARNING indexer search.contoso.com 15900 systemmsg api_queue: The operation with session id 82 and size of 78643200 bytes is larger than the configured max api queue size of 52428800 bytes. Please reduce batch size or increase max api queue size for optimal performance

Cause:

The default setting for the API Queue Size is 50mb. While working with large documents, a single document could effectively fill up the API queue if it is larger than the specified maxQueueSize limit.

Resolution :

As a best practice Microsoft recommends increasing the maxQueueSize (API Queue Size) as needed, but not to exceed a max setting of 500mb. If one is unable index content with a maxQueueSize of 500mb, try reducing the sizes and/or truncate the documents to work with a smaller API Queue Size

More Information :

If you require a larger then default Indexer API Queue, it can be adjusted with the following procedure:

1. From the AdminServer Node, edit the file %FASTSEARCH%\etc\config_data\RTSearch\webcluster\rtsearchrc.xml

2. Add/modify the attribute "maxQueueSize” to the appropriate setting:

Example: maxQueueSize=”78643200”

3. Reload the configuration from the AdminServer Node:

Syntax: nctrl reloadcfg

4. Restart of all indexer processes.

Use "nctrl stop indexer" and "nctrl start indexer" from all indexer nodes or via the System Management tab in the ESP Admin GUI.

Important Note: As always, make sure the backup indexer nodes are stopped first, and then Master Indexer Nodes are restarted. Once this is completed, the indexing services on the backup nodes can be started.

Reference: http://support.microsoft.com/kb/2535170

FAST ESP : Web Analyzer module dead

Posted: September 23, 2011 in FAST ESP

The following is seen in the Webanalyzer logs (%FASTSEARCH%\var\log\webanalyzer\):
Traceback (most recent call last):
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzer.py", line 56, in ?
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzerCore.py", line 3128, in go
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzerCore.py", line 3162, in WADriver
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzerCore.py", line 3305, in ProcessingDriver
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzerCore.py", line 3884, in ProcessView
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzerCore.py", line 841, in start
File "C:\d\cruise\builds\active\common\webanalyzer\src\WebAnalyzerCore.py", line 894, in _add_tasks

The following is seen in the all.log (%FASTSEARCH%\var\log\syslogs\):
[2011-07-14 00:57:40] INFO : webanalyzer (mailto:webanalyzer@NP2APPL648V) : systemmsg: WebAnalyzer started (PID: 5164) with 1 workers
[2011-07-14 00:58:01] WARNING : configserver (mailto:configserver@NP2APPL648V:16005) : systemmsg: Module (WebAnalyzer) at aa.xx.yy.com:16700 is not responding
[2011-07-14 00:58:09] WARNING : nodecontroller (mailto:nodecontroller@NP2APPL648V:16015) : systemmsg: Process webanalyzer was not running, restarting it

Resolution :

This issue is caused by the webanalyzer not shutting down gracefully. When this happens, there can be problems parsing the cache files (.swp) in %FASTSEARCH%\data\webanalyzer\config when it starts again.

To resolve this issue follow the steps below to clear the cache files:
1. Stop the Web analyzer:
nctrl stop webanalyzer
2. Delete or rename the *.xml.swp files located in the %FASTSEARCH%\data\webanalyzer\config\ directory.
3. Start the Web analyzer:
nctrl start webanalyzer

After updating the index-profile, the following behavior is observed:

In fdispatch-15700.log:

[2007-12-10 16:33:25] WARNING : fdispatch: System inconsistency, /usr/fast/esp/data/data_index/normalized.10/index.cf has 11 catalogs, while /usr/fast/esp/var/searchctrl/etc/index.cf file has 12 catalogs.

[2007-12-10 16:38:34] ERROR : fdispatch: Document summary too short, couldn’t unpack
[2007-12-10 16:38:34] ERROR : fdispatch: Document summary too short, couldn’t unpack

Resolution

The messages generally indicate that files created during an index-profile update have not updated properly, thus causing inconsistency on the system.

To determine if the files were incorrectly generated by the configserver, do the following:

  1. Stop FAST ESP (nctrl stop).
  2. Clear the contents of %FASTSEARCH%\var\searchctrl\etc\* and %FASTSEARCH%\var\etc\*. (The files will be regenerated.)
  3. Start FAST ESP (nctrl start).