Configuring a crawler to search your local portal site

Configure and run a search crawler on your local portal site to gather information and create a search collection that enables your users to search your portal site.

1. Set the crawler user ID. Set a dedicated crawler user ID for crawling the portal site content source. Proceed as follows:
   1. Define the crawler user ID by using the Manage Users and Groups portlet. Proceed as follows:
      Note: It is of benefit to define a dedicated crawler user ID. The pre-configured default portal site search uses the default administrator user ID wpsadmin with the default password of that user ID for the crawler. If you changed the default administrator user ID during your portal installation, the crawler uses that default user ID. If you have changed the password for the wpsadmin or other administrative user ID, or if you changed the default administrator user ID to an ID other than wpsadmin, or if you want to use a separate user ID, you need to set the crawler user ID.
   2. Set the preferred language of the portal site crawler user ID to match the language of the portal site search collection that it crawls. (If you do this after you started a crawl on the portal site search collection, you need to reset the portal site collection. Refer to Resetting the default search collection.)
   3. Edit the portal site collection content source and fill in the crawler user ID and its password. To do this, proceed as follows:
      1. Click Administration > Manage Search > Search Collections.
      2. Select Default Search Collection from the Search Collection list.
      3. In the Content Source Name list of the search collection, click the Portal Content Source search collection.
      4. Click the Edit icon next to the Portal Content Source collection name.
      5. Select the Security tab.
      6. Click the Edit icon next to the security realm that you want to modify.
      7. Type the crawler user ID and password into the appropriate fields.
      8. Click Update.
      9. Click Save to save your changes.
   4. Optional: For content sources of type Web Site, you can configure the crawler to follow external links from inside the portal. To do this, modify the value in the field Levels of links to follow under the tab General Parameters. Set the level to a value higher than 1. In addition, you can configure filters for those external links from the Filters tab. The default filter suppresses all links that point back to portal pages. The default filter is displayed only after you save the configuration of the content source.
2. Start the initial crawl. Start the initial crawl on the portal site content source:
   1. Click Administration > Manage Search > Search Collections.
   2. In the search collection list, click Default Search Collection.
   3. Click the Start Crawler icon (right-pointing arrow) next to the Portal content source name.
3. Configure regular crawls. If you want regular crawls on the portal site content source, perform either of the following tasks:
   • Enable the default scheduler. To do this, proceed as follows:
     1. Click the View Content Source Schedulers icon next to the collection name.
     2. In the Manage Schedulers page, click Disabled. This changes the status of the scheduler to Enabled and displays a confirmation message.
   • Set up your own scheduler. To do this, proceed as follows:
     1. Click the Edit icon for the content source.
        Note: You can have only one schedule at a time. Therefore, to create your own schedule, you first have to delete the existing schedule.
     2. Select the Schedulers tab.
     3. Configure your own scheduler as required. For more details about how to do this, refer to the Manage Search portlet help.
     4. Click Save to save your changes.