Apostrophe 1.5: legacy documentation

Still using Apostrophe 1.5? Check out Apostrophe 2 for your future projects.

apostrophe Mysql Search Plugin

Apostrophe MySQL Search Plugin

This plugin replaces the default Zend Lucene-based search capabilities in Apostrophe with a search engine implemented entirely in MySQL and Doctrine queries. This search engine implements a subclass of aSearchService, so it can be easily enabled via app.yml in any Apostrophe project.

Advantages and Disadvantages

aMysqlSearch has the following advantages over the default search engine:

  • Searches are fast.
  • Memory usage is lower.
  • Zend Lucene does a lousy job managing permissions for folders and ties your site to the filesystem. aMysqlSearch does not have these issues because data storage is in the database.
  • Since your search index is in your MySQL database it is easier to pick up your site and move it with a single mysqldump (plus moving your uploads folder).

aMysqlSearch has the following disadvantages:

  • Initially building an index for a large preexisting site takes a long time, perhaps longer than Zend. Indexing pages later "on the fly" as they are edited is reasonably fast.
  • There is no advanced search syntax. The search engine searches for the words the user types; documents with more matches rank higher. We'll be tweaking the results a bit, but we won't be implementing memory-hungry features or features that can't be built with Doctrine queries. This is meant to be the simple, low-overhead option.
  • Overrides of the addSearchResults method no longer work. That technique is now deprecated. However, there are two other ways to add other kinds of search results to your site's global search results:

(1) override a/searchBefore and/or a/searchAfter to create search sidebars that provide teaser search results for other data types that users can click on to get more information on another page. This is easy and user-friendly and we recommend it over trying to merge non-page results with pages.

(2) Mirror your content to an Apostrophe virtual page with a slug that is a valid Symfony URL which displays (or redirects to) the item in question. Search results recognize such virtual pages and include them in search results. This is how blog articles are included in the main Apostrophe search results. We don't recommend it for things that aren't very page-like. Technique #1 is easier and better in most cases.

Installation

To enable this search engine:

1. Check out the plugin to your plugins folder. As always we recommend the use of svn:externals:

svn propedit svn:externals plugins

Add this line:

apostropheMysqlSearchPlugin http://svn.apostrophenow.org/plugins/apostropheMysqlSearchPlugin/trunk

Then just svn update your plugins folder. (If you don't use svn for your project you'll have to svn co the plugin instead.)

2. Enable the plugin. Edit config/ProjectConfiguration.class.php and add apostropheMysqlSearchPlugin to the END of the list of plugins. In particular, it must appear AFTER any plugins that introduce models you'll want to have search capabilities for.

WARNING: THIS PLUGIN MUST BE ENABLED *AFTER* apostrophePlugin! OTHERWISE YOU WILL *BREAK* YOUR PROJECT by autogenerating classes in the wrong plugin folder.

3. Edit app.yml and set the search service class name:

all:
  a:
    search_service_class: aMysqlSearch

Also, add your table to app_aToolkit_indexes:

all:
  aToolkit:
    indexes:
      - 'aPage'
      - 'aMediaItem'
      - 'teMember'

4. Rebuild your model classes:

./symfony doctrine:build --all-classes

5. Run symfony cc

6. Run apostrophe:migrate to create the new tables containing the search index:

./symfony apostrophe:migrate

(If you are building a brand new site and don't have any content yet, you can build your database and load fixtures instead.)

Note: if you have added search for custom database models beyond aPage and aMediaItem, you will need to read the next section before rebuilding your search index.

7. Rebuild your search index:

./symfony apostrophe:rebuild-search-index

8. Test out search. Everything should work normally for sitewide search, media search and blog search.

Adding Search For Other Model Classes

You probably don't need to do this. Instead  use the new `mirrorForSearch` functionality.

If you don't want to mirror content to virtual pages for simple inclusion in standard Apostrophe search results, you can use the search service to search for information in other model classes besides aPage and aMediaItem. To do that, just add the following to your schema, replacing references to aPage or a_page with references to your model class and its table name:

aPageToASearchDocument:
  columns:
    a_search_document_id:
      type: integer
    # Must be your table name followed by _id
    a_page_id:
      type: integer
  relations:
    aSearchDocument:
      local: a_search_document_id
      foreign: id
      class: aSearchDocument
      onDelete: cascade
    aPage:
      local: a_page_id
      foreign: id
      class: aPage
      onDelete: cascade
  options:
    symfony:
      form:   false
      filter: false
  
aPage:
  relations:
    aSearchDocuments:
      class: aSearchDocument
      refClass: aPageToASearchDocument
      foreign: a_search_document_id
      local: a_page_id

Note that we add a new relation to the aPage model here. You would move that into the declaration of your model or, if the model comes from a plugin's schema, just extend that in your project-level schema, which is exactly what we're doing here (aPage originally comes from the apostrophePlugin schema, but you can add additional relations in other plugins as long as they are declared later or at the project level).

If you already have made your model classes compatible with the old aZendSearch API, you may be able to stop reading at this point. To ease the transition, there is basic backwards compatibility with the older API. The backwards compatibility feature works well, as long as you don't get fancy and use the methods of the Zend search hit objects rather than just looking at the properties you set when calling updateLuceneIndex. However you can get more efficient results and use custom Doctrine criteria if you migrate to the newer technique described below.

Once you have made the above additions, you can add calls to the search service to appropriate methods of your model class. For example, in postSave you might call:

aTools::$searchService->update(
  array(
    'item' => $this,
    'text' => $item->getBody(), 
    'info' => array('summary' => $item->getSummary()), 
    'culture' => aTools::getUserCulture()));

You don't have to specify a culture if you do not save different internationalizations of the same content on a per-culture basis.

In `postDelete you might call:

aTools::$searchService->delete(
  array(
    'item' => $this
  )
);

Searching In Your Own Code

Now that your data is in the search index, how do you search for it? Just use the Doctrine-friendly search API:

$q = Doctrine::getTable('Dog')->createQuery('d');
aTools::$searchService->addSearchToQuery($q, $this->getParameter('search'), array('culture' => aTools::getUserCulture());
$q->addWhere('lots of other criteria that are important to you, unrelated to search');
$results = $q->execute(); // Or fetchArray(), or use an sfDoctrinePager...

IMPORTANT: leave out the culture if the data was indexed without a culture in the first place.

Warning: Not Everybody Has aTools::$searchService

For simplicity the examples here assume that the plugin, or some other plugin that implements aSearchService, is installed and configured. Older projects might still be using the old Zend Search implementation, which does not use that interface. For backwards compatibility it's possible to update the search index via the old aZendSearch APIs and to search it that way as well as long as you don't get too fancy. See PluginaPage for examples, if you're brave.

Is that all?

There's more, but not too much more. You can pass more than one text field to be indexed and give them different weights, and you can use the 'info' field to serialize some data to be easily retrieved with each search result, which is good for displaying summaries without the overhead of object hydration. But that's about it. We've kept it intentionally simple. See  aSearchService for complete documentation of the API.