Search

Overview

There are four components (coincidentally, the same number of Ninja Turtles) whose powers combine to provide you fully comprehensive powers of search.

1. Forms
2. Results
3. Indexes
4. Drivers

Forms

The search form is the entry point to your site search. Search forms are basic, vanilla HTML forms with a text or search input named q submitting to any URL with a search:results tag in its view template.

You can create that page however you wish: it could be an entry, a custom route, or something even fancier we didn't think of.
This Laracasts video shows how to setup search quickly.

<form action="/search/results">
    <input type="search" name="q" placeholder="Search">
    <button type="submit">Go find it!</button>
</form>

Results

Results are powered by the Search Results tag. The tag will look for that q input sent by the form as a query string in the URL (e.g. /search/results?q=where's%20the%20beef).

Inside the tag you have access to all the content and variables from each result.

{{ search:results index="default" }}
    {{ if no_results }}
        <h2>No results found for {{ get:q }}.</h2>
    {{ else }}
        <a href="{{ url }}">
            <h2>{{ title }}</h2>
            <p>{{ description | truncate:180 }}</p>
        </a>
    {{ /if }}
{{ /search:results }}

<statamic:search:results
  index="default"
  as="results"
>
  @forelse ($results as $result)
    <a href="{{ $result->url }}">
        <h2>{{ $result->title }}</h2>
        <p>{{ Statamic::modify($result->description)->truncate(180) }}</p>
    </a>
  @empty
    <h2>No results found for {{ get('q') }}.</h2>
  @endforelse
</statamic:search:results>

The tag has a lot more fine-tuned control available, like renaming the query parameter, filtering on fields and collections, and so on. You can read more about it in the search results tag docs.

Indexes

A search index is an ephemeral copy of your content, optimized for speed and performance while executing search queries. Without indexes, each search would require a scan of every entry in your site. Not an efficient way to go about it, but still technically possible.

Indexes are configured in config/statamic/search.php and you can create as many as you want. Each index can hold different pieces of content — and any one piece of content may be stored in any number of indexes.

Hot Tip!

An index is a collection of records, each representing a single search item. A record might be an entry, a taxonomy term, or even a user.

Your site's default index includes only the title from all collections. The default config looks like this:

'default' => [
    'driver' => 'local',
    'searchables' => 'all',
    'fields' => ['title'],
],

Search a specific index

The index you wish you to search can be specified as a parameter on your search results tag.

{{ search:results index="docs" }} ... {{ /search:results }}

<statamic:search:results
  index="docs"
>
  ...
</statamic:search:results>

Searchables

The searchables value determines what items are contained in a given index. By passing an array of searchable values you can customize your index however you'd like. For example, to index all blog posts and news articles together, you can do this:

'searchables' => ['collection:blog', 'collection:news']

Possible options include:

• all
• collection:*
• collection:{collection handle}
• taxonomy:{taxonomy handle}
• assets:{container handle}
• users
• Custom ones may be added by addons. Read about creating custom searchables

Filtering searchables

You may choose to allow only a subset of searchable items.

For example, you may want to have a toggle field that controls whether an entry gets indexed or not. You can specify a closure with that logic.

'searchables' => ['collection:blog'],
'filter' => function ($item) {
    return $item->status() === 'published' && ! $item->exclude_from_search;
}

Now, only published entries that don't have the exclude_from_search toggle field enabled will be indexed.

Heads up

Your filter will override any native filters. For example, entries will filter out drafts by default. If your filter doesn't also remove drafts, they could be indexed.

Alternatively you can specify a class to handle the filtering. This is useful when you want to cache your config using php artisan config:cache.

'filter' => \App\SearchFilters\BlogFilter::class,

namespace App\SearchFilters;

class BlogFilter
{
    public function handle($item)
    {
        return $item->status() === 'published' && ! $item->exclude_from_search;
    }
}

Records & fields

The best practice for creating search indexes is to simplify your record structure as much as possible. Each record should contain only enough information to be discoverable on its own, and no more. You can customize this record by deciding which fields are included in the index.

Transforming fields

By default, the data in the entry/term/etc that corresponds to the fields you've selected will be stored in the index. However, you're able to tailor the values exactly how you want using transformers.

Each transformer is a closure that would correspond to a field in your index's fields array.

'fields' => ['title', 'address'],
'transformers' => [

    // Return a value to store in the index.
    'title' => function ($title) {
        return ucfirst($title);
    },

    // Return an array of values to be stored.
    // These will all be separate searchable fields in the index.
    // $value is the current value
    // $searchable is the object that $value has been plucked from
    'address' => function ($value, $searchable) {
        return [
            '_geoloc' => $value['geolocation'],
            'location' => $value['location'],
            'region' => $value['region'],
        ];
    }
]

Alternatively you can specify a class to handle the transformation. This is useful when you want to cache your config using php artisan config:cache.

'fields' => ['title', 'address'],
'transformers' => [
    'title' => \App\SearchTransformers\MyTransfomer::class,
]

namespace App\SearchTransformers;

class MyTransformer
{
    public function handle($value, $field, $searchable)
    {
        // $value is the current value
        // $field is the index from the transformers array
        // $searchable is the object that $value has been plucked from

        return ucfirst($value);
    }
}

Updating indexes

Whenever you save an item in the Control Panel it will automatically update any appropriate indexes. If you edit content by hand, you can tell Statamic to scan for new and updated records via the command line.

# Select which indexes to update
php please search:update

# Update a specific index
php please search:update name

# Update all indexes
php please search:update --all

Connecting indexes

When a search is performed in the control panel (in collections, taxonomies, or asset containers, for example), Statamic will search the configured index for that content type.

If an index hasn't been defined or specified, Statamic will perform a less efficient, generic search. It'll be slower and less relevant, but better than nothing.

You can define which search index will be used by adding it to the respective YAML config file:

# content/collections/blog.yaml
title: Blog
search_index: blog

About entries

After specifying that an index contains entries from a collection (in searchables), you must also specify the index in the collection config itself because collections and entries can be in multiple indexes.

Also, since draft entries are not included in search indexes by default, you'll want to include them for your collection-linked index. You can add a filter that allows everything.

 'articles' => [
     'driver' => 'local',
     'searchables' => ['collection:articles'],
+    'filter' => fn () => true, 
 ]

Localization

You may choose to use separate indexes to store localized content. For example, English entries go in one index, French entries go in another, and so on.

Take these site and search configs for example:

# resources/sites.yaml
en:
  url: /
fr:
  url: /fr/
de:
  url: /de/

// config/statamic/search.php
'indexes' => [
    'default' => [
        'driver' => 'local',
        'searchables' => 'all',
    ]
]

By default, all entries will go into the default index, regardless of what site they're in. You can enable localization by setting the sites you want.

 'indexes' => [
     'default' => [
         'driver' => 'local',
         'searchables' => 'all',
+        'sites' => ['en', 'fr'], // You can also use "all" 
     ]
 ]

This will create dynamic indexes named after the specified sites:

• default_en
• default_fr

If you have a localized index and include searchables that do not support localization (like assets or users), they will appear in each localized index.

Drivers

Statamic takes a "driver" based approach to search engines. Drivers are interchangeable so you can gain new features or integrate with 3rd party services without ever having to change your data or frontend.

The native local driver is simple and requires no additional configuration, while the included algolia driver makes it super simple to integrate with Algolia, one of the leading search as a service providers.

You can build your own custom search drivers or look at the Addon Marketplace to see what the community has already created.

Local

The local driver (aka "Comb") uses JSON to store key/value pairs, mapping fields to the content IDs they belong to. It lacks advanced features you would see in a service like Algolia, but hey, It Just Works™. It's a great way to get a search started quickly.

Settings

You may provide local driver specific settings in a settings array.

'driver' => 'local',
'searchables' => 'all',
'min_characters' => 3,
'use_stemming' => true,

• match_weights: An array of weights for each field to use when calculating relevance scores. Defaults to:

  [
      'partial_word' => 1,
      'partial_first_word' => 2,
      'partial_word_start' => 1,
      'partial_first_word_start' => 2,
      'whole_word' => 5,
      'whole_first_word' => 5,
      'partial_whole' => 2,
      'partial_whole_start' => 2,
      'whole' => 10,
  ]
  

• min_characters: The minimum number of characters required in a search query. Defaults to 1.
• min_word_characters: The minimum number of characters required in a word in a search query. Defaults to 2.
• score_threshold: The minimum score required for a result to be included in the search results. Defaults to 1.
• property_weights: An array of weights for each property to use when calculating relevance scores. Defaults to [].
• query_mode: The query mode to use when searching (e.g. "whole", "words", "boolean"). Defaults to boolean.
• use_stemming: Whether to use stemming when searching (e.g. "jumping" matches "jump"). Defaults to false.
• use_alternates: Whether to use alternate spellings when searching (e.g. "color" matches "colour"). Defaults to false.
• include_full_query: Whether to include the full search query in the search results. Defaults to true.
• stop_words: An array of stop words to exclude from the search query. Defaults to [].
• limit: Whether to limit the number of results returned. Defaults to null.
• enable_too_many_results: Whether to enable a warning when too many results are returned. Defaults to false.
• sort_by_score: Whether to sort the search results by relevance score. Defaults to true.
• group_by_category: Whether to group the search results by category. Defaults to false.
• exclude_properties: An array of properties to exclude from the search results. Defaults to [].
• include_properties: An array of properties to include in the search results. Defaults to [].

Algolia

Algolia is a full-featured search and navigation cloud service. They offer fast and relevant search with results in under 100 ms (99% under 20 ms). Results are prioritized and displayed using a customizable ranking formula.

'default' => [
    'driver' => 'algolia',
    'searchables' => 'all',
],

To set up the Algolia driver, create an account on their site, drop your API credentials into your .env, and install the composer dependency.

ALGOLIA_APP_ID=your-algolia-app-id

ALGOLIA_SECRET=your-algolia-admin-key

composer require algolia/algoliasearch-client-php:^3.4

Statamic will automatically create and sync your indexes as you create and modify entries once you kick off the initial index creation by running the command php please search:update.

Settings

You may provide Algolia-specific settings in a settings array.

'driver' => 'algolia',
'searchables' => 'all',
'settings' => [ 
    'attributesForFaceting' => [
        'filterOnly(post_tags)',
        'filterOnly(post_categories)',
    ],
    'customRanking' => [
        'asc(attribute1)',
        'desc(attribute2)',
        'typo',
        'words'
    ]
] 

Templating with Algolia

We recommend using the Javascript implementation to communicate directly with them for the frontend of your site. This bypasses Statamic entirely in the request lifecycle and is incredibly fast.

Extras

Config cascade

You can add values into the defaults array, which will cascade down to all the indexes, regardless of which driver they use.

You can also add values to the drivers array, which will cascade down to any indexes using that respective driver. A good use case for this is to share API credentials across indexes.

Any values you add to an individual index will only be applied there.

Digging deeper

Search is split into a handful of different parts behind the scenes.

• An Index class will exist for each configured index. It will know how to send data to and from the underlying driver.
• Searchable classes are the things that can be indexed and searched. (e.g. an Entry)
• ProvidesSearchables classes (or just "Providers") are classes that define all the applicable searchable items. (e.g. an Entries provider gives Entry instances.)
• Result classes are wrappers that allow Statamic to use the searchable objects in a consistent way. (e.g. each result should have an identifier, type, Control Panel URL, etc)

Custom Searchables

In order to allow searching of custom items, you must create a provider and make your object implement Searchable.

In the following examples, we'll assume you are wanting to store products as Eloquent models.

Implement Searchable

The object you want to make searchable should implement both the Statamic\Contracts\Data\Augmentable and Statamic\Contracts\Search\Searchable interfaces. To make things easier, you can import the HasAugmentedData and Searchable traits which will handle most of the heavy lifting for you.

use Illuminate\Database\Eloquent\Model;
use Statamic\Contracts\Data\Augmentable;
use Statamic\Contracts\Search\Result as ResultContract;
use Statamic\Contracts\Search\Searchable as SearchableContract;
use Statamic\Data\HasAugmentedData;
use Statamic\Search\Result;
use Statamic\Search\Searchable;

class Product extends Model implements Augmentable, ContainsQueryableValues, SearchableContract
{
    use HasAugmentedData, Searchable;

    /**
     * The identifier that will be used in search indexes.
     */
    public function getSearchReference(): string
    {
        return 'product::'.$this->id;
    }

    /**
     * The indexed value for a given field.
     */
    public function getSearchValue(string $field)
    {
        return $this->$field;
    }

    /**
     * Convert to a search result class.
     * You can use the Result class, or feel free to create your own.
     */
    public function toSearchResult(): ResultContract
    {
        return new Result($this, 'product');
    }

/**
     * Returns an array of the model's attributes to be used in augmentation.
     */
    public function augmentedArrayData()
    {
        return $this->attributesToArray();
    }
}

When you're making a searchable from an Eloquent model, you'll need to override some offset/__call methods to prevent conflicts between Eloquent and Statamic's implementations of those methods:

/**
 * These methods exist on both Eloquent's Model class and in Statamic's HasAugmentedInstance trait.
 * To prevent conflicts, we need to override these methods and force them to call Eloquent's method.
 */

public function offsetGet($offset): mixed
{
    return parent::offsetGet($offset);
}

public function offsetSet($offset, $value): void
{
    parent::offsetSet($offset, $value);
}

public function offsetUnset($offset): void
{
    parent::offsetUnset($offset);
}

public function offsetExists($offset): bool
{
    return parent::offsetExists($offset);
}

public function __call($method, $parameters)
{
    return parent::__call($method, $parameters);
}

Create Provider

You should have a class that implements ProvidesSearchables, however it's even easier for you to extend Provider.

use Statamic\Search\Searchables\Provider;

class ProductProvider extends Provider
{
    /**
     * The handle used within search configs.
     *
     * e.g. For 'searchables' => ['collection:blog', 'products:hats', 'users']
     *      they'd be 'collection', 'products', and 'users'.
     */
    protected static $handle = 'products';

    /**
     * The prefix used in each Searchable's reference.
     *
     * e.g. For 'entry::123', it would be 'entry'.
     */
    protected static $referencePrefix = 'product';

    /**
     * Convert an array of keys to the actual objects.
     * The keys will be searchable references with the prefix removed.
     */
    public function find(array $keys): Collection
    {
        return Product::find($keys);
    }

    /**
     * Get a collection of all searchables.
     */
    public function provide(): Collection
    {
        return Product::all();

        // If you wanted to allow subsets of products, you could specify them in your
        // config then retrieve them appropriately here using keys.
        // e.g. 'searchables' => ['products:hats', 'products:shoes'],
        // $this->keys would be ['keys', 'hats'].
        return Product::whereIn('type', $this->keys)->get();
    }

    /**
     * Check if a given object belongs to this provider.
     */
    public function contains($searchable): bool
    {
        return $searchable instanceof Product;
    }
}

Register the Provider

In order to use the provider, first register it within a service provider, and then it will be available to your search config by its handle.

use Statamic\Facades\Search;

public function boot()
{
    ProductProvider::register();
}

  // config/statamic/search.php
  
  'indexes' => [
      'products_and_blog_posts' => [
          'driver' => 'local',
          'searchables' => [
 +            'products', 
              'collections:blog'
          ],
         'fields' => ['title']
     ]
 ]

Event Listeners

You will want to update the indexes when you create, edit, or delete your searchable items.

Continuing with the Eloquent example, we can hook into model events in your service provider.

use Illuminate\Support\Facades\Event;
use Statamic\Facades\Search;

Event::listen('eloquent.saved: App\Models\Product', function ($model) {
    Search::updateWithinIndexes($model);
});

Event::listen('eloquent.deleted: App\Models\Product', function ($model) {
    Search::deleteFromIndexes($model);
});

The updateWithinIndexes method will update the record in all appropriate indexes. If a filter determines that the record should be removed (e.g. if it changed into a draft), it'll remove it.

The deleteFromIndexes method will remove it from all appropriate indexes.

Custom Index Drivers

Statamic comes with two native search index drivers: Comb and Algolia. Comb is our "local" driver, where indexes are stored as json files. Algolia integrates with the service using their API.

For this example, we'll integrate with a fictional service called FastSearch.

Create Index

You should have a class that extends Index.

use Statamic\Search\Index;

class FastSearchIndex extends Index
{
    private $client;

    public function __construct(FastSearchClient $client, $name, array $config, string $locale = null)
    {
        // In this example, we'll accept a fictional client class that will perform API requests.
        // If you have a constructor, don't forget to construct the parent class too.
        $this->client = $client;
        parent::__construct($name, $config, $locale);
    }

    /**
     * Return a query builder that will perform the search.
     */
    public function search($query)
    {
        return (new FastSearchQuery($this))->query($query);
    }

    /**
     * Check whether the index actually exists.
     * i.e. Does it exist in the service, or as a json file, etc.
     */
    public function exists()
    {
        $this->client->indexExists($this->name);
    }

    /**
     * Insert items into the index.
     */
    protected function insertDocuments(Documents $documents)
    {
        $this->client->insertObjects($documents->all());
    }

    /**
     * Delete an item from the index.
     */
    public function delete($document)
    {
        $this->client->deleteObject($document);
    }

    /**
     * Delete the entire index.
     */
    protected function deleteIndex()
    {
        $this->client->deleteIndex($this->name);
    }
}

Register Index

public function boot()
{
    Search::extend('fast', function ($app, $config, $name) {
        $client = new FastSearchApiClient('api-key');
        return new FastSearchIndex($client, $name, $config);
    });
}

Create Query Builder

In the index class, the search method wanted a query builder. You can create a class that extends our own, which only requires you to define a single method.

<?php

namespace App\Search;

use Statamic\Search\QueryBuilder;
use Statamic\Support\Str;

class CustomSearchQuery extends QueryBuilder
{
    /**
     * Get search results as an array.
     * e.g. [
     *  ['title' => 'One', 'search_score' => 500],
     *  ['title' => 'Two', 'search_score' => 400],
     * ]
     */
    public function getSearchResults()
    {
        $results = $this->index->searchUsingApi($query);

        // Statamic will expects a search_score to be in each result.
        // Some services like Algolia don't have scores and will just return them in order.
        // This is a trick to set the scores in sequential order, highest first.
        return $results->map(function ($result, $i) use ($results) {
            $result['search_score'] = $results->count() - $i;

            return $result;
        });
    }
}

This getSearchResults method is used in the parent class in order to allow basic filtering and other query methods. Of course, you are free to build as much of your own query builder as you like.