Generate autocomplete suggestions based on what your users search
🍊 Battle-tested at Instacart
Add this line to your application’s Gemfile:
gem "autosuggest"Start with a hash of queries and their popularity, like the number of users who have searched it.
top_queries = {
"bananas" => 353,
"apples" => 213,
"oranges" => 140
}With Searchjoy, you can do:
top_queries = Searchjoy::Search.group(:normalized_query)
.having("COUNT(DISTINCT user_id) >= 5").distinct.count(:user_id)Then pass them to Autosuggest.
autosuggest = Autosuggest::Generator.new(top_queries)Stemming is used to detect duplicates like apple and apples.
Specify the stemming language (defaults to english) with:
autosuggest = Autosuggest::Generator.new(top_queries, language: "spanish")The most popular query is preferred by default. To override this, use:
autosuggest.prefer ["apples"]To fix false positives, use:
autosuggest.not_duplicates [["straws", "straus"]]We tried open-source libraries like Aspell and Hunspell but quickly realized we needed to build a corpus specific to our application.
There are two ways to build the corpus, which can be used together.
- Add words
autosuggest.parse_words Product.pluck(:name)Use the min option to only add words that appear multiple times.
- Add concepts
autosuggest.add_concept "brand", Brand.pluck(:name)Profanity is blocked by default. Add custom words with:
autosuggest.block_words ["boom"]Generate suggestions with:
suggestions = autosuggest.suggestionsSave suggestions in your database or another data store.
With Rails, you can generate a simple model with:
rails generate autosuggest:suggestions
rails db:migrateAnd update suggestions with:
now = Time.now
records = suggestions.map { |s| s.slice(:query, :score).merge(updated_at: now) }
Autosuggest::Suggestion.transaction do
Autosuggest::Suggestion.upsert_all(records, unique_by: :query)
Autosuggest::Suggestion.where("updated_at < ?", now).delete_all
endLeave out unique_by for MySQL, and use activerecord-import for upserts with Rails < 6.
Use a JavaScript autocomplete library like typeahead.js to show suggestions in the UI.
If you only have a few thousand suggestions, it’s much faster to load them all at once instead of as a user types (eliminates network requests).
With Rails, you can load all suggestions with:
Autosuggest::Suggestion.order(score: :desc).pluck(:query)And suggestions matching user input with:
input = params[:query]
Autosuggest::Suggestion
.order(score: :desc)
.where("query LIKE ?", "%#{Autosuggest::Suggestion.sanitize_sql_like(input.downcase)}%")
.pluck(:query)You can also cache suggestions for performance.
Rails.cache.fetch("suggestions", expires_in: 5.minutes) do
Autosuggest::Suggestion.order(score: :desc).pluck(:query)
endYou may want to have someone manually approve suggestions:
Autosuggest::Suggestion.where(status: "approved")Or filter suggestions without results:
Autosuggest::Suggestion.find_each do |suggestion|
suggestion.results_count = Product.search(suggestion.query, load: false).count
suggestion.save! if suggestion.changed?
end
Autosuggest::Suggestion.where("results_count > 0")You can add additional fields to your model/data store to accomplish this.
top_queries = Searchjoy::Search.group(:normalized_query)
.having("COUNT(DISTINCT user_id) >= 5").distinct.count(:user_id)
product_names = Product.pluck(:name)
brand_names = Brand.pluck(:name)
autosuggest = Autosuggest::Generator.new(top_queries)
autosuggest.parse_words product_names
autosuggest.add_concept "brand", brand_names
autosuggest.prefer brand_names
autosuggest.not_duplicates [["straws", "straus"]]
autosuggest.block_words ["boom"]
suggestions = autosuggest.suggestions
now = Time.now
records = suggestions.map { |s| s.slice(:query, :score).merge(updated_at: now) }
Autosuggest::Suggestion.transaction do
Autosuggest::Suggestion.upsert_all(records, unique_by: :query)
Autosuggest::Suggestion.where("updated_at < ?", now).delete_all
endSuggestions are now filtered by default, and only the query and score are returned. Get all queries and fields with:
autosuggest.suggestions(filter: false)View the changelog
Everyone is encouraged to help improve this project. Here are a few ways you can help:
- Report bugs
- Fix bugs and submit pull requests
- Write, clarify, or fix documentation
- Suggest or add new features
To get started with development:
git clone https://github.com/ankane/autosuggest.git
cd autosuggest
bundle install
bundle exec rake test
