BatchLoader
This package provides a generic lazy batching mechanism to avoid N+1 DB queries, HTTP queries, etc.
Contents
Highlights
- Generic utility to avoid N+1 DB queries, HTTP requests, etc.
- Adapted Elixir implementation of the battle-tested tools like Haskell Haxl, JS DataLoader, Ruby BatchLoader.
- Convenient and flexible integration with Ecto Schemas.
- Allows inlining the code without defining extra named functions, unlike Absinthe Batch.
- Allows using batching with any data sources, not just DB, unlike Absinthe DataLoader.
Usage
Let's imagine that we have a Post GraphQL type defined with Absinthe:
defmodule MyApp.PostType do
use Absinthe.Schema.Notation
alias MyApp.Repo
object :post_type do
field :title, :string
field :user, :user_type do
resolve(fn post, _, _ ->
user = post |> Ecto.assoc(:user) |> Repo.one() # N+1 DB requests
{:ok, user}
end)
end
end
endThis will produce N+1 DB requests if we send this GraphQL request:
query {
posts {
title
user { # N+1 request per each post
name
}
}
}Ecto Resolve Association
We can get rid of the N+1 DB requests by loading all Users for all Posts at once in.
All we have to do is to use resolve_assoc function by passing the Ecto associations name:
import BatchLoader.Absinthe, only: [resolve_assoc: 1]
field :user, :user_type, resolve: resolve_assoc(:user)Set the default repo in your config.exs file:
config :batch_loader, :default_repo, MyApp.RepoAnd finally, add BatchLoader.Absinthe.Plugin plugin to the GraphQL schema.
This will allow to lazily collect information about all users which need to be loaded and then batch them all together:
defmodule MyApp.Schema do
use Absinthe.Schema
import_types MyApp.PostType
def plugins do
[BatchLoader.Absinthe.Plugin] ++ Absinthe.Plugin.defaults()
end
endEcto Load Association
You can use load_assoc to load Ecto associations in the existing schema:
import BatchLoader.Absinthe, only: [load_assoc: 3]
field :author, :string do
resolve(fn post, _, _ ->
load_assoc(post, :user, fn user ->
{:ok, user.name}
end)
end)
endEcto Preload Association
You can use preload_assoc to preload Ecto associations in the existing schema:
import BatchLoader.Absinthe, only: [preload_assoc: 3]
field :title, :string do
resolve(fn post, _, _ ->
preload_assoc(post, :user, fn post_with_user ->
{:ok, "#{post_with_user.title} - #{post_with_user.user.name}"}
end)
end)
endDIY Batching
You can also use BatchLoader to batch in the resolve function manually, for example, to fix N+1 HTTP requests:
field :user, :user_type do
resolve(fn post, _, _ ->
BatchLoader.Absinthe.for(post.user_id, &resolved_users_by_user_ids/1)
end)
end
def resolved_users_by_user_ids(user_ids) do
MyApp.HttpClient.users(user_ids) # load all users at once
|> Enum.map(fn user -> {user.id, {:ok, user}} end) # return "{user.id, result}" tuples
endAlternatively, you can simply inline the batch function:
field :user, :user_type do
resolve(fn post, _, _ ->
BatchLoader.Absinthe.for(post.user_id, fn user_ids ->
MyApp.HttpClient.users(user_ids)
|> Enum.map(fn user -> {user.id, {:ok, user}} end)
end)
end)
endCustomization
- To specify default resolve Absinthe values:
BatchLoader.Absinthe.for(post.user_id, &resolved_users_by_user_ids/1, default_value: {:error, "NOT FOUND"})- To use custom callback function:
BatchLoader.Absinthe.for(post.user_id, &users_by_user_ids/1, callback: fn user ->
{:ok, user.name}
end)- To use custom Ecto repos:
BatchLoader.Absinthe.resolve_assoc(:user, repo: AnotherRepo)
BatchLoader.Absinthe.preload_assoc(post, :user, &callback/1, repo: AnotherRepo)- To pass custom options to
Ecto.Repo.preload:
BatchLoader.Absinthe.resolve_assoc(:user, preload_opts: [prefix: nil])
BatchLoader.Absinthe.preload_assoc(post, :user, &callback/1, preload_opts: [prefix: nil])Installation
Add batch_loader to your list of dependencies in mix.exs:
def deps do
[
{:batch_loader, "~> 0.1.0-beta.6"}
]
endTesting
make install
make test
