GraphQL DataLoader: Batch Loading, Caching

Quick answer. DataLoader fixes GraphQL's N+1 problem by batching and deduplicating all loads within one event-loop tick, turning 100 per-post author lookups into a single WHERE id IN (...) query. It also caches per request, and integrates with Prisma and Apollo Server using nested loaders for relationships. The N+1 problem is GraphQL's most common performance pitfall. A query for 100 posts that each have an author triggers 1 query for posts + 100 queries for authors — 101 database round-trips instead of 2. DataLoader solves this by batching and deduplicating all loads within a single event loop tick: those 100 author lookups become a single WHERE id IN (...) query.

This post covers DataLoader patterns from the ground up: basic batch loading, per-request caching, nested loaders for relationships, and production integration with Prisma and Apollo Server.

How DataLoader Works

GraphQL resolves fields concurrently within a single request.
Without DataLoader:

  post[0].author → db.user.findUnique({ where: { id: 'u1' } })
  post[1].author → db.user.findUnique({ where: { id: 'u2' } })
  post[2].author → db.user.findUnique({ where: { id: 'u1' } })  ← duplicate!
  ...100 more queries

With DataLoader (single event loop tick):

  post[0].author → loader.load('u1')  ┐
  post[1].author → loader.load('u2')  ├── batched → db.user.findMany({ where: { id: IN ['u1','u2'] } })
  post[2].author → loader.load('u1')  ┘  (deduplicated: u1 only fetched once)
  → 1 query total

---

1. Basic DataLoader Setup

npm install dataloader

// src/lib/graphql/loaders/user.loader.ts
import DataLoader from 'dataloader';
import { db } from '../../db';

// Batch function: receives array of keys, must return array of values
// in the SAME ORDER as keys (or Error for missing items)
async function batchLoadUsers(ids: readonly string[]): Promise<Array<User | Error>> {
  const users = await db.user.findMany({
    where: { id: { in: ids as string[] } },
  });

  // Build a map for O(1) lookup
  const userMap = new Map(users.map((u) => [u.id, u]));

  // Return in the same order as input ids
  // DataLoader requires this — mismatched order causes wrong cache hits
  return ids.map(
    (id) => userMap.get(id) ?? new Error(`User not found: ${id}`)
  );
}

export function createUserLoader() {
  return new DataLoader<string, User>(batchLoadUsers, {
    cache: true,          // Per-request cache (default: true)
    maxBatchSize: 1000,   // Max keys per batch
    batchScheduleFn: (callback) => setTimeout(callback, 0), // Next tick
  });
}

Context Integration with Apollo Server

// src/graphql/context.ts
import { createUserLoader } from './loaders/user.loader';
import { createPostLoader } from './loaders/post.loader';
import { createCommentCountLoader } from './loaders/comment-count.loader';

export interface GraphQLContext {
  userId: string | null;
  tenantId: string | null;
  loaders: {
    user: ReturnType<typeof createUserLoader>;
    post: ReturnType<typeof createPostLoader>;
    commentCount: ReturnType<typeof createCommentCountLoader>;
  };
}

// Create FRESH loaders for EACH request (don't share loaders across requests!)
export function createContext(req: Request): GraphQLContext {
  return {
    userId: req.headers.get('x-user-id'),
    tenantId: req.headers.get('x-tenant-id'),
    loaders: {
      user: createUserLoader(),
      post: createPostLoader(),
      commentCount: createCommentCountLoader(),
    },
  };
}

// Apollo Server setup
const server = new ApolloServer<GraphQLContext>({
  schema,
});

const handler = startStandaloneServer(server, {
  context: async ({ req }) => createContext(req as Request),
});

---

2. Resolver Usage

// src/graphql/resolvers/post.resolvers.ts
import { Resolvers } from '../generated/types';

export const postResolvers: Resolvers<GraphQLContext> = {
  Query: {
    posts: async (_parent, { limit = 20, cursor }, ctx) => {
      return db.post.findMany({
        take: limit + 1,
        cursor: cursor ? { id: cursor } : undefined,
        skip: cursor ? 1 : 0,
        orderBy: { createdAt: 'desc' },
      });
    },
  },

  Post: {
    // Each of these calls loader.load() — batched automatically
    author: async (post, _args, ctx) => {
      return ctx.loaders.user.load(post.authorId);
    },

    commentCount: async (post, _args, ctx) => {
      return ctx.loaders.commentCount.load(post.id);
    },

    // Tags: many-to-many — use a different loader pattern
    tags: async (post, _args, ctx) => {
      return ctx.loaders.postTags.load(post.id);
    },
  },

  User: {
    // Nested: load posts for each user — still batched!
    posts: async (user, { limit = 5 }, ctx) => {
      return ctx.loaders.userPosts.load({ userId: user.id, limit });
    },
  },
};

---

3. Advanced Loader Patterns

One-to-Many: Posts by Author

For one-to-many relationships, the batch function groups results by the parent key.

// src/lib/graphql/loaders/post.loader.ts — posts by authorId
import DataLoader from 'dataloader';
import { db } from '../../db';

async function batchLoadPostsByAuthor(
  authorIds: readonly string[]
): Promise<Post[][]> {
  const posts = await db.post.findMany({
    where: { authorId: { in: authorIds as string[] } },
    orderBy: { createdAt: 'desc' },
  });

  // Group by authorId
  const postsByAuthor = new Map<string, Post[]>();
  for (const authorId of authorIds) {
    postsByAuthor.set(authorId, []);
  }
  for (const post of posts) {
    postsByAuthor.get(post.authorId)!.push(post);
  }

  // Return in same order as input
  return authorIds.map((id) => postsByAuthor.get(id) ?? []);
}

export function createUserPostsLoader() {
  return new DataLoader<string, Post[]>(batchLoadPostsByAuthor);
}

Aggregate Loader: Comment Counts

// src/lib/graphql/loaders/comment-count.loader.ts
import DataLoader from 'dataloader';
import { db } from '../../db';

async function batchLoadCommentCounts(
  postIds: readonly string[]
): Promise<number[]> {
  // Single query: COUNT grouped by postId
  const counts = await db.$queryRaw<Array<{ post_id: string; count: bigint }>>`
    SELECT post_id, COUNT(*) as count
    FROM comments
    WHERE post_id = ANY(${postIds as string[]}::uuid[])
    GROUP BY post_id
  `;

  const countMap = new Map(counts.map((r) => [r.post_id, Number(r.count)]));

  return postIds.map((id) => countMap.get(id) ?? 0);
}

export function createCommentCountLoader() {
  return new DataLoader<string, number>(batchLoadCommentCounts);
}

Compound Key Loader (with Arguments)

When the same resolver can be called with different arguments, encode them into the key.

// src/lib/graphql/loaders/user-posts.loader.ts
interface UserPostsKey {
  userId: string;
  limit: number;
  status?: string;
}

// Encode compound key as string for DataLoader
function encodeKey(key: UserPostsKey): string {
  return JSON.stringify({ userId: key.userId, limit: key.limit, status: key.status ?? 'published' });
}

async function batchLoadUserPosts(
  keys: readonly UserPostsKey[]
): Promise<Post[][]> {
  // Group keys by their parameters (limit/status) to minimize queries
  const groups = new Map<string, string[]>();

  for (const key of keys) {
    const paramKey = JSON.stringify({ limit: key.limit, status: key.status ?? 'published' });
    if (!groups.has(paramKey)) groups.set(paramKey, []);
    groups.get(paramKey)!.push(key.userId);
  }

  const resultMap = new Map<string, Post[]>();

  for (const [paramKey, userIds] of groups) {
    const { limit, status } = JSON.parse(paramKey);

    const posts = await db.post.findMany({
      where: {
        authorId: { in: userIds },
        status,
      },
      orderBy: { createdAt: 'desc' },
      take: limit * userIds.length, // Over-fetch, then trim per user
    });

    // Group by authorId and trim to limit
    const byAuthor = new Map<string, Post[]>();
    for (const userId of userIds) byAuthor.set(userId, []);
    for (const post of posts) {
      const authorPosts = byAuthor.get(post.authorId)!;
      if (authorPosts.length < limit) authorPosts.push(post);
    }

    for (const [userId, userPosts] of byAuthor) {
      resultMap.set(JSON.stringify({ userId, limit, status }), userPosts);
    }
  }

  return keys.map((key) => resultMap.get(encodeKey(key)) ?? []);
}

export function createUserPostsLoader() {
  return new DataLoader<UserPostsKey, Post[]>(batchLoadUserPosts, {
    // Custom cache key function for compound keys
    cacheKeyFn: encodeKey,
  });
}

---

4. DataLoader with Prisma Batch Optimization

// src/lib/graphql/loaders/factories.ts
// Centralized loader factory with Prisma

import DataLoader from 'dataloader';
import { PrismaClient } from '@prisma/client';

// Generic loader factory for simple ID lookups
export function createIdLoader<T extends { id: string }>(
  fetchFn: (ids: string[]) => Promise<T[]>
): DataLoader<string, T> {
  return new DataLoader<string, T>(async (ids) => {
    const items = await fetchFn([...ids]);
    const map = new Map(items.map((item) => [item.id, item]));
    return ids.map((id) => map.get(id) ?? new Error(`Not found: ${id}`));
  });
}

// Usage:
// const userLoader = createIdLoader((ids) =>
//   db.user.findMany({ where: { id: { in: ids } } })
// );

// Loader with tenant isolation
export function createTenantScopedLoader<T extends { id: string }>(
  tenantId: string,
  fetchFn: (ids: string[], tenantId: string) => Promise<T[]>
): DataLoader<string, T | null> {
  return new DataLoader<string, T | null>(async (ids) => {
    const items = await fetchFn([...ids], tenantId);
    const map = new Map(items.map((item) => [item.id, item]));
    // Return null (not Error) for not-found — caller decides if that's an error
    return ids.map((id) => map.get(id) ?? null);
  });
}

---

5. Cache Invalidation and Prime

DataLoader's per-request cache means you don't need manual invalidation — each request gets a fresh cache. But within a request, you can prime the cache after mutations.

// Mutation resolver: update user, then prime cache with new value
export const mutationResolvers = {
  Mutation: {
    updateUser: async (_parent, { id, input }, ctx: GraphQLContext) => {
      const updated = await db.user.update({
        where: { id },
        data: input,
      });

      // Prime the cache: next load(id) returns this value without a DB hit
      ctx.loaders.user.prime(id, updated);

      return updated;
    },

    deletePost: async (_parent, { id }, ctx: GraphQLContext) => {
      await db.post.delete({ where: { id } });

      // Clear specific key from cache
      ctx.loaders.post.clear(id);
      // Or clear all: ctx.loaders.post.clearAll();

      return { success: true };
    },
  },
};

---

6. Testing DataLoaders

// src/lib/graphql/loaders/__tests__/user.loader.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { createUserLoader } from '../user.loader';
import { db } from '../../../db';

vi.mock('../../../db', () => ({
  db: {
    user: {
      findMany: vi.fn(),
    },
  },
}));

describe('UserLoader', () => {
  beforeEach(() => {
    vi.clearAllMocks();
  });

  it('batches multiple loads into a single DB query', async () => {
    const mockUsers = [
      { id: 'u1', name: 'Alice', email: 'alice@example.com' },
      { id: 'u2', name: 'Bob', email: 'bob@example.com' },
    ];

    vi.mocked(db.user.findMany).mockResolvedValue(mockUsers as any);

    const loader = createUserLoader();

    // Load multiple users "concurrently" (same tick)
    const [user1, user2, user1Again] = await Promise.all([
      loader.load('u1'),
      loader.load('u2'),
      loader.load('u1'), // duplicate — should use cache
    ]);

    expect(user1).toEqual(mockUsers[0]);
    expect(user2).toEqual(mockUsers[1]);
    expect(user1Again).toEqual(mockUsers[0]);

    // Only ONE DB query despite 3 load() calls
    expect(db.user.findMany).toHaveBeenCalledTimes(1);
    expect(db.user.findMany).toHaveBeenCalledWith({
      where: { id: { in: ['u1', 'u2'] } }, // Deduplicated
    });
  });

  it('returns Error for missing user', async () => {
    vi.mocked(db.user.findMany).mockResolvedValue([]);

    const loader = createUserLoader();
    const result = await loader.load('missing-id').catch((e) => e);

    expect(result).toBeInstanceOf(Error);
    expect(result.message).toContain('missing-id');
  });
});

---

Performance Impact

---

Next Steps

• GraphQL Persisted Queries and Performance Optimization
• GraphQL Federation: Microservices with a Unified Schema
• GraphQL Subscriptions: Real-Time Updates with WebSockets
• PostgreSQL Performance: Indexes, Query Plans, and Connection Pooling
• React Query Server State: useQuery, Optimistic Updates, and Infinite Scroll

---

Inside Viprasol

Building a GraphQL API that's hitting N+1 performance walls? We implement DataLoader patterns across your resolver tree, identify batch loading opportunities with query analysis, and integrate with Prisma for optimal database utilization — turning 100-query waterfalls into single batched fetches.

Talk to our team → | See our web development services →

GraphQL DataLoader Batching, Caching, and the N+1 Problem: What the Official Docs Recommend

If you have read the graphql dataloader batching caching n+1 official docs, the core idea is simple: defer individual loads, collect the keys requested within a single tick of the event loop, then resolve them with one batched query. This is how DataLoader eliminates the N+1 problem, where a list resolver fires one database call per item instead of a single grouped fetch. Per-request caching adds a second win, deduplicating repeated key lookups so the same entity is never loaded twice in one operation.

In practice, create a fresh loader instance per request to keep the cache scoped and avoid stale data leaking between users. At Viprasol Tech, our engineers wire batch functions, key ordering, and cache invalidation correctly so your GraphQL API stays fast under real load, with full ownership end to end.