GraphQL DataLoader: Batch Loading, Caching, and N+1 Prevention in TypeScript

The N+1 problem is GraphQL's most common performance pitfall. A query for 100 posts that each have an author triggers 1 query for posts + 100 queries for authors — 101 database round-trips instead of 2. DataLoader solves this by batching and deduplicating all loads within a single event loop tick: those 100 author lookups become a single WHERE id IN (...) query.

This post covers DataLoader patterns from the ground up: basic batch loading, per-request caching, nested loaders for relationships, and production integration with Prisma and Apollo Server.

How DataLoader Works

GraphQL resolves fields concurrently within a single request.
Without DataLoader:

  post[0].author → db.user.findUnique({ where: { id: 'u1' } })
  post[1].author → db.user.findUnique({ where: { id: 'u2' } })
  post[2].author → db.user.findUnique({ where: { id: 'u1' } })  ← duplicate!
  ...100 more queries

With DataLoader (single event loop tick):

  post[0].author → loader.load('u1')  ┐
  post[1].author → loader.load('u2')  ├── batched → db.user.findMany({ where: { id: IN ['u1','u2'] } })
  post[2].author → loader.load('u1')  ┘  (deduplicated: u1 only fetched once)
  → 1 query total

---

1. Basic DataLoader Setup

npm install dataloader

// src/lib/graphql/loaders/user.loader.ts
import DataLoader from 'dataloader';
import { db } from '../../db';

// Batch function: receives array of keys, must return array of values
// in the SAME ORDER as keys (or Error for missing items)
async function batchLoadUsers(ids: readonly string[]): Promise<Array<User | Error>> {
  const users = await db.user.findMany({
    where: { id: { in: ids as string[] } },
  });

  // Build a map for O(1) lookup
  const userMap = new Map(users.map((u) => [u.id, u]));

  // Return in the same order as input ids
  // DataLoader requires this — mismatched order causes wrong cache hits
  return ids.map(
    (id) => userMap.get(id) ?? new Error(`User not found: ${id}`)
  );
}

export function createUserLoader() {
  return new DataLoader<string, User>(batchLoadUsers, {
    cache: true,          // Per-request cache (default: true)
    maxBatchSize: 1000,   // Max keys per batch
    batchScheduleFn: (callback) => setTimeout(callback, 0), // Next tick
  });
}

Context Integration with Apollo Server

// src/graphql/context.ts
import { createUserLoader } from './loaders/user.loader';
import { createPostLoader } from './loaders/post.loader';
import { createCommentCountLoader } from './loaders/comment-count.loader';

export interface GraphQLContext {
  userId: string | null;
  tenantId: string | null;
  loaders: {
    user: ReturnType<typeof createUserLoader>;
    post: ReturnType<typeof createPostLoader>;
    commentCount: ReturnType<typeof createCommentCountLoader>;
  };
}

// Create FRESH loaders for EACH request (don't share loaders across requests!)
export function createContext(req: Request): GraphQLContext {
  return {
    userId: req.headers.get('x-user-id'),
    tenantId: req.headers.get('x-tenant-id'),
    loaders: {
      user: createUserLoader(),
      post: createPostLoader(),
      commentCount: createCommentCountLoader(),
    },
  };
}

// Apollo Server setup
const server = new ApolloServer<GraphQLContext>({
  schema,
});

const handler = startStandaloneServer(server, {
  context: async ({ req }) => createContext(req as Request),
});

---

2. Resolver Usage

// src/graphql/resolvers/post.resolvers.ts
import { Resolvers } from '../generated/types';

export const postResolvers: Resolvers<GraphQLContext> = {
  Query: {
    posts: async (_parent, { limit = 20, cursor }, ctx) => {
      return db.post.findMany({
        take: limit + 1,
        cursor: cursor ? { id: cursor } : undefined,
        skip: cursor ? 1 : 0,
        orderBy: { createdAt: 'desc' },
      });
    },
  },

  Post: {
    // Each of these calls loader.load() — batched automatically
    author: async (post, _args, ctx) => {
      return ctx.loaders.user.load(post.authorId);
    },

    commentCount: async (post, _args, ctx) => {
      return ctx.loaders.commentCount.load(post.id);
    },

    // Tags: many-to-many — use a different loader pattern
    tags: async (post, _args, ctx) => {
      return ctx.loaders.postTags.load(post.id);
    },
  },

  User: {
    // Nested: load posts for each user — still batched!
    posts: async (user, { limit = 5 }, ctx) => {
      return ctx.loaders.userPosts.load({ userId: user.id, limit });
    },
  },
};

---

3. Advanced Loader Patterns

One-to-Many: Posts by Author

For one-to-many relationships, the batch function groups results by the parent key.

// src/lib/graphql/loaders/post.loader.ts — posts by authorId
import DataLoader from 'dataloader';
import { db } from '../../db';

async function batchLoadPostsByAuthor(
  authorIds: readonly string[]
): Promise<Post[][]> {
  const posts = await db.post.findMany({
    where: { authorId: { in: authorIds as string[] } },
    orderBy: { createdAt: 'desc' },
  });

  // Group by authorId
  const postsByAuthor = new Map<string, Post[]>();
  for (const authorId of authorIds) {
    postsByAuthor.set(authorId, []);
  }
  for (const post of posts) {
    postsByAuthor.get(post.authorId)!.push(post);
  }

  // Return in same order as input
  return authorIds.map((id) => postsByAuthor.get(id) ?? []);
}

export function createUserPostsLoader() {
  return new DataLoader<string, Post[]>(batchLoadPostsByAuthor);
}

Aggregate Loader: Comment Counts

// src/lib/graphql/loaders/comment-count.loader.ts
import DataLoader from 'dataloader';
import { db } from '../../db';

async function batchLoadCommentCounts(
  postIds: readonly string[]
): Promise<number[]> {
  // Single query: COUNT grouped by postId
  const counts = await db.$queryRaw<Array<{ post_id: string; count: bigint }>>`
    SELECT post_id, COUNT(*) as count
    FROM comments
    WHERE post_id = ANY(${postIds as string[]}::uuid[])
    GROUP BY post_id
  `;

  const countMap = new Map(counts.map((r) => [r.post_id, Number(r.count)]));

  return postIds.map((id) => countMap.get(id) ?? 0);
}

export function createCommentCountLoader() {
  return new DataLoader<string, number>(batchLoadCommentCounts);
}

Compound Key Loader (with Arguments)

When the same resolver can be called with different arguments, encode them into the key.

// src/lib/graphql/loaders/user-posts.loader.ts
interface UserPostsKey {
  userId: string;
  limit: number;
  status?: string;
}

// Encode compound key as string for DataLoader
function encodeKey(key: UserPostsKey): string {
  return JSON.stringify({ userId: key.userId, limit: key.limit, status: key.status ?? 'published' });
}

async function batchLoadUserPosts(
  keys: readonly UserPostsKey[]
): Promise<Post[][]> {
  // Group keys by their parameters (limit/status) to minimize queries
  const groups = new Map<string, string[]>();

  for (const key of keys) {
    const paramKey = JSON.stringify({ limit: key.limit, status: key.status ?? 'published' });
    if (!groups.has(paramKey)) groups.set(paramKey, []);
    groups.get(paramKey)!.push(key.userId);
  }

  const resultMap = new Map<string, Post[]>();

  for (const [paramKey, userIds] of groups) {
    const { limit, status } = JSON.parse(paramKey);

    const posts = await db.post.findMany({
      where: {
        authorId: { in: userIds },
        status,
      },
      orderBy: { createdAt: 'desc' },
      take: limit * userIds.length, // Over-fetch, then trim per user
    });

    // Group by authorId and trim to limit
    const byAuthor = new Map<string, Post[]>();
    for (const userId of userIds) byAuthor.set(userId, []);
    for (const post of posts) {
      const authorPosts = byAuthor.get(post.authorId)!;
      if (authorPosts.length < limit) authorPosts.push(post);
    }

    for (const [userId, userPosts] of byAuthor) {
      resultMap.set(JSON.stringify({ userId, limit, status }), userPosts);
    }
  }

  return keys.map((key) => resultMap.get(encodeKey(key)) ?? []);
}

export function createUserPostsLoader() {
  return new DataLoader<UserPostsKey, Post[]>(batchLoadUserPosts, {
    // Custom cache key function for compound keys
    cacheKeyFn: encodeKey,
  });
}

---

4. DataLoader with Prisma Batch Optimization

// src/lib/graphql/loaders/factories.ts
// Centralized loader factory with Prisma

import DataLoader from 'dataloader';
import { PrismaClient } from '@prisma/client';

// Generic loader factory for simple ID lookups
export function createIdLoader<T extends { id: string }>(
  fetchFn: (ids: string[]) => Promise<T[]>
): DataLoader<string, T> {
  return new DataLoader<string, T>(async (ids) => {
    const items = await fetchFn([...ids]);
    const map = new Map(items.map((item) => [item.id, item]));
    return ids.map((id) => map.get(id) ?? new Error(`Not found: ${id}`));
  });
}

// Usage:
// const userLoader = createIdLoader((ids) =>
//   db.user.findMany({ where: { id: { in: ids } } })
// );

// Loader with tenant isolation
export function createTenantScopedLoader<T extends { id: string }>(
  tenantId: string,
  fetchFn: (ids: string[], tenantId: string) => Promise<T[]>
): DataLoader<string, T | null> {
  return new DataLoader<string, T | null>(async (ids) => {
    const items = await fetchFn([...ids], tenantId);
    const map = new Map(items.map((item) => [item.id, item]));
    // Return null (not Error) for not-found — caller decides if that's an error
    return ids.map((id) => map.get(id) ?? null);
  });
}

---

5. Cache Invalidation and Prime

DataLoader's per-request cache means you don't need manual invalidation — each request gets a fresh cache. But within a request, you can prime the cache after mutations.

// Mutation resolver: update user, then prime cache with new value
export const mutationResolvers = {
  Mutation: {
    updateUser: async (_parent, { id, input }, ctx: GraphQLContext) => {
      const updated = await db.user.update({
        where: { id },
        data: input,
      });

      // Prime the cache: next load(id) returns this value without a DB hit
      ctx.loaders.user.prime(id, updated);

      return updated;
    },

    deletePost: async (_parent, { id }, ctx: GraphQLContext) => {
      await db.post.delete({ where: { id } });

      // Clear specific key from cache
      ctx.loaders.post.clear(id);
      // Or clear all: ctx.loaders.post.clearAll();

      return { success: true };
    },
  },
};

---

6. Testing DataLoaders

// src/lib/graphql/loaders/__tests__/user.loader.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { createUserLoader } from '../user.loader';
import { db } from '../../../db';

vi.mock('../../../db', () => ({
  db: {
    user: {
      findMany: vi.fn(),
    },
  },
}));

describe('UserLoader', () => {
  beforeEach(() => {
    vi.clearAllMocks();
  });

  it('batches multiple loads into a single DB query', async () => {
    const mockUsers = [
      { id: 'u1', name: 'Alice', email: 'alice@example.com' },
      { id: 'u2', name: 'Bob', email: 'bob@example.com' },
    ];

    vi.mocked(db.user.findMany).mockResolvedValue(mockUsers as any);

    const loader = createUserLoader();

    // Load multiple users "concurrently" (same tick)
    const [user1, user2, user1Again] = await Promise.all([
      loader.load('u1'),
      loader.load('u2'),
      loader.load('u1'), // duplicate — should use cache
    ]);

    expect(user1).toEqual(mockUsers[0]);
    expect(user2).toEqual(mockUsers[1]);
    expect(user1Again).toEqual(mockUsers[0]);

    // Only ONE DB query despite 3 load() calls
    expect(db.user.findMany).toHaveBeenCalledTimes(1);
    expect(db.user.findMany).toHaveBeenCalledWith({
      where: { id: { in: ['u1', 'u2'] } }, // Deduplicated
    });
  });

  it('returns Error for missing user', async () => {
    vi.mocked(db.user.findMany).mockResolvedValue([]);

    const loader = createUserLoader();
    const result = await loader.load('missing-id').catch((e) => e);

    expect(result).toBeInstanceOf(Error);
    expect(result.message).toContain('missing-id');
  });
});

---

Performance Impact

---

See Also

• GraphQL Persisted Queries and Performance Optimization
• GraphQL Federation: Microservices with a Unified Schema
• GraphQL Subscriptions: Real-Time Updates with WebSockets
• PostgreSQL Performance: Indexes, Query Plans, and Connection Pooling
• React Query Server State: useQuery, Optimistic Updates, and Infinite Scroll

---

Working With Viprasol

Building a GraphQL API that's hitting N+1 performance walls? We implement DataLoader patterns across your resolver tree, identify batch loading opportunities with query analysis, and integrate with Prisma for optimal database utilization — turning 100-query waterfalls into single batched fetches.

Talk to our team → | See our web development services →