MongoDB Aggregation Pipeline Tutorial

Introduction

The aggregation pipeline is MongoDB's answer to complex reporting and analytics queries — the equivalent of SQL's GROUP BY, joins, and computed columns, but expressed as a sequence of stages that each transform the data flowing through them. Once you understand the pipeline as "documents flowing through a series of steps," building sophisticated queries becomes a matter of composing simple, well-understood stages.

The Basic Idea: A Pipeline of Stages

db.orders.aggregate([
  { $match: { status: "completed" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
]);

Each stage receives the documents produced by the previous stage (or the raw collection, for the first stage) and passes its own output forward. This example filters completed orders, groups them by customer while summing their order amounts, and sorts the results by total spend, descending.

$match: Filtering Documents

$match behaves like the filter you would pass to find(), and should almost always come as early as possible in the pipeline to reduce the number of documents later stages must process:

{ $match: { status: "completed", amount: { $gt: 100 } } }

$group: Aggregating by Key

$group is the core of most reporting pipelines. The _id field of the stage specifies what to group by, and every other field defines a computed value using an accumulator:

db.orders.aggregate([
  {
    $group: {
      _id: "$customerId",
      totalSpent: { $sum: "$amount" },
      orderCount: { $sum: 1 },
      averageOrder: { $avg: "$amount" },
      maxOrder: { $max: "$amount" },
    },
  },
]);

Common accumulators include $sum, $avg, $min, $max, $push (collect values into an array), and $addToSet (collect unique values into an array).

$project: Reshaping Documents

$project controls exactly which fields appear in the output, and can also compute new fields:

db.orders.aggregate([
  {
    $project: {
      customerId: 1,
      amount: 1,
      amountWithTax: { $multiply: ["$amount", 1.18] },
      _id: 0,
    },
  },
]);

$sort and $limit: Ordering and Trimming Results

db.orders.aggregate([
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 10 }, // top 10 customers by spend
]);

$lookup: Joining Collections

MongoDB is not a relational database, but $lookup performs a left-outer-join-style operation between two collections when data is spread across them:

db.orders.aggregate([
  {
    $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customerDetails",
    },
  },
  { $unwind: "$customerDetails" }, // flatten the joined array into an object
]);

$lookup returns an array (even when there is only one match), so $unwind is commonly used afterward to flatten that array into a single embedded document per result.

A Complete Reporting Example

Combining several stages produces a realistic sales report: total revenue per product category, for completed orders only, sorted highest first:

db.orders.aggregate([
  { $match: { status: "completed" } },
  { $unwind: "$items" },
  {
    $group: {
      _id: "$items.category",
      revenue: { $sum: { $multiply: ["$items.price", "$items.qty"] } },
      unitsSold: { $sum: "$items.qty" },
    },
  },
  { $sort: { revenue: -1 } },
  { $project: { category: "$_id", revenue: 1, unitsSold: 1, _id: 0 } },
]);

Performance Considerations

• Place $match (and $sort where possible) as early as possible so MongoDB can use indexes and reduce the working set before expensive stages.
• Avoid $lookup on very large foreign collections without a supporting index on the foreign field; it can be a significant performance bottleneck.
• Use $project early to drop unneeded fields before heavier stages like $group, reducing the amount of data each stage has to process.
• For pipelines with predictable repeated use, consider precomputing and storing aggregate values instead of recalculating them on every request.

Best Practices

• Build pipelines incrementally, checking the output of each stage before adding the next, especially for complex reports.
• Name grouped fields clearly in $project so consumers of the API do not have to guess what _id represents after a $group.
• Use $facet when you need multiple independent aggregations (for example, paginated results plus a total count) from a single query.
• Test aggregation pipelines against realistic data volumes, since behavior and performance can differ significantly from a small development dataset.

Common Mistakes to Avoid

• Placing $match after expensive stages instead of before them, forcing MongoDB to process far more documents than necessary.
• Forgetting that $lookup always returns an array, and trying to access joined fields directly without $unwind.
• Omitting an initial value or forgetting accumulators inside $group, resulting in unexpected null or missing fields.
• Building overly long pipelines that are difficult to debug — breaking a report into smaller, named pipeline stages (or separate queries) can be easier to maintain.

Multiple Views of the Same Data with $facet

Sometimes you need several different aggregations computed from the exact same filtered dataset in one round trip — for example, a paginated list of results alongside the total count for pagination controls. $facet runs multiple sub-pipelines in parallel over the same input:

db.orders.aggregate([
  { $match: { status: "completed" } },
  {
    $facet: {
      paginatedResults: [
        { $sort: { createdAt: -1 } },
        { $skip: 20 },
        { $limit: 10 },
      ],
      totalCount: [{ $count: "count" }],
      revenueByMonth: [
        { $group: { _id: { $month: "$createdAt" }, total: { $sum: "$amount" } } },
      ],
    },
  },
]);

This single query returns an object with three independent keys — paginatedResults, totalCount, and revenueByMonth — each computed from its own sub-pipeline but sharing the same initial $match-filtered input, avoiding the need to query the collection three separate times just to assemble one dashboard view. The main trade-off is that $facet processes the full matched dataset once per sub-pipeline internally, so it is best suited to moderately sized result sets rather than aggregating across an entire massive collection with no upstream filtering.

Indexing to Support Aggregation

An aggregation pipeline benefits from indexes exactly the same way a regular query does, but only for stages that can actually use one — primarily an early $match or $sort stage. Placing $match as close to the start of the pipeline as possible lets MongoDB use an existing index to filter documents before any of the more expensive grouping or reshaping stages run, dramatically reducing the amount of data those later stages have to process:

db.orders.createIndex({ status: 1, createdAt: -1 });

db.orders.aggregate([
  { $match: { status: "completed" } }, // uses the index above
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
]);

Stages after the first $group or $project that reshapes documents generally cannot use indexes at all, since the documents flowing through the pipeline at that point are no longer the original indexed documents — they are computed intermediate results. This is why pipeline stage order matters for performance as much as for correctness: filtering and sorting early, before reshaping, gives MongoDB the best chance to use an index and avoid scanning far more documents than the final result actually needs.

Conclusion

The aggregation pipeline turns MongoDB from a simple document store into a genuine analytics engine. By composing focused stages — filter, group, reshape, join, sort — you can build reports and computed views that would otherwise require pulling data out and processing it in application code. Start with $match and $group, and layer in $lookup and $project as your reporting needs grow.