MongoDB Cheatsheet 11 — Change Streams

MongoDB change streams.

Basic

with db.users.watch() as stream:
    for change in stream:
        print(change)

Watches for inserts, updates, deletes.

Operation types

insert
update
replace
delete
invalidate (collection dropped)

Filter

pipeline = [{"$match": {"operationType": "insert"}}]
with db.users.watch(pipeline) as stream:
    for c in stream:
        process(c)

Watch DB or cluster

db.watch([...])              # entire db
client.watch([...])           # entire cluster

Resume

token = None
while True:
    try:
        with db.users.watch(resume_after=token) as stream:
            for change in stream:
                process(change)
                token = change["_id"]
    except Exception:
        time.sleep(5)

Resume from last token after disconnect.

Persist token

Save token to DB / Redis after each batch. On restart, resume from saved.

Full document

stream = db.users.watch(full_document="updateLookup")

For updates: include current full doc (extra query).

Pre/post images (newer Mongo)

db.runCommand({
    collMod: "users",
    changeStreamPreAndPostImages: { enabled: true }
})

stream = db.users.watch(
    full_document="required",
    full_document_before_change="required",
)

Useful for “what changed” diffs.

Use cases

CDC to Elasticsearch / data warehouse.
Cache invalidation.
Event-driven workflows.
Audit log.
Sync to other systems.

Node.js

const stream = collection.watch([
    { $match: { operationType: "insert" } }
]);

for await (const change of stream) {
    process(change);
}

Performance

Each change stream = one cursor on primary’s oplog.
Many parallel streams = oplog pressure.
Filter early via pipeline.

Resume token expiry

If gone too long → token might be unresumable. Reasons:

Oplog too small.
Token from another node post-failover.

Mitigations:

Big oplog.
Use start_after (more lenient) instead of resume_after.

Common mistakes

No resume token persistence → lose changes on restart.
Heavy work in handler → can’t keep up.
Single change stream as single point of failure.
Forgetting that delete event has only _id, not full doc.

Basic#

Operation types#

Filter#

Watch DB or cluster#

Resume#

Persist token#

Full document#

Pre/post images (newer Mongo)#

Use cases#

Node.js#

Performance#

Resume token expiry#

Common mistakes#

Read this next#