#Data Tasks API Reference

dataTasks uses one DataTaskJob configuration for release-scoped index checks, filtered data synchronization, local field edits, reviewed preview, affected-scope backup, and restore.

Start with Production Data Migration to choose the right tool. See Production Rollout for the complete release sequence.

#Entry Point and Four Methods

Use the named ESM export:

import MonSQLize, { dataTasks, type DataTaskJob } from 'monsqlize';

Read the same export from the CommonJS package object:

const { dataTasks } = require('monsqlize');

dataTasks is not an instance method. There is no task runner or second step-based configuration.

#Complete Configuration

const development = new MonSQLize({
  type: 'mongodb',
  databaseName: 'development',
  config: { uri: process.env.SOURCE_MONGODB_URI! }
});

const production = new MonSQLize({
  type: 'mongodb',
  databaseName: 'production',
  config: { uri: process.env.TARGET_MONGODB_URI! }
});

await Promise.all([development.connect(), production.connect()]);

const job = {
  name: 'release-2026-07-feature-modules',
  description: 'Publish the feature modules required by this release',
  source: development,
  target: production,
  targetEnvironment: 'production',
  collections: [{
    name: 'feature_modules',
    targetName: 'feature_modules',
    indexes: [
      { key: { code: 1 }, options: { unique: true } },
      { key: { release: 1, enabled: 1 } }
    ],
    data: {
      filter: { release: '2026-07' },
      projection: { _id: 1, code: 1, legacyName: 1, enabled: 1, developmentOnly: 1 },
      identity: { mode: 'fields', fields: ['code'] },
      strategy: 'upsert',
      rename: { legacyName: 'name' },
      set: { schemaVersion: 2 },
      unset: ['developmentOnly'],
      batchSize: 500,
      maxDocuments: 10_000
    },
    verify: {
      mode: 'full',
      fields: ['code', 'name', 'enabled', 'schemaVersion']
    }
  }],
  backup: {
    dir: './.monsqlize/data-tasks',
    format: 'extended-jsonl',
    compression: 'gzip',
    retentionDays: 7,
    maxBytes: 256 * 1024 * 1024
  },
  lock: { ttlMs: 120_000, waitTimeoutMs: 0 }
} satisfies DataTaskJob;

SDK jobs can use two connected monSQLize instances; the caller owns their lifecycle. CLI jobs must use two MonSQLizeOptions values. dataTasks closes connections that it creates.

Close both SDK instances in finally before the process exits. See examples/docs/data-tasks.ts for the complete runnable lifecycle.

#Job Parameters

Every collection needs non-empty indexes, data, or both.

#Collection and Index Parameters

Preview calls listIndexes() first for every target collection and classifies each declaration as existing, missing, or conflict. Apply creates only missing indexes. It never drops, renames, or rebuilds a conflicting index.

identity.mode: 'fields' requires an exact non-partial unique index in the target or in the same collection's declarations. Proving a missing unique index requires a final-target-image scan. The task blocks when that scan exceeds 10,000 documents or an index shape cannot be proved exactly.

#Data Parameters

Data task source reads use a bounded internal stream and do not inherit the ordinary findLimit default. Planning reads at most maxDocuments + 1 source documents so concurrent growth fails closed instead of silently truncating or loading an unbounded result.

Field edits run in this order: filter -> projection -> rename -> set -> unset -> diff. They are deliberately limited to deterministic edits of a few release fields:

• Paths cannot touch _id, identity fields, or overlap one another.
• __proto__, prototype, and constructor path segments are rejected.
• Rename blocks when the destination already contains a different value.
• Dotted paths block when an intermediate value is not an object.
• set accepts BSON-serializable literals, not functions, random values, or update expressions.

#Identity and _id

Business-key matching:

identity: { mode: 'fields', fields: ['tenantId', 'code'] }

Updates preserve the target _id; inserts let the target MongoDB generate _id. Use this when environments assign IDs independently.

Source-ID matching:

identity: { mode: 'source-id', conflictBy: ['tenantId', 'code'] }

Inserts preserve source _id, and existing documents update only when _id matches. conflictBy detects the same business key under another _id and blocks preview.

#Verification, Backup, and Lock

Index, identity, count, manifest, and actual-write checks are always full. verify.mode only changes user-selected field content verification.

#Preview and Apply

const preview = await dataTasks.preview(job, {
  sampleSize: 10,
  approvalTtlMs: 15 * 60 * 1000
});

if (!preview.passed || !preview.approval) throw new Error(preview.errors.join('; '));
const result = await dataTasks.apply(job, { approval: preview.approval });

Preview performs no writes. It returns index states, source/insert/update/unchanged/conflict, change samples, backup document count, and estimated bytes. Approval binds the normalized job, patched source image, target preimages, target indexes, and expiry.

Apply runs in this order:

1. Acquire the optional target lease.
2. Rebuild the full plan and reject approval on any hash drift.
3. Write and read back the affected-scope backup.
4. Read indexes again and create only reviewed missing indexes.
5. Record each batch as pending before ordered data writes.
6. Use target-preimage CAS filters to avoid overwriting concurrent changes.
7. Read back the exact expected after-image and verify fields, identity, indexes, and manifest.
8. Record final status and release the lease.

Indexes and cross-collection writes are not one MongoDB transaction. A mid-run error returns failed or partial, preserves the manifest, and never restores automatically.

#Restore

const restorePreview = await dataTasks.previewRestore(result.backup);
if (!restorePreview.passed || !restorePreview.approval) throw new Error(restorePreview.errors.join('; '));

const restored = await dataTasks.restore(result.backup, {
  approval: restorePreview.approval
});

Restore preview verifies that the target still matches the applied after-image and resolves pending operations when a process stopped between a database write and manifest checkpoint. Ambiguous or later target changes block with RESTORE_DRIFT.

Restore creates a safety backup before writing. Document operations use exact current-image filters, and index recovery only touches indexes actually created by the task whose definitions remain unchanged. The safety backup can itself be previewed and restored to undo a restore.

#CLI

The task file must directly export the DataTaskJob:

module.exports = {
  name: 'release-settings',
  source: { type: 'mongodb', databaseName: 'development', config: { uri: process.env.SOURCE_URI } },
  target: { type: 'mongodb', databaseName: 'production', config: { uri: process.env.TARGET_URI } },
  targetEnvironment: 'production',
  collections: [{
    name: 'settings',
    data: { all: true, identity: { mode: 'source-id', conflictBy: ['code'] } }
  }],
  backup: { dir: './.monsqlize/data-tasks' }
};

monsqlize data-task preview --task ./tasks/release.cjs --out preview.json --json
monsqlize data-task apply --task ./tasks/release.cjs --approval preview.json --out result.json --json
monsqlize data-task preview-restore --task ./tasks/release.cjs --backup ./path/manifest.json --out restore-preview.json --json
monsqlize data-task restore --task ./tasks/release.cjs --backup ./path/manifest.json --approval restore-preview.json --json

Invalid configuration, stale approval, lock conflict, partial/failed status, checksum errors, and restore drift exit with code 1.

#Errors and Boundaries

DataTaskJobError exposes code, phase, and optional collection. Stable codes are INVALID_JOB, IDENTITY_CONFLICT, INDEX_CONFLICT, APPROVAL_STALE, BACKUP_FAILED, LOCK_NOT_ACQUIRED, LOCK_LOST, APPLY_PARTIAL, RESTORE_DRIFT, and RESTORE_FAILED.

• A backup is an affected-scope rollback package, not a full backup or point-in-time restore.
• dataTasks does not synchronize Model schema, copy every source index, maintain migration directories, or run ongoing CDC.
• Use MongoDB native or managed backup tools for full moves and disaster recovery.
• Use Change Stream sync for ongoing changes after the initial backfill.

#Related Documents

• Production Data Migration
• Production Rollout
• Create indexes in bulk
• Change Stream Sync