Data Integrity

This guide covers unique constraints, versioning with optimistic concurrency, and idempotency — mechanisms for ensuring data correctness in concurrent environments.

Unique Constraints

Unique constraints ensure that a field (or combination of fields) has a unique value across all items of an entity type. In DynamoDB, there are no built-in unique indexes — effect-dynamodb enforces uniqueness using sentinel items and transactional writes.

Declaration

const Users = Entity.make({
  model: User,
  entityType: "User",
  primaryKey: {
    pk: { field: "pk", composite: ["userId"] },
    sk: { field: "sk", composite: [] },
  },
  timestamps: true,
  unique: {
    email: ["email"], // single-field uniqueness
    tenantEmail: ["tenantId", "email"], // compound uniqueness
    username: ["username"], // another single-field
  },
})

How It Works

For each unique constraint, the system creates a sentinel item — a separate DynamoDB item whose existence proves that a value is taken.

Sentinel key format:

PK: $myapp#v1#user.email#alice@example.com
SK: $myapp#v1#user.email

The sentinel contains a back-reference to the entity’s primary key, enabling cleanup on delete.

Transactional Enforcement

All mutations involving unique constraints are automatically wrapped in DynamoDB transactions. Sentinel operations are sparse — they are emitted only for constraints whose composing fields are all present on the record (see Sparse Constraints below).

Operation	Transaction Items
Put	Entity item + sentinel per unique field whose composites are all set (condition: attribute_not_exists(pk))
Update (unique field unchanged)	Entity item only (no sentinel operations)
Update (undefined → defined)	Entity item + put new sentinel
Update (defined → undefined)	Entity item + delete old sentinel
Update (defined → defined, changed)	Entity item + delete old sentinel + put new sentinel
Delete	Entity item + delete sentinel per unique field whose composites were set

Error Handling

When a unique value is already taken, the write fails with UniqueConstraintViolation:

  // #region unique-error-handling
  yield* db.entities.Users.put({
    userId: "u-1",
    email: "alice@example.com",
    username: "alice",
    tenantId: "t-acme",
    displayName: "Alice",
  })

  // Same email → UniqueConstraintViolation
  const duplicateResult = yield* db.entities.Users.put({
    userId: "u-2",
    email: "alice@example.com",
    username: "bob",
    tenantId: "t-acme",
    displayName: "Bob",
  })
    .asEffect()
    .pipe(
      Effect.catchTag("UniqueConstraintViolation", (e) => {
        // e.entityType === "User"
        // e.constraint === "email" or "username"
        // e.fields === { email: "alice@example.com" }
        return Effect.succeed(
          `Blocked: constraint="${e.constraint}", fields=${JSON.stringify(e.fields)}`,
        )
      }),
    )
  // Blocked: constraint="email", fields={"email":"alice@example.com"}

Compound Constraints

Compound constraints combine multiple fields. The sentinel key includes all field values:

unique: {
  tenantEmail: ["tenantId", "email"],
}
// Sentinel PK: $myapp#v1#user.tenantemail#t-acme#alice@example.com

This allows alice@example.com to exist in different tenants but not twice within the same tenant.

Sparse Constraints

A unique constraint is sparse with respect to its composing fields: a sentinel is only written when every field is present on the record. If any composite is undefined or null, the constraint is silently skipped — the record is excluded from that constraint, just as a sparse GSI excludes records with missing key composites.

This matters for any unique constraint that references an optional model field:

class Vehicle extends Schema.Class<Vehicle>("Vehicle")({
  vehicleId: Schema.String,
  accountId: Schema.String,
  name: Schema.NonEmptyString,
  deviceBinding: Schema.optional(Schema.String),  // optional
  transponderId: Schema.optional(Schema.String),  // optional
}) {}

const Vehicles = Entity.make({
  model: Vehicle,
  entityType: "Vehicle",
  primaryKey: { pk: { field: "pk", composite: ["vehicleId"] }, sk: { field: "sk", composite: [] } },
  unique: {
    nameInAccount: ["accountId", "name"],   // required composites — always enforced
    deviceBinding: ["deviceBinding"],        // sparse — only enforced when set
    transponderId: ["transponderId"],        // sparse — only enforced when set
  },
})

Two records can coexist with the same constraint left unset:

// Both succeed — no sentinel is written for deviceBinding/transponderId on either record
yield* db.entities.Vehicles.create({ vehicleId: "v-1", accountId: "acct-1", name: "Truck A" })
yield* db.entities.Vehicles.create({ vehicleId: "v-2", accountId: "acct-1", name: "Truck B" })

Update transitions claim and release the sentinel as the field becomes set or unset:

// undefined → defined: the new value is now reserved
yield* db.entities.Vehicles.update({ vehicleId: "v-1" }).set({ deviceBinding: "device-xyz" })

// Another record trying to claim the same value is blocked
const conflict = yield* db.entities.Vehicles.update({ vehicleId: "v-2" })
  .set({ deviceBinding: "device-xyz" })
  .asEffect()
  .pipe(Effect.catchTag("UniqueConstraintViolation", (e) => Effect.succeed(e.constraint)))
// "deviceBinding"

// defined → undefined: the value is released and can be claimed elsewhere
yield* db.entities.Vehicles.update({ vehicleId: "v-1" }).pipe(Entity.remove(["deviceBinding"]))
yield* db.entities.Vehicles.update({ vehicleId: "v-2" }).set({ deviceBinding: "device-xyz" })
// Succeeds — v-1 released the value

Sparse semantics mean you can declare a unique constraint on any optional field without worrying that records leaving that field unset will collide on a synthesized “missing value” key.

Versioning

When versioned is enabled on an Entity, every item carries a version field that auto-increments on each write.

Declaration

The simplest form just adds a version field with no retention:

const SimpleEntity = Entity.make({
  // ...
  versioned: true,  // adds version field, no retention
})

With version retention and optional TTL:

const VersionedUsers = Entity.make({
  model: User,
  entityType: "VersionedUser",
  primaryKey: {
    pk: { field: "pk", composite: ["userId"] },
    sk: { field: "sk", composite: [] },
  },
  timestamps: true,
  versioned: {
    retain: true, // keep version history
    ttl: Duration.days(90), // auto-expire old versions
  },
  unique: {
    email: ["email"],
    username: ["username"],
  },
})

You can also customize the version field name:

const CustomEntity = Entity.make({
  // ...
  versioned: { field: "revision", retain: true },
})

How It Works

• Put sets version = 1
• Update uses SET version = version + 1 (DynamoDB atomic counter)
• The version field appears in Entity.Record<E> (readable) but not in Entity.Input<E> or Entity.Update<E> (system-managed)

Version Retention

With retain: true, every mutation stores a snapshot of the previous state as a separate DynamoDB item:

PK: $myapp#v1#user#u-1  SK: $myapp#v1#user           ← current (version 4)
PK: $myapp#v1#user#u-1  SK: $myapp#v1#user#v#0000001  ← snapshot v1
PK: $myapp#v1#user#u-1  SK: $myapp#v1#user#v#0000002  ← snapshot v2
PK: $myapp#v1#user#u-1  SK: $myapp#v1#user#v#0000003  ← snapshot v3

All versions are co-located (same partition key) for efficient access. Version numbers are zero-padded for correct lexicographic sort order.

Querying Versions

// Get a specific version snapshot
const v1 = yield* db.entities.VersionedUsers.getVersion({ userId: "v-1" }, 1)

// Query version history (most recent first)
const history = yield* db.entities.VersionedUsers.collect(
  VersionedUsers.versions({ userId: "v-1" }),
)

TTL on Snapshots

With ttl: Duration.days(90), version snapshots get a DynamoDB TTL attribute. DynamoDB automatically deletes expired items. The current item is never TTL’d — only historical snapshots.

Optimistic Concurrency

Versioning enables optimistic locking. Pass expectedVersion on update to ensure no one else has modified the item since you read it.

  // #region optimistic-concurrency
  // Create a versioned user
  const user = yield* db.entities.VersionedUsers.put({
    userId: "v-1",
    email: "versioned@example.com",
    username: "versioned-alice",
    tenantId: "t-acme",
    displayName: "Alice",
  })
  // user.version === 1

  // Update twice to build history
  yield* db.entities.VersionedUsers.update({ userId: "v-1" }).set({ displayName: "Alice V2" })
  // version is now 2

  const current = yield* db.entities.VersionedUsers.update({ userId: "v-1" }).set({
    displayName: "Alice V3",
  })
  // current.version === 3

  // Update with optimistic lock — must match current version
  yield* db.entities.VersionedUsers.update({ userId: "v-1" })
    .set({ displayName: "Alice V4" })
    .expectedVersion(3)
  // Succeeds: version was 3, now 4

If another process updated the item between your read and write, the version will have changed and the update fails:

  // #region optimistic-lock-error
  // Stale version → OptimisticLockError
  const lockResult = yield* db.entities.VersionedUsers.update({ userId: "v-1" })
    .set({ displayName: "Stale Update" })
    .expectedVersion(2)
    .asEffect()
    .pipe(
      Effect.catchTag("OptimisticLockError", (e) =>
        Effect.succeed(`Blocked: expected version ${e.expectedVersion}, actual ${e.actualVersion}`),
      ),
    )
  // Blocked: expected version 2, actual 4

Under the Hood

The entity adds a ConditionExpression to the DynamoDB update:

ConditionExpression: #version = :expected
ExpressionAttributeNames: { "#version": "version" }
ExpressionAttributeValues: { ":expected": { N: "5" } }

If the condition fails, DynamoDB returns ConditionalCheckFailedException, which the entity maps to OptimisticLockError.

Idempotency

Idempotency keys prevent duplicate processing of the same request. In effect-dynamodb, this is implemented as a unique constraint with TTL — no new concept needed.

Declaration

const Payments = Entity.make({
  model: Payment,
  entityType: "Payment",
  primaryKey: {
    pk: { field: "pk", composite: ["paymentId"] },
    sk: { field: "sk", composite: [] },
  },
  timestamps: true,
  unique: {
    idempotencyKey: { fields: ["idempotencyKey"], ttl: Duration.hours(1) },
  },
})

The Payment model is a standard Schema.Class:

class Payment extends Schema.Class<Payment>("Payment")({
  paymentId: Schema.String,
  amount: Schema.Number,
  currency: Schema.String,
  idempotencyKey: Schema.String,
}) {}

How It Works

1. Client sends a request with a unique idempotencyKey (e.g., a UUID).
2. The entity creates a sentinel item for the key.
3. If the same idempotencyKey is sent again within the TTL window, the sentinel already exists and the write fails with UniqueConstraintViolation.
4. After the TTL expires, DynamoDB deletes the sentinel, allowing the key to be reused.

  // #region idempotency-usage
  const requestId = "idem-abc-123"

  // First request — succeeds
  const payment = yield* db.entities.Payments.put({
    paymentId: "pay-001",
    amount: 99.99,
    currency: "USD",
    idempotencyKey: requestId,
  })

  // Retry with same idempotency key → blocked
  const retry = yield* db.entities.Payments.put({
    paymentId: "pay-002",
    amount: 99.99,
    currency: "USD",
    idempotencyKey: requestId,
  })
    .asEffect()
    .pipe(
      Effect.catchTag("UniqueConstraintViolation", (e) =>
        e.constraint === "idempotencyKey"
          ? Effect.succeed("Duplicate request prevented")
          : Effect.fail(e),
      ),
    )
  // The sentinel has a TTL of 1 hour
  // After expiry, the same key can be reused

Transaction Item Budget

DynamoDB limits transactions to 100 items. When multiple features are enabled, each mutation consumes more of this budget:

Mutation	Items Used
Simple put (no features)	1
Put + 1 unique constraint	2
Put + 2 unique constraints + version snapshot	4
Update (unique field changed) + version snapshot	4
Soft delete + 2 sentinels + version snapshot	5

Plan your entity design accordingly. For entities with many unique constraints, ensure the total transaction items stay well under 100.

What’s Next?

• Lifecycle — Soft delete, restore, purge, and version retention
• Advanced — Rich updates, batch operations, conditional writes