Indexes & Collections

This guide covers how to define access patterns using indexes, how key generation works, and how to use collections for cross-entity queries.

Entity Primary Key

Entities define their primary key and GSI indexes together. Collections are auto-discovered from entities sharing the same collection name on a GSI.

const Tasks = Entity.make({
  model: Task,
  entityType: "Task",
  primaryKey: {
    pk: { field: "pk", composite: ["taskId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    byProject: {
      name: "gsi1",
      pk: { field: "gsi1pk", composite: ["projectId"] },
      sk: { field: "gsi1sk", composite: ["priority"] },
    },
    byAssignee: {
      name: "gsi2",
      pk: { field: "gsi2pk", composite: ["employeeId"] },
      sk: { field: "gsi2sk", composite: ["priority"] },
    },
  },
  timestamps: true,
})

Anatomy of a Primary Key

primaryKey: {
  pk: {
    field: "pk",                         // Physical DynamoDB attribute
    composite: ["taskId"],               // Model attributes -> partition key
  },
  sk: {
    field: "sk",
    composite: [],                       // Empty = one item per partition key
  },
}

Property	Type	Required	Description
pk.field	string	Yes	Physical DynamoDB attribute name
pk.composite	string[]	Yes	Model attributes composing the partition key
sk.field	string	Yes	Physical DynamoDB attribute name
sk.composite	string[]	Yes	Model attributes composing the sort key

Access Patterns via the Primary Index

Every entity exposes a .primary(...) query accessor on the bound client — the primary index is treated symmetrically with GSI indexes. Pass the required PK composites (and optionally one or more SK composites) to return a BoundQuery:

// `Memberships` primary key: pk = orgId, sk = userId
//
// List all items under a shared primary partition key (PK only)
const allMembers = yield* db.entities.Memberships.primary({
  orgId: "org-acme",
}).collect()

// Narrow by partial SK composite (begins_with prefix match)
const acmeAdmins = yield* db.entities.Memberships.primary({
  orgId: "org-acme",
})
  .filter({ role: "admin" })
  .collect()

This is the natural access pattern for shared-PK single-table designs — for example a join table that holds many channel grants under one accountId. Use .get(fullKey) when you want a strongly-consistent single-item fetch by the full composite key; use .primary(partialKey) when you want a Query that returns every item in the partition (or a prefix-matched subset).

GSI Access Patterns via Entity Indexes

GSI access patterns are defined directly on the entity via the indexes property. Each index specifies the physical GSI, PK composites, and SK composites — see the task-entity block above for the canonical shape.

Logical Names as API Surface

Collection names become query accessors on the typed client:

// Logical names become query accessors — no physical index names in code
const alphaTasks = yield* basic.entities.Tasks.byProject({ projectId: "proj-alpha" }).collect()

const aliceTasks = yield* basic.entities.Tasks.byAssignee({ employeeId: "emp-alice" }).collect()

You never reference physical index names (gsi1, gsi2) in application code.

Key Generation

effect-dynamodb generates DynamoDB key values automatically from your composite declarations. You declare which attributes compose each key — the system handles how.

Format

$<schema>#v<version>#<prefix>#<attrName>_<value>#<attrName>_<value>

Each composite value is prefixed with its attribute name (separated by _) and segments are joined with #. Given DynamoSchema({ name: "myapp", version: 1 }) and entityType: "Task":

Composite	Generated Key
["taskId"] with value "t-001"	$myapp#v1#task#taskid_t-001
["projectId", "status"] with values "proj-alpha", "active"	$myapp#v1#task#projectid_proj-alpha#status_active
[] (empty)	$myapp#v1#task

Empty Composites

Both pk.composite and sk.composite can be empty arrays. The entity type prefix guarantees no collisions between entity types.

pk	sk	Use Case
[]	[]	Singleton (config, counters)
["userId"]	[]	One item per user
[]	["userId"]	All users in one partition
["tenantId"]	["userId"]	Users partitioned by tenant

Attribute Serialization

Type	Key Value
string	As-is
number	Zero-padded
DateTime.Utc	ISO 8601
boolean	"true" / "false"
Branded string	Underlying value

Casing Resolution

Casing is applied uniformly across the entire generated key — schema name, version prefix, entity type, collection name, and composite attribute values. Resolution order:

1. Index-level casing (highest priority)
2. Schema-level casing (default: "lowercase")

This means "Emp-Alice" and "emp-alice" produce identical keys under the default "lowercase" setting. The original attribute value casing is preserved on the stored attribute itself — only the composed key string is cased.

// With casing: "lowercase", entityType: "Employee", employeeId: "Emp-Alice"
// Composed PK:      $myapp#v1#employee#emp-alice
// Stored attribute: employeeId = "Emp-Alice"  (original casing preserved)

Collections

Collections group multiple entity types that share a partition key, enabling cross-entity queries. Two modes are supported: isolated (default) and clustered. Isolated keeps each entity’s sort-key namespace separate so single-entity scans remain efficient; clustered places the collection name above each entity’s sort key so all members are physically interleaved (required for sub-collections).

Isolated Collections

Each entity owns its sort key prefix. Collection queries use only the partition key — no sort key condition.

const IsolatedEmployees = Entity.make({
  model: Employee,
  entityType: "Employee",
  primaryKey: {
    pk: { field: "pk", composite: ["employeeId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    departmentStaff: {
      collection: "departmentStaff",
      name: "gsi1",
      pk: { field: "gsi1pk", composite: ["department"] },
      sk: { field: "gsi1sk", composite: ["hireDate"] },
    },
  },
  timestamps: true,
})

class Equipment extends Schema.Class<Equipment>("Equipment")({
  equipmentId: Schema.String,
  department: Schema.String,
  name: Schema.String,
  purchaseDate: Schema.String,
}) {}

const Equipments = Entity.make({
  model: Equipment,
  entityType: "Equipment",
  primaryKey: {
    pk: { field: "pk", composite: ["equipmentId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    departmentStaff: {
      collection: "departmentStaff",
      name: "gsi1",
      pk: { field: "gsi1pk", composite: ["department"] },
      sk: { field: "gsi1sk", composite: ["purchaseDate"] },
    },
  },
  timestamps: true,
})

Sort key format (isolated):

$myapp#v1#employee_1#hiredate_2020-01-15        <- Employee entity owns its SK
$myapp#v1#equipment_1#purchasedate_2023-06-01   <- Equipment entity owns its SK

Query behavior:

Query	Mechanism	Returns
Everything	PK only (no SK condition)	Employee + Equipment
Employees only	PK + begins_with(SK, "$myapp#v1#employee_1")	Employee

Best for: High-volume single-entity queries where cross-entity queries are occasional.

Clustered Collections

The collection name sits at the top of every entity’s sort key. All entities share this prefix.

const ClusteredEmployees = Entity.make({
  model: Employee,
  entityType: "Employee",
  primaryKey: {
    pk: { field: "pk", composite: ["employeeId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    tenantMembers: {
      collection: "tenantMembers",
      name: "gsi1",
      pk: { field: "gsi1pk", composite: ["tenantId"] },
      sk: { field: "gsi1sk", composite: ["department", "hireDate"] },
      type: "clustered",
    },
  },
  timestamps: true,
})

const ClusteredTasks = Entity.make({
  model: Task,
  entityType: "Task",
  primaryKey: {
    pk: { field: "pk", composite: ["taskId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    tenantMembers: {
      collection: "tenantMembers",
      name: "gsi1",
      pk: { field: "gsi1pk", composite: ["tenantId"] },
      sk: { field: "gsi1sk", composite: ["projectId", "taskId"] },
      type: "clustered",
    },
  },
  timestamps: true,
})

Sort key format (clustered):

$myapp#v1#tenantmembers#employee_1#department_engineering#hiredate_2020-01-15
$myapp#v1#tenantmembers#task_1#projectid_proj-alpha#taskid_t-001
^^^^^^^^^^^^^^^^^^^^^^^^^^
collection prefix (shared)

Query behavior:

Query	Mechanism	Returns
Everything	PK + begins_with(SK, "$myapp#v1#tenantmembers")	Employee + Task
Employees only	PK + begins_with(SK, "$myapp#v1#tenantmembers#employee_1")	Employee
Tasks only	PK + begins_with(SK, "$myapp#v1#tenantmembers#task_1")	Task

Best for: Cross-entity queries, relationship-dense data.

Hierarchical Sub-Collections

True parent/child sub-collections nest the collection hierarchy in the sort key so a begins_with query at the parent level returns the parent’s items and every descendant. Two requirements:

1. Use the array form on the entity index: collection: ["parent", "child"] (deeper levels add more elements).
2. Use type: "clustered" so the SK puts the hierarchy above the entity-type prefix.

In the example below, Employee lives at the parent level (["contributions"]) while Task and ProjectMember live one level deeper (["contributions", "assignments"]). All three share gsi2 keyed on employeeId:

// True hierarchical sub-collections via the array form `collection: ["parent", "child"]`
// + `type: "clustered"`. The full hierarchy is written into the SK so a begins_with
// query at the parent level matches every descendant — querying "contributions"
// returns SubEmployee + SubTasks + SubProjectMembers, while "assignments" returns
// only SubTasks + SubProjectMembers.
const SubEmployee = Entity.make({
  model: Employee,
  entityType: "Employee",
  primaryKey: {
    pk: { field: "pk", composite: ["employeeId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    contributions: {
      collection: ["contributions"],
      type: "clustered",
      name: "gsi2",
      pk: { field: "gsi2pk", composite: ["employeeId"] },
      sk: { field: "gsi2sk", composite: ["department"] },
    },
  },
  timestamps: true,
})

const SubTasks = Entity.make({
  model: Task,
  entityType: "Task",
  primaryKey: {
    pk: { field: "pk", composite: ["taskId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    assignments: {
      collection: ["contributions", "assignments"],
      type: "clustered",
      name: "gsi2",
      pk: { field: "gsi2pk", composite: ["employeeId"] },
      sk: { field: "gsi2sk", composite: ["projectId", "taskId"] },
    },
  },
  timestamps: true,
})

const SubProjectMembers = Entity.make({
  model: ProjectMember,
  entityType: "ProjectMember",
  primaryKey: {
    pk: { field: "pk", composite: ["employeeId", "projectId"] },
    sk: { field: "sk", composite: [] },
  },
  indexes: {
    assignments: {
      collection: ["contributions", "assignments"],
      type: "clustered",
      name: "gsi2",
      pk: { field: "gsi2pk", composite: ["employeeId"] },
      sk: { field: "gsi2sk", composite: ["projectId"] },
    },
  },
  timestamps: true,
})

SK shape written by each entity:

Entity	SK
SubEmployee	$myapp#v1#contributions#employee_1#department_engineering
SubTasks	$myapp#v1#contributions#assignments#task_1#projectid_p-α#taskid_t-001
SubProjectMembers	$myapp#v1#contributions#assignments#projectmember_1#projectid_p-α

Both descendant SKs start with the parent prefix $myapp#v1#contributions, so a query at the parent level returns all three. The child query uses the longer prefix $myapp#v1#contributions#assignments and returns only the two descendant entities.

Query at any depth:

Collection	begins_with prefix	Returns
contributions	$myapp#v1#contributions	Employee + Task + ProjectMember
assignments	$myapp#v1#contributions#assignments	Task + ProjectMember

// Parent — includes everything
const contributions = yield* db.collections
  .contributions({ employeeId: "emp-alice" })
  .collect()
// { SubEmployee: Employee[], SubTasks: Task[], SubProjectMembers: ProjectMember[] }

// Child — only descendants
const assignments = yield* db.collections
  .assignments({ employeeId: "emp-alice" })
  .collect()
// { SubTasks: Task[], SubProjectMembers: ProjectMember[] }

When to use sub-collections

Sub-collections are an advanced pattern — they cost a clustered SK layout and require every entity in the hierarchy to share the same physical GSI. Reach for them when you need a single query to return multiple entity types in a strict parent/child relationship without doing N queries. For independent collections that just happen to share an index, use a single-element string (collection: "name") instead.

Defining Collections

Collections are defined by adding a collection property to entity indexes. Entities sharing the same collection name on the same physical GSI are automatically grouped into a collection by DynamoClient.make():

// On EmployeeEntity
indexes: {
  tenantMembers: {
    collection: "tenantMembers",
    name: "gsi1",
    pk: { field: "gsi1pk", composite: ["tenantId"] },
    sk: { field: "gsi1sk", composite: ["department", "hireDate"] },
    type: "clustered",
  },
}

// On TaskEntity — same collection name + same GSI
indexes: {
  tenantMembers: {
    collection: "tenantMembers",
    name: "gsi1",
    pk: { field: "gsi1pk", composite: ["tenantId"] },
    sk: { field: "gsi1sk", composite: ["projectId", "taskId"] },
    type: "clustered",
  },
}

When you call DynamoClient.make({ entities, tables }), the client auto-discovers that both entities share the tenantMembers collection and exposes db.collections.tenantMembers(...) as a cross-entity query accessor.

Validation Rules

• All entities in a collection must share the same PK composite on that index
• All entities sharing an index must agree on the type (no mixing isolated/clustered)
• Sub-collections sharing an index must use the same PK and SK fields
• No duplicate entity types within a single collection

Worked Example: Multi-Tenant Project Management

Access Patterns

Pattern	Collection	PK	SK
Get employee by ID	primary	employeeId	—
Get task by ID	primary	taskId	—
Employees in tenant	TenantMembers (gsi1)	tenantId	department, hireDate
Tasks in tenant	TenantMembers (gsi1)	tenantId	projectId, taskId
All tenant items	TenantMembers (gsi1)	tenantId	—
Employee by email	ByEmail (gsi2)	email	—
Tasks by assignee	ByAssignee (gsi2)	employeeId	priority

DynamoDB Items

Each composite value in a key is prefixed with its attribute name (e.g., employeeid_emp-alice). Structural parts (schema, entity type, collection) are cased according to the schema/index casing rules.

pk	sk	gsi1pk	gsi1sk	edd_e
$myapp#v1#employee#employeeid_emp-alice	$myapp#v1#employee	$myapp#v1#tenantmembers#tenantid_t-acme	$myapp#v1#tenantmembers#employee_1#department_engineering#hiredate_2024-01-15	Employee
$myapp#v1#task#taskid_t-001	$myapp#v1#task	$myapp#v1#tenantmembers#tenantid_t-acme	$myapp#v1#tenantmembers#task_1#projectid_proj-alpha#taskid_t-001	Task
$myapp#v1#employee#employeeid_emp-bob	$myapp#v1#employee	$myapp#v1#tenantmembers#tenantid_t-acme	$myapp#v1#tenantmembers#employee_1#department_sales#hiredate_2023-06-01	Employee

What’s Next?

• indexPolicy — Sparse & Preserved GSIs — Per-GSI sparse/preserve semantics for partial updates and hybrid writers
• Queries — Query by index, filter by sort key composites, paginate results
• Data Integrity — Unique constraints and optimistic concurrency
• Lifecycle — Soft delete and version retention