Run tasks in workers

On this page

Use a worker pool to run exported functions in parallel, with work isolated from the main application. Declare the worker entry before building the application. Each worker keeps its own module state, VM, heap, and event loop.

Use a pool for request/result tasks. Use a long-lived Worker when you need a message protocol or an independent event loop. The public maligator:workers API does not require a surface flag; worker execution requires a threaded host.

Use an ES module project for the .ts files below.

Create a pool

Save both files in the same directory:

tasks.ts

import type { TaskContext } from "maligator:workers";

export function sum(context: TaskContext, values: Array<number>): number {
	let total = 0;
	for (const value of values) {
		context.throwIfCancelled();
		total += value;
	}
	return total;
}

export async function waitForCancellation(context: TaskContext): Promise<void> {
	context.throwIfCancelled();
	await new Promise<void>((resolve) => {
		context.signal.addEventListener("abort", () => resolve(), { once: true });
	});
	context.throwIfCancelled();
}

pool.ts

import { createPool, createWorkerUrl } from "maligator:workers";

const tasks = createWorkerUrl<typeof import("./tasks.ts")>(
	"./tasks.ts",
	import.meta.url,
);
const pool = createPool(tasks, { size: 2, maxQueuedTasks: 4 });

try {
	await pool.ready;
	console.log(await pool.run("sum", [[1, 2, 3]]));

	const inputs: Array<[Array<number>]> = [[[1, 2]], [[3, 4]]];
	for await (const total of pool.map("sum", inputs, { window: 2 })) {
		console.log(total);
	}

	const controller = new AbortController();
	const pending = pool.run("waitForCancellation", [], {
		signal: controller.signal,
	});
	const cancelled = pending.then(
		() => false,
		(reason: unknown) => reason === controller.signal.reason,
	);
	controller.abort();
	console.log(await cancelled);
} finally {
	await pool.close();
}

shell

maligator run pool.ts

The application prints 6, then 3 and 7, then true for the cancellation check. The finally block drains and joins the pool.

createWorkerUrl takes a static specifier and import.meta.url. The compiler bundles the worker graph into the application image. Workers do not load arbitrary source files at runtime. The erased generic describes the module you expect; it is not runtime type validation.

Submit arguments and receive results

A task is an exported function whose first argument is TaskContext. Pass only the remaining arguments to run. In the example, [[1, 2, 3]] is a one-element argument tuple containing an array.

Admission failures throw synchronously. An accepted task returns a promise that resolves with its result or rejects with its failure. Await pool.ready to catch entry startup failures before submitting work.

Each worker executes one task until the task's returned promise settles. Keep independent work in different tasks; an await inside one task does not free that worker to run another pool task. A worker failure does not replay accepted tasks.

Bound pending work

Set a pool size and queue bounds that fit your workload. Defaults and ranges are in PoolOptions. A full queue rejects admission with QueueFullError; increasing its size also increases retained inputs.

map reads an iterable with a bounded window and yields results in input order. Its window counts pulled inputs and buffered results together. A slow early input can delay later results even if those tasks have completed.

A size-one pool preserves serial dispatch, but its module state still lives in a separate isolate. Closing the pool stops new admission and drains accepted work. To cancel it, see Cancel work and shut down.

Continue with Messages and transfers to choose how values cross isolate boundaries.