Compare commits

...
3 Commits
Author SHA1 Message Date
Vanilagy 634186fd5e Bump minor 2026-07-22 15:31:39 +02:00
Vanilagy ae5a5838d6 Add ability to pause conversions or step them deliberately 2026-07-22 15:30:37 +02:00
f41eef0937 Add composable conversions
* Add non-owning conversions via ConversionOptions.ownsOutput

A conversion with ownsOutput: false only adds tracks to the output and
drives their media data; starting, finalizing, and metadata tags remain
the caller's responsibility. This lets multiple conversions and
directly-added user tracks compose on a single Output (see upstream
issue #436).

- ownsOutput: false allows a pre-populated output (state must still be
  'pending') and seeds track-capacity accounting from existing tracks
- execute() requires the output to be started and never finalizes it
- cancel() closes only the conversion's own sources, releasing internal
  synchronizer waiters, and leaves the output usable
- tags cannot be combined with ownsOutput: false
- isValid requires at least one contributed track instead of the
  format's minimum track counts

Prototype for API discussion; default (owning) behavior is unchanged.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AJdfnbY9AFh9i9dKgtrj6E

* Add external-audio example using a non-owning conversion

Demonstrates composing a user-owned audio track (synthesized voiceover
via OfflineAudioContext + AudioBufferSource) onto a picked video with
Conversion.init({ ownsOutput: false }), including progress reporting
and playback/download of the result.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AJdfnbY9AFh9i9dKgtrj6E

* Release synchronizer waiters when canceling during output finalization

A non-owning conversion's cancel() previously no-oped entirely when the
output (owned by someone else) was already finalizing or finalized,
leaving pump loops parked in the track synchronizer and hanging
execute() forever. Now it still marks the conversion canceled and
releases parked waiters in that state, without force-closing sources
(finalization owns flushing them at that point).

Also adds coverage: non-owning onProgress monotonicity, canceling one
of two sibling conversions, capacity seeding across sequential inits,
exact metadata exclusivity, and cancel-before-execute.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01AJdfnbY9AFh9i9dKgtrj6E

* Document non-owning conversions on the converting-media-files guide page

Adds the doc section requested in #436: what ownsOutput: false does, the
required choreography (add tracks -> output.start() before execute() ->
run the conversion concurrently with your own sources -> finalize), the
cancellation split (conversion.cancel() leaves the output alive; cancel
both for a full abort and tear the output down on error paths), isValid
semantics in this mode, and the tags restriction with the
setMetadataTags() alternative. Also cross-links the fresh-output rule to
the new section. VitePress build passes with dead-link checking on.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* Clean up conversion logic, add Output.tracks and .hasEnoughTracks(), move new conversion tests around, remove external audio example

* non-owning -> composable, and update docs

* Update

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Vanilagy <[email protected]>
2026-07-22 08:42:19 +00:00
19 changed files with 930 additions and 175 deletions
+124 -2
View File
@@ -22,7 +22,7 @@ It has the following features:
- Audio up/downmixing
- User-defined video & audio processing
The conversion API was built to be simple, versatile and extremely performant.
The conversion API was built to be simple, versatile, composable and performant.
## Basic usage
@@ -63,7 +63,7 @@ await conversion.execute();
That's it! A `Conversion` simply takes an instance of `Input` and `Output`, then reads the data from the input and writes it to the output. If you're unfamiliar with [`Input`](./reading-media-files) and [`Output`](./writing-media-files), check out their respective guides.
::: info
The `Output` passed to the `Conversion` must be *fresh*; that is, it must have no added tracks or metadata tags and be in the `'pending'` state (not started yet).
The `Output` passed to the `Conversion` must be *fresh*; that is, it must have no added tracks or metadata tags and be in the `'pending'` state (not started yet). This requirement is relaxed for [composable conversions](#composable-conversions), which allows you to combine the conversion with other tracks.
:::
Unconfigured, the conversion process handles all the details automatically, such as:
@@ -114,6 +114,43 @@ await conversion.cancel(); // Resolves once the conversion is canceled
This automatically frees up all resources used by the conversion process and will cause any ongoing call to `execute` to throw a `ConversionCanceledError`.
If the conversion is [composable](#composable-conversions), the corresponding `Output` is not canceled and remains usable after conversion cancellation.
### Pausing a conversion
You can pause a conversion mid-execution and resume it later. For this, pass a pause signal to the `execute` method:
```ts
const controller = new AbortController();
button.onclick = () => controller.abort();
await conversion.execute({
pauseSignal: controller.signal,
});
if (conversion.state === 'idle') {
// Paused before completion
} else if (conversion.state === 'done') {
// Ran to completion
}
```
An unfinished conversion can simply be resumed with another call to `execute`:
```ts
await conversion.execute();
```
### Partial execution
Instead of running a conversion in full, you can execute it only until a certain timestamp is reached:
```ts
await conversion.execute({
// Pauses execution once an output timestamp of 10 seconds is reached
until: 10,
});
```
The conversion can then be resumed and continued by calling `execute` again. This feature is especially useful for [composable conversions](#composable-conversions).
## Video options
You can set the `video` property in the conversion options to configure the converter's behavior for video tracks. The options are:
@@ -506,6 +543,91 @@ conversion.utilizedTracks; // => InputTrack[]
```
A track may appear multiple times in this list when [fan-out](#track-fan-out) produces multiple output tracks from it.
## Composable conversions
By default, a `Conversion` takes full ownership of its `Output`: it requires a fresh output, then starts it, adds data, and finalizes it for you. Sometimes, however, you want a conversion to be just *one* of several contributors to a single output file - for example, to keep an input's video track while attaching your own, externally-produced audio track. For this, set `composable: true`.
A composable conversion only adds its own tracks to the output and pumps their media data while `execute()` runs. Everything else about the output's lifecycle is yours: you add any additional tracks, set any metadata tags, and call `start()` and `finalize()` yourself. This enables you to add additional tracks outside of the conversion, or even have multiple conversions target a single `Output`.
To use it, initialize everything, then start the `Output`, and then execute the conversion:
```ts
import {
Input,
Output,
Mp4OutputFormat,
BufferTarget,
Conversion,
AudioBufferSource,
} from 'mediabunny';
const input = new Input({ ... });
const output = new Output({
format: new Mp4OutputFormat(),
target: new BufferTarget(),
});
// Use the conversion only to copy over the video
const conversion = await Conversion.init({
input,
output,
audio: { discard: true },
composable: true,
});
// Add our own audio track directly
const audioSource = new AudioBufferSource({ codec: 'aac', bitrate: 128e3 });
output.addAudioTrack(audioSource);
// Start the output
await output.start();
// Run the conversion concurrently with feeding our own audio
await Promise.all([
conversion.execute(),
audioSource.add(myAudioBuffer).then(() => audioSource.close()),
]);
// Finalize the output
await output.finalize();
```
### Running in lockstep
To prevent high memory usage due to buffering needs, it's important to add media data at roughly the same speed across all tracks. To achieve this, you can step the conversion deliberately by calling `execute` multiple times:
```ts
await output.start();
for (using sample of generateAudioSamples()) {
await audioSource.add(sample);
await conversion.execute({ until: sample.timestamp });
}
// Convert whatever's left
await conversion.execute();
await output.finalize();
```
When running multiple composable conversions that target the same output, you can use a pattern like this:
```ts
await output.start();
for (let until = 1; true; until += 1) {
await Promise.all([
conversion1.execute({ until }),
conversion2.execute({ until }),
]);
if (conversion1.state === 'done' && conversion2.state === 'done') {
break;
}
}
await output.finalize();
```
## Converting live streams
Live inputs, like HLS live streams, can also be used with the Conversion API. In this case, by default, the conversion will run until the live stream has ended.
+42
View File
@@ -627,6 +627,48 @@ await conversion.execute();
// Conversion is complete
```
## Combine multiple files into one
```ts
import {
Input,
Output,
Conversion,
} from 'mediabunny';
// Let's take the video track from one file...
const videoInput = new Input(...);
// ...and the audio track from another
const audioInput = new Input(...);
const output = new Output(...);
const videoConversion = await Conversion.init({
input: videoInput,
output,
composable: true, // Ensure the conversion doesn't own the output
audio: { discard: true },
});
const audioConversion = await Conversion.init({
input: audioInput,
output,
composable: true,
video: { discard: true },
});
await output.start();
await Promise.all([
videoConversion.execute(),
audioConversion.execute(),
]);
await output.finalize();
// Conversion is complete
```
::: info
See [Composable conversions](./converting-media-files#composable-conversions) for the full documentation.
:::
## Reading HLS playlists
```ts
+9 -9
View File
@@ -1,12 +1,12 @@
{
"name": "mediabunny",
"version": "1.50.9",
"version": "1.51.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "mediabunny",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"workspaces": [
".",
@@ -12954,7 +12954,7 @@
},
"packages/aac-encoder": {
"name": "@mediabunny/aac-encoder",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"devDependencies": {
"@types/emscripten": "^1.40.1"
@@ -12969,7 +12969,7 @@
},
"packages/ac3": {
"name": "@mediabunny/ac3",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"devDependencies": {
"@types/emscripten": "^1.40.1"
@@ -12984,7 +12984,7 @@
},
"packages/flac-encoder": {
"name": "@mediabunny/flac-encoder",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"devDependencies": {
"@types/emscripten": "^1.40.1"
@@ -12999,7 +12999,7 @@
},
"packages/mp3-encoder": {
"name": "@mediabunny/mp3-encoder",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"devDependencies": {
"@types/emscripten": "^1.40.1"
@@ -13014,7 +13014,7 @@
},
"packages/prores": {
"name": "@mediabunny/prores",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"dependencies": {
"turbores": "^1.2.2"
@@ -13029,10 +13029,10 @@
},
"packages/server": {
"name": "@mediabunny/server",
"version": "1.50.9",
"version": "1.51.0",
"license": "MPL-2.0",
"dependencies": {
"@mediabunny/prores": "^1.50.9",
"@mediabunny/prores": "^1.51.0",
"node-av": "^6.0.0"
},
"funding": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "mediabunny",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "Pure TypeScript media toolkit for reading, writing, and converting media files, directly in the browser.",
"type": "module",
"workspaces": [
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/aac-encoder",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "AAC encoder extension for Mediabunny, based on FFmpeg.",
"main": "./dist/bundles/mediabunny-aac-encoder.mjs",
"module": "./dist/bundles/mediabunny-aac-encoder.mjs",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/ac3",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "AC-3 and E-AC-3 (Dolby Digital) decoder and encoder extension for Mediabunny, based on FFmpeg.",
"main": "./dist/bundles/mediabunny-ac3.mjs",
"module": "./dist/bundles/mediabunny-ac3.mjs",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/flac-encoder",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "FLAC encoder extension for Mediabunny, based on libFLAC.",
"main": "./dist/bundles/mediabunny-flac-encoder.mjs",
"module": "./dist/bundles/mediabunny-flac-encoder.mjs",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/mp3-encoder",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "MP3 encoder extension for Mediabunny, based on LAME.",
"main": "./dist/bundles/mediabunny-mp3-encoder.mjs",
"module": "./dist/bundles/mediabunny-mp3-encoder.mjs",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/prores",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "Apple ProRes decoder extension for Mediabunny, based on TurboRes.",
"main": "./dist/bundles/mediabunny-prores.mjs",
"module": "./dist/bundles/mediabunny-prores.mjs",
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/server",
"author": "Vanilagy",
"version": "1.50.9",
"version": "1.51.0",
"description": "Adds full video and audio decoder and encoder support to Mediabunny for use in server-side environments (Node, Bun, Deno). Based on NodeAV.",
"main": "./dist/bundles/mediabunny-server.cjs",
"module": "./dist/bundles/mediabunny-server.mjs",
@@ -35,7 +35,7 @@
},
"dependencies": {
"node-av": "^6.0.0",
"@mediabunny/prores": "^1.50.9"
"@mediabunny/prores": "^1.51.0"
},
"peerDependencies": {
"mediabunny": "^1.45.0"
+336 -131
View File
@@ -48,7 +48,7 @@ import {
promiseWithResolvers,
Rotation,
} from './misc';
import { Output, OutputTrackGroup, TrackType } from './output';
import { Output, OutputTrackGroup } from './output';
import { Mp4OutputFormat } from './output-format';
import {
AudioSample,
@@ -146,6 +146,18 @@ export type ConversionOptions = {
* want to keep the console output clean.
*/
showWarnings?: boolean;
/**
* Whether this conversion is composable, defaults to `false`. A non-composable conversion takes full ownership of
* the output: it requires a fresh output and controls its entire lifecycle, meaning it starts it, writes its
* metadata tags, and finalizes it.
*
* A composable conversion only adds tracks to the output and drives their media data; starting and finalizing
* the output is an outside responsibility. This is useful when only some output tracks should be driven by a
* conversion, and other are to be driven manually. Additionally, it can be used to have multiple conversions target
* the same output.
*/
composable?: boolean;
};
/**
@@ -527,6 +539,33 @@ export type DiscardedTrack = {
trackOptions: ConversionVideoOptions | ConversionAudioOptions;
};
/**
* Options for controlling a single call to {@link Conversion.execute}.
* @group Conversion
* @public
*/
export type ConversionExecuteOptions = {
/**
* The timestamp in seconds, in the output's timescale, until which the conversion should advance. Defaults to
* `Infinity`, meaning the conversion runs until the end.
*
* This field is especially useful for composable conversions, as it allows you to advance the conversion in
* lockstep with other media data sources.
*/
until?: number;
/**
* A signal that, when triggered, pauses the conversion as soon as possible.
*/
pauseSignal?: AbortSignal;
};
type TrackPump = {
done: boolean;
resolvers: ReturnType<typeof promiseWithResolvers<void>>;
wake: (() => void) | null;
start: () => void;
};
/**
* Represents a media file conversion process, used to convert one media file into another. In addition to conversion,
* this class can be used to resize and rotate video, resample audio, drop tracks, or trim to a specific time range.
@@ -539,6 +578,16 @@ export class Conversion {
/** The output file. */
readonly output: Output;
/**
* The current state of the conversion.
*
* - `'idle'`: The conversion is not currently executing and isn't done; `execute` can be called.
* - `'executing'`: A call to `execute` is currently running.
* - `'canceled'`: The conversion has been canceled and can no longer be executed.
* - `'done'`: The conversion has run to completion. Subsequent calls to `execute` do nothing.
*/
state: 'idle' | 'executing' | 'canceled' | 'done' = 'idle';
/** @internal */
_options: ConversionOptions;
/** @internal */
@@ -546,15 +595,6 @@ export class Conversion {
/** @internal */
_endTimestamp!: number;
/** @internal */
_addedCounts: Record<TrackType, number> = {
video: 0,
audio: 0,
subtitle: 0,
};
/** @internal */
_totalTrackCount = 0;
/** @internal */
_nextOutputTrackId = 0;
/** @internal */
@@ -563,26 +603,25 @@ export class Conversion {
_outputOwnTrackGroups: (OutputTrackGroup | null)[] = [];
/** @internal */
_trackPromises: Promise<void>[] = [];
_trackPumps: TrackPump[] = [];
/** @internal */
_composable = false;
/** @internal */
_started: Promise<void>;
/** @internal */
_start: () => void;
/** @internal */
_executed = false;
/** @internal */
_executionUntil = Infinity;
/** @internal */
_pauseRequested = false;
/** @internal */
_synchronizer = new TrackSynchronizer();
_synchronizer = new TrackSynchronizer(this);
/** @internal */
_totalDuration: number | null = null;
/** @internal */
_maxTimestamps = new Map<number, number>(); // Track ID -> timestamp
/** @internal */
_canceled = false;
/**
* A callback that is fired whenever the conversion progresses. Gets passed as first argument a number between
* 0 and 1, indicating the completion of the conversion. Note that a progress of 1 doesn't necessarily mean the
@@ -600,7 +639,8 @@ export class Conversion {
/**
* Whether this conversion, as it has been configured, is valid and can be executed. If this field is `false`, check
* the `discardedTracks` field for reasons.
* the `discardedTracks` field for reasons. Composable conversions are always valid, even if they utilize
* zero tracks.
*
* Note: a conversion having discarded tracks does not automatically mean it is invalid; if the remaining, utilized
* tracks make for a valid output file, the conversion is still allowed.
@@ -642,12 +682,30 @@ export class Conversion {
'options.tracks, when provided, must be either \'all\' or \'primary\'.',
);
}
if (
options.output._tracks.length > 0
|| Object.keys(options.output._metadataTags).length > 0
|| options.output.state !== 'pending'
) {
throw new TypeError('options.output must be fresh: no tracks or metadata tags added and not started.');
if (options.composable !== undefined && typeof options.composable !== 'boolean') {
throw new TypeError('options.composable, when provided, must be a boolean.');
}
const composable = options.composable ?? false;
if (!composable) {
if (
options.output.tracks.length > 0
|| Object.keys(options.output._metadataTags).length > 0
|| options.output.state !== 'pending'
) {
throw new TypeError('options.output must be fresh: no tracks or metadata tags added and not started.');
}
} else {
if (options.tags !== undefined) {
throw new TypeError(
'options.tags cannot be set by a composable conversion; set metadata directly on the output'
+ ' instead.',
);
}
if (options.output.state !== 'pending') {
throw new TypeError('options.output must not have been started yet.');
}
}
if (options.video !== undefined && typeof options.video !== 'function') {
@@ -704,12 +762,9 @@ export class Conversion {
}
this._options = options;
this._composable = composable;
this.input = options.input;
this.output = options.output;
const { promise: started, resolve: start } = promiseWithResolvers();
this._started = started;
this._start = start;
}
/** @internal */
@@ -857,7 +912,7 @@ export class Conversion {
const options = filteredTrackOptions[i]!;
for (const option of options) {
if (this._totalTrackCount === outputTrackCounts.total.max) {
if (this.output.tracks.length === outputTrackCounts.total.max) {
this.discardedTracks.push({
track,
reason: 'max_track_count_reached',
@@ -866,7 +921,12 @@ export class Conversion {
continue;
}
if (this._addedCounts[track.type] === outputTrackCounts[track.type].max) {
const addedCountOfType = this.output.tracks.reduce(
(count, t) => count + (t.type === track.type ? 1 : 0),
0,
);
if (addedCountOfType === outputTrackCounts[track.type].max) {
this.discardedTracks.push({
track,
reason: 'max_track_count_of_type_reached',
@@ -906,39 +966,44 @@ export class Conversion {
}
}
// Now, let's deal with metadata tags
// Now, let's deal with metadata tags. A composable conversion does not touch the output's metadata tags; that
// remains the responsibility of whoever owns the output.
const inputTags = await this.input.getMetadataTags();
let outputTags: MetadataTags;
if (!this._composable) {
const inputTags = await this.input.getMetadataTags();
let outputTags: MetadataTags;
if (this._options.tags) {
const result = typeof this._options.tags === 'function'
? await this._options.tags(inputTags)
: this._options.tags;
validateMetadataTags(result);
if (this._options.tags) {
const result = typeof this._options.tags === 'function'
? await this._options.tags(inputTags)
: this._options.tags;
validateMetadataTags(result);
outputTags = result;
} else {
outputTags = inputTags;
outputTags = result;
} else {
outputTags = inputTags;
}
// Somewhat dirty but pragmatic
const inputAndOutputFormatMatch = inputFormat.mimeType === this.output.format.mimeType;
const rawTagsAreUnchanged = inputTags.raw === outputTags.raw;
if (inputTags.raw && rawTagsAreUnchanged && !inputAndOutputFormatMatch) {
// If the input and output formats aren't the same, copying over raw metadata tags makes no sense and
// only results in junk tags, so let's cut them out.
delete outputTags.raw;
}
this.output.setMetadataTags(outputTags);
}
// Somewhat dirty but pragmatic
const inputAndOutputFormatMatch = inputFormat.mimeType === this.output.format.mimeType;
const rawTagsAreUnchanged = inputTags.raw === outputTags.raw;
if (inputTags.raw && rawTagsAreUnchanged && !inputAndOutputFormatMatch) {
// If the input and output formats aren't the same, copying over raw metadata tags makes no sense and only
// results in junk tags, so let's cut them out.
delete outputTags.raw;
}
this.output.setMetadataTags(outputTags);
// Let's check if the conversion can actually be executed
this.isValid = this._totalTrackCount >= outputTrackCounts.total.min
&& this._addedCounts.video >= outputTrackCounts.video.min
&& this._addedCounts.audio >= outputTrackCounts.audio.min
&& this._addedCounts.subtitle >= outputTrackCounts.subtitle.min;
if (!this._composable) {
this.isValid = this.output.hasEnoughTracks();
} else {
// Checking Output start validity is not up to us. We consider even zero-track conversions to be valid
this.isValid = true;
}
if (this._options.showWarnings ?? true) {
const warnElements: unknown[] = [];
@@ -1041,11 +1106,24 @@ export class Conversion {
}
/**
* Executes the conversion process. Resolves once conversion is complete.
* Executes the conversion process and resolves when the conversion is complete. When
* {@link ConversionExecuteOptions.until} is provided, the conversion will be suspended once that output timestamp
* is reached and can be resumed with another call to `execute`. An ongoing execution may also be suspended via
* {@link ConversionExecuteOptions.pauseSignal}.
*
* Will throw if `isValid` is `false`.
* Execution will throw if `isValid` is `false`.
*/
async execute() {
async execute(options: ConversionExecuteOptions = {}) {
if (!options || typeof options !== 'object') {
throw new TypeError('options must be an object.');
}
if (options.until !== undefined && (typeof options.until !== 'number' || Number.isNaN(options.until))) {
throw new TypeError('options.until, when provided, must be a number.');
}
if (options.pauseSignal !== undefined && !(options.pauseSignal instanceof AbortSignal)) {
throw new TypeError('options.pauseSignal, when provided, must be an AbortSignal.');
}
if (!this.isValid) {
throw new Error(
'Cannot execute this conversion because its output configuration is invalid. Make sure to always check'
@@ -1054,63 +1132,123 @@ export class Conversion {
);
}
if (this._executed) {
throw new Error('Conversion cannot be executed twice.');
}
this._executed = true;
for (const id of this._outputTrackIds) {
this._synchronizer.declareTrack(id);
if (this.state === 'executing') {
throw new Error('Cannot call execute() while a previous call to execute() is still running.');
}
if (this.onProgress) {
// Compute duration using only the utilized tracks
const uniqueUtilizedTracks = new Set(this.utilizedTracks);
const durationPromises = [...uniqueUtilizedTracks].map(async (track) => {
if (await track.isLive()) {
return Infinity; // Upper bound (assuming no universe heat death)
}
if (this.state === 'canceled') {
throw new ConversionCanceledError();
}
return (await track.getDurationFromMetadata()) ?? (await track.computeDuration());
});
const duration = Math.max(0, ...await Promise.all(durationPromises));
if (this.state === 'done') {
// The conversion already ran to completion, nothing left to do
return;
}
this._computeProgress = true;
this._totalDuration = Math.min(
duration - this._startTimestamp,
this._endTimestamp - this._startTimestamp,
if (this._composable && this.output.state === 'pending') {
throw new Error(
'A composable conversion requires the output to be started. Call start() on the output before executing'
+ ' the conversion.',
);
}
for (const id of this._outputTrackIds) {
this._maxTimestamps.set(id, 0);
this.state = 'executing';
this._executionUntil = options.until ?? Infinity;
this._pauseRequested = options.pauseSignal?.aborted ?? false;
const onPause = () => {
if (this.state !== 'executing') {
return;
}
this.onProgress?.(0, 0);
this._pauseRequested = true;
// Release any pumps stuck in the synchronizer so they can reach their next checkpoint and suspend
this._synchronizer.resolveAll();
};
options.pauseSignal?.addEventListener('abort', onPause);
for (const pump of this._trackPumps) {
if (!pump.done) {
pump.resolvers = promiseWithResolvers();
}
}
await this.output.start();
this._start();
if (!this._executed) {
this._executed = true;
for (const id of this._outputTrackIds) {
this._synchronizer.declareTrack(id);
}
if (this.onProgress) {
// Compute duration using only the utilized tracks
const uniqueUtilizedTracks = new Set(this.utilizedTracks);
const durationPromises = [...uniqueUtilizedTracks].map(async (track) => {
if (await track.isLive()) {
return Infinity; // Upper bound (assuming no universe heat death)
}
return (await track.getDurationFromMetadata()) ?? (await track.computeDuration());
});
const duration = Math.max(0, ...await Promise.all(durationPromises));
this._computeProgress = true;
this._totalDuration = Math.min(
duration - this._startTimestamp,
this._endTimestamp - this._startTimestamp,
);
for (const id of this._outputTrackIds) {
this._maxTimestamps.set(id, 0);
}
this.onProgress?.(0, 0);
}
if (!this._composable) {
await this.output.start();
}
for (const pump of this._trackPumps) {
pump.start();
}
} else {
// Wake all suspended track pumps
for (const pump of this._trackPumps) {
pump.wake?.();
}
}
try {
await Promise.all(this._trackPromises);
await Promise.all(this._trackPumps.map(x => x.resolvers.promise));
} catch (error) {
if (!this._canceled) {
if ((this.state as Conversion['state']) !== 'canceled') {
// Make sure to cancel to stop other encoding processes and clean up resources
void this.cancel();
}
throw error;
} finally {
options.pauseSignal?.removeEventListener('abort', onPause);
}
if (this._canceled) {
if ((this.state as Conversion['state']) === 'canceled') {
throw new ConversionCanceledError();
}
await this.output.finalize();
const isDone = this._trackPumps.every(x => x.done);
this.state = isDone ? 'done' : 'idle';
if (this._computeProgress) {
const minTimestamp = Math.min(...this._maxTimestamps.values());
this.onProgress?.(1, minTimestamp);
if (isDone) {
if (!this._composable) {
await this.output.finalize();
}
if (this._computeProgress) {
const minTimestamp = Math.min(...this._maxTimestamps.values());
this.onProgress?.(1, minTimestamp);
}
}
}
@@ -1119,17 +1257,27 @@ export class Conversion {
* Does nothing if the conversion is already complete.
*/
async cancel() {
if (this.output.state === 'finalizing' || this.output.state === 'finalized') {
if (this.state === 'done') {
return;
}
if (this._canceled) {
if (this.state === 'canceled') {
Logging._warn('Conversion already canceled.');
return;
}
this._canceled = true;
await this.output.cancel();
this.state = 'canceled';
// Wake all suspended track pumps so they can wind down
for (const pump of this._trackPumps) {
pump.wake?.();
}
this._synchronizer.resolveAll();
if (!this._composable) {
await this.output.cancel();
}
}
/** @internal */
@@ -1210,16 +1358,14 @@ export class Conversion {
const source = new EncodedVideoPacketSource(sourceCodec);
videoSource = source;
this._trackPromises.push((async () => {
await this._started;
this._registerTrackPump(async (pump) => {
const sink = new EncodedPacketSink(track);
const decoderConfig = await track.getDecoderConfig();
const meta: EncodedVideoChunkMetadata = { decoderConfig: decoderConfig ?? undefined };
for await (const packet of sink.packets(undefined, undefined, { verifyKeyPackets: true })) {
if (this._canceled) {
return;
if (this.state === 'canceled') {
break;
}
if (packet.timestamp >= this._endTimestamp) {
@@ -1240,11 +1386,13 @@ export class Conversion {
if (this._synchronizer.shouldWait(outputTrackId, modifiedPacket.timestamp)) {
await this._synchronizer.wait(modifiedPacket.timestamp);
}
await this._checkpoint(pump, modifiedPacket.timestamp);
}
source.close();
this._synchronizer.closeTrack(outputTrackId);
})());
});
} else {
// We need to decode & reencode the video
@@ -1370,14 +1518,12 @@ export class Conversion {
const source = new VideoSampleSource(encodingConfig);
videoSource = source;
this._trackPromises.push((async () => {
await this._started;
this._registerTrackPump(async (pump) => {
const sink = new VideoSampleSink(track);
for await (using sample of sink.samples(this._startTimestamp, this._endTimestamp)) {
if (this._canceled) {
return;
if (this.state === 'canceled') {
break;
}
const adjustedSampleTimestamp = Math.max(sample.timestamp - this._startTimestamp, 0);
@@ -1391,16 +1537,20 @@ export class Conversion {
if (this._synchronizer.shouldWait(outputTrackId, lastSampleTimestamp)) {
await this._synchronizer.wait(lastSampleTimestamp);
}
await this._checkpoint(pump, lastSampleTimestamp);
}
}
source.close();
this._synchronizer.closeTrack(outputTrackId);
})());
});
}
let ownGroup: OutputTrackGroup | null = null;
if (!trackOptions.group) {
if (!trackOptions.group && !this._composable) {
// Create per-track groups to replicate the input's pairability graph. Don't do this for composable
// conversions.
ownGroup = new OutputTrackGroup();
}
@@ -1414,8 +1564,6 @@ export class Conversion {
rotation: outputTrackRotation,
group: ownGroup ?? trackOptions.group,
});
this._addedCounts.video++;
this._totalTrackCount++;
this.utilizedTracks.push(track);
this._outputTrackIds.push(outputTrackId);
@@ -1465,16 +1613,14 @@ export class Conversion {
const source = new EncodedAudioPacketSource(sourceCodec);
audioSource = source;
this._trackPromises.push((async () => {
await this._started;
this._registerTrackPump(async (pump) => {
const sink = new EncodedPacketSink(track);
const decoderConfig = await track.getDecoderConfig();
const meta: EncodedAudioChunkMetadata = { decoderConfig: decoderConfig ?? undefined };
for await (const packet of sink.packets()) {
if (this._canceled) {
return;
if (this.state === 'canceled') {
break;
}
if (packet.timestamp >= this._endTimestamp) {
@@ -1492,11 +1638,13 @@ export class Conversion {
if (this._synchronizer.shouldWait(outputTrackId, modifiedPacket.timestamp)) {
await this._synchronizer.wait(modifiedPacket.timestamp);
}
await this._checkpoint(pump, modifiedPacket.timestamp);
}
source.close();
this._synchronizer.closeTrack(outputTrackId);
})());
});
} else {
// We need to decode & reencode the audio
@@ -1589,13 +1737,11 @@ export class Conversion {
const source = new AudioSampleSource(encodingConfig);
audioSource = source;
this._trackPromises.push((async () => {
await this._started;
this._registerTrackPump(async (pump) => {
const sink = new AudioSampleSink(track);
for await (using sample of sink.samples(this._startTimestamp, this._endTimestamp)) {
if (this._canceled) {
return;
if (this.state === 'canceled') {
break;
}
if (needsPadding) {
@@ -1618,7 +1764,9 @@ export class Conversion {
sampleRate: originalSampleRate,
timestamp: 0,
});
await this._registerAudioSample(silentSample, source, outputTrackId, () => lastSampleTimestamp);
await this._registerAudioSample(
pump, silentSample, source, outputTrackId, () => lastSampleTimestamp,
);
needsPadding = false;
}
@@ -1654,16 +1802,20 @@ export class Conversion {
// Offset the timestamp as needed
finalSample.setTimestamp(finalSample.timestamp - this._startTimestamp);
await this._registerAudioSample(finalSample, source, outputTrackId, () => lastSampleTimestamp);
await this._registerAudioSample(
pump, finalSample, source, outputTrackId, () => lastSampleTimestamp,
);
}
source.close();
this._synchronizer.closeTrack(outputTrackId);
})());
});
}
let ownGroup: OutputTrackGroup | null = null;
if (!trackOptions.group) {
if (!trackOptions.group && !this._composable) {
// Create per-track groups to replicate the input's pairability graph. Don't do this for composable
// conversions.
ownGroup = new OutputTrackGroup();
}
@@ -1675,8 +1827,6 @@ export class Conversion {
disposition: await track.getDisposition(),
group: ownGroup ?? trackOptions.group,
});
this._addedCounts.audio++;
this._totalTrackCount++;
this.utilizedTracks.push(track);
this._outputTrackIds.push(outputTrackId);
@@ -1685,6 +1835,7 @@ export class Conversion {
/** @internal */
async _registerAudioSample(
pump: TrackPump,
sample: AudioSample,
source: AudioSampleSource,
outputTrackId: number,
@@ -1700,6 +1851,39 @@ export class Conversion {
if (this._synchronizer.shouldWait(outputTrackId, lastSampleTimestamp)) {
await this._synchronizer.wait(lastSampleTimestamp);
}
await this._checkpoint(pump, lastSampleTimestamp);
}
}
/** @internal */
_registerTrackPump(fn: (pump: TrackPump) => Promise<void>) {
const pump: TrackPump = {
done: false,
resolvers: promiseWithResolvers(),
wake: null,
start: () => {
void fn(pump).then(() => {
pump.done = true;
pump.resolvers.resolve();
}, (error) => {
pump.resolvers.reject(error);
});
},
};
this._trackPumps.push(pump);
}
/** @internal */
async _checkpoint(pump: TrackPump, timestamp: number) {
while (this.state !== 'canceled' && (timestamp >= this._executionUntil || this._pauseRequested)) {
// We've reached the target; signal it and suspend until the next execution wakes us up
pump.resolvers.resolve();
const { promise, resolve } = promiseWithResolvers();
pump.wake = resolve;
await promise;
}
}
@@ -1747,12 +1931,17 @@ const MAX_TIMESTAMP_GAP = 1; // in seconds
* slowest consumer.
*/
class TrackSynchronizer {
conversion: Conversion;
maxTimestamps = new Map<number, number>(); // Track ID -> timestamp
resolvers: {
timestamp: number;
resolve: () => void;
}[] = [];
constructor(conversion: Conversion) {
this.conversion = conversion;
}
declareTrack(trackId: number) {
this.maxTimestamps.set(trackId, 0);
}
@@ -1764,6 +1953,15 @@ class TrackSynchronizer {
this.maxTimestamps.set(trackId, Math.max(timestamp, currentValue));
const newMin = this.computeMinAndMaybeResolve();
if (
this.conversion.state === 'canceled'
|| this.conversion._pauseRequested
|| timestamp >= this.conversion._executionUntil
) {
// No point in throttling consumers that are about to suspend or wind down anyway
return false;
}
return timestamp - newMin > MAX_TIMESTAMP_GAP; // Should wait if it is too far ahead of the slowest consumer
}
@@ -1783,6 +1981,13 @@ class TrackSynchronizer {
this.computeMinAndMaybeResolve();
}
resolveAll() {
for (const entry of this.resolvers) {
entry.resolve();
}
this.resolvers.length = 0;
}
computeMinAndMaybeResolve() {
let newMin = Infinity;
for (const [, timestamp] of this.maxTimestamps) {
+5 -5
View File
@@ -148,8 +148,8 @@ export class HlsMuxer extends Muxer {
async start(): Promise<void> {
const release = await this.mutex.acquire();
const someRelative = this.output._tracks.some(t => t.metadata.isRelativeToUnixEpoch);
const someNotRelative = this.output._tracks.some(t => !t.metadata.isRelativeToUnixEpoch);
const someRelative = this.output.tracks.some(t => t.metadata.isRelativeToUnixEpoch);
const someNotRelative = this.output.tracks.some(t => !t.metadata.isRelativeToUnixEpoch);
if (someRelative && someNotRelative) {
throw new Error(
'All tracks must agree on `relativeToUnixEpoch`: some tracks are relative to the Unix epoch and some'
@@ -180,14 +180,14 @@ export class HlsMuxer extends Muxer {
let keyPacketsOnlyPairingWarned = false;
// First, let's build the "sibling" groups induced by track pairability
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
if (track.type === 'video') {
hasVideo = true;
}
const pairableGroups = new Map<MediaCodec, OutputTrack[]>();
for (const otherTrack of this.output._tracks) {
for (const otherTrack of this.output.tracks) {
if (track === otherTrack) {
continue;
}
@@ -264,7 +264,7 @@ export class HlsMuxer extends Muxer {
const unpairedAudioTracks: OutputTrack[] = [];
// Now, create the top-level variant streams
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
const assignedGroupKeys = groupAssignment.get(track);
if (assignedGroupKeys) {
assert(assignedGroupKeys.length > 0);
+1
View File
@@ -287,6 +287,7 @@ export {
type ConversionOptions,
type ConversionVideoOptions,
type ConversionAudioOptions,
type ConversionExecuteOptions,
ConversionCanceledError,
type DiscardedTrack,
} from './conversion';
+3 -3
View File
@@ -248,7 +248,7 @@ export class IsobmffMuxer extends Muxer {
this.initBoxWriter = new IsobmffBoxWriter(initWriter);
}
const holdsAvc = this.output._tracks.some(x => x.isVideoTrack() && x.source._codec === 'avc');
const holdsAvc = this.output.tracks.some(x => x.isVideoTrack() && x.source._codec === 'avc');
// Write the header
{
@@ -282,7 +282,7 @@ export class IsobmffMuxer extends Muxer {
// We're write at finalization
} else if (this.fastStart === 'reserve') {
// Validate that all tracks have set maximumPacketCount
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
if (track.metadata.maximumPacketCount === undefined) {
throw new Error(
'All tracks must specify maximumPacketCount in their metadata when using'
@@ -312,7 +312,7 @@ export class IsobmffMuxer extends Muxer {
}
private allTracksAreKnown() {
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
if (!track.source._closed && !this.trackDatas.some(x => x.track === track)) {
return false; // We haven't seen a sample from this open track yet
}
+1 -1
View File
@@ -691,7 +691,7 @@ export class MatroskaMuxer extends Muxer {
}
private allTracksAreKnown() {
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
if (!track.source._closed && !this.trackDatas.some(x => x.track === track)) {
return false; // We haven't seen a sample from this open track yet
}
+1 -1
View File
@@ -451,7 +451,7 @@ export class MpegTsMuxer extends Muxer {
}
private allTracksAreKnown() {
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
if (!track.source._closed && !this.trackDatas.some(x => x.track === track)) {
return false;
}
+1 -1
View File
@@ -297,7 +297,7 @@ export class OggMuxer extends Muxer {
}
allTracksAreKnown() {
for (const track of this.output._tracks) {
for (const track of this.output.tracks) {
if (!track.source._closed && !this.trackDatas.some(x => x.track === track)) {
return false; // We haven't seen a sample from this open track yet
}
+41 -13
View File
@@ -372,6 +372,11 @@ export class Output<
*/
readonly defaultTrackGroup = new OutputTrackGroup();
/**
* The tracks that have been added to this output. Treat it as a readonly field; to add tracks, use the methods.
*/
readonly tracks: OutputTrack[] = [];
/** @internal */
private _initTarget: T | (() => MaybePromise<T>) | null;
/** @internal */
@@ -383,8 +388,6 @@ export class Output<
/** @internal */
_rootWriterPromise: Promise<Writer> | null = null;
/** @internal */
_tracks: OutputTrack[] = [];
/** @internal */
_startPromise: Promise<void> | null = null;
/** @internal */
_cancelPromise: Promise<void> | null = null;
@@ -619,7 +622,7 @@ export class Output<
metadataCopy.group ??= this.defaultTrackGroup;
return this._addTrack(new OutputVideoTrack(
this._tracks.length + 1, this, source, metadataCopy,
this.tracks.length + 1, this, source, metadataCopy,
));
}
@@ -634,7 +637,7 @@ export class Output<
metadataCopy.group ??= this.defaultTrackGroup;
return this._addTrack(new OutputAudioTrack(
this._tracks.length + 1, this, source, metadataCopy,
this.tracks.length + 1, this, source, metadataCopy,
));
}
@@ -649,7 +652,7 @@ export class Output<
metadataCopy.group ??= this.defaultTrackGroup;
return this._addTrack(new OutputSubtitleTrack(
this._tracks.length + 1, this, source, metadataCopy,
this.tracks.length + 1, this, source, metadataCopy,
));
}
@@ -680,7 +683,7 @@ export class Output<
// Verify maximum track count constraints
const supportedTrackCounts = this.format.getSupportedTrackCounts();
const presentTracksOfThisType = this._tracks.reduce(
const presentTracksOfThisType = this.tracks.reduce(
(count, t) => count + (t.type === track.type ? 1 : 0),
0,
);
@@ -694,7 +697,7 @@ export class Output<
);
}
const maxTotalCount = supportedTrackCounts.total.max;
if (this._tracks.length === maxTotalCount) {
if (this.tracks.length === maxTotalCount) {
throw new Error(
`${this.format._name} does not support more than ${maxTotalCount} tracks`
+ `${maxTotalCount === 1 ? '' : 's'} in total.`,
@@ -748,12 +751,37 @@ export class Output<
}
}
this._tracks.push(track);
this.tracks.push(track);
track.source._connectedTrack = track;
return track;
}
/**
* Whether the output has enough tracks (of the correct type) to be started, based on the requirements of the output
* format.
*/
hasEnoughTracks() {
const supportedTrackCounts = this.format.getSupportedTrackCounts();
for (const trackType of ALL_TRACK_TYPES) {
const presentTracksOfThisType = this.tracks.reduce(
(count, track) => count + (track.type === trackType ? 1 : 0),
0,
);
const minCount = supportedTrackCounts[trackType].min;
if (presentTracksOfThisType < minCount) {
return false;
}
}
const totalMinCount = supportedTrackCounts.total.min;
if (this.tracks.length < totalMinCount) {
return false;
}
return true;
}
/**
* Starts the creation of the output file. This method should be called after all tracks have been added. Only after
* the output has started can media samples be added to the tracks.
@@ -764,7 +792,7 @@ export class Output<
// Verify minimum track count constraints
const supportedTrackCounts = this.format.getSupportedTrackCounts();
for (const trackType of ALL_TRACK_TYPES) {
const presentTracksOfThisType = this._tracks.reduce(
const presentTracksOfThisType = this.tracks.reduce(
(count, track) => count + (track.type === trackType ? 1 : 0),
0,
);
@@ -780,7 +808,7 @@ export class Output<
}
}
const totalMinCount = supportedTrackCounts.total.min;
if (this._tracks.length < totalMinCount) {
if (this.tracks.length < totalMinCount) {
throw new Error(
totalMinCount === supportedTrackCounts.total.max
? (`${this.format._name} requires exactly ${totalMinCount} track`
@@ -807,7 +835,7 @@ export class Output<
try {
await this._muxer.start();
const promises = this._tracks.map(track => track.source._start());
const promises = this.tracks.map(track => track.source._start());
await Promise.all(promises);
} finally {
release();
@@ -850,7 +878,7 @@ export class Output<
const release = await this._mutex.acquire();
try {
const promises = this._tracks.map(x => x.source._flushOrWaitForOngoingClose(true)); // Force close
const promises = this.tracks.map(x => x.source._flushOrWaitForOngoingClose(true)); // Force close
await Promise.all(promises);
await Promise.all([...this._unfinalizedTargets].map(target => target._close()));
@@ -883,7 +911,7 @@ export class Output<
const release = await this._mutex.acquire();
try {
const promises = this._tracks.map(x => x.source._flushOrWaitForOngoingClose(false));
const promises = this.tracks.map(x => x.source._flushOrWaitForOngoingClose(false));
await Promise.all(promises);
await this._muxer.finalize();
+358 -1
View File
@@ -11,7 +11,7 @@ import { Output, OutputTrackGroup } from '../../src/output.js';
import { BufferSource, CustomPathedSource, UrlSource } from '../../src/source.js';
import { expect, test } from 'vitest';
import { BufferTarget, PathedTarget } from '../../src/target.js';
import { Conversion } from '../../src/conversion.js';
import { Conversion, ConversionCanceledError } from '../../src/conversion.js';
import { assert } from '../../src/misc.js';
import { InputVideoTrack } from '../../src/input-track.js';
import { CanvasSource, EncodedAudioPacketSource } from '../../src/media-source.js';
@@ -402,3 +402,360 @@ test('Fractional audio sample boundary', async () => {
});
await conversion.execute();
});
test('Non-composable conversion requires a fresh output', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
output.addAudioTrack(new EncodedAudioPacketSource('aac')); // Makes the output non-fresh
await expect(Conversion.init({ input, output })).rejects.toThrow(/must be fresh/);
});
test('Composable init works on an output that already has a track, but not on a started one', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
output.addAudioTrack(new EncodedAudioPacketSource('aac')); // A user-added track
const conversion = await Conversion.init({
input,
output,
composable: true,
audio: { discard: true }, // Only contribute the video track
showWarnings: false,
});
expect(conversion.isValid).toBe(true);
expect(conversion.utilizedTracks).toHaveLength(1);
expect(conversion.utilizedTracks[0]!.type).toBe('video');
const startedOutput = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
startedOutput.addAudioTrack(new EncodedAudioPacketSource('aac'));
await startedOutput.start();
await expect(Conversion.init({ input, output: startedOutput, composable: true }))
.rejects.toThrow(/not have been started/);
await startedOutput.cancel();
});
test('Composable conversion rejects tags', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const makeOutput = () => new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
await expect(Conversion.init({
input,
output: makeOutput(),
composable: true,
tags: { title: 'Not allowed' },
})).rejects.toThrow(/tags cannot be set by a composable conversion/);
});
test('Composable conversion composes with a user-added track', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({
input,
output,
composable: true,
audio: { discard: true }, // The user provides their own audio track
showWarnings: false,
});
expect(conversion.utilizedTracks).toHaveLength(1);
const audioSource = new EncodedAudioPacketSource('aac');
output.addAudioTrack(audioSource);
await output.start();
await Promise.all([
conversion.execute(),
(async () => {
await addAacPackets(audioSource, 5);
audioSource.close();
})(),
]);
// The composable conversion must not have finalized the output
expect(output.state).toBe('started');
await output.finalize();
expect(output.state).toBe('finalized');
using result = new Input({ source: new BufferSource(output.target.buffer!), formats: ALL_FORMATS });
const tracks = await result.getTracks();
expect(tracks.map(t => t.type).sort()).toEqual(['audio', 'video']);
const videoTrack = await result.getPrimaryVideoTrack();
const audioTrack = await result.getPrimaryAudioTrack();
expect(videoTrack).not.toBeNull();
expect(audioTrack).not.toBeNull();
expect(await videoTrack!.getCodec()).toBe('avc');
expect(await audioTrack!.getCodec()).toBe('aac');
expect(await videoTrack!.computeDuration()).toBeGreaterThan(4);
});
test('Two composable conversions compose into one output', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const videoConversion = await Conversion.init({
input,
output,
composable: true,
audio: { discard: true },
showWarnings: false,
});
const audioConversion = await Conversion.init({
input,
output,
composable: true,
video: { discard: true },
showWarnings: false,
});
expect(videoConversion.utilizedTracks).toHaveLength(1);
expect(videoConversion.utilizedTracks[0]!.type).toBe('video');
expect(audioConversion.utilizedTracks).toHaveLength(1);
expect(audioConversion.utilizedTracks[0]!.type).toBe('audio');
await output.start();
await Promise.all([videoConversion.execute(), audioConversion.execute()]);
expect(output.state).toBe('started');
await output.finalize();
using result = new Input({ source: new BufferSource(output.target.buffer!), formats: ALL_FORMATS });
const tracks = await result.getTracks();
expect(tracks.map(t => t.type).sort()).toEqual(['audio', 'video']);
expect(await (await result.getPrimaryVideoTrack())!.getCodec()).toBe('avc');
expect(await (await result.getPrimaryAudioTrack())!.getCodec()).toBe('aac');
});
test('Composable conversion does not write metadata tags', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
// Sanity check: this input carries metadata tags that a non-composable conversion would copy over
const inputTags = await input.getMetadataTags();
expect(inputTags.comment).toBeDefined();
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({
input,
output,
composable: true,
audio: { discard: true },
showWarnings: false,
});
// The conversion must not have touched the output's metadata tags
expect(Object.keys(output._metadataTags)).toHaveLength(0);
const audioSource = new EncodedAudioPacketSource('aac');
output.addAudioTrack(audioSource);
// The user sets their own tags; these must survive
output.setMetadataTags({ comment: 'User-owned' });
await output.start();
await Promise.all([
conversion.execute(),
(async () => {
await addAacPackets(audioSource, 5);
audioSource.close();
})(),
]);
await output.finalize();
using result = new Input({ source: new BufferSource(output.target.buffer!), formats: ALL_FORMATS });
const outTags = await result.getMetadataTags();
// Only the user's tag is present; the input's tags were not copied
expect(outTags.comment).toBe('User-owned');
});
test('Canceling a composable conversion leaves the output usable', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({
input,
output,
composable: true,
audio: { discard: true },
showWarnings: false,
});
const audioSource = new EncodedAudioPacketSource('aac');
output.addAudioTrack(audioSource);
await output.start();
const executePromise = conversion.execute();
void conversion.cancel();
expect(conversion.state).toBe('canceled');
await expect(executePromise).rejects.toBeInstanceOf(ConversionCanceledError);
// The output must not have been canceled by the composable conversion
expect(output.state).toBe('started');
// The user's own track can still finish, and the output can still be finalized
await addAacPackets(audioSource, 2);
audioSource.close();
await output.finalize();
expect(output.state).toBe('finalized');
using result = new Input({ source: new BufferSource(output.target.buffer!), formats: ALL_FORMATS });
const audioTrack = await result.getPrimaryAudioTrack();
expect(audioTrack).not.toBeNull();
expect(await audioTrack!.getCodec()).toBe('aac');
});
test('Track capacity works correctly with composable conversions', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new WavOutputFormat(), target: new BufferTarget() });
// The user already occupies the single audio slot that WAVE allows
output.addAudioTrack(new EncodedAudioPacketSource('pcm-s16'));
const conversion = await Conversion.init({
input,
output,
composable: true,
showWarnings: false,
});
// The conversion's audio track has no room left, so it gets discarded
expect(conversion.isValid).toBe(true);
expect(conversion.utilizedTracks).toHaveLength(0);
expect(conversion.discardedTracks).toHaveLength(2);
// WAVE allows only one track in total, so the total-count check fires before the per-type one
expect(conversion.discardedTracks[0]!.reason).toBe('max_track_count_reached');
});
test('Blank execute', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({ input, output });
expect(conversion.state).toBe('idle');
const promise = conversion.execute();
expect(conversion.state).toBe('executing');
await promise;
expect(conversion.state).toBe('done');
expect(output.state).toBe('finalized');
await conversion.execute();
expect(conversion.state).toBe('done');
});
test('Stepwise until', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({ input, output });
await conversion.execute({ until: 2 });
expect(conversion.state).toBe('idle');
expect(output.state).toBe('started');
await conversion.execute({ until: 4 });
expect(conversion.state).toBe('idle');
await conversion.execute({ until: 6 });
expect(conversion.state).toBe('done');
expect(output.state).toBe('finalized');
await conversion.execute();
expect(conversion.state).toBe('done');
using result = new Input({ source: new BufferSource(output.target.buffer!), formats: ALL_FORMATS });
const videoTrack = await result.getPrimaryVideoTrack();
expect(await videoTrack!.computeDuration()).toBeGreaterThan(4);
});
test('Pause signal', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({ input, output });
const controller = new AbortController();
conversion.onProgress = (progress) => {
if (progress >= 0.5 && !controller.signal.aborted) {
controller.abort();
}
};
await conversion.execute({ pauseSignal: controller.signal });
expect(conversion.state).toBe('idle');
expect(output.state).toBe('started');
await conversion.execute();
expect(conversion.state).toBe('done');
expect(output.state).toBe('finalized');
await conversion.execute();
expect(conversion.state).toBe('done');
});
test('Pre-signaled pause signal', async () => {
using input = new Input({
source: new UrlSource('/video.mp4'),
formats: ALL_FORMATS,
});
const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });
const conversion = await Conversion.init({ input, output });
const controller = new AbortController();
controller.abort();
await conversion.execute({ pauseSignal: controller.signal });
expect(conversion.state).toBe('idle');
expect(output.state).toBe('started');
await conversion.execute();
expect(conversion.state).toBe('done');
expect(output.state).toBe('finalized');
await conversion.execute();
expect(conversion.state).toBe('done');
});