arcodange/mediabunny
1
0
Fork 0
mirror of https://github.com/arcodange-org/mediabunny.git synced 2026-09-27 19:03:46 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add custom quality factors and quantizer bitrate mode (#448)

Browse Source 
* Add custom quality factors and quantizer bitrate mode (closes #327)

* Fix quantizer mode fallback and per-frame quantizer edge cases

* Clean up quantizer mode tests

* Quantizer implementation

* Add quality tests

* Improve conversion codec error message, make test more lenient

* Add quantizer support blog post

---------

Co-authored-by: Vanilagy <[email protected]>
This commit is contained in:
Don Carignan
2026-07-30 17:24:59 +02:00
committed by GitHub
co-authored by Vanilagy
parent 1a99b0371d
commit c3df2a24e5
48 changed files with 1372 additions and 371 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/guide/media-sources.md
+66 -29
View File
@@ -50,9 +50,8 @@ All video sources that handle encoding internally require you to specify a `Vide
```ts
type VideoEncodingConfig = {
codec: VideoCodec;
bitrate: number | Quality;
quality: Quality;
alpha?: 'discard' | 'keep';
bitrateMode?: 'constant' | 'variable';
latencyMode?: 'quality' | 'realtime';
keyFrameInterval?: number;
fullCodecString?: string;
@@ -84,11 +83,10 @@ type VideoEncodingConfig = {
};
```
- `codec`: The [video codec](./supported-formats-and-codecs#video-codecs) used for encoding.
- `bitrate`: The target number of bits per second. Alternatively, this can be a [subjective quality](#subjective-qualities).
- `quality`: The desired quality of the encoded video. See [Encoding quality](#encoding-quality).
- `alpha`: What to do with alpha data contained in the video samples.
- `'discard'` (default): Only the samples' color data is kept; the video is opaque.
- `'keep'`: The samples' alpha data is also encoded as side data. Make sure to pair this mode with a container format that supports transparency (such as WebM or Matroska).
- `bitrateMode`: Can be used to control constant vs. variable bitrate.
- `latencyMode`: The latency mode as specified by the WebCodecs API. Browsers default to `quality`. Media stream-driven video sources will automatically use the `realtime` setting.
- `keyFrameInterval`: The maximum interval in seconds between two adjacent key frames. Defaults to 2 seconds. More frequent key frames improve seeking behavior but increase file size. When using multiple video tracks, this value should be set to the same value for all tracks.
- `fullCodecString`: Allows you to optionally specify the full codec string used by the video encoder, as specified in the [Mediabunny Codec Registry](/codec-registry/overview). For example, you may set it to `'avc1.42001f'` when using AVC. Keep in mind that the codec string must still match the codec specified in `codec`. If you don't set this field, a codec string will be generated automatically.
@@ -117,8 +115,7 @@ All audio sources that handle encoding internally require you to specify an `Aud
```ts
type AudioEncodingConfig = {
codec: AudioCodec;
bitrate?: number | Quality;
bitrateMode?: 'constant' | 'variable';
quality?: Quality;
fullCodecString?: string;
transform?: {
@@ -139,8 +136,7 @@ type AudioEncodingConfig = {
};
```
- `codec`: The [audio codec](./supported-formats-and-codecs#audio-codecs) used for encoding. Can be omitted for uncompressed PCM codecs.
- `bitrate`: The target number of bits per second. Alternatively, this can be a [subjective quality](#subjective-qualities).
- `bitrateMode`: Can be used to control constant vs. variable bitrate.
- `quality`: The desired quality of the encoded audio; unused for PCM codecs. See [Encoding quality](#encoding-quality).
- `fullCodecString`: Allows you to optionally specify the full codec string used by the audio encoder, as specified in the [Mediabunny Codec Registry](/codec-registry/overview). For example, you may set it to `'mp4a.40.2'` when using AAC. Keep in mind that the codec string must still match the codec specified in `codec`. If you don't set this field, a codec string will be generated automatically.
- `transform`: Optional transformations to apply to the audio samples before they are passed to the encoder.
- `numberOfChannels`: The desired number of output channels to up/downmix to.
@@ -149,18 +145,59 @@ type AudioEncodingConfig = {
- `onEncodedPacket`: Called for each successfully encoded packet. Useful for determining encoding progress.
- `onEncoderConfig`: Called when the internal encoder config, as used by the WebCodecs API, is created. You can use this to introspect the full codec string.
### Subjective qualities
### Encoding quality
Mediabunny provides five subjective quality options as an alternative to manually providing a bitrate. From a subjective quality, a bitrate will be calculated internally based on the codec and track information (width, height, sample rate, ...).
Mediabunny provides the [`Quality`](../api/Quality) class as a way to describe the desired encoding quality, i.e. how much the media should be compressed.
#### Qualitative quality
Qualitative qualities are defined abstractly will automatically map to an underlying bitrate or quantizer based on codec and media parameters (such as video dimensions).
```ts
import {
QUALITY_VERY_LOW,
QUALITY_LOW,
QUALITY_MEDIUM,
QUALITY_HIGH,
QUALITY_VERY_HIGH,
} from 'mediabunny';
import { Quality } from 'mediabunny';
// A Quality can be created from these five pre-defined values
new Quality('very-low');
new Quality('low');
new Quality('medium');
new Quality('high');
new Quality('very-high');
// Or from a quality value from 0 to 1. Here, 0 represents the worst quality and
// 1 the best.
new Quality(0.6);
```
#### Quantitative quality
Quantitative qualities allow explicit control over compression parameters.
```ts
import { Quality } from 'mediabunny';
// 1 Mbps bitrate (variable)
new Quality({
bitrate: 1e6,
});
// 1 Mbps bitrate (constant)
new Quality({
bitrate: 1e6,
bitrateMode: 'constant',
});
// Specific quantizer value to use. The valid range of values depends on each
// codec and is defined in the Mediabunny Codec Registry. Throws if quantizer
// mode is unavailable.
new Quality({
quantizer: 32,
});
// Quantizer and bitrate combined, with bitrate used as a fallback
new Quality({
quantizer: 32,
bitrate: 1e6,
});
```
## Video sources
@@ -172,11 +209,11 @@ Video sources feed data to video tracks on an `Output`. They all extend the abst
This source takes [video samples](./packets-and-samples#videosample), encodes them, and passes the encoded data to the output.
```ts
import { VideoSampleSource } from 'mediabunny';
import { VideoSampleSource, Quality } from 'mediabunny';
const sampleSource = new VideoSampleSource({
codec: 'avc',
bitrate: 1e6,
quality: new Quality({ bitrate: 1e6 }),
});
await sampleSource.add(videoSample);
@@ -191,11 +228,11 @@ await sampleSource.add(videoSample, { keyFrame: true });
This source simplifies a common pattern: A single canvas is repeatedly updated in a render loop and each frame is added to the output file.
```ts
import { CanvasSource, QUALITY_MEDIUM } from 'mediabunny';
import { CanvasSource, Quality } from 'mediabunny';
const canvasSource = new CanvasSource(canvasElement, {
codec: 'av1',
bitrate: QUALITY_MEDIUM,
quality: new Quality('medium'),
});
await canvasSource.add(0.0, 0.1); // Timestamp, duration (in seconds)
@@ -211,7 +248,7 @@ await canvasSource.add(0.3, 0.1, { keyFrame: true });
This is a source for use with the [Media Capture and Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Media_Capture_and_Streams_API). Use this source if you want to pipe a real-time video source (such as a webcam or screen recording) to an output file.
```ts
import { MediaStreamVideoTrackSource } from 'mediabunny';
import { MediaStreamVideoTrackSource, Quality } from 'mediabunny';
// Get the user's screen
const stream = await navigator.mediaDevices.getDisplayMedia({ video: true });
@@ -219,7 +256,7 @@ const videoTrack = stream.getVideoTracks()[0];
const videoTrackSource = new MediaStreamVideoTrackSource(videoTrack, {
codec: 'vp9',
bitrate: 1e7,
quality: new Quality({ bitrate: 1e7 }),
});
// Make sure to allow any internal errors to properly bubble up
@@ -345,11 +382,11 @@ Audio sources feed data to audio tracks on an `Output`. They all extend the abst
This source takes [audio samples](./packets-and-samples#audiosample), encodes them, and passes the encoded data to the output.
```ts
import { AudioSampleSource } from 'mediabunny';
import { AudioSampleSource, Quality } from 'mediabunny';
const sampleSource = new AudioSampleSource({
codec: 'aac',
bitrate: 128e3,
quality: new Quality({ bitrate: 128e3 }),
});
await sampleSource.add(audioSample);
@@ -361,11 +398,11 @@ audioSample.close(); // If it's not needed anymore
This source directly accepts instances of `AudioBuffer` as data, simplifying usage with the Web Audio API. The first AudioBuffer will be played at timestamp 0, and any subsequent AudioBuffer will be appended after all previous AudioBuffers.
```ts
import { AudioBufferSource, QUALITY_MEDIUM } from 'mediabunny';
import { AudioBufferSource, Quality } from 'mediabunny';
const bufferSource = new AudioBufferSource({
codec: 'opus',
bitrate: QUALITY_MEDIUM,
quality: new Quality('medium'),
});
await bufferSource.add(audioBuffer1);
@@ -378,7 +415,7 @@ await bufferSource.add(audioBuffer3);
This is a source for use with the [Media Capture and Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Media_Capture_and_Streams_API). Use this source if you want to pipe a real-time audio source (such as a microphone or audio from the user's computer) to an output file.
```ts
import { MediaStreamAudioTrackSource } from 'mediabunny';
import { MediaStreamAudioTrackSource, Quality } from 'mediabunny';
// Get the user's microphone
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
@@ -386,7 +423,7 @@ const audioTrack = stream.getAudioTracks()[0];
const audioTrackSource = new MediaStreamAudioTrackSource(audioTrack, {
codec: 'opus',
bitrate: 128e3,
quality: new Quality({ bitrate: 128e3 }),
});
// Make sure to allow any internal errors to properly bubble up