mirror of
https://github.com/arcodange-org/mediabunny.git
synced 2026-09-27 02:43:48 +02:00
Add Mediabunny Codec Registry
This commit is contained in:
Vanilagy
@@ -42,6 +42,12 @@ export default withMermaid({
|
||||
{ text: 'Examples', link: '/examples', activeMatch: '/examples' },
|
||||
{ text: 'Sponsors', link: '/#sponsors', activeMatch: '/#sponsors' },
|
||||
{ text: 'License', link: 'https://github.com/Vanilagy/mediabunny#license' },
|
||||
{
|
||||
text: 'More',
|
||||
items: [
|
||||
{ text: 'Codec Registry', link: '/codec-registry/overview' },
|
||||
],
|
||||
},
|
||||
],
|
||||
|
||||
sidebar: {
|
||||
@@ -93,6 +99,40 @@ export default withMermaid({
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any, @typescript-eslint/no-unsafe-assignment
|
||||
'/api': apiRoutes as any,
|
||||
|
||||
'/codec-registry': [
|
||||
{
|
||||
text: 'Codec registry',
|
||||
items: [
|
||||
{ text: 'Overview', link: '/codec-registry/overview' },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: 'Video',
|
||||
items: [
|
||||
{ text: 'AVC (H.264)', link: '/codec-registry/avc' },
|
||||
{ text: 'HEVC (H.265)', link: '/codec-registry/hevc' },
|
||||
{ text: 'VP8', link: '/codec-registry/vp8' },
|
||||
{ text: 'VP9', link: '/codec-registry/vp9' },
|
||||
{ text: 'AV1', link: '/codec-registry/av1' },
|
||||
],
|
||||
},
|
||||
{
|
||||
text: 'Audio',
|
||||
items: [
|
||||
{ text: 'AAC', link: '/codec-registry/aac' },
|
||||
{ text: 'Opus', link: '/codec-registry/opus' },
|
||||
{ text: 'MP3', link: '/codec-registry/mp3' },
|
||||
{ text: 'Vorbis', link: '/codec-registry/vorbis' },
|
||||
{ text: 'FLAC', link: '/codec-registry/flac' },
|
||||
{ text: 'AC-3', link: '/codec-registry/ac3' },
|
||||
{ text: 'E-AC-3', link: '/codec-registry/eac3' },
|
||||
{ text: 'Linear PCM', link: '/codec-registry/pcm' },
|
||||
{ text: 'μ-law PCM', link: '/codec-registry/ulaw' },
|
||||
{ text: 'A-law PCM', link: '/codec-registry/alaw' },
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
|
||||
socialLinks: [
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# AAC codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Advanced Audio Coding (AAC) audio codec, specified in [ISO/IEC 14496-3](https://www.iso.org/standard/76383.html).
|
||||
|
||||
An AAC bitstream can have either of two formats:
|
||||
- _AAC_ (raw), where packets contain raw AAC frames (syntax element `raw_data_block()`) as defined in [ISO/IEC 14496-3](https://www.iso.org/standard/76383.html) Section 4.4.2.1. Here, codec metadata is provided out-of-band.
|
||||
- _ADTS_, where packets contain ADTS frames as defined in [ISO/IEC 14496-3](https://www.iso.org/standard/76383.html) Section 1.A.3.2. Here, codec metadata is provided in-band in each ADTS frame header.
|
||||
|
||||
All packets within the bitstream must have the same format.
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'aac'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
If the bitstream is in the _AAC_ (raw) format, the packet's data must be a raw AAC frame (syntax element `raw_data_block()`) as defined in [ISO/IEC 14496-3](https://www.iso.org/standard/76383.html) Section 4.4.2.1.
|
||||
|
||||
If the bitstream is in the _ADTS_ format, the packet's data must be an ADTS frame as defined in [ISO/IEC 14496-3](https://www.iso.org/standard/76383.html) Section 1.A.3.2.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
The following fully qualified codec strings are recognized:
|
||||
|
||||
- `'mp4a.40.2'` — MPEG-4 AAC-LC
|
||||
- `'mp4a.40.02'` — MPEG-4 AAC-LC (leading zero for Aud-OTI compatibility)
|
||||
- `'mp4a.40.5'` — MPEG-4 HE-AAC v1 (AAC-LC + SBR)
|
||||
- `'mp4a.40.05'` — MPEG-4 HE-AAC v1 (leading zero for Aud-OTI compatibility)
|
||||
- `'mp4a.40.29'` — MPEG-4 HE-AAC v2 (AAC-LC + SBR + PS)
|
||||
- `'mp4a.67'` — MPEG-2 AAC-LC
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
If the bitstream is in the _AAC_ (raw) format, `description` must be an `AudioSpecificConfig` as defined in [ISO/IEC 14496-3](https://www.iso.org/standard/76383.html) Section 1.6.2.1.
|
||||
|
||||
If the bitstream is in the _ADTS_ format, `description` must be undefined.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# AC-3 codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Dolby Digital (AC-3) audio codec, specified in [ETSI TS 102 366](https://www.etsi.org/deliver/etsi_ts/102300_102399/102366/01.04.01_60/ts_102366v010401p.pdf).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'ac3'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a sync frame (syntactic element `syncframe()`) as defined in Section 4.3 of [ETSI TS 102 366](https://www.etsi.org/deliver/etsi_ts/102300_102399/102366/01.04.01_60/ts_102366v010401p.pdf), beginning with the sync word `0x0B77`.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'ac-3'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# A-law PCM codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The A-law companded PCM audio codec, specified in [ITU-T G.711](https://www.itu.int/rec/T-REC-G.711) Tables 1a and 1b.
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'alaw'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a sequence of bytes of arbitrary length (divisible by the channel count), where each byte is an A-law encoded PCM sample. For multichannel audio, samples from different channels are interleaved.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'alaw'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,33 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Video codec" />
|
||||
|
||||
# AV1 codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The AOMedia Video 1 (AV1) video codec, specified in the [AV1 Bitstream & Decoding Process Specification](https://aomediacodec.github.io/av1-spec/).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'av1'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must comply with the low-overhead bitstream format as defined in Section 5 of the [AV1 Bitstream & Decoding Process Specification](https://aomediacodec.github.io/av1-spec/).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
If the packet's type is `'key'`, then the packet is expected to contain a frame with `frame_type` of `KEY_FRAME`, as defined in Section 6.8.2 of the [AV1 Bitstream & Decoding Process Specification](https://aomediacodec.github.io/av1-spec/).
|
||||
|
||||
## `VideoDecoderConfig` codec string
|
||||
|
||||
The full codec string begins with the prefix `'av01.'`, with a variable-length suffix as specified in Section 5 of the [AV1 Codec ISO Media File Format Binding](https://aomediacodec.github.io/av1-isobmff/).
|
||||
|
||||
## `VideoDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,41 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Video codec" />
|
||||
|
||||
# AVC (H.264) codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Advanced Video Coding (H.264) video codec, specified in [Rec. ITU-T H.264](https://www.itu.int/rec/T-REC-H.264) / [ISO/IEC 14496-10](https://www.iso.org/standard/87574.html).
|
||||
|
||||
An AVC bitstream can have either of two formats:
|
||||
- _Canonical_ (length-prefixed), as defined in [ISO/IEC 14496-15](https://www.iso.org/standard/89118.html) Section 5.3.2. Here, video parameter sets (SPS/PPS) are provided out-of-band.
|
||||
- _Annex B_, as defined in [Rec. ITU-T H.264](https://www.itu.int/rec/T-REC-H.264) Annex B. Here, video parameter sets (SPS/PPS) must be provided in-band in the respective NALUs.
|
||||
|
||||
All packets within the bitstream must have the same format.
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'avc'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be an access unit as defined in [Rec. ITU-T H.264](https://www.itu.int/rec/T-REC-H.264) Section 7.4.1.2, in either _canonical_ or _Annex B_ format.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
If the packet's type is `'key'`, then the packet is expected to contain a primary coded picture from which decoding can begin. Additionally, if the bitstream's format is _Annex B_, then this packet is also expected to contain the necessary video parameter sets to initialize the decoder.
|
||||
|
||||
## `VideoDecoderConfig` codec string
|
||||
|
||||
The full codec string begins with the prefix `'avc1.'` or `'avc3.'`, with a suffix of 6 characters as described respectively in Section 3.4 of [RFC 6381](https://www.rfc-editor.org/rfc/rfc6381) and Section 5.4.1 of [ISO/IEC 14496-15](https://www.iso.org/standard/89118.html).
|
||||
|
||||
## `VideoDecoderConfig` description
|
||||
|
||||
If the bitstream is in the _canonical_ (length-prefixed) format, `description` must be an `AVCDecoderConfigurationRecord` as defined in [ISO/IEC 14496-15](https://www.iso.org/standard/89118.html) Section 5.3.3.1.
|
||||
|
||||
If the bitstream is in the _Annex B_ format, `description` must be undefined.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# E-AC-3 codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Dolby Digital Plus (E-AC-3) audio codec, specified in [ETSI TS 102 366](https://www.etsi.org/deliver/etsi_ts/102300_102399/102366/01.04.01_60/ts_102366v010401p.pdf).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'eac3'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a sync frame as defined in Section E.1.2.0 of [ETSI TS 102 366](https://www.etsi.org/deliver/etsi_ts/102300_102399/102366/01.04.01_60/ts_102366v010401p.pdf), beginning with the sync word `0x0B77`.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'ec-3'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# FLAC codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Free Lossless Audio Codec (FLAC), specified in the [FLAC Format Specification](https://xiph.org/flac/format.html).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'flac'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a FLAC frame as described in the [FLAC Format Specification](https://xiph.org/flac/format.html).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'flac'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` must contain the bytes `0x66 0x4C 0x61 0x43` (the ASCII string `'fLaC'`), followed by a `STREAMINFO` metadata block as defined in Section 7 of the [FLAC Format Specification](https://xiph.org/flac/format.html).
|
||||
@@ -0,0 +1,41 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Video codec" />
|
||||
|
||||
# HEVC (H.265) codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The High Efficiency Video Coding (H.265) video codec, specified in [Rec. ITU-T H.265](https://www.itu.int/rec/T-REC-H.265) / [ISO/IEC 23008-2](https://www.iso.org/standard/75484.html).
|
||||
|
||||
An HEVC bitstream can have either of two formats:
|
||||
- _Canonical_ (length-prefixed), as defined in [ISO/IEC 14496-15](https://www.iso.org/standard/89118.html) Section 8.3.2. Here, video parameter sets (VPS/SPS/PPS) are provided out-of-band.
|
||||
- _Annex B_, as defined in [Rec. ITU-T H.265](https://www.itu.int/rec/T-REC-H.265) Annex B. Here, video parameter sets (VPS/SPS/PPS) must be provided in-band in the respective NALUs.
|
||||
|
||||
All packets within the bitstream must have the same format.
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'hevc'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be an access unit as defined in [Rec. ITU-T H.265](https://www.itu.int/rec/T-REC-H.265) Section 7.4.2.4, containing exactly one base layer coded picture, in either _canonical_ or _Annex B_ format.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
If the packet's type is `'key'`, then the packet is expected to contain an IDR, CRA, or BLA picture. Additionally, if the bitstream's format is _Annex B_, then this packet is also expected to contain the necessary video parameter sets to initialize the decoder.
|
||||
|
||||
## `VideoDecoderConfig` codec string
|
||||
|
||||
The full codec string begins with the prefix `'hev1.'` or `'hvc1.'`, with a variable-length suffix of four dot-separated fields as specified in Section E.3 of [ISO/IEC 14496-15](https://www.iso.org/standard/89118.html).
|
||||
|
||||
## `VideoDecoderConfig` description
|
||||
|
||||
If the bitstream is in the _canonical_ (length-prefixed) format, `description` must be an `HEVCDecoderConfigurationRecord` as defined in [ISO/IEC 14496-15](https://www.iso.org/standard/89118.html) Section 8.3.3.1.
|
||||
|
||||
If the bitstream is in the _Annex B_ format, `description` must be undefined.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# MP3 codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The MP3 audio codec (MPEG-1/2 Audio Layer III), specified in [ISO/IEC 13818-3](https://www.iso.org/standard/26797.html).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'mp3'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be an MP3 frame as described in Section 2.4.2.2 of [ISO/IEC 13818-3](https://www.iso.org/standard/26797.html).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'mp3'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# Opus codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Opus audio codec, specified in [RFC 6716](https://www.rfc-editor.org/rfc/rfc6716).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'opus'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be an Opus packet as described in Section 3 of [RFC 6716](https://www.rfc-editor.org/rfc/rfc6716).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'opus'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
If present, `description` must be an Identification Header as defined in Section 5.1 of [RFC 7845](https://www.rfc-editor.org/rfc/rfc7845).
|
||||
@@ -0,0 +1,26 @@
|
||||
# Mediabunny Codec Registry
|
||||
|
||||
The Mediabunny Codec Registry formalizes the precise definitions of all video and audio codecs supported by Mediabunny. More specifically, for any given codec, it describes the format that `EncodedPacket`, `VideoDecoderConfig` and `AudioDecoderConfig` must adhere to. All packets coming out of or going into Mediabunny are expected to adhere to this registry.
|
||||
|
||||
The registry is an extension of the [WebCodecs Codec Registry](https://www.w3.org/TR/webcodecs-codec-registry/). Mediabunny's registry matches that of WebCodecs for all codecs supported by both.
|
||||
|
||||
## Video codecs
|
||||
|
||||
- [AVC (H.264)](./avc)
|
||||
- [HEVC (H.265)](./hevc)
|
||||
- [VP8](./vp8)
|
||||
- [VP9](./vp9)
|
||||
- [AV1](./av1)
|
||||
|
||||
## Audio codecs
|
||||
|
||||
- [AAC](./aac)
|
||||
- [Opus](./opus)
|
||||
- [MP3](./mp3)
|
||||
- [Vorbis](./vorbis)
|
||||
- [FLAC](./flac)
|
||||
- [AC-3](./ac3)
|
||||
- [E-AC-3](./eac3)
|
||||
- [Linear PCM](./pcm)
|
||||
- [μ-law PCM](./ulaw)
|
||||
- [A-law PCM](./alaw)
|
||||
@@ -0,0 +1,44 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# Linear PCM codecs registration
|
||||
|
||||
## Description
|
||||
|
||||
A family of linear pulse-code modulation (PCM) audio codecs of various bit depths and byte orders.
|
||||
|
||||
## Codec IDs
|
||||
|
||||
| Codec ID | Description |
|
||||
| --- | --- |
|
||||
| `'pcm-u8'` | Unsigned 8-bit integer |
|
||||
| `'pcm-s8'` | Signed 8-bit integer |
|
||||
| `'pcm-s16'` | Signed 16-bit integer, little-endian |
|
||||
| `'pcm-s16be'` | Signed 16-bit integer, big-endian |
|
||||
| `'pcm-s24'` | Signed 24-bit integer, little-endian |
|
||||
| `'pcm-s24be'` | Signed 24-bit integer, big-endian |
|
||||
| `'pcm-s32'` | Signed 32-bit integer, little-endian |
|
||||
| `'pcm-s32be'` | Signed 32-bit integer, big-endian |
|
||||
| `'pcm-f32'` | 32-bit float, little-endian |
|
||||
| `'pcm-f32be'` | 32-bit float, big-endian |
|
||||
| `'pcm-f64'` | 64-bit float, little-endian |
|
||||
| `'pcm-f64be'` | 64-bit float, big-endian |
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a sequence of bytes of arbitrary length (divisible by the size of one frame), with each sample occupying the number of bits defined by the codec ID. For multichannel audio, samples from different channels are interleaved.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
The codec string is the same as the codec ID (e.g. `'pcm-s16'`, `'pcm-f32'`).
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# μ-law PCM codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The μ-law companded PCM audio codec, specified in [ITU-T G.711](https://www.itu.int/rec/T-REC-G.711) Tables 2a and 2b.
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'ulaw'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a sequence of bytes of arbitrary length (divisible by the channel count), where each byte is a μ-law encoded PCM sample. For multichannel audio, samples from different channels are interleaved.
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'ulaw'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Audio codec" />
|
||||
|
||||
# Vorbis codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The Vorbis audio codec, specified in the [Vorbis I Specification](https://xiph.org/vorbis/doc/Vorbis_I_spec.html).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'vorbis'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be an audio packet as described in Section 4.3 of the [Vorbis I Specification](https://xiph.org/vorbis/doc/Vorbis_I_spec.html).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
The packet's type is always `'key'`.
|
||||
|
||||
## `AudioDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'vorbis'
|
||||
```
|
||||
|
||||
## `AudioDecoderConfig` description
|
||||
|
||||
`description` must contain Vorbis codec setup data in Xiph extradata format: the `page_segments` field, followed by the `segment_table` field, followed by the three Vorbis header packets (identification header, comments header, and setup header) as defined in Section 4.2 of the [Vorbis I Specification](https://xiph.org/vorbis/doc/Vorbis_I_spec.html).
|
||||
@@ -0,0 +1,35 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Video codec" />
|
||||
|
||||
# VP8 codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The VP8 video codec, specified in [RFC 6386](https://www.rfc-editor.org/rfc/rfc6386).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'vp8'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a frame as described in Section 4 and Annex A of [RFC 6386](https://www.rfc-editor.org/rfc/rfc6386).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
If the packet's type is `'key'`, then the packet is expected to contain a frame where `key_frame` is `true`, as defined in Section 19.1 of [RFC 6386](https://www.rfc-editor.org/rfc/rfc6386).
|
||||
|
||||
## `VideoDecoderConfig` codec string
|
||||
|
||||
```ts
|
||||
'vp8'
|
||||
```
|
||||
|
||||
## `VideoDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -0,0 +1,33 @@
|
||||
<script setup>
|
||||
import { VPBadge } from 'vitepress/theme'
|
||||
</script>
|
||||
|
||||
<VPBadge type="info" text="Video codec" />
|
||||
|
||||
# VP9 codec registration
|
||||
|
||||
## Description
|
||||
|
||||
The VP9 video codec, specified in the [VP9 Bitstream & Decoding Process Specification](https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp9-bitstream-specification-v0.6-20160331-draft.pdf).
|
||||
|
||||
## Codec ID
|
||||
|
||||
```ts
|
||||
'vp9'
|
||||
```
|
||||
|
||||
## `EncodedPacket` data
|
||||
|
||||
The packet's data must be a frame as described in Section 6 of the [VP9 Bitstream & Decoding Process Specification](https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp9-bitstream-specification-v0.6-20160331-draft.pdf).
|
||||
|
||||
## `EncodedPacket` type
|
||||
|
||||
If the packet's type is `'key'`, then the packet is expected to contain a frame with `frame_type` of `KEY_FRAME`, as defined in Section 7.2 of the [VP9 Bitstream & Decoding Process Specification](https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp9-bitstream-specification-v0.6-20160331-draft.pdf).
|
||||
|
||||
## `VideoDecoderConfig` codec string
|
||||
|
||||
The full codec string begins with the prefix `'vp09.'`, with a variable-length suffix as specified in the [VP Codec ISO Media File Format Binding](https://www.webmproject.org/vp9/mp4/).
|
||||
|
||||
## `VideoDecoderConfig` description
|
||||
|
||||
`description` is not used for this codec.
|
||||
@@ -74,7 +74,7 @@ type VideoEncodingConfig = {
|
||||
- `bitrateMode`: Can be used to control constant vs. variable bitrate.
|
||||
- `latencyMode`: The latency mode as specified by the WebCodecs API. Browsers default to `quality`. Media stream-driven video sources will automatically use the `realtime` setting.
|
||||
- `keyFrameInterval`: The maximum interval in seconds between two adjacent key frames. Defaults to 5 seconds. More frequent key frames improve seeking behavior but increase file size. When using multiple video tracks, this value should be set to the same value for all tracks.
|
||||
- `fullCodecString`: Allows you to optionally specify the full codec string used by the video encoder, as specified in the [WebCodecs Codec Registry](https://www.w3.org/TR/webcodecs-codec-registry/). For example, you may set it to `'avc1.42001f'` when using AVC. Keep in mind that the codec string must still match the codec specified in `codec`. If you don't set this field, a codec string will be generated automatically.
|
||||
- `fullCodecString`: Allows you to optionally specify the full codec string used by the video encoder, as specified in the [Mediabunny Codec Registry](/codec-registry/overview). For example, you may set it to `'avc1.42001f'` when using AVC. Keep in mind that the codec string must still match the codec specified in `codec`. If you don't set this field, a codec string will be generated automatically.
|
||||
- `hardwareAcceleration`: A hint that configures the hardware acceleration method of this codec. This is best left on `'no-preference'`.
|
||||
- `scalabilityMode`: An encoding scalability mode identifier as defined by [WebRTC-SVC](https://w3c.github.io/webrtc-svc/#scalabilitymodes*).
|
||||
- `contentHint`: An encoding video content hint as defined by [mst-content-hint](https://w3c.github.io/mst-content-hint/#video-content-hints).
|
||||
@@ -104,7 +104,7 @@ type AudioEncodingConfig = {
|
||||
- `codec`: The [audio codec](./supported-formats-and-codecs#audio-codecs) used for encoding. Can be omitted for uncompressed PCM codecs.
|
||||
- `bitrate`: The target number of bits per second. Alternatively, this can be a [subjective quality](#subjective-qualities).
|
||||
- `bitrateMode`: Can be used to control constant vs. variable bitrate.
|
||||
- `fullCodecString`: Allows you to optionally specify the full codec string used by the audio encoder, as specified in the [WebCodecs Codec Registry](https://www.w3.org/TR/webcodecs-codec-registry/). For example, you may set it to `'mp4a.40.2'` when using AAC. Keep in mind that the codec string must still match the codec specified in `codec`. If you don't set this field, a codec string will be generated automatically.
|
||||
- `fullCodecString`: Allows you to optionally specify the full codec string used by the audio encoder, as specified in the [Mediabunny Codec Registry](/codec-registry/overview). For example, you may set it to `'mp4a.40.2'` when using AAC. Keep in mind that the codec string must still match the codec specified in `codec`. If you don't set this field, a codec string will be generated automatically.
|
||||
- `onEncodedPacket`: Called for each successfully encoded packet. Useful for determining encoding progress.
|
||||
- `onEncoderConfig`: Called when the internal encoder config, as used by the WebCodecs API, is created. You can use this to introspect the full codec string.
|
||||
|
||||
@@ -240,7 +240,7 @@ await packetSource.add(firstPacket, {
|
||||
});
|
||||
```
|
||||
|
||||
`codec`, `codedWidth`, and `codedHeight` are required for all codecs, whereas `description` is required for some codecs. Additional fields, such as `colorSpace`, are optional. The [WebCodecs Codec Registry](https://www.w3.org/TR/webcodecs-codec-registry/) specifies the formats of `codec` and `description` for each video codec, which you must adhere to.
|
||||
`codec`, `codedWidth`, and `codedHeight` are required for all codecs, whereas `description` is required for some codecs. Additional fields, such as `colorSpace`, are optional. The [Mediabunny Codec Registry](/codec-registry/overview) specifies the formats of `codec` and `description` for each video codec, which you **must** adhere to.
|
||||
|
||||
#### B-frames
|
||||
|
||||
@@ -397,7 +397,7 @@ await packetSource.add(firstPacket, {
|
||||
});
|
||||
```
|
||||
|
||||
`codec`, `numberOfChannels`, and `sampleRate` are required for all codecs, whereas `description` is required for some codecs. The [WebCodecs Codec Registry](https://www.w3.org/TR/webcodecs-codec-registry/) specifies the formats of `codec` and `description` for each audio codec, which you must adhere to.
|
||||
`codec`, `numberOfChannels`, and `sampleRate` are required for all codecs, whereas `description` is required for some codecs. The [Mediabunny Codec Registry](/codec-registry/overview) specifies the formats of `codec` and `description` for each audio codec, which you must adhere to.
|
||||
|
||||
## Subtitle sources
|
||||
|
||||
|
||||
@@ -94,6 +94,8 @@ constructor(
|
||||
);
|
||||
```
|
||||
|
||||
When creating a packet for a given codec, you *must* adhere to the data format specified in the [Mediabunny Codec Registry](/codec-registry/overview).
|
||||
|
||||
::: info
|
||||
You probably won't ever need to set `sequenceNumber` or `byteLength` in the constructor.
|
||||
:::
|
||||
|
||||
@@ -129,7 +129,7 @@ track.codec; // => MediaCodec | null
|
||||
```
|
||||
This field is `null` when the track's codec couldn't be recognized or is not supported by Mediabunny. See [Codecs](./supported-formats-and-codecs#codecs) for the full list of supported codecs. When Mediabunny doesn't recognize the format, you can still use the `internalCodecId` field to figure out the codec of the track, although its format depends on the container format used and is not homogenized by Mediabunny.
|
||||
|
||||
You can also extract the full codec parameter string from the track, as specified in the [WebCodecs Codec Registry](https://www.w3.org/TR/webcodecs-codec-registry/):
|
||||
You can also extract the full codec parameter string from the track, as specified in the [Mediabunny Codec Registry](/codec-registry/overview):
|
||||
```ts
|
||||
await track.getCodecParameterString(); // => 'avc1.42001f'
|
||||
```
|
||||
|
||||
@@ -21,6 +21,8 @@ Mediabunny supports a wide range of video, audio, and subtitle codecs. More spec
|
||||
|
||||
The availability of the codecs provided by the WebCodecs API depends on the browser and thus cannot be guaranteed by this library. Mediabunny provides [special utility functions](#querying-codec-encodability) to check which codecs are able to be encoded. You can also specify [custom coders](#custom-coders) to provide your own encoder/decoder implementation if the browser doesn't support the codec natively.
|
||||
|
||||
For precise definitions of each codec including the corresponding packet format, please refer to the [Mediabunny Codec Registry](/codec-registry/overview).
|
||||
|
||||
::: info
|
||||
Mediabunny ships with built-in decoders and encoders for all audio PCM codecs, meaning they are always supported.
|
||||
:::
|
||||
@@ -40,8 +42,8 @@ Mediabunny ships with built-in decoders and encoders for all audio PCM codecs, m
|
||||
- `'mp3'` - MP3
|
||||
- `'vorbis'` - Vorbis
|
||||
- `'flac'` - Free Lossless Audio Codec (FLAC)
|
||||
- `'ac3'` - Dolby Digital (AC-3)
|
||||
- `'eac3'` - Dolby Digital Plus (E-AC-3)
|
||||
- `'ac3'` - Dolby Digital (AC-3) [^1]
|
||||
- `'eac3'` - Dolby Digital Plus (E-AC-3) [^1]
|
||||
- `'pcm-u8'` - 8-bit unsigned PCM
|
||||
- `'pcm-s8'` - 8-bit signed PCM
|
||||
- `'pcm-s16'` - 16-bit little-endian signed PCM
|
||||
@@ -57,6 +59,8 @@ Mediabunny ships with built-in decoders and encoders for all audio PCM codecs, m
|
||||
- `'ulaw'` - μ-law PCM
|
||||
- `'alaw'` - A-law PCM
|
||||
|
||||
[^1]: AC-3 and E-AC-3 are not natively supported by WebCodecs. To encode or decode these codecs, you must provide a [custom coder](#custom-coders).
|
||||
|
||||
### Subtitle codecs
|
||||
|
||||
- `'webvtt'` - WebVTT
|
||||
@@ -65,7 +69,7 @@ Mediabunny ships with built-in decoders and encoders for all audio PCM codecs, m
|
||||
|
||||
Not all codecs can be used with all containers. The following table specifies the supported codec-container combinations:
|
||||
|
||||
| | .mp4 | .mov | .mkv | .webm[^1] | .ogg | .mp3 | .wav | .aac | .flac | .ts |
|
||||
| | .mp4 | .mov | .mkv | .webm[^2] | .ogg | .mp3 | .wav | .aac | .flac | .ts |
|
||||
|:--------------:|:--------:|:-----:|:-----:|:---------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|
||||
| `'avc'` | ✓ | ✓ | ✓ | | | | | | | ✓ |
|
||||
| `'hevc'` | ✓ | ✓ | ✓ | | | | | | | ✓ |
|
||||
@@ -77,8 +81,8 @@ Not all codecs can be used with all containers. The following table specifies th
|
||||
| `'mp3'` | ✓ | ✓ | ✓ | | | ✓ | | | | ✓ |
|
||||
| `'vorbis'` | ✓ | ✓ | ✓ | ✓ | ✓ | | | | | |
|
||||
| `'flac'` | ✓ | ✓ | ✓ | | | | | | ✓ | |
|
||||
| `'ac3'`[^2] | ✓ | ✓ | ✓ | | | | | | | ✓ |
|
||||
| `'eac3'`[^2] | ✓ | ✓ | ✓ | | | | | | | ✓ |
|
||||
| `'ac3'` | ✓ | ✓ | ✓ | | | | | | | ✓ |
|
||||
| `'eac3'` | ✓ | ✓ | ✓ | | | | | | | ✓ |
|
||||
| `'pcm-u8'` | | ✓ | ✓ | | | | ✓ | | | |
|
||||
| `'pcm-s8'` | | ✓ | | | | | | | | |
|
||||
| `'pcm-s16'` | ✓ | ✓ | ✓ | | | | ✓ | | | |
|
||||
@@ -96,8 +100,7 @@ Not all codecs can be used with all containers. The following table specifies th
|
||||
| `'webvtt'`[^3] | (✓) | | (✓) | (✓) | | | | | | |
|
||||
|
||||
|
||||
[^1]: WebM only supports a small subset of the codecs supported by Matroska. However, this library can technically read all codecs from a WebM that are supported by Matroska.
|
||||
[^2]: AC-3 and E-AC-3 are not natively supported by WebCodecs. To encode or decode these codecs, you must provide a [custom coder](#custom-coders).
|
||||
[^2]: WebM only supports a small subset of the codecs supported by Matroska. However, this library can technically read all codecs from a WebM that are supported by Matroska.
|
||||
[^3]: WebVTT can only be written, not read.
|
||||
|
||||
## Querying codec encodability
|
||||
|
||||
Reference in New Issue
Block a user