Compare commits

..
13 Commits
Author SHA1 Message Date
Vanilagy a46e9ee9ec Fix flaky FLAC metadata reading 2025-09-19 14:29:41 +02:00
Vanilagy 23181a9ff6 Also run tests in release script 2025-09-19 14:07:16 +02:00
Vanilagy 88f9b64331 Fix faultily written Xing frame 2025-09-19 14:03:46 +02:00
Vanilagy eb1ce86187 Fix faulty MP3 bitrate and frame size calculation (fixes #134) 2025-09-19 13:20:36 +02:00
Vanilagy e88bc15b0e Bump minor 2025-09-18 23:43:11 +02:00
420423bc0f feat: Add support for custom MP4 metadata with mdta format (#132)
* feat: Add support for custom MP4 metadata with mdta format

- Add metadataFormat option ('mdir' | 'mdta') to IsobmffOutputFormatOptions
- Implement complete mdta format with keys box generation
- Enable custom metadata fields readable by ffmpeg/ffprobe
- Maintain backward compatibility with default mdir format

* Refactor ISOBMFF metadata tag writing logic

* Autism

---------

Co-authored-by: tanfu <[email protected]>
Co-authored-by: Vanilagy <[email protected]>
2025-09-18 23:41:08 +02:00
d426d36386 FLAC container support (#95)
* add a test

* recognize as input format

* scaffold flac demuxer

* implement getting metadata

* Implement mime type

* read all metadata + deduplicate stubs

* Read first packet

* copyright headers

* read the first packet

* Get entire first packet, work on advancing

* iterate over all samples

* testable with bun

* stub out metadata support

* parse descriptive metadata

* All in 1 file

seems more appropriate to the philosophy

* some parameters are not needed anymore all within 1 class

* skip over bytes we are sure are not the syncword

* run prettier

* timestamp is determined based on passed blocks, not maximumBlockSize

* no binary search needed!

* simplifications

* Finish demuxer reading sequentially

* more tests + add a file with a seektable

* don't throw if (this.audioInfo.minimumBlockSize !== this.audioInfo.maximumBlockSize

* Add docs

* Update format compatibility table

* Simplification

* confirm conversion is working

* Finish

* Resolve TODO comment

* Support images (read-only)

* Returning description as Uint8Array

* Cleanup of demuxer

* Misc renames

* Object on same line

* Resolve first batch of comments

* Throw errors on corrupt blocks, correctly use requestSlice()

* Fix description field

* Explain why last frame is a bit shorter

* Add FLAC to README

* Put track backings below demuxer

* convert to methods

* Reorder container checking

* getBlockSize() -> readBlockSize()

* bitStream -> bitstream

* better naming for bytes

* use .skip()

* Don't return blockSize twice in readFlacFrameHeader

* Use enum + switch to distinguish Flac block types

* Use else-if

* Use else-if

* Compressed switch statement

* readCodedNumber

* Update flac-misc.ts

* We don't need the bits variable at all

* reorder functions in flac-demuxer

* Handle gracefully not being able to load another sample

* Update flac-demuxer.ts

* blockingbit null

* use async instead of promise.resolve

* use binary search

* Replace recursion with while loop

* Add mutex to getPacket()

* Load more data not in getPacketAtIndex, but outside

* Apply suggestion from @Vanilagy

Co-authored-by: David P. <[email protected]>

* computeDuration() reads last packet

* Update flac-demuxer.ts

* share vorbis comment reading logic

* reuse vorbis comment writing logic, set vendor always to "Mediabunny"

* flush after writign

* Use FileSlice.tempFromBytes

* assert !== null

* fixing nitpicks

* apply suggestions

* fix ogg

* We are now muxing images

* apply suggestion

* Update src/flac/flac-muxer.ts

Co-authored-by: David P. <[email protected]>

* apply suggestion

* readSampleRate()

* seek outside writeHeader()

* mention vorbis metadata +  add to metadatatags comment

* should be able to -> can

* Add test in metadata tags

* Add test for packets being byte identical after remuxing

* compare to null

* no casting to uint8array

* make test pass

* Update flac-muxer.ts

* Call validateAudioChunkMetadata() and validateAndNormalizeTimestamp()

* `onFrame` option

* emit frames using onFrame

* Run prettier over files

* Fix FLAC PICTURE block logic, small other changes

* Update docs

* Remove .only modifier

* fix remuxing and add test

* Don't throw error if parsing fails in header, since hitting a syncword might just be coincidential

* Fix remaining type errors

---------

Co-authored-by: David P. <[email protected]>
2025-09-18 21:05:09 +02:00
Vanilagy 7664d48f70 Merge branch 'main' of https://github.com/Vanilagy/mediabunny 2025-09-18 15:14:01 +02:00
Vanilagy 4fd90cf73b Add missing block scope condition 2025-09-18 15:14:01 +02:00
David P.andGitHub d0c0a9221f Merge pull request #128 from JonnyBurger/headless-mode
Enable headless mode for browser tests
2025-09-18 11:10:12 +02:00
Jonny Burger 4c313ad2a6 Enable headless mode for browser tests 2025-09-18 10:55:42 +02:00
Vanilagy f340e19baa Clarify docs 2025-09-18 10:11:27 +02:00
Vanilagy 407dd05953 Make Matroska block decoding lazy 2025-09-18 10:00:03 +02:00
38 changed files with 2473 additions and 610 deletions
+3
View File
@@ -46,6 +46,9 @@ jobs:
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm run test
- name: Run build
run: npm run build
+1 -1
View File
@@ -51,7 +51,7 @@ Mediabunny is a JavaScript library for reading, writing, and converting media fi
Core features include:
- **Wide format support**: Read and write MP4, MOV, WebM, MKV, WAVE, MP3, Ogg, ADTS
- **Wide format support**: Read and write MP4, MOV, WebM, MKV, WAVE, MP3, Ogg, ADTS, FLAC
- **Built-in encoding & decoding**: Supports 25+ video, audio, and subtitle codecs, hardware-accelerated using the WebCodecs API
- **High precision**: Fine-grained, microsecond-accurate reading and writing operations
- **Conversion API**: Easy-to-use API with features such as transmuxing, transcoding, resizing, rotation, cropping, resampling, trimming, and more
+7 -5
View File
@@ -24,7 +24,7 @@
chunked: true,
chunkSize: 2**20
});
const outputFormat = new Mediabunny.Mp4OutputFormat({});
const outputFormat = new Mediabunny.Mp3OutputFormat({});
const button = document.createElement('button');
button.textContent = 'Cancel';
@@ -54,7 +54,8 @@
source
}),
output,
audio: {
audio: (_, n) => ({
discard: n > 1,
//codec: 'pcm-s16',
//sampleRate: 16000,
//numberOfChannels: 1,
@@ -65,7 +66,7 @@
//sampleRate: 4000
//discard: true
//forceTranscode: true,
},
}),
/*
video: {
discard: true,
@@ -91,6 +92,7 @@
},
*/
video: () => ({
//discard: true,
//discard: true,
//crop: {
// left: 0,
@@ -142,8 +144,8 @@
}
},
trim: {
start: 10,
end: 20
//start: 10,
//end: 20
},
});
console.log(conversion);
+16
View File
@@ -14,6 +14,21 @@
source: new Mediabunny.BlobSource(file),
});
console.log(await input.computeDuration());
return;
const audioTrack = await input.getPrimaryAudioTrack();
const sink = new Mediabunny.EncodedPacketSink(audioTrack);
for await (const packet of sink.packets()) {
console.log(packet)
}
console.log("Done")
/*
const videoTrack = await input.getPrimaryVideoTrack();
const sink = new Mediabunny.VideoSampleSink(videoTrack);
const sink2 = new Mediabunny.EncodedPacketSink(videoTrack);
@@ -37,6 +52,7 @@
break;
}
}
*/
/*
const sink = new Mediabunny.EncodedPacketSink(videoTrack);
+4
View File
@@ -30,6 +30,8 @@ import {
MP3, // MP3 input format singleton
WAVE, // WAVE input format singleton
OGG, // Ogg input format singleton
ADTS, // ADTS input format singleton
FLAC, // FLAC input format singleton
} from 'mediabunny';
```
@@ -76,6 +78,8 @@ In addition to singletons, input format classes are structured hierarchically:
- `Mp3InputFormat`
- `WaveInputFormat`
- `OggInputFormat`
- `AdtsInputFormat`
- `FlacInputFormat`
This means you can also perform input format checks using `instanceof` instead of `===` comparisons. For example:
```ts
+1 -1
View File
@@ -12,7 +12,7 @@ Here's a long list of stuff this library does:
- Converting media files
- Hardware-accelerated decoding & encoding (via the WebCodecs API)
- Support for multiple video, audio and subtitle tracks
- Read & write support for many container formats (.mp4, .mov, .webm, .mkv, .mp3, .wav, .ogg, .aac), including variations such as MP4 with Fast Start, fragmented MP4, or streamable Matroska
- Read & write support for many container formats (.mp4, .mov, .webm, .mkv, .mp3, .wav, .ogg, .aac, .flac), including variations such as MP4 with Fast Start, fragmented MP4, or streamable Matroska
- Support for 25 different codecs
- Lazy, optimized, on-demand file reading
- Input and output streaming, arbitrary file size support
+29 -1
View File
@@ -64,6 +64,7 @@ The following options are available:
type IsobmffOutputFormatOptions = {
fastStart?: false | 'in-memory' | 'reserve' | 'fragmented';
minimumFragmentDuration?: number;
metadataFormat?: 'mdir' | 'mdta' | 'udta' | 'auto';
onFtyp?: (data: Uint8Array, position: number) => unknown;
onMoov?: (data: Uint8Array, position: number) => unknown;
@@ -94,6 +95,12 @@ type IsobmffOutputFormatOptions = {
The default option; it behaves like `'in-memory'` when using [`BufferTarget`](./writing-media-files#buffertarget) and like `false` otherwise.
- `minimumFragmentDuration`\
Only relevant when `fastStart` is `'fragmented'`. Sets the minimum duration in seconds a fragment must have to be finalized and written to the file. Defaults to 1 second.
- `metadataFormat`\
The metadata format to use for writing metadata tags.
- `'auto'` (default): Behaves like `'mdir'` for MP4 and like `'udta'` for QuickTime, matching FFmpeg's default behavior.
- `'mdir'`: Write tags into `moov/udta/meta` using the 'mdir' handler format.
- `'mdta'`: Write tags into `moov/udta/meta` using the 'mdta' handler format, equivalent to FFmpeg's `use_metadata_tags` flag. This allows for custom keys of arbitrary length.
- `'udta'`: Write tags directly into `moov/udta`.
- `onFtyp`\
Will be called once the ftyp (File Type) box of the output file has been written.
- `onMoov`\
@@ -267,4 +274,25 @@ type AdtsOutputFormatOptions = {
};
```
- `onFrame`\
Will be called for each ADTS frame that is written.
Will be called for each ADTS frame that is written.
## FLAC
This output format creates FLAC (.flac) files.
```ts
import { Output, FlacOutputFormat } from 'mediabunny';
const output = new Output({
format: new FlacOutputFormat(options),
// ...
});
```
The following options are available:
```ts
type FlacOutputFormatOptions = {
onFrame?: (data: Uint8Array, position: number) => unknown;
};
```
- `onFrame`\
Will be called for each FLAC frame that is written.
+28 -27
View File
@@ -12,6 +12,7 @@ Mediabunny supports many commonly used media container formats, all of which are
- MP3 (.mp3)
- WAVE (.wav)
- ADTS (.aac)
- FLAC (.flac)
## Codecs
@@ -61,33 +62,33 @@ Mediabunny ships with built-in decoders and encoders for all audio PCM codecs, m
Not all codecs can be used with all containers. The following table specifies the supported codec-container combinations:
| | .mp4 | .mov | .mkv | .webm[^1] | .ogg | .mp3 | .wav | .aac |
|:--------------:|:--------:|:-----:|:-----:|:---------:|:-----:|:-----:|:-----:|:-----:|
| `'avc'` | ✓ | ✓ | ✓ | | | | | |
| `'hevc'` | ✓ | ✓ | ✓ | | | | | |
| `'vp8'` | ✓ | ✓ | ✓ | ✓ | | | | |
| `'vp9'` | ✓ | ✓ | ✓ | ✓ | | | | |
| `'av1'` | ✓ | ✓ | ✓ | ✓ | | | | |
| `'aac'` | ✓ | ✓ | ✓ | | | | | ✓ |
| `'opus'` | ✓ | ✓ | ✓ | ✓ | ✓ | | | |
| `'mp3'` | ✓ | ✓ | ✓ | | | ✓ | | |
| `'vorbis'` | ✓ | ✓ | ✓ | ✓ | ✓ | | | |
| `'flac'` | ✓ | ✓ | ✓ | | | | | |
| `'pcm-u8'` | | ✓ | ✓ | | | | ✓ | |
| `'pcm-s8'` | | ✓ | | | | | | |
| `'pcm-s16'` | ✓ | ✓ | ✓ | | | | ✓ | |
| `'pcm-s16be'` | ✓ | ✓ | ✓ | | | | | |
| `'pcm-s24'` | ✓ | ✓ | ✓ | | | | ✓ | |
| `'pcm-s24be'` | ✓ | ✓ | ✓ | | | | | |
| `'pcm-s32'` | ✓ | ✓ | ✓ | | | | ✓ | |
| `'pcm-s32be'` | ✓ | ✓ | ✓ | | | | | |
| `'pcm-f32'` | ✓ | ✓ | ✓ | | | | ✓ | |
| `'pcm-f32be'` | ✓ | ✓ | | | | | | |
| `'pcm-f64'` | ✓ | ✓ | ✓ | | | | | |
| `'pcm-f64be'` | ✓ | ✓ | | | | | | |
| `'ulaw'` | | ✓ | | | | | ✓ | |
| `'alaw'` | | ✓ | | | | | ✓ | |
| `'webvtt'`[^2] | (✓) | | (✓) | (✓) | | | | |
| | .mp4 | .mov | .mkv | .webm[^1] | .ogg | .mp3 | .wav | .aac | .flac |
|:--------------:|:--------:|:-----:|:-----:|:---------:|:-----:|:-----:|:-----:|:-----:|:-----:|
| `'avc'` | ✓ | ✓ | ✓ | | | | | | |
| `'hevc'` | ✓ | ✓ | ✓ | | | | | | |
| `'vp8'` | ✓ | ✓ | ✓ | ✓ | | | | | |
| `'vp9'` | ✓ | ✓ | ✓ | ✓ | | | | | |
| `'av1'` | ✓ | ✓ | ✓ | ✓ | | | | | |
| `'aac'` | ✓ | ✓ | ✓ | | | | | ✓ | |
| `'opus'` | ✓ | ✓ | ✓ | ✓ | ✓ | | | | |
| `'mp3'` | ✓ | ✓ | ✓ | | | ✓ | | | |
| `'vorbis'` | ✓ | ✓ | ✓ | ✓ | ✓ | | | | |
| `'flac'` | ✓ | ✓ | ✓ | | | | | | ✓ |
| `'pcm-u8'` | | ✓ | ✓ | | | | ✓ | | |
| `'pcm-s8'` | | ✓ | | | | | | | |
| `'pcm-s16'` | ✓ | ✓ | ✓ | | | | ✓ | | |
| `'pcm-s16be'` | ✓ | ✓ | ✓ | | | | | | |
| `'pcm-s24'` | ✓ | ✓ | ✓ | | | | ✓ | | |
| `'pcm-s24be'` | ✓ | ✓ | ✓ | | | | | | |
| `'pcm-s32'` | ✓ | ✓ | ✓ | | | | ✓ | | |
| `'pcm-s32be'` | ✓ | ✓ | ✓ | | | | | | |
| `'pcm-f32'` | ✓ | ✓ | ✓ | | | | ✓ | | |
| `'pcm-f32be'` | ✓ | ✓ | | | | | | | |
| `'pcm-f64'` | ✓ | ✓ | ✓ | | | | | | |
| `'pcm-f64be'` | ✓ | ✓ | | | | | | | |
| `'ulaw'` | | ✓ | | | | | ✓ | | |
| `'alaw'` | | ✓ | | | | | ✓ | | |
| `'webvtt'`[^2] | (✓) | | (✓) | (✓) | | | | | |
[^1]: WebM only supports a small subset of the codecs supported by Matroska. However, this library can technically read all codecs from a WebM that are supported by Matroska.
+2
View File
@@ -123,6 +123,8 @@ output.setMetadataTags({
});
```
If you add metadata tags, you must call this method *before* starting the `Output`.
For more info on which tags you can write, see [`MetadataTags`](../api/MetadataTags).
## Starting an output
+6 -6
View File
@@ -1,12 +1,12 @@
{
"name": "mediabunny",
"version": "1.16.0",
"version": "1.17.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "mediabunny",
"version": "1.16.0",
"version": "1.17.3",
"license": "MPL-2.0",
"workspaces": [
"packages/*"
@@ -7749,9 +7749,9 @@
}
},
"node_modules/mediabunny": {
"version": "1.15.2",
"resolved": "https://registry.npmjs.org/mediabunny/-/mediabunny-1.15.2.tgz",
"integrity": "sha512-OvIuSK10wNdz+7Dd4bmeq/IWkYG0bkm33FQb29vmyDztpr47ru8XSxYansa3EHLy6lLEm4rUADUENOcmrnN0jg==",
"version": "1.17.2",
"resolved": "https://registry.npmjs.org/mediabunny/-/mediabunny-1.17.2.tgz",
"integrity": "sha512-I7l/WDv7howETh0Vhc9J8Y0XZ1Z1qUpUEcud05Eqf/MeaaOT8FGb744sTXN6U4Rk5ZxFYRz7g3JHcA1XvsFkWg==",
"license": "MPL-2.0",
"peer": true,
"workspaces": [
@@ -12242,7 +12242,7 @@
},
"packages/mp3-encoder": {
"name": "@mediabunny/mp3-encoder",
"version": "1.16.0",
"version": "1.17.3",
"license": "MPL-2.0",
"devDependencies": {
"@types/emscripten": "^1.40.1"
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "mediabunny",
"author": "Vanilagy",
"version": "1.16.0",
"version": "1.17.3",
"description": "Pure TypeScript media toolkit for reading, writing, and converting media files, directly in the browser.",
"type": "module",
"workspaces": [
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@mediabunny/mp3-encoder",
"author": "Vanilagy",
"version": "1.16.0",
"version": "1.17.3",
"description": "MP3 encoder extension for Mediabunny, based on LAME.",
"main": "./dist/bundles/mediabunny-mp3-encoder.mjs",
"module": "./dist/bundles/mediabunny-mp3-encoder.mjs",
+3 -1
View File
@@ -28,7 +28,9 @@ class Mp3Encoder extends CustomAudioEncoder {
static override supports(codec: AudioCodec, config: AudioDecoderConfig): boolean {
return codec === 'mp3'
&& (config.numberOfChannels === 1 || config.numberOfChannels === 2)
&& Object.values(SAMPLING_RATES).some(x => x.includes(config.sampleRate));
&& Object.values(SAMPLING_RATES).some(x =>
x === config.sampleRate || (x / 2) === config.sampleRate || (x / 4) === config.sampleRate,
);
}
async init() {
+43 -43
View File
@@ -7,32 +7,20 @@
*/
export const FRAME_HEADER_SIZE = 4;
export const SAMPLING_RATES = [44100, 48000, 32000];
export const KILOBIT_RATES = [
// lowSamplingFrequency === 0
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, // layer = 0
-1, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, -1, // layer 1
-1, 32, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, -1, // layer = 2
-1, 32, 64, 96, 128, 160, 192, 224, 256, 288, 320, 352, 384, 416, 448, -1, // layer = 3
// These are in kbps:
export const MPEG_V1_BITRATES: Record<number, number[]> = {
// Layer 3
1: [-1, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, -1],
// Layer 2
2: [-1, 32, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, -1],
// Layer 1
3: [-1, 32, 64, 96, 128, 160, 192, 224, 256, 288, 320, 352, 384, 416, 448, -1],
};
export const MPEG_V2_BITRATES: Record<number, number[]> = {
// Layer 3
1: [-1, 32, 48, 56, 64, 80, 96, 112, 128, 144, 160, 176, 192, 224, 256, -1],
// Layer 2
2: [-1, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160, -1],
// Layer 1
3: [-1, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160, -1],
};
export const SAMPLING_RATES: Record<number, number[]> = {
// MPEG Version 2.5
0: [11025, 12000, 8000, -1],
// MPEG Version 2 (ISO/IEC 13818-3)
2: [22050, 24000, 16000, -1],
// MPEG Version 1 (ISO/IEC 11172-3)
3: [44100, 48000, 32000, -1],
};
// lowSamplingFrequency === 1
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, // layer = 0
-1, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160, -1, // layer = 1
-1, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160, -1, // layer = 2
-1, 32, 48, 56, 64, 80, 96, 112, 128, 144, 160, 176, 192, 224, 256, -1, // layer = 3
];
/** 'Xing' */
export const XING = 0x58696e67;
@@ -54,12 +42,21 @@ export type FrameHeader = {
audioSamplesInFrame: number;
};
export const computeMp3FrameSize = (layer: number, bitrate: number, sampleRate: number, padding: number) => {
if (layer === 3) {
// Layer 1
return Math.floor((12 * bitrate / sampleRate + padding) * 4);
} else {
return Math.floor((144 * bitrate / sampleRate) + padding);
export const computeMp3FrameSize = (
lowSamplingFrequency: number,
layer: number,
bitrate: number,
sampleRate: number,
padding: number,
) => {
if (layer === 0) {
return 0; // Not expected that this is hit
} else if (layer === 1) {
return Math.round(144 * bitrate / (sampleRate << lowSamplingFrequency)) + padding;
} else if (layer === 2) {
return Math.round(144 * bitrate / sampleRate) + padding;
} else { // layer === 3
return (Math.round(12 * bitrate / sampleRate) + padding) * 4;
}
};
@@ -93,11 +90,21 @@ export const readFrameHeader = (word: number, remainingBytes: number | null): {
return { header: null, bytesAdvanced: 1 };
}
let lowSamplingFrequency = 0;
let mpeg25 = 0;
if (secondByte & (1 << 4)) {
lowSamplingFrequency = (secondByte & (1 << 3)) ? 0 : 1;
} else {
lowSamplingFrequency = 1;
mpeg25 = 1;
}
const mpegVersionId = (secondByte >> 3) & 0x3;
const layer = (secondByte >> 1) & 0x3;
const bitrateIndex = (thirdByte >> 4) & 0xf;
const frequencyIndex = (thirdByte >> 2) & 0x3;
const frequencyIndex = ((thirdByte >> 2) & 0x3) % 3;
const padding = (thirdByte >> 1) & 0x1;
const channel = (fourthByte >> 6) & 0x3;
@@ -106,21 +113,14 @@ export const readFrameHeader = (word: number, remainingBytes: number | null): {
const original = (fourthByte >> 2) & 0x1;
const emphasis = fourthByte & 0x3;
const kilobitRate = mpegVersionId === 3
? MPEG_V1_BITRATES[layer]?.[bitrateIndex]
: MPEG_V2_BITRATES[layer]?.[bitrateIndex];
if (!kilobitRate || kilobitRate === -1) {
const kilobitRate = KILOBIT_RATES[lowSamplingFrequency * 16 * 4 + layer * 16 + bitrateIndex]!;
if (kilobitRate === -1) {
return { header: null, bytesAdvanced: 1 };
}
const bitrate = kilobitRate * 1000;
const sampleRate = SAMPLING_RATES[mpegVersionId]?.[frequencyIndex];
if (!sampleRate || sampleRate === -1) {
return { header: null, bytesAdvanced: 1 };
}
const frameLength = computeMp3FrameSize(layer, bitrate, sampleRate, padding);
const sampleRate = SAMPLING_RATES[frequencyIndex]! >> (lowSamplingFrequency + mpeg25);
const frameLength = computeMp3FrameSize(lowSamplingFrequency, layer, bitrate, sampleRate, padding);
if (remainingBytes !== null && remainingBytes < frameLength) {
// The frame doesn't fit into the rest of the file
+347 -1
View File
@@ -11,14 +11,21 @@ import { InputVideoTrack } from './input-track';
import {
assert,
assertNever,
base64ToBytes,
Bitstream,
bytesToBase64,
keyValueIterator,
getUint24,
last,
readExpGolomb,
readSignedExpGolomb,
textDecoder,
textEncoder,
toDataView,
toUint8Array,
} from './misc';
import { EncodedPacket, PacketType } from './packet';
import { MetadataTags } from './tags';
// References for AVC/HEVC code:
// ISO 14496-15
@@ -116,7 +123,7 @@ const findNalUnitsInLengthPrefixed = (packetData: Uint8Array, lengthSize: 1 | 2
} else if (lengthSize === 2) {
nalUnitLength = dataView.getUint16(offset, false);
} else if (lengthSize === 3) {
nalUnitLength = (dataView.getUint16(offset, false) << 8) + dataView.getUint8(offset + 2);
nalUnitLength = getUint24(dataView, offset, false);
} else if (lengthSize === 4) {
nalUnitLength = dataView.getUint32(offset, false);
} else {
@@ -1571,3 +1578,342 @@ export const determineVideoPacketType = async (
};
}
};
export enum FlacBlockType {
STREAMINFO = 0,
VORBIS_COMMENT = 4,
PICTURE = 6,
}
export const readVorbisComments = (bytes: Uint8Array, metadataTags: MetadataTags) => {
// https://datatracker.ietf.org/doc/html/rfc7845#section-5.2
const commentView = toDataView(bytes);
let commentPos = 0;
const vendorStringLength = commentView.getUint32(commentPos, true);
commentPos += 4;
const vendorString = textDecoder.decode(
bytes.subarray(commentPos, commentPos + vendorStringLength),
);
commentPos += vendorStringLength;
if (vendorStringLength > 0) {
// Expose the vendor string in the raw metadata
metadataTags.raw ??= {};
metadataTags.raw['vendor'] ??= vendorString;
}
const listLength = commentView.getUint32(commentPos, true);
commentPos += 4;
// Loop over all metadata tags
for (let i = 0; i < listLength; i++) {
const stringLength = commentView.getUint32(commentPos, true);
commentPos += 4;
const string = textDecoder.decode(
bytes.subarray(commentPos, commentPos + stringLength),
);
commentPos += stringLength;
const separatorIndex = string.indexOf('=');
if (separatorIndex === -1) {
continue;
}
const key = string.slice(0, separatorIndex).toUpperCase();
const value = string.slice(separatorIndex + 1);
metadataTags.raw ??= {};
metadataTags.raw[key] ??= value;
switch (key) {
case 'TITLE': {
metadataTags.title ??= value;
}; break;
case 'DESCRIPTION': {
metadataTags.description ??= value;
}; break;
case 'ARTIST': {
metadataTags.artist ??= value;
}; break;
case 'ALBUM': {
metadataTags.album ??= value;
}; break;
case 'ALBUMARTIST': {
metadataTags.albumArtist ??= value;
}; break;
case 'COMMENT': {
metadataTags.comment ??= value;
}; break;
case 'LYRICS': {
metadataTags.lyrics ??= value;
}; break;
case 'TRACKNUMBER': {
const parts = value.split('/');
const trackNum = Number.parseInt(parts[0]!, 10);
const tracksTotal = parts[1] && Number.parseInt(parts[1], 10);
if (Number.isInteger(trackNum) && trackNum > 0) {
metadataTags.trackNumber ??= trackNum;
}
if (tracksTotal && Number.isInteger(tracksTotal) && tracksTotal > 0) {
metadataTags.tracksTotal ??= tracksTotal;
}
}; break;
case 'TRACKTOTAL': {
const tracksTotal = Number.parseInt(value, 10);
if (Number.isInteger(tracksTotal) && tracksTotal > 0) {
metadataTags.tracksTotal ??= tracksTotal;
}
}; break;
case 'DISCNUMBER': {
const parts = value.split('/');
const discNum = Number.parseInt(parts[0]!, 10);
const discsTotal = parts[1] && Number.parseInt(parts[1], 10);
if (Number.isInteger(discNum) && discNum > 0) {
metadataTags.discNumber ??= discNum;
}
if (discsTotal && Number.isInteger(discsTotal) && discsTotal > 0) {
metadataTags.discsTotal ??= discsTotal;
}
}; break;
case 'DISCTOTAL': {
const discsTotal = Number.parseInt(value, 10);
if (Number.isInteger(discsTotal) && discsTotal > 0) {
metadataTags.discsTotal ??= discsTotal;
}
}; break;
case 'DATE': {
const date = new Date(value);
if (!Number.isNaN(date.getTime())) {
metadataTags.date ??= date;
}
}; break;
case 'GENRE': {
metadataTags.genre ??= value;
}; break;
case 'METADATA_BLOCK_PICTURE': {
// https://datatracker.ietf.org/doc/rfc9639/ Section 8.8
const decoded = base64ToBytes(value);
const view = toDataView(decoded);
const pictureType = view.getUint32(0, false);
const mediaTypeLength = view.getUint32(4, false);
const mediaType = String.fromCharCode(...decoded.subarray(8, 8 + mediaTypeLength)); // ASCII
const descriptionLength = view.getUint32(8 + mediaTypeLength, false);
const description = textDecoder.decode(decoded.subarray(
12 + mediaTypeLength,
12 + mediaTypeLength + descriptionLength,
));
const dataLength = view.getUint32(mediaTypeLength + descriptionLength + 28);
const data = decoded.subarray(
mediaTypeLength + descriptionLength + 32,
mediaTypeLength + descriptionLength + 32 + dataLength,
);
metadataTags.images ??= [];
metadataTags.images.push({
data,
mimeType: mediaType,
kind: pictureType === 3 ? 'coverFront' : pictureType === 4 ? 'coverBack' : 'unknown',
name: undefined,
description: description || undefined,
});
}; break;
}
}
};
export const createVorbisComments = (headerBytes: Uint8Array, tags: MetadataTags, writeImages: boolean) => {
// https://datatracker.ietf.org/doc/html/rfc7845#section-5.2
const commentHeaderParts: Uint8Array[] = [
headerBytes,
];
const vendorString = 'Mediabunny';
const encodedVendorString = textEncoder.encode(vendorString);
let currentBuffer = new Uint8Array(4 + encodedVendorString.length);
let currentView = new DataView(currentBuffer.buffer);
currentView.setUint32(0, encodedVendorString.length, true);
currentBuffer.set(encodedVendorString, 4);
commentHeaderParts.push(currentBuffer);
const writtenTags = new Set<string>();
const addCommentTag = (key: string, value: string) => {
const joined = `${key}=${value}`;
const encoded = textEncoder.encode(joined);
currentBuffer = new Uint8Array(4 + encoded.length);
currentView = new DataView(currentBuffer.buffer);
currentView.setUint32(0, encoded.length, true);
currentBuffer.set(encoded, 4);
commentHeaderParts.push(currentBuffer);
writtenTags.add(key);
};
for (const { key, value } of keyValueIterator(tags)) {
switch (key) {
case 'title': {
addCommentTag('TITLE', value);
}; break;
case 'description': {
addCommentTag('DESCRIPTION', value);
}; break;
case 'artist': {
addCommentTag('ARTIST', value);
}; break;
case 'album': {
addCommentTag('ALBUM', value);
}; break;
case 'albumArtist': {
addCommentTag('ALBUMARTIST', value);
}; break;
case 'genre': {
addCommentTag('GENRE', value);
}; break;
case 'date': {
const rawVersion = tags.raw?.['DATE'] ?? tags.raw?.['date'];
if (rawVersion && typeof rawVersion === 'string') {
addCommentTag('DATE', rawVersion);
} else {
addCommentTag('DATE', value.toISOString().slice(0, 10));
}
}; break;
case 'comment': {
addCommentTag('COMMENT', value);
}; break;
case 'lyrics': {
addCommentTag('LYRICS', value);
}; break;
case 'trackNumber': {
addCommentTag('TRACKNUMBER', value.toString());
}; break;
case 'tracksTotal': {
addCommentTag('TRACKTOTAL', value.toString());
}; break;
case 'discNumber': {
addCommentTag('DISCNUMBER', value.toString());
}; break;
case 'discsTotal': {
addCommentTag('DISCTOTAL', value.toString());
}; break;
case 'images': {
// For example, in .flac, we put the pictures in a different section,
// not in the Vorbis comment header.
if (!writeImages) {
break;
}
for (const image of value) {
// https://datatracker.ietf.org/doc/rfc9639/ Section 8.8
const pictureType = image.kind === 'coverFront' ? 3 : image.kind === 'coverBack' ? 4 : 0;
const encodedMediaType = new Uint8Array(image.mimeType.length);
for (let i = 0; i < image.mimeType.length; i++) {
encodedMediaType[i] = image.mimeType.charCodeAt(i);
}
const encodedDescription = textEncoder.encode(image.description ?? '');
const buffer = new Uint8Array(
4 // Picture type
+ 4 // MIME type length
+ encodedMediaType.length // MIME type
+ 4 // Description length
+ encodedDescription.length // Description
+ 16 // Width, height, color depth, number of colors
+ 4 // Picture data length
+ image.data.length, // Picture data
);
const view = toDataView(buffer);
view.setUint32(0, pictureType, false);
view.setUint32(4, encodedMediaType.length, false);
buffer.set(encodedMediaType, 8);
view.setUint32(8 + encodedMediaType.length, encodedDescription.length, false);
buffer.set(encodedDescription, 12 + encodedMediaType.length);
// Skip a bunch of fields (width, height, color depth, number of colors)
view.setUint32(
28 + encodedMediaType.length + encodedDescription.length, image.data.length, false,
);
buffer.set(
image.data,
32 + encodedMediaType.length + encodedDescription.length,
);
const encoded = bytesToBase64(buffer);
addCommentTag('METADATA_BLOCK_PICTURE', encoded);
}
}; break;
case 'raw': {
// Handled later
}; break;
default: assertNever(key);
}
}
if (tags.raw) {
for (const key in tags.raw) {
const value = tags.raw[key] ?? tags.raw[key.toLowerCase()];
if (key === 'vendor' || value == null || writtenTags.has(key)) {
continue;
}
if (typeof value === 'string') {
addCommentTag(key, value);
}
}
}
const listLengthBuffer = new Uint8Array(4);
toDataView(listLengthBuffer).setUint32(0, writtenTags.size, true);
commentHeaderParts.splice(2, 0, listLengthBuffer); // Insert after the header and vendor section
// Merge all comment header parts into a single buffer
const commentHeaderLength = commentHeaderParts.reduce((a, b) => a + b.length, 0);
const commentHeader = new Uint8Array(commentHeaderLength);
let pos = 0;
for (const part of commentHeaderParts) {
commentHeader.set(part, pos);
pos += part.length;
}
return commentHeader;
};
+699
View File
@@ -0,0 +1,699 @@
/*!
* Copyright (c) 2025-present, Vanilagy and contributors
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at https://mozilla.org/MPL/2.0/.
*/
import { FlacBlockType, readVorbisComments } from '../codec-data';
import { Demuxer } from '../demuxer';
import { Input } from '../input';
import { InputAudioTrack, InputAudioTrackBacking } from '../input-track';
import { PacketRetrievalOptions } from '../media-sink';
import {
assert,
AsyncMutex,
binarySearchLessOrEqual,
Bitstream,
textDecoder,
UNDETERMINED_LANGUAGE,
} from '../misc';
import { EncodedPacket, PLACEHOLDER_DATA } from '../packet';
import {
FileSlice,
readBytes,
Reader,
readU24Be,
readU32Be,
readU8,
} from '../reader';
import { MetadataTags } from '../tags';
import {
calculateCrc8,
readBlockSize,
getBlockSizeOrUncommon,
readCodedNumber,
readSampleRate,
getSampleRateOrUncommon,
} from './flac-misc';
type FlacAudioInfo = {
numberOfChannels: number;
sampleRate: number;
totalSamples: number;
minimumBlockSize: number;
maximumBlockSize: number;
minimumFrameSize: number;
maximumFrameSize: number;
description: Uint8Array;
};
type Sample = {
blockOffset: number;
blockSize: number;
byteOffset: number;
byteSize: number;
};
type NextFlacFrameResult = {
num: number;
blockSize: number;
sampleRate: number;
size: number;
isLastFrame: boolean;
};
export class FlacDemuxer extends Demuxer {
reader: Reader;
loadedSamples: Sample[] = []; // All samples from the start of the file to lastLoadedPos
metadataPromise: Promise<void> | null = null;
track: InputAudioTrack | null = null;
metadataTags: MetadataTags = {};
audioInfo: FlacAudioInfo | null = null;
lastLoadedPos: number | null = null;
blockingBit: number | null = null;
readingMutex = new AsyncMutex();
lastSampleLoaded = false;
constructor(input: Input) {
super(input);
this.reader = input._reader;
}
override async computeDuration(): Promise<number> {
await this.readMetadata();
assert(this.track);
return this.track.computeDuration();
}
override async getMetadataTags(): Promise<MetadataTags> {
await this.readMetadata();
return this.metadataTags;
}
async getTracks() {
await this.readMetadata();
assert(this.track);
return [this.track];
}
async getMimeType() {
return 'audio/flac';
}
async readMetadata() {
let currentPos = 4; // Skip 'fLaC'
return (this.metadataPromise ??= (async () => {
while (
this.reader.fileSize === null
|| currentPos < this.reader.fileSize
) {
let sizeSlice = this.reader.requestSlice(currentPos, 4);
if (sizeSlice instanceof Promise) sizeSlice = await sizeSlice;
currentPos += 4;
if (sizeSlice === null) {
throw new Error(
`Metadata block at position ${currentPos} is too small! Corrupted file.`,
);
}
assert(sizeSlice);
const byte = readU8(sizeSlice); // first bit: isLastMetadata, remaining 7 bits: metaBlockType
const size = readU24Be(sizeSlice);
const isLastMetadata = (byte & 0x80) !== 0;
const metaBlockType = byte & 0x7f;
switch (metaBlockType) {
case FlacBlockType.STREAMINFO: {
// Parse streaminfo block
// https://www.rfc-editor.org/rfc/rfc9639.html#section-8.2
let streamInfoBlock = this.reader.requestSlice(
currentPos,
size,
);
if (streamInfoBlock instanceof Promise) streamInfoBlock = await streamInfoBlock;
assert(streamInfoBlock);
if (streamInfoBlock === null) {
throw new Error(
`StreamInfo block at position ${currentPos} is too small! Corrupted file.`,
);
}
const streamInfoBytes = readBytes(streamInfoBlock, 34);
const bitstream = new Bitstream(streamInfoBytes);
const minimumBlockSize = bitstream.readBits(16);
const maximumBlockSize = bitstream.readBits(16);
const minimumFrameSize = bitstream.readBits(24);
const maximumFrameSize = bitstream.readBits(24);
const sampleRate = bitstream.readBits(20);
const numberOfChannels = bitstream.readBits(3) + 1;
bitstream.readBits(5); // bitsPerSample - 1
const totalSamples = bitstream.readBits(36);
// https://www.w3.org/TR/webcodecs-flac-codec-registration/#audiodecoderconfig-description
// description is required, and has to be the following:
// 1. The bytes 0x66 0x4C 0x61 0x43 ("fLaC" in ASCII)
// 2. A metadata block (called the STREAMINFO block) as described in section 7 of [FLAC]
// 3. Optionaly (sic) other metadata blocks, that are not used by the specification
bitstream.skipBits(16 * 8); // md5 hash
const description = new Uint8Array(42);
// 1. "fLaC"
description.set(new Uint8Array([0x66, 0x4c, 0x61, 0x43]), 0);
// 2. STREAMINFO block
description.set(new Uint8Array([128, 0, 0, 34]), 4);
// 3. Other metadata blocks
description.set(streamInfoBytes, 8);
this.audioInfo = {
numberOfChannels,
sampleRate,
totalSamples,
minimumBlockSize,
maximumBlockSize,
minimumFrameSize,
maximumFrameSize,
description,
};
this.track = new InputAudioTrack(new FlacAudioTrackBacking(this));
break;
}
case FlacBlockType.VORBIS_COMMENT: {
// Parse vorbis comment block
// https://www.rfc-editor.org/rfc/rfc9639.html#name-vorbis-comment
let vorbisCommentBlock = this.reader.requestSlice(
currentPos,
size,
);
if (vorbisCommentBlock instanceof Promise) vorbisCommentBlock = await vorbisCommentBlock;
assert(vorbisCommentBlock);
readVorbisComments(
readBytes(vorbisCommentBlock, size),
this.metadataTags,
);
break;
}
case FlacBlockType.PICTURE: {
// Parse picture block
// https://www.rfc-editor.org/rfc/rfc9639.html#name-picture
let pictureBlock = this.reader.requestSlice(
currentPos,
size,
);
if (pictureBlock instanceof Promise) pictureBlock = await pictureBlock;
assert(pictureBlock);
const pictureType = readU32Be(pictureBlock);
const mediaTypeLength = readU32Be(pictureBlock);
const mediaType = textDecoder.decode(
readBytes(pictureBlock, mediaTypeLength),
);
const descriptionLength = readU32Be(pictureBlock);
const description = textDecoder.decode(
readBytes(pictureBlock, descriptionLength),
);
pictureBlock.skip(4 + 4 + 4 + 4); // Skip width, height, color depth, number of indexed colors
const dataLength = readU32Be(pictureBlock);
const data = readBytes(pictureBlock, dataLength);
this.metadataTags.images ??= [];
this.metadataTags.images.push({
data,
mimeType: mediaType,
// https://www.rfc-editor.org/rfc/rfc9639.html#table13
kind:
pictureType === 3
? 'coverFront'
: pictureType === 4
? 'coverBack'
: 'unknown',
description,
});
break;
}
default:
break;
}
currentPos += size;
if (isLastMetadata) {
this.lastLoadedPos = currentPos;
break;
}
}
})());
}
async readNextFlacFrame({
startPos,
isFirstPacket,
}: {
startPos: number;
isFirstPacket: boolean;
}): Promise<NextFlacFrameResult | null> {
assert(this.audioInfo);
// we expect that there are at least `minimumFrameSize` bytes left in the file
// Ideally we also want to validate the next header is valid
// to throw out an accidential sync word
// The shortest valid FLAC header I can think of, based off the code
// of readFlacFrameHeader:
// 4 bytes used for bitstream from syncword to bit depth
// 1 byte coded number
// (uncommon values, no bytes read)
// 1 byte crc
// --> 6 bytes
const minimumHeaderLength = 6;
// If we read everything in readFlacFrameHeader, we read 16 bytes
const maximumHeaderSize = 16;
const maximumSliceLength
= this.audioInfo.maximumFrameSize + maximumHeaderSize;
const slice = await this.reader.requestSliceRange(
startPos,
this.audioInfo.minimumFrameSize,
maximumSliceLength,
);
if (!slice) {
return null;
}
const frameHeader = this.readFlacFrameHeader({
slice,
isFirstPacket: isFirstPacket,
});
if (!frameHeader) {
return null;
}
// We don't know exactly how long the packet is, we only know the `minimumFrameSize` and `maximumFrameSize`
// The packet is over if the next 2 bytes are the sync word followed by a valid header
// or the end of the file is reached
// The next sync word is expected at earliest when `minimumFrameSize` is reached,
// we can skip over anything before that
slice.filePos = startPos + this.audioInfo.minimumFrameSize;
while (true) {
// Reached end of the file, packet is over
if (slice.filePos > slice.end - minimumHeaderLength) {
return {
num: frameHeader.num,
blockSize: frameHeader.blockSize,
sampleRate: frameHeader.sampleRate,
size: slice.end - startPos,
isLastFrame: true,
};
}
const nextByte = readU8(slice);
if (nextByte === 0xff) {
const byteAfterNextByte = readU8(slice);
const expected = this.blockingBit === 1 ? 0b1111_1001 : 0b1111_1000;
if (byteAfterNextByte !== expected) {
slice.skip(-1);
continue;
}
slice.skip(-2);
const lengthIfNextFlacFrameHeaderIsLegit = slice.filePos - startPos;
const nextIsLegit = this.readFlacFrameHeader({
slice,
isFirstPacket: false,
});
if (!nextIsLegit) {
slice.skip(-1);
continue;
}
return {
num: frameHeader.num,
blockSize: frameHeader.blockSize,
sampleRate: frameHeader.sampleRate,
size: lengthIfNextFlacFrameHeaderIsLegit,
isLastFrame: false,
};
}
}
}
readFlacFrameHeader({
slice,
isFirstPacket,
}: {
slice: FileSlice;
isFirstPacket: boolean;
}) {
// In this function, generally it is not safe to throw errors.
// We might end up here because we stumbled upon a syncword,
// but the data might not actually be a FLAC frame, it might be random bitstream
// data, in that case we should return null and continue.
const startOffset = slice.filePos;
// https://www.rfc-editor.org/rfc/rfc9639.html#section-9.1
// Each frame MUST start on a byte boundary and start with the 15-bit frame
// sync code 0b111111111111100. Following the sync code is the blocking strategy
// bit, which MUST NOT change during the audio stream.
const bytes = readBytes(slice, 4);
const bitstream = new Bitstream(bytes);
const bits = bitstream.readBits(15);
if (bits !== 0b111111111111100) {
// This cannot be a valid FLAC frame, must start with the syncword
return null;
}
if (this.blockingBit === null) {
assert(isFirstPacket);
const newBlockingBit = bitstream.readBits(1);
this.blockingBit = newBlockingBit;
} else if (this.blockingBit === 1) {
assert(!isFirstPacket);
const newBlockingBit = bitstream.readBits(1);
if (newBlockingBit !== 1) {
// This cannot be a valid FLAC frame, expected 1 but got 0
return null;
}
} else if (this.blockingBit === 0) {
assert(!isFirstPacket);
const newBlockingBit = bitstream.readBits(1);
if (newBlockingBit !== 0) {
// This cannot be a valid FLAC frame, expected 0 but got 1
return null;
}
} else {
throw new Error('Invalid blocking bit');
}
const blockSizeOrUncommon = getBlockSizeOrUncommon(bitstream.readBits(4));
if (!blockSizeOrUncommon) {
// This cannot be a valid FLAC frame, the syncword was just coincidental
return null;
}
assert(this.audioInfo);
const sampleRateOrUncommon = getSampleRateOrUncommon(
bitstream.readBits(4),
this.audioInfo.sampleRate,
);
if (!sampleRateOrUncommon) {
// This cannot be a valid FLAC frame, the syncword was just coincidental
return null;
}
bitstream.readBits(4); // channel count
bitstream.readBits(3); // bit depth
const reservedZero = bitstream.readBits(1); // reserved zero
if (reservedZero !== 0) {
// This cannot be a valid FLAC frame, the syncword was just coincidental
return null;
}
const num = readCodedNumber(slice);
const blockSize = readBlockSize(slice, blockSizeOrUncommon);
const sampleRate = readSampleRate(slice, sampleRateOrUncommon);
if (sampleRate === null) {
// This cannot be a valid FLAC frame, the syncword was just coincidental
return null;
}
const size = slice.filePos - startOffset;
const crc = readU8(slice);
slice.skip(-size);
slice.skip(-1);
const crcCalculated = calculateCrc8(readBytes(slice, size));
if (crc !== crcCalculated) {
// Maybe this wasn't a FLAC frame at all, the syncword was just coincidentally
// in the bitstream
return null;
}
return { num, blockSize, sampleRate };
}
async advanceReader() {
await this.readMetadata();
assert(this.lastLoadedPos !== null);
assert(this.audioInfo);
const startPos = this.lastLoadedPos;
const frame = await this.readNextFlacFrame({
startPos,
isFirstPacket: this.loadedSamples.length === 0,
});
if (!frame) {
// Unexpected case, failed to read next FLAC frame
// handling gracefully
this.lastSampleLoaded = true;
return;
}
const lastSample = this.loadedSamples[this.loadedSamples.length - 1];
const blockOffset = lastSample
? lastSample.blockOffset + lastSample.blockSize
: 0;
const sample: Sample = {
blockOffset,
blockSize: frame.blockSize,
byteOffset: startPos,
byteSize: frame.size,
};
this.lastLoadedPos = this.lastLoadedPos + frame.size;
this.loadedSamples.push(sample);
if (frame.isLastFrame) {
this.lastSampleLoaded = true;
return;
}
}
}
class FlacAudioTrackBacking implements InputAudioTrackBacking {
constructor(public demuxer: FlacDemuxer) {}
getId() {
return 1;
}
getCodec() {
return 'flac' as const;
}
getInternalCodecId(): string | number | Uint8Array | null {
return null;
}
getNumberOfChannels() {
assert(this.demuxer.audioInfo);
return this.demuxer.audioInfo.numberOfChannels;
}
async computeDuration() {
const lastPacket = await this.getPacket(Infinity, { metadataOnly: true });
return (lastPacket?.timestamp ?? 0) + (lastPacket?.duration ?? 0);
}
getSampleRate() {
assert(this.demuxer.audioInfo);
return this.demuxer.audioInfo.sampleRate;
}
getName(): string | null {
return null;
}
getLanguageCode() {
return UNDETERMINED_LANGUAGE;
}
getTimeResolution() {
assert(this.demuxer.audioInfo);
return this.demuxer.audioInfo.sampleRate;
}
async getFirstTimestamp() {
return 0;
}
async getDecoderConfig(): Promise<AudioDecoderConfig | null> {
assert(this.demuxer.audioInfo);
return {
codec: 'flac' as const,
numberOfChannels: this.demuxer.audioInfo.numberOfChannels,
sampleRate: this.demuxer.audioInfo.sampleRate,
description: this.demuxer.audioInfo.description,
};
}
async getPacket(
timestamp: number,
options: PacketRetrievalOptions,
): Promise<EncodedPacket | null> {
assert(this.demuxer.audioInfo);
if (timestamp < 0) {
throw new Error('Timestamp cannot be negative');
}
const release = await this.demuxer.readingMutex.acquire();
try {
while (true) {
const packetIndex = binarySearchLessOrEqual(
this.demuxer.loadedSamples,
timestamp,
x => x.blockOffset / this.demuxer.audioInfo!.sampleRate,
);
if (packetIndex === -1) {
await this.demuxer.advanceReader();
continue;
}
const packet = this.demuxer.loadedSamples[packetIndex]!;
const sampleTimestamp
= packet.blockOffset / this.demuxer.audioInfo.sampleRate;
const sampleDuration
= packet.blockSize / this.demuxer.audioInfo.sampleRate;
if (sampleTimestamp + sampleDuration <= timestamp) {
if (this.demuxer.lastSampleLoaded) {
return this.getPacketAtIndex(
this.demuxer.loadedSamples.length - 1,
options,
);
}
await this.demuxer.advanceReader();
continue;
}
return this.getPacketAtIndex(packetIndex, options);
}
} finally {
release();
}
}
async getNextPacket(
packet: EncodedPacket,
options: PacketRetrievalOptions,
): Promise<EncodedPacket | null> {
const release = await this.demuxer.readingMutex.acquire();
try {
const nextIndex = packet.sequenceNumber + 1;
if (
this.demuxer.lastSampleLoaded
&& nextIndex >= this.demuxer.loadedSamples.length
) {
return null;
}
// Ensure the next sample exists
while (
nextIndex >= this.demuxer.loadedSamples.length
&& !this.demuxer.lastSampleLoaded
) {
await this.demuxer.advanceReader();
}
return this.getPacketAtIndex(nextIndex, options);
} finally {
release();
}
}
getKeyPacket(
timestamp: number,
options: PacketRetrievalOptions,
): Promise<EncodedPacket | null> {
return this.getPacket(timestamp, options);
}
getNextKeyPacket(
packet: EncodedPacket,
options: PacketRetrievalOptions,
): Promise<EncodedPacket | null> {
return this.getNextPacket(packet, options);
}
async getPacketAtIndex(
sampleIndex: number,
options: PacketRetrievalOptions,
): Promise<EncodedPacket | null> {
const rawSample = this.demuxer.loadedSamples[sampleIndex];
if (!rawSample) {
return null;
}
let data: Uint8Array;
if (options.metadataOnly) {
data = PLACEHOLDER_DATA;
} else {
let slice = this.demuxer.reader.requestSlice(
rawSample.byteOffset,
rawSample.byteSize,
);
if (slice instanceof Promise) slice = await slice;
if (!slice) {
return null; // Data didn't fit into the rest of the file
}
data = readBytes(slice, rawSample.byteSize);
}
assert(this.demuxer.audioInfo);
const timestamp = rawSample.blockOffset / this.demuxer.audioInfo.sampleRate;
const duration = rawSample.blockSize / this.demuxer.audioInfo.sampleRate;
return new EncodedPacket(
data,
'key',
timestamp,
duration,
sampleIndex,
rawSample.byteSize,
);
}
async getFirstPacket(
options: PacketRetrievalOptions,
): Promise<EncodedPacket | null> {
// Ensure the next sample exists
while (
this.demuxer.loadedSamples.length === 0
&& !this.demuxer.lastSampleLoaded
) {
await this.demuxer.advanceReader();
}
return this.getPacketAtIndex(0, options);
}
}
+164
View File
@@ -0,0 +1,164 @@
/*!
* Copyright (c) 2025-present, Vanilagy and contributors
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at https://mozilla.org/MPL/2.0/.
*/
import { assert, assertNever, Bitstream } from '../misc';
import { FileSlice, readBytes, readU16Be, readU8 } from '../reader';
type BlockSizeOrUncommon = number | 'uncommon-u16' | 'uncommon-u8';
type SampleRateOrUncommon =
| number
| 'uncommon-u8'
| 'uncommon-u16'
| 'uncommon-u16-10';
// https://www.rfc-editor.org/rfc/rfc9639.html#name-block-size-bits
export const getBlockSizeOrUncommon = (bits: number): BlockSizeOrUncommon | null => {
if (bits === 0b0000) {
return null;
} else if (bits === 0b0001) {
return 192;
} else if (bits >= 0b0010 && bits <= 0b0101) {
return 144 * 2 ** bits;
} else if (bits === 0b0110) {
return 'uncommon-u8';
} else if (bits === 0b0111) {
return 'uncommon-u16';
} else if (bits >= 0b1000 && bits <= 0b1111) {
return 2 ** bits;
} else {
return null;
}
};
// https://www.rfc-editor.org/rfc/rfc9639.html#name-sample-rate-bits
export const getSampleRateOrUncommon = (
sampleRateBits: number,
streamInfoSampleRate: number,
): SampleRateOrUncommon | null => {
switch (sampleRateBits) {
case 0b0000: return streamInfoSampleRate;
case 0b0001: return 88200;
case 0b0010: return 176400;
case 0b0011: return 192000;
case 0b0100: return 8000;
case 0b0101: return 16000;
case 0b0110: return 22050;
case 0b0111: return 24000;
case 0b1000: return 32000;
case 0b1001: return 44100;
case 0b1010: return 48000;
case 0b1011: return 96000;
case 0b1100: return 'uncommon-u8';
case 0b1101: return 'uncommon-u16';
case 0b1110: return 'uncommon-u16-10';
default: return null;
}
};
// https://www.rfc-editor.org/rfc/rfc9639.html#name-coded-number
export const readCodedNumber = (fileSlice: FileSlice): number => {
let ones = 0;
const bitstream1 = new Bitstream(readBytes(fileSlice, 1));
while (bitstream1.readBits(1) === 1) {
ones++;
}
if (ones === 0) {
return bitstream1.readBits(7);
}
const bitArray: number[] = [];
const extraBytes = ones - 1;
const bitstream2 = new Bitstream(readBytes(fileSlice, extraBytes));
const firstByteBits = 8 - ones - 1;
for (let i = 0; i < firstByteBits; i++) {
bitArray.unshift(bitstream1.readBits(1));
}
for (let i = 0; i < extraBytes; i++) {
for (let j = 0; j < 8; j++) {
const val = bitstream2.readBits(1);
if (j < 2) {
continue;
}
bitArray.unshift(val);
}
}
const encoded = bitArray.reduce((acc, bit, index) => {
return acc | (bit << index);
}, 0);
return encoded;
};
export const readBlockSize = (
slice: FileSlice,
blockSizeBits: BlockSizeOrUncommon,
) => {
if (blockSizeBits === 'uncommon-u16') {
return readU16Be(slice) + 1;
} else if (blockSizeBits === 'uncommon-u8') {
return readU8(slice) + 1;
} else if (typeof blockSizeBits === 'number') {
return blockSizeBits;
} else {
assertNever(blockSizeBits);
assert(false);
}
};
export const readSampleRate = (
slice: FileSlice,
sampleRateOrUncommon: SampleRateOrUncommon,
) => {
if (sampleRateOrUncommon === 'uncommon-u16') {
return readU16Be(slice);
}
if (sampleRateOrUncommon === 'uncommon-u16-10') {
return readU16Be(slice) * 10;
}
if (sampleRateOrUncommon === 'uncommon-u8') {
return readU8(slice);
}
if (typeof sampleRateOrUncommon === 'number') {
return sampleRateOrUncommon;
}
return null;
};
// https://www.rfc-editor.org/rfc/rfc9639.html#section-9.1.1
export const calculateCrc8 = (data: Uint8Array) => {
const polynomial = 0x07; // x^8 + x^2 + x^1 + x^0
let crc = 0x00; // Initialize CRC to 0
for (const byte of data) {
crc ^= byte; // XOR byte into least significant byte of crc
for (let i = 0; i < 8; i++) {
// For each bit in the byte
if ((crc & 0x80) !== 0) {
// If the leftmost bit (MSB) is set
crc = (crc << 1) ^ polynomial; // Shift left and XOR with polynomial
} else {
crc <<= 1; // Just shift left
}
crc &= 0xff; // Ensure CRC remains 8-bit
}
}
return crc;
};
+320
View File
@@ -0,0 +1,320 @@
/*!
* Copyright (c) 2025-present, Vanilagy and contributors
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at https://mozilla.org/MPL/2.0/.
*/
import { validateAudioChunkMetadata } from '../codec';
import { createVorbisComments, FlacBlockType } from '../codec-data';
import {
assert,
Bitstream,
textEncoder,
toDataView,
toUint8Array,
} from '../misc';
import { Muxer } from '../muxer';
import { Output, OutputAudioTrack } from '../output';
import { FlacOutputFormat } from '../output-format';
import { EncodedPacket } from '../packet';
import { FileSlice, readBytes } from '../reader';
import { AttachedImage, metadataTagsAreEmpty } from '../tags';
import { Writer } from '../writer';
import {
readBlockSize,
getBlockSizeOrUncommon,
readCodedNumber,
} from './flac-misc';
const FLAC_HEADER = new Uint8Array([0x66, 0x4c, 0x61, 0x43]); // 'fLaC'
const STREAMINFO_SIZE = 38;
const STREAMINFO_BLOCK_SIZE = 34;
export class FlacMuxer extends Muxer {
private writer: Writer;
private metadataWritten = false;
private blockSizes: number[] = [];
private frameSizes: number[] = [];
private sampleRate: number | null = null;
private channels: number | null = null;
private bitsPerSample: number | null = null;
private format: FlacOutputFormat;
constructor(output: Output, format: FlacOutputFormat) {
super(output);
this.writer = output._writer;
this.format = format;
}
async start() {
this.writer.write(FLAC_HEADER);
}
writeHeader({
bitsPerSample,
minimumBlockSize,
maximumBlockSize,
minimumFrameSize,
maximumFrameSize,
sampleRate,
channels,
totalSamples,
}: {
minimumBlockSize: number;
maximumBlockSize: number;
minimumFrameSize: number;
maximumFrameSize: number;
sampleRate: number;
channels: number;
bitsPerSample: number;
totalSamples: number;
}) {
assert(this.writer.getPos() === 4);
const hasMetadata = !metadataTagsAreEmpty(this.output._metadataTags);
const headerBitstream = new Bitstream(new Uint8Array(4));
headerBitstream.writeBits(1, Number(!hasMetadata)); // isLastMetadata
headerBitstream.writeBits(7, FlacBlockType.STREAMINFO); // metaBlockType = streaminfo
headerBitstream.writeBits(24, STREAMINFO_BLOCK_SIZE); // size
this.writer.write(headerBitstream.bytes);
const contentBitstream = new Bitstream(new Uint8Array(18));
contentBitstream.writeBits(16, minimumBlockSize);
contentBitstream.writeBits(16, maximumBlockSize);
contentBitstream.writeBits(24, minimumFrameSize);
contentBitstream.writeBits(24, maximumFrameSize);
contentBitstream.writeBits(20, sampleRate);
contentBitstream.writeBits(3, channels - 1);
contentBitstream.writeBits(5, bitsPerSample - 1);
// Bitstream operations are only safe until 32bit, breaks when using 36 bits
// Splitting up into writing 4 0 bits and then 32 bits is safe
// This is safe for audio up to (2 ** 32 / 44100 / 3600) -> 27 hours
// Not implementing support for more than 32 bits now
if (totalSamples >= 2 ** 32) {
throw new Error('This muxer only supports writing up to 2 ** 32 samples');
}
contentBitstream.writeBits(4, 0);
contentBitstream.writeBits(32, totalSamples);
this.writer.write(contentBitstream.bytes);
// The MD5 hash is calculated from decoded audio data, but we do not have access
// to it here. We are allowed to set 0:
// "A value of 0 signifies that the value is not known."
// https://www.rfc-editor.org/rfc/rfc9639.html#name-streaminfo
this.writer.write(new Uint8Array(16));
}
writePictureBlock(picture: AttachedImage) {
// Header size:
// 4 bytes: picture type
// 4 bytes: media type length
// x bytes: media type
// 4 bytes: description length
// y bytes: description
// 1 bytes: width
// 1 bytes: height
// 1 bytes: color depth
// 1 bytes: number of indexed colors
// 4 bytes: picture data length
// z bytes: picture data
// Total: 20 + x + y + z
const headerSize
= 32
+ picture.mimeType.length
+ (picture.description?.length ?? 0)
+ picture.data.length;
const header = new Uint8Array(headerSize);
let offset = 0;
const dataView = toDataView(header);
dataView.setUint32(
offset,
picture.kind === 'coverFront' ? 3 : picture.kind === 'coverBack' ? 4 : 0,
);
offset += 4;
dataView.setUint32(offset, picture.mimeType.length);
offset += 4;
header.set(textEncoder.encode(picture.mimeType), 8);
offset += picture.mimeType.length;
dataView.setUint32(offset, picture.description?.length ?? 0);
offset += 4;
header.set(textEncoder.encode(picture.description ?? ''), offset);
offset += picture.description?.length ?? 0;
offset += 4 + 4 + 4 + 4; // setting width, height, color depth, number of indexed colors to 0
dataView.setUint32(offset, picture.data.length);
offset += 4;
header.set(picture.data, offset);
offset += picture.data.length;
assert(offset === headerSize);
const headerBitstream = new Bitstream(new Uint8Array(4));
headerBitstream.writeBits(1, 0); // Last metadata block -> false, will be continued by vorbis comment
headerBitstream.writeBits(7, FlacBlockType.PICTURE); // Type -> Picture
headerBitstream.writeBits(24, headerSize);
this.writer.write(headerBitstream.bytes);
this.writer.write(header);
}
writeVorbisCommentAndPictureBlock() {
this.writer.seek(STREAMINFO_SIZE + FLAC_HEADER.byteLength);
if (metadataTagsAreEmpty(this.output._metadataTags)) {
this.metadataWritten = true;
return;
}
const pictures = this.output._metadataTags.images ?? [];
for (const picture of pictures) {
this.writePictureBlock(picture);
}
const vorbisComment = createVorbisComments(
new Uint8Array(0),
this.output._metadataTags,
false,
);
const headerBitstream = new Bitstream(new Uint8Array(4));
headerBitstream.writeBits(1, 1); // Last metadata block -> true
headerBitstream.writeBits(7, FlacBlockType.VORBIS_COMMENT); // Type -> Vorbis comment
headerBitstream.writeBits(24, vorbisComment.length);
this.writer.write(headerBitstream.bytes);
this.writer.write(vorbisComment);
this.metadataWritten = true;
}
async getMimeType() {
return 'audio/flac';
}
async addEncodedVideoPacket() {
throw new Error('FLAC does not support video.');
}
async addEncodedAudioPacket(
track: OutputAudioTrack,
packet: EncodedPacket,
meta?: EncodedAudioChunkMetadata,
): Promise<void> {
const release = await this.mutex.acquire();
validateAudioChunkMetadata(meta);
assert(meta);
assert(meta.decoderConfig);
assert(meta.decoderConfig.description);
try {
this.validateAndNormalizeTimestamp(
track,
packet.timestamp,
packet.type === 'key',
);
if (this.sampleRate === null) {
this.sampleRate = meta.decoderConfig.sampleRate;
}
if (this.channels === null) {
this.channels = meta.decoderConfig.numberOfChannels;
}
if (this.bitsPerSample === null) {
const descriptionBitstream = new Bitstream(
toUint8Array(meta.decoderConfig.description),
);
// skip 'fLaC' + block size + frame size + sample rate + number of channels
// See demuxer for the exact structure
descriptionBitstream.skipBits(103 + 64);
const bitsPerSample = descriptionBitstream.readBits(5) + 1;
this.bitsPerSample = bitsPerSample;
}
if (!this.metadataWritten) {
this.writeVorbisCommentAndPictureBlock();
}
const slice = FileSlice.tempFromBytes(packet.data);
readBytes(slice, 2);
const bytes = readBytes(slice, 2);
const bitstream = new Bitstream(bytes);
const blockSizeOrUncommon = getBlockSizeOrUncommon(bitstream.readBits(4));
if (blockSizeOrUncommon === null) {
throw new Error('Invalid FLAC frame: Invalid block size.');
}
readCodedNumber(slice); // num
const blockSize = readBlockSize(slice, blockSizeOrUncommon);
this.blockSizes.push(blockSize);
this.frameSizes.push(packet.data.length);
const startPos = this.writer.getPos();
this.writer.write(packet.data);
if (this.format._options.onFrame) {
this.format._options.onFrame(packet.data, startPos);
}
await this.writer.flush();
} finally {
release();
}
}
override addSubtitleCue(): Promise<void> {
throw new Error('FLAC does not support subtitles.');
}
async finalize(): Promise<void> {
const release = await this.mutex.acquire();
let minimumBlockSize = Infinity;
let maximumBlockSize = 0;
let minimumFrameSize = Infinity;
let maximumFrameSize = 0;
let totalSamples = 0;
for (let i = 0; i < this.blockSizes.length; i++) {
minimumFrameSize = Math.min(minimumFrameSize, this.frameSizes[i]!);
maximumFrameSize = Math.max(maximumFrameSize, this.frameSizes[i]!);
maximumBlockSize = Math.max(maximumBlockSize, this.blockSizes[i]!);
totalSamples += this.blockSizes[i]!;
// Excluding the last frame from block size calculation
// https://www.rfc-editor.org/rfc/rfc9639.html#name-streaminfo
// "The minimum block size (in samples) used in the stream, excluding the last block."
const isLastFrame = i === this.blockSizes.length - 1;
if (isLastFrame) {
continue;
}
minimumBlockSize = Math.min(minimumBlockSize, this.blockSizes[i]!);
}
assert(this.sampleRate !== null);
assert(this.channels !== null);
assert(this.bitsPerSample !== null);
this.writer.seek(4);
this.writeHeader({
minimumBlockSize,
maximumBlockSize,
minimumFrameSize,
maximumFrameSize,
sampleRate: this.sampleRate,
channels: this.channels,
bitsPerSample: this.bitsPerSample,
totalSamples,
});
release();
}
}
+4
View File
@@ -21,6 +21,8 @@ export {
OutputFormat,
AdtsOutputFormat,
AdtsOutputFormatOptions,
FlacOutputFormat,
FlacOutputFormatOptions,
IsobmffOutputFormat,
IsobmffOutputFormatOptions,
MkvOutputFormat,
@@ -130,6 +132,7 @@ export {
QuickTimeInputFormat,
WaveInputFormat,
WebMInputFormat,
FlacInputFormat,
ALL_FORMATS,
ADTS,
MATROSKA,
@@ -139,6 +142,7 @@ export {
QTFF,
WAVE,
WEBM,
FLAC,
} from './input-format';
export {
Input,
+40 -1
View File
@@ -28,6 +28,7 @@ import { WaveDemuxer } from './wave/wave-demuxer';
import { MAX_FRAME_HEADER_SIZE, MIN_FRAME_HEADER_SIZE, readFrameHeader } from './adts/adts-reader';
import { AdtsDemuxer } from './adts/adts-demuxer';
import { readAscii } from './reader';
import { FlacDemuxer } from './flac/flac-demuxer';
/**
* Base class representing an input media file format.
@@ -394,6 +395,37 @@ export class OggInputFormat extends InputFormat {
return 'application/ogg';
}
}
/**
* FLAC file format.
*
* Do not instantiate this class; use the {@link FLAC} singleton instead.
*
* @group Input formats
* @public
*/
export class FlacInputFormat extends InputFormat {
/** @internal */
async _canReadInput(input: Input) {
let slice = input._reader.requestSlice(0, 4);
if (slice instanceof Promise) slice = await slice;
if (!slice) return false;
return readAscii(slice, 4) === 'fLaC';
}
get name() {
return 'FLAC';
}
get mimeType() {
return 'audio/flac';
}
/** @internal */
_createDemuxer(input: Input): Demuxer {
return new FlacDemuxer(input);
}
}
/**
* ADTS file format.
@@ -492,10 +524,17 @@ export const OGG = new OggInputFormat();
*/
export const ADTS = new AdtsInputFormat();
/**
* FLAC input format singleton.
* @group Input formats
* @public
*/
export const FLAC = new FlacInputFormat();
/**
* List of all input format singletons. If you don't need to support all input formats, you should specify the
* formats individually for better tree shaking.
* @group Input formats
* @public
*/
export const ALL_FORMATS: InputFormat[] = [MP4, QTFF, MATROSKA, WEBM, WAVE, OGG, MP3, ADTS];
export const ALL_FORMATS: InputFormat[] = [MP4, QTFF, MATROSKA, WEBM, WAVE, OGG, FLAC, MP3, ADTS];
+101 -59
View File
@@ -1274,14 +1274,18 @@ export const vtta = (notes: string) => box('vtta', [...textEncoder.encode(notes)
const udta = (muxer: IsobmffMuxer) => {
const boxes: Box[] = [];
const metadataFormat = muxer.format._options.metadataFormat ?? 'auto';
const metadataTags = muxer.output._metadataTags;
// Depending on the format, metadata tags are written differently
if (muxer.isQuickTime) {
if (metadataFormat === 'mdir' || (metadataFormat === 'auto' && !muxer.isQuickTime)) {
const metaBox = metaMdir(metadataTags);
if (metaBox) boxes.push(metaBox);
} else if (metadataFormat === 'mdta') {
const metaBox = metaMdta(metadataTags);
if (metaBox) boxes.push(metaBox);
} else if (metadataFormat === 'udta' || (metadataFormat === 'auto' && muxer.isQuickTime)) {
addQuickTimeMetadataTagBoxes(boxes, muxer.output._metadataTags);
} else {
const metaBox = meta(muxer.output._metadataTags);
if (metaBox) {
boxes.push(metaBox);
}
}
if (boxes.length === 0) {
@@ -1381,9 +1385,14 @@ const DATA_BOX_MIME_TYPE_MAP: Record<string, number> = {
'image/bmp': 27,
};
/** Metadata Box */
const meta = (tags: MetadataTags) => {
const boxes: Box[] = [];
/**
* Generates key-value metadata for inclusion in the "meta" box.
*/
const generateMetadataPairs = (tags: MetadataTags, isMdta: boolean) => {
const pairs: {
key: string;
value: Box;
}[] = [];
// https://exiftool.org/TagNames/QuickTime.html (QuickTime ItemList Tags)
// This is the metadata format used for MP4 files
@@ -1391,39 +1400,42 @@ const meta = (tags: MetadataTags) => {
for (const { key, value } of keyValueIterator(tags)) {
switch (key) {
case 'title': {
boxes.push(metadataTagStringBoxLong('©nam', value));
pairs.push({ key: isMdta ? 'title' : '©nam', value: dataStringBoxLong(value) });
}; break;
case 'description': {
boxes.push(metadataTagStringBoxLong('©des', value));
pairs.push({ key: isMdta ? 'description' : '©des', value: dataStringBoxLong(value) });
}; break;
case 'artist': {
boxes.push(metadataTagStringBoxLong('©ART', value));
pairs.push({ key: isMdta ? 'artist' : '©ART', value: dataStringBoxLong(value) });
}; break;
case 'album': {
boxes.push(metadataTagStringBoxLong('©alb', value));
pairs.push({ key: isMdta ? 'album' : '©alb', value: dataStringBoxLong(value) });
}; break;
case 'albumArtist': {
boxes.push(metadataTagStringBoxLong('aART', value));
pairs.push({ key: isMdta ? 'album_artist' : 'aART', value: dataStringBoxLong(value) });
}; break;
case 'comment': {
boxes.push(metadataTagStringBoxLong('©cmt', value));
pairs.push({ key: isMdta ? 'comment' : '©cmt', value: dataStringBoxLong(value) });
}; break;
case 'genre': {
boxes.push(metadataTagStringBoxLong('©gen', value));
pairs.push({ key: isMdta ? 'genre' : '©gen', value: dataStringBoxLong(value) });
}; break;
case 'lyrics': {
boxes.push(metadataTagStringBoxLong('©lyr', value));
pairs.push({ key: isMdta ? 'lyrics' : '©lyr', value: dataStringBoxLong(value) });
}; break;
case 'date': {
boxes.push(metadataTagStringBoxLong('©day', value.toISOString().slice(0, 10)));
pairs.push({
key: isMdta ? 'date' : '©day',
value: dataStringBoxLong(value.toISOString().slice(0, 10)),
});
}; break;
case 'images': {
@@ -1432,40 +1444,45 @@ const meta = (tags: MetadataTags) => {
continue;
}
boxes.push(box('covr', undefined, [
box('data', [
u32(DATA_BOX_MIME_TYPE_MAP[image.mimeType] ?? 0), // Type indicator
u32(0), // Locale indicator
Array.from(image.data), // Kinda slow, hopefully temp
]),
]));
pairs.push({ key: 'covr', value: box('data', [
u32(DATA_BOX_MIME_TYPE_MAP[image.mimeType] ?? 0), // Type indicator
u32(0), // Locale indicator
Array.from(image.data), // Kinda slow, hopefully temp
]) });
}
}; break;
case 'trackNumber': {
boxes.push(box('trkn', undefined, [
box('data', [
if (isMdta) {
const string = tags.tracksTotal !== undefined
? `${value}/${tags.tracksTotal}`
: value.toString();
pairs.push({ key: 'track', value: dataStringBoxLong(string) });
} else {
pairs.push({ key: 'trkn', value: box('data', [
u32(0), // 8 bytes empty
u32(0),
u16(0), // Empty
u16(value),
u16(tags.tracksTotal ?? 0),
u16(0), // Empty
]),
]));
]) });
}
}; break;
case 'discNumber': {
boxes.push(box('disc', undefined, [
box('data', [
if (!isMdta) {
// Only written for mdir
pairs.push({ key: 'disc', value: box('data', [
u32(0), // 8 bytes empty
u32(0),
u16(0), // Empty
u16(value),
u16(tags.discsTotal ?? 0),
u16(0), // Empty
]),
]));
]) });
}
}; break;
case 'tracksTotal':
@@ -1484,49 +1501,74 @@ const meta = (tags: MetadataTags) => {
if (tags.raw) {
for (const key in tags.raw) {
const value = tags.raw[key];
if (value == null || key.length !== 4 || boxes.some(x => x.type === key)) {
if (value == null || (!isMdta && key.length !== 4) || pairs.some(x => x.key === key)) {
continue;
}
if (typeof value === 'string') {
boxes.push(metadataTagStringBoxLong(key, value));
pairs.push({ key, value: dataStringBoxLong(value) });
} else if (value instanceof Uint8Array) {
boxes.push(box(key, undefined, [
box('data', [
u32(0), // Type indicator
u32(0), // Locale indicator
Array.from(value),
]),
]));
pairs.push({ key, value: box('data', [
u32(0), // Type indicator
u32(0), // Locale indicator
Array.from(value),
]) });
} else if (value instanceof RichImageData) {
boxes.push(box(key, undefined, [
box('data', [
u32(DATA_BOX_MIME_TYPE_MAP[value.mimeType] ?? 0), // Type indicator
u32(0), // Locale indicator
Array.from(value.data), // Kinda slow, hopefully temp
]),
]));
pairs.push({ key, value: box('data', [
u32(DATA_BOX_MIME_TYPE_MAP[value.mimeType] ?? 0), // Type indicator
u32(0), // Locale indicator
Array.from(value.data), // Kinda slow, hopefully temp
]) });
}
}
}
if (boxes.length === 0) {
return pairs;
};
/** Metadata Box (mdir format) */
const metaMdir = (tags: MetadataTags) => {
const pairs = generateMetadataPairs(tags, false);
if (pairs.length === 0) {
return null;
}
// fullBox format
return fullBox('meta', 0, 0, undefined, [
hdlr(false, 'mdir', '', 'appl'),
box('ilst', undefined, boxes),
hdlr(false, 'mdir', '', 'appl'), // mdir handler
box('ilst', undefined, pairs.map(pair => box(pair.key, undefined, [pair.value]))), // Item list without keys box
]);
};
const metadataTagStringBoxLong = (name: string, value: string) => {
return box(name, undefined, [
box('data', [
u32(1), // Type indicator (UTF-8)
u32(0), // Locale indicator
...textEncoder.encode(value),
]),
/** Metadata Box (mdta format with keys box) */
const metaMdta = (tags: MetadataTags) => {
const pairs = generateMetadataPairs(tags, true);
if (pairs.length === 0) {
return null;
}
// box without version and flags
return box('meta', undefined, [
hdlr(false, 'mdta', ''), // mdta handler
fullBox('keys', 0, 0, [
u32(pairs.length),
], pairs.map(pair => box('mdta', [ // Hacky since these aren't boxes technically, but if not box why box-shaped?
...textEncoder.encode(pair.key),
]))),
box('ilst', undefined, pairs.map((pair, i) => {
const boxName = String.fromCharCode(...u32(i + 1));
return box(boxName, undefined, [pair.value]);
})),
]);
};
const dataStringBoxLong = (value: string) => {
return box('data', [
u32(1), // Type indicator (UTF-8)
u32(0), // Locale indicator
...textEncoder.encode(value),
]);
};
+4 -3
View File
@@ -24,6 +24,7 @@ import {
AvcDecoderConfigurationRecord,
extractAv1CodecInfoFromPacket,
extractVp9CodecInfoFromPacket,
FlacBlockType,
HevcDecoderConfigurationRecord,
Vp9CodecInfo,
} from '../codec-data';
@@ -1451,7 +1452,7 @@ export class IsobmffDemuxer extends Demuxer {
const type = flagAndType & BLOCK_TYPE_MASK;
// It's a STREAMINFO block; let's extract the actual sample rate and channel count
if (type === 0) {
if (type === FlacBlockType.STREAMINFO) {
slice.skip(10);
// Extract sample rate and channel count
@@ -2322,7 +2323,7 @@ export class IsobmffDemuxer extends Demuxer {
}; break;
case 'trkn': {
if (data instanceof Uint8Array) {
if (data instanceof Uint8Array && data.length >= 6) {
const view = toDataView(data);
const trackNumber = view.getUint16(2, false);
@@ -2339,7 +2340,7 @@ export class IsobmffDemuxer extends Demuxer {
case 'disc':
case 'disk': {
if (data instanceof Uint8Array) {
if (data instanceof Uint8Array && data.length >= 6) {
const view = toDataView(data);
const discNumber = view.getUint16(2, false);
+1 -1
View File
@@ -138,7 +138,7 @@ export const intoTimescale = (timeInSeconds: number, timescale: number, round =
};
export class IsobmffMuxer extends Muxer {
private format: IsobmffOutputFormat;
format: IsobmffOutputFormat;
private writer: Writer;
private boxWriter: IsobmffBoxWriter;
private fastStart: NonNullable<IsobmffOutputFormatOptions['fastStart']>;
+51 -31
View File
@@ -116,7 +116,7 @@ type Cluster = {
};
type ClusterTrackData = {
track: InternalTrack | null;
track: InternalTrack;
startTimestamp: number;
endTimestamp: number;
firstKeyFrameTimestamp: number | null;
@@ -141,6 +141,7 @@ type ClusterBlock = {
referencedTimestamps: number[];
data: Uint8Array;
lacing: BlockLacing;
decoded: boolean;
};
type CuePoint = {
@@ -667,7 +668,7 @@ export class MatroskaDemuxer extends Demuxer {
const nextEntry = trackData.presentationTimestamps[i + 1]!;
currentBlock.duration = nextEntry.timestamp - currentBlock.timestamp;
} else if (currentBlock.duration === 0) {
if (track?.defaultDuration != null) {
if (track.defaultDuration != null) {
if (currentBlock.lacing === BlockLacing.None) {
currentBlock.duration = track.defaultDuration;
} else {
@@ -695,13 +696,11 @@ export class MatroskaDemuxer extends Demuxer {
trackData.startTimestamp = firstBlock.timestamp;
trackData.endTimestamp = lastBlock.timestamp + lastBlock.duration;
if (track) {
insertSorted(track.clusters, cluster, x => x.elementStartPos);
insertSorted(track.clusters, cluster, x => x.elementStartPos);
const hasKeyFrame = trackData.firstKeyFrameTimestamp !== null;
if (hasKeyFrame) {
insertSorted(track.clustersWithKeyFrame, cluster, x => x.elementStartPos);
}
const hasKeyFrame = trackData.firstKeyFrameTimestamp !== null;
if (hasKeyFrame) {
insertSorted(track.clustersWithKeyFrame, cluster, x => x.elementStartPos);
}
}
@@ -714,8 +713,13 @@ export class MatroskaDemuxer extends Demuxer {
getTrackDataInCluster(cluster: Cluster, trackNumber: number) {
let trackData = cluster.trackData.get(trackNumber);
if (!trackData) {
const track = cluster.segment.tracks.find(x => x.id === trackNumber);
if (!track) {
return null;
}
trackData = {
track: cluster.segment.tracks.find(x => x.id === trackNumber) ?? null,
track,
startTimestamp: 0,
endTimestamp: 0,
firstKeyFrameTimestamp: null,
@@ -728,7 +732,7 @@ export class MatroskaDemuxer extends Demuxer {
return trackData;
}
expandLacedBlocks(blocks: ClusterBlock[], track: InternalTrack | null) {
expandLacedBlocks(blocks: ClusterBlock[], track: InternalTrack) {
// https://www.matroska.org/technical/notes.html#block-lacing
for (let blockIndex = 0; blockIndex < blocks.length; blockIndex++) {
@@ -737,6 +741,12 @@ export class MatroskaDemuxer extends Demuxer {
continue;
}
// Decode the block data if it hasn't been decoded yet (needed for lacing expansion)
if (!originalBlock.decoded) {
originalBlock.data = this.decodeBlockData(track, originalBlock.data);
originalBlock.decoded = true;
}
const slice = FileSlice.tempFromBytes(originalBlock.data);
const frameSizes: number[] = [];
@@ -819,7 +829,7 @@ export class MatroskaDemuxer extends Demuxer {
const frameSize = frameSizes[i]!;
const frameData = readBytes(slice, frameSize);
const blockDuration = originalBlock.duration || (frameCount * (track?.defaultDuration ?? 0));
const blockDuration = originalBlock.duration || (frameCount * (track.defaultDuration ?? 0));
// Distribute timestamps evenly across the block duration
const frameTimestamp = originalBlock.timestamp + (blockDuration * i / frameCount);
@@ -832,6 +842,7 @@ export class MatroskaDemuxer extends Demuxer {
referencedTimestamps: originalBlock.referencedTimestamps,
data: frameData,
lacing: BlockLacing.None,
decoded: true,
});
}
@@ -976,6 +987,7 @@ export class MatroskaDemuxer extends Demuxer {
if (this.currentTrack.decodingInstructions.some((instruction) => {
return instruction.data?.type !== 'decompress'
|| instruction.scope !== ContentEncodingScope.Block
|| instruction.data.algorithm !== ContentCompAlgo.HeaderStripping;
})) {
console.warn(`Track #${this.currentTrack.id} has an unsupported content encoding; dropping.`);
@@ -1331,18 +1343,17 @@ export class MatroskaDemuxer extends Demuxer {
const trackNumber = readVarInt(slice);
if (trackNumber === null) break;
const trackData = this.getTrackDataInCluster(this.currentCluster, trackNumber);
if (!trackData) break; // Not a track we care about
const relativeTimestamp = readI16Be(slice);
const flags = readU8(slice);
const isKeyFrame = !!(flags & 0x80);
const lacing = (flags >> 1) & 0x3 as BlockLacing; // If the block is laced, we'll expand it later
const trackData = this.getTrackDataInCluster(this.currentCluster, trackNumber);
let blockData = readBytes(slice, size - (slice.filePos - dataStartPos));
if (trackData.track) {
blockData = this.decodeBlockData(trackData.track, blockData);
}
const blockData = readBytes(slice, size - (slice.filePos - dataStartPos));
const hasDecodingInstructions = trackData.track.decodingInstructions.length > 0;
trackData.blocks.push({
timestamp: relativeTimestamp, // We'll add the cluster's timestamp to this later
@@ -1351,6 +1362,7 @@ export class MatroskaDemuxer extends Demuxer {
referencedTimestamps: [],
data: blockData,
lacing,
decoded: !hasDecodingInstructions,
});
}; break;
@@ -1374,17 +1386,16 @@ export class MatroskaDemuxer extends Demuxer {
const trackNumber = readVarInt(slice);
if (trackNumber === null) break;
const trackData = this.getTrackDataInCluster(this.currentCluster, trackNumber);
if (!trackData) break;
const relativeTimestamp = readI16Be(slice);
const flags = readU8(slice);
const lacing = (flags >> 1) & 0x3 as BlockLacing; // If the block is laced, we'll expand it later
const trackData = this.getTrackDataInCluster(this.currentCluster, trackNumber);
let blockData = readBytes(slice, size - (slice.filePos - dataStartPos));
if (trackData.track) {
blockData = this.decodeBlockData(trackData.track, blockData);
}
const blockData = readBytes(slice, size - (slice.filePos - dataStartPos));
const hasDecodingInstructions = trackData.track.decodingInstructions.length > 0;
this.currentBlock = {
timestamp: relativeTimestamp, // We'll add the cluster's timestamp to this later
@@ -1393,6 +1404,7 @@ export class MatroskaDemuxer extends Demuxer {
referencedTimestamps: [],
data: blockData,
lacing,
decoded: !hasDecodingInstructions,
};
trackData.blocks.push(this.currentBlock);
}; break;
@@ -1600,24 +1612,26 @@ export class MatroskaDemuxer extends Demuxer {
}
decodeBlockData(track: InternalTrack, rawData: Uint8Array) {
assert(track.decodingInstructions.length > 0); // This method shouldn't be called otherwise
let currentData = rawData;
// In the vast number of cases there are exactly zero decoding instructions
for (let i = 0; i < track.decodingInstructions.length; i++) {
const instruction = track.decodingInstructions[i]!;
for (const instruction of track.decodingInstructions) {
assert(instruction.data);
switch (instruction.data.type) {
case 'decompress': {
switch (instruction.data.algorithm) {
case ContentCompAlgo.HeaderStripping: {
const prefix = instruction.data.settings ?? new Uint8Array(0);
const newData = new Uint8Array(prefix.length + currentData.length);
if (instruction.data.settings && instruction.data.settings.length > 0) {
const prefix = instruction.data.settings;
const newData = new Uint8Array(prefix.length + currentData.length);
newData.set(prefix, 0);
newData.set(currentData, prefix.length);
newData.set(prefix, 0);
newData.set(currentData, prefix.length);
currentData = newData;
currentData = newData;
}
}; break;
default: {
@@ -1959,6 +1973,12 @@ abstract class MatroskaTrackBacking implements InputTrackBacking {
const block = trackData.blocks[blockIndex];
assert(block);
// Perform lazy decoding if needed
if (!block.decoded) {
block.data = this.internalTrack.demuxer.decodeBlockData(this.internalTrack, block.data);
block.decoded = true;
}
const data = options.metadataOnly ? PLACEHOLDER_DATA : block.data;
const timestamp = block.timestamp / this.internalTrack.segment.timestampFactor;
const duration = block.duration / this.internalTrack.segment.timestampFactor;
+9 -2
View File
@@ -117,9 +117,16 @@ export class Mp3Demuxer extends Demuxer {
this.firstFrameHeader = header;
}
const sampleDuration = header.audioSamplesInFrame / header.sampleRate;
if (header.sampleRate !== this.firstFrameHeader.sampleRate) {
console.warn(
`MP3 changed sample rate mid-file: ${this.firstFrameHeader.sampleRate} Hz to ${header.sampleRate} Hz.`
+ ` Might be a bug, so please report this file.`,
);
}
const sampleDuration = header.audioSamplesInFrame / this.firstFrameHeader.sampleRate;
const sample: Sample = {
timestamp: this.nextTimestampInSamples / header.sampleRate,
timestamp: this.nextTimestampInSamples / this.firstFrameHeader.sampleRate,
duration: sampleDuration,
dataStart: result.startPos,
dataSize: header.totalSize,
+1
View File
@@ -84,6 +84,7 @@ export class Mp3Muxer extends Muxer {
mpegVersionId: header.mpegVersionId,
layer: header.layer,
frequencyIndex: header.frequencyIndex,
sampleRate: header.sampleRate,
channel: header.channel,
modeExtension: header.modeExtension,
copyright: header.copyright,
+79 -71
View File
@@ -12,9 +12,7 @@ import {
computeMp3FrameSize,
encodeSynchsafe,
getXingOffset,
MPEG_V1_BITRATES,
MPEG_V2_BITRATES,
SAMPLING_RATES,
KILOBIT_RATES,
XING,
} from '../../shared/mp3-misc';
import { Id3V2TextEncoding } from './mp3-reader';
@@ -23,6 +21,7 @@ export type XingFrameData = {
mpegVersionId: number;
layer: number;
frequencyIndex: number;
sampleRate: number;
channel: number;
modeExtension: number;
copyright: number;
@@ -62,6 +61,83 @@ export class Mp3Writer {
this.writer.write(this.helper.subarray(0, text.length));
}
writeXingFrame(data: XingFrameData) {
const startPos = this.writer.getPos();
const firstByte = 0xff;
const secondByte = 0xe0 | (data.mpegVersionId << 3) | (data.layer << 1);
let lowSamplingFrequency: number;
if (data.mpegVersionId & 2) {
lowSamplingFrequency = (data.mpegVersionId & 1) ? 0 : 1;
} else {
lowSamplingFrequency = 1;
}
const padding = 0;
const neededBytes = 155;
let bitrateIndex = -1;
const bitrateOffset = lowSamplingFrequency * 16 * 4 + data.layer * 16;
// Let's find the lowest bitrate for which the frame size is sufficiently large to fit all the data
for (let i = 0; i < 16; i++) {
const kbr = KILOBIT_RATES[bitrateOffset + i]!;
const size = computeMp3FrameSize(lowSamplingFrequency, data.layer, 1000 * kbr, data.sampleRate, padding);
if (size >= neededBytes) {
bitrateIndex = i;
break;
}
}
if (bitrateIndex === -1) {
throw new Error('No suitable bitrate found.');
}
const thirdByte = (bitrateIndex << 4) | (data.frequencyIndex << 2) | padding << 1;
const fourthByte = (data.channel << 6)
| (data.modeExtension << 4)
| (data.copyright << 3)
| (data.original << 2)
| data.emphasis;
this.helper[0] = firstByte;
this.helper[1] = secondByte;
this.helper[2] = thirdByte;
this.helper[3] = fourthByte;
this.writer.write(this.helper.subarray(0, 4));
const xingOffset = getXingOffset(data.mpegVersionId, data.channel);
this.writer.seek(startPos + xingOffset);
this.writeU32(XING);
let flags = 0;
if (data.frameCount !== null) {
flags |= 1;
}
if (data.fileSize !== null) {
flags |= 2;
}
if (data.toc !== null) {
flags |= 4;
}
this.writeU32(flags);
this.writeU32(data.frameCount ?? 0);
this.writeU32(data.fileSize ?? 0);
this.writer.write(data.toc ?? new Uint8Array(100));
const kilobitRate = KILOBIT_RATES[bitrateOffset + bitrateIndex]!;
const frameSize = computeMp3FrameSize(
lowSamplingFrequency, data.layer, 1000 * kilobitRate, data.sampleRate, padding,
);
this.writer.seek(startPos + frameSize);
}
writeSynchsafeU32(value: number) {
this.writeU32(encodeSynchsafe(value));
}
@@ -170,72 +246,4 @@ export class Mp3Writer {
this.writer.write(imageData);
}
writeXingFrame(data: XingFrameData) {
const startPos = this.writer.getPos();
const firstByte = 0xff;
const secondByte = 0xe0 | (data.mpegVersionId << 3) | (data.layer << 1);
const bitrateGroup = data.mpegVersionId === 3 ? MPEG_V1_BITRATES : MPEG_V2_BITRATES;
const bitrates = bitrateGroup?.[data.layer];
if (!bitrates) {
throw new Error('Invalid MPEG version and layer combination.');
}
const sampleRate = SAMPLING_RATES[data.mpegVersionId]?.[data.frequencyIndex];
if (!sampleRate || sampleRate === -1) {
throw new Error('Invalid MPEG version and frequency index combination.');
}
const padding = 0;
const neededBytes = 155;
// Let's find the lowest bitrate for which the frame size is sufficiently large to fit all the data
const bitrateIndex = bitrates.findIndex((kbr) => {
return computeMp3FrameSize(data.layer, 1000 * kbr, sampleRate, padding) >= neededBytes;
});
if (bitrateIndex === -1) {
throw new Error('No suitable bitrate found.');
}
const thirdByte = (bitrateIndex << 4) | (data.frequencyIndex << 2) | padding << 1;
const fourthByte = (data.channel << 6)
| (data.modeExtension << 4)
| (data.copyright << 3)
| (data.original << 2)
| data.emphasis;
this.helper[0] = firstByte;
this.helper[1] = secondByte;
this.helper[2] = thirdByte;
this.helper[3] = fourthByte;
this.writer.write(this.helper.subarray(0, 4));
const xingOffset = getXingOffset(data.mpegVersionId, data.channel);
this.writer.seek(startPos + xingOffset);
this.writeU32(XING);
let flags = 0;
if (data.frameCount !== null) {
flags |= 1;
}
if (data.fileSize !== null) {
flags |= 2;
}
if (data.toc !== null) {
flags |= 4;
}
this.writeU32(flags);
this.writeU32(data.frameCount ?? 0);
this.writeU32(data.fileSize ?? 0);
this.writer.write(data.toc ?? new Uint8Array(100));
const frameSize = computeMp3FrameSize(data.layer, 1000 * bitrates[bitrateIndex]!, sampleRate, padding);
this.writer.seek(startPos + frameSize);
}
}
+3 -161
View File
@@ -7,7 +7,7 @@
*/
import { OPUS_SAMPLE_RATE } from '../codec';
import { parseModesFromVorbisSetupPacket, parseOpusIdentificationHeader } from '../codec-data';
import { parseModesFromVorbisSetupPacket, parseOpusIdentificationHeader, readVorbisComments } from '../codec-data';
import { Demuxer } from '../demuxer';
import { Input } from '../input';
import { InputAudioTrack, InputAudioTrackBacking } from '../input-track';
@@ -16,12 +16,10 @@ import { MetadataTags } from '../tags';
import {
assert,
AsyncMutex,
base64ToBytes,
binarySearchLessOrEqual,
findLast,
last,
roundToPrecision,
textDecoder,
toDataView,
UNDETERMINED_LANGUAGE,
} from '../misc';
@@ -223,7 +221,7 @@ export class OggDemuxer extends Demuxer {
modeBlockflags: parseModesFromVorbisSetupPacket(thirdPacket.data).modeBlockflags,
};
this.readVorbisComments(secondPacket.data.subarray(7)); // Skip header type and 'vorbis'
readVorbisComments(secondPacket.data.subarray(7), this.metadataTags); // Skip header type and 'vorbis'
}
async readOpusMetadata(firstPacket: Packet, bitstream: LogicalBitstream) {
@@ -255,163 +253,7 @@ export class OggDemuxer extends Demuxer {
preSkip: header.preSkip,
};
this.readVorbisComments(secondPacket.data.subarray(8)); // Skip 'OpusTags'
}
readVorbisComments(bytes: Uint8Array) {
// https://datatracker.ietf.org/doc/html/rfc7845#section-5.2
const commentView = toDataView(bytes);
let commentPos = 0;
const vendorStringLength = commentView.getUint32(commentPos, true);
commentPos += 4;
const vendorString = textDecoder.decode(
bytes.subarray(commentPos, commentPos + vendorStringLength),
);
commentPos += vendorStringLength;
if (vendorStringLength > 0) {
// Expose the vendor string in the raw metadata
this.metadataTags.raw ??= {};
this.metadataTags.raw['vendor'] ??= vendorString;
}
const listLength = commentView.getUint32(commentPos, true);
commentPos += 4;
// Loop over all metadata tags
for (let i = 0; i < listLength; i++) {
const stringLength = commentView.getUint32(commentPos, true);
commentPos += 4;
const string = textDecoder.decode(
bytes.subarray(commentPos, commentPos + stringLength),
);
commentPos += stringLength;
const separatorIndex = string.indexOf('=');
if (separatorIndex === -1) {
continue;
}
const key = string.slice(0, separatorIndex).toUpperCase();
const value = string.slice(separatorIndex + 1);
this.metadataTags.raw ??= {};
this.metadataTags.raw[key] ??= value;
switch (key) {
case 'TITLE': {
this.metadataTags.title ??= value;
}; break;
case 'DESCRIPTION': {
this.metadataTags.description ??= value;
}; break;
case 'ARTIST': {
this.metadataTags.artist ??= value;
}; break;
case 'ALBUM': {
this.metadataTags.album ??= value;
}; break;
case 'ALBUMARTIST': {
this.metadataTags.albumArtist ??= value;
}; break;
case 'COMMENT': {
this.metadataTags.comment ??= value;
}; break;
case 'LYRICS': {
this.metadataTags.lyrics ??= value;
}; break;
case 'TRACKNUMBER': {
const parts = value.split('/');
const trackNum = Number.parseInt(parts[0]!, 10);
const tracksTotal = parts[1] && Number.parseInt(parts[1], 10);
if (Number.isInteger(trackNum) && trackNum > 0) {
this.metadataTags.trackNumber ??= trackNum;
}
if (tracksTotal && Number.isInteger(tracksTotal) && tracksTotal > 0) {
this.metadataTags.tracksTotal ??= tracksTotal;
}
}; break;
case 'TRACKTOTAL': {
const tracksTotal = Number.parseInt(value, 10);
if (Number.isInteger(tracksTotal) && tracksTotal > 0) {
this.metadataTags.tracksTotal ??= tracksTotal;
}
}; break;
case 'DISCNUMBER': {
const parts = value.split('/');
const discNum = Number.parseInt(parts[0]!, 10);
const discsTotal = parts[1] && Number.parseInt(parts[1], 10);
if (Number.isInteger(discNum) && discNum > 0) {
this.metadataTags.discNumber ??= discNum;
}
if (discsTotal && Number.isInteger(discsTotal) && discsTotal > 0) {
this.metadataTags.discsTotal ??= discsTotal;
}
}; break;
case 'DISCTOTAL': {
const discsTotal = Number.parseInt(value, 10);
if (Number.isInteger(discsTotal) && discsTotal > 0) {
this.metadataTags.discsTotal ??= discsTotal;
}
}; break;
case 'DATE': {
const date = new Date(value);
if (!Number.isNaN(date.getTime())) {
this.metadataTags.date ??= date;
}
}; break;
case 'GENRE': {
this.metadataTags.genre ??= value;
}; break;
case 'METADATA_BLOCK_PICTURE': {
// https://datatracker.ietf.org/doc/rfc9639/ Section 8.8
const decoded = base64ToBytes(value);
const view = toDataView(decoded);
const pictureType = view.getUint32(0, false);
const mediaTypeLength = view.getUint32(4, false);
const mediaType = String.fromCharCode(...decoded.subarray(8, 8 + mediaTypeLength)); // ASCII
const descriptionLength = view.getUint32(8 + mediaTypeLength, false);
const description = textDecoder.decode(decoded.subarray(
12 + mediaTypeLength,
12 + mediaTypeLength + descriptionLength,
));
const dataLength = view.getUint32(mediaTypeLength + descriptionLength + 28);
const data = decoded.subarray(
mediaTypeLength + descriptionLength + 32,
mediaTypeLength + descriptionLength + 32 + dataLength,
);
this.metadataTags.images ??= [];
this.metadataTags.images.push({
data,
mimeType: mediaType,
kind: pictureType === 3 ? 'coverFront' : pictureType === 4 ? 'coverBack' : 'unknown',
name: undefined,
description: description || undefined,
});
}; break;
}
}
readVorbisComments(secondPacket.data.subarray(8), this.metadataTags); // Skip 'OpusTags'
}
async readPacket(startPage: Page, startSegmentIndex: number): Promise<Packet | null> {
+3 -178
View File
@@ -7,15 +7,11 @@
*/
import { OPUS_SAMPLE_RATE, validateAudioChunkMetadata } from '../codec';
import { parseModesFromVorbisSetupPacket, parseOpusIdentificationHeader } from '../codec-data';
import { createVorbisComments, parseModesFromVorbisSetupPacket, parseOpusIdentificationHeader } from '../codec-data';
import {
assert,
assertNever,
bytesToBase64,
keyValueIterator,
promiseWithResolvers,
setInt64,
textEncoder,
toDataView,
toUint8Array,
} from '../misc';
@@ -199,7 +195,7 @@ export class OggMuxer extends Muxer {
commentHeaderHeader[5] = 0x69; // 'i'
commentHeaderHeader[6] = 0x73; // 's'
const commentHeader = this.createVorbisComments(commentHeaderHeader);
const commentHeader = createVorbisComments(commentHeaderHeader, this.output._metadataTags, true);
trackData.packetQueue.push({
data: identificationHeader,
@@ -239,7 +235,7 @@ export class OggMuxer extends Muxer {
const commentHeaderHeaderView = toDataView(commentHeaderHeader);
commentHeaderHeaderView.setUint32(0, 0x4f707573, false); // 'Opus'
commentHeaderHeaderView.setUint32(4, 0x54616773, false); // 'Tags'
const commentHeader = this.createVorbisComments(commentHeaderHeader);
const commentHeader = createVorbisComments(commentHeaderHeader, this.output._metadataTags, true);
trackData.packetQueue.push({
data: identificationHeader,
@@ -259,177 +255,6 @@ export class OggMuxer extends Muxer {
}
}
createVorbisComments(headerBytes: Uint8Array) {
// https://datatracker.ietf.org/doc/html/rfc7845#section-5.2
const tags = this.output._metadataTags;
const commentHeaderParts = [
headerBytes,
];
let vendorString = '';
if (typeof tags.raw?.['vendor'] === 'string') {
vendorString = tags.raw?.['vendor'];
}
const encodedVendorString = textEncoder.encode(vendorString);
let currentBuffer = new Uint8Array(4 + encodedVendorString.length);
let currentView = new DataView(currentBuffer.buffer);
currentView.setUint32(0, encodedVendorString.length, true);
currentBuffer.set(encodedVendorString, 4);
commentHeaderParts.push(currentBuffer);
const writtenTags = new Set<string>();
const addCommentTag = (key: string, value: string) => {
const joined = `${key}=${value}`;
const encoded = textEncoder.encode(joined);
currentBuffer = new Uint8Array(4 + encoded.length);
currentView = new DataView(currentBuffer.buffer);
currentView.setUint32(0, encoded.length, true);
currentBuffer.set(encoded, 4);
commentHeaderParts.push(currentBuffer);
writtenTags.add(key);
};
for (const { key, value } of keyValueIterator(tags)) {
switch (key) {
case 'title': {
addCommentTag('TITLE', value);
}; break;
case 'description': {
addCommentTag('DESCRIPTION', value);
}; break;
case 'artist': {
addCommentTag('ARTIST', value);
}; break;
case 'album': {
addCommentTag('ALBUM', value);
}; break;
case 'albumArtist': {
addCommentTag('ALBUMARTIST', value);
}; break;
case 'genre': {
addCommentTag('GENRE', value);
}; break;
case 'date': {
addCommentTag('DATE', value.toISOString().slice(0, 10));
}; break;
case 'comment': {
addCommentTag('COMMENT', value);
}; break;
case 'lyrics': {
addCommentTag('LYRICS', value);
}; break;
case 'trackNumber': {
addCommentTag('TRACKNUMBER', value.toString());
}; break;
case 'tracksTotal': {
addCommentTag('TRACKTOTAL', value.toString());
}; break;
case 'discNumber': {
addCommentTag('DISCNUMBER', value.toString());
}; break;
case 'discsTotal': {
addCommentTag('DISCTOTAL', value.toString());
}; break;
case 'images': {
for (const image of value) {
// https://datatracker.ietf.org/doc/rfc9639/ Section 8.8
const pictureType = image.kind === 'coverFront' ? 3 : image.kind === 'coverBack' ? 4 : 0;
const encodedMediaType = new Uint8Array(image.mimeType.length);
for (let i = 0; i < image.mimeType.length; i++) {
encodedMediaType[i] = image.mimeType.charCodeAt(i);
}
const encodedDescription = textEncoder.encode(image.description ?? '');
const buffer = new Uint8Array(
4 // Picture type
+ 4 // MIME type length
+ encodedMediaType.length // MIME type
+ 4 // Description length
+ encodedDescription.length // Description
+ 16 // Width, height, color depth, number of colors
+ 4 // Picture data length
+ image.data.length, // Picture data
);
const view = toDataView(buffer);
view.setUint32(0, pictureType, false);
view.setUint32(4, encodedMediaType.length, false);
buffer.set(encodedMediaType, 8);
view.setUint32(8 + encodedMediaType.length, encodedDescription.length, false);
buffer.set(encodedDescription, 12 + encodedMediaType.length);
// Skip a bunch of fields (width, height, color depth, number of colors)
view.setUint32(
28 + encodedMediaType.length + encodedDescription.length, image.data.length, false,
);
buffer.set(
image.data,
32 + encodedMediaType.length + encodedDescription.length,
);
const encoded = bytesToBase64(buffer);
addCommentTag('METADATA_BLOCK_PICTURE', encoded);
}
}; break;
case 'raw': {
// Handled later
}; break;
default: assertNever(key);
}
}
if (tags.raw) {
for (const key in tags.raw) {
const value = tags.raw[key];
if (key === 'vendor' || value == null || writtenTags.has(key)) {
continue;
}
if (typeof value === 'string') {
addCommentTag(key, value);
}
}
}
const listLengthBuffer = new Uint8Array(4);
toDataView(listLengthBuffer).setUint32(0, writtenTags.size, true);
commentHeaderParts.splice(2, 0, listLengthBuffer); // Insert after the header and vendor section
// Merge all comment header parts into a single buffer
const commentHeaderLength = commentHeaderParts.reduce((a, b) => a + b.length, 0);
const commentHeader = new Uint8Array(commentHeaderLength);
let pos = 0;
for (const part of commentHeaderParts) {
commentHeader.set(part, pos);
pos += part.length;
}
return commentHeader;
}
async addEncodedAudioPacket(track: OutputAudioTrack, packet: EncodedPacket, meta?: EncodedAudioChunkMetadata) {
const release = await this.mutex.acquire();
+92
View File
@@ -18,6 +18,7 @@ import {
VIDEO_CODECS,
VideoCodec,
} from './codec';
import { FlacMuxer } from './flac/flac-muxer';
import { IsobmffMuxer } from './isobmff/isobmff-muxer';
import { MatroskaMuxer } from './matroska/matroska-muxer';
import { MediaSource } from './media-source';
@@ -139,6 +140,18 @@ export type IsobmffOutputFormatOptions = {
*/
minimumFragmentDuration?: number;
/**
* The metadata format to use for writing metadata tags.
*
* - `'auto'` (default): Behaves like `'mdir'` for MP4 and like `'udta'` for QuickTime, matching FFmpeg's default
* behavior.
* - `'mdir'`: Write tags into `moov/udta/meta` using the 'mdir' handler format.
* - `'mdta'`: Write tags into `moov/udta/meta` using the 'mdta' handler format, equivalent to FFmpeg's
* `use_metadata_tags` flag. This allows for custom keys of arbitrary length.
* - `'udta'`: Write tags directly into `moov/udta`.
*/
metadataFormat?: 'auto' | 'mdir' | 'mdta' | 'udta';
/**
* Will be called once the ftyp (File Type) box of the output file has been written.
*
@@ -215,6 +228,14 @@ export abstract class IsobmffOutputFormat extends OutputFormat {
if (options.onMoof !== undefined && typeof options.onMoof !== 'function') {
throw new TypeError('options.onMoof, when provided, must be a function.');
}
if (
options.metadataFormat !== undefined
&& !['mdir', 'mdta', 'udta', 'auto'].includes(options.metadataFormat)
) {
throw new TypeError(
'options.metadataFormat, when provided, must be either \'auto\', \'mdir\', \'mdta\', or \'udta\'.',
);
}
super();
@@ -827,3 +848,74 @@ export class AdtsOutputFormat extends OutputFormat {
return false;
}
}
/**
* FLAC-specific output options.
* @group Output formats
* @public
*/
export type FlacOutputFormatOptions = {
/**
* Will be called for each FLAC frame that is written.
*
* @param data - The raw bytes.
* @param position - The byte offset of the data in the file.
*/
onFrame?: (data: Uint8Array, position: number) => unknown;
};
/**
* FLAC file format.
* @group Output formats
* @public
*/
export class FlacOutputFormat extends OutputFormat {
/** @internal */
_options: FlacOutputFormatOptions;
/** Creates a new {@link FlacOutputFormat} configured with the specified `options`. */
constructor(options: FlacOutputFormatOptions = {}) {
if (!options || typeof options !== 'object') {
throw new TypeError('options must be an object.');
}
super();
this._options = options;
}
/** @internal */
_createMuxer(output: Output) {
return new FlacMuxer(output, this);
}
/** @internal */
get _name() {
return 'FLAC';
}
getSupportedTrackCounts(): TrackCountLimits {
return {
video: { min: 0, max: 0 },
audio: { min: 1, max: 1 },
subtitle: { min: 0, max: 0 },
total: { min: 1, max: 1 },
};
}
get fileExtension() {
return '.flac';
}
get mimeType() {
return 'audio/flac';
}
getSupportedCodecs(): MediaCodec[] {
return ['flac'];
}
get supportsVideoRotationMetadata() {
return false;
}
}
+4 -4
View File
@@ -197,7 +197,7 @@ export class Output<
this._muxer = options.format._createMuxer(this);
}
/** Adds a video track to the output with the given source. Must be called before output is started. */
/** Adds a video track to the output with the given source. Can only be called before the output is started. */
addVideoTrack(source: VideoSource, metadata: VideoTrackMetadata = {}) {
if (!(source instanceof VideoSource)) {
throw new TypeError('source must be a VideoSource.');
@@ -221,7 +221,7 @@ export class Output<
this._addTrack('video', source, metadata);
}
/** Adds an audio track to the output with the given source. Must be called before output is started. */
/** Adds an audio track to the output with the given source. Can only be called before the output is started. */
addAudioTrack(source: AudioSource, metadata: AudioTrackMetadata = {}) {
if (!(source instanceof AudioSource)) {
throw new TypeError('source must be an AudioSource.');
@@ -231,7 +231,7 @@ export class Output<
this._addTrack('audio', source, metadata);
}
/** Adds a subtitle track to the output with the given source. Must be called before output is started. */
/** Adds a subtitle track to the output with the given source. Can only be called before the output is started. */
addSubtitleTrack(source: SubtitleSource, metadata: SubtitleTrackMetadata = {}) {
if (!(source instanceof SubtitleSource)) {
throw new TypeError('source must be a SubtitleSource.');
@@ -245,7 +245,7 @@ export class Output<
* Sets descriptive metadata tags about the media file, such as title, author, date, or cover art. When called
* multiple times, only the metadata from the last call will be used.
*
* Must be called before output is started.
* Can only be called before the output is started.
*/
setMetadataTags(tags: MetadataTags) {
validateMetadataTags(tags);
+12 -4
View File
@@ -6,7 +6,7 @@
* file, You can obtain one at https://mozilla.org/MPL/2.0/.
*/
import { assert, clamp, MaybePromise, toDataView } from './misc';
import { assert, clamp, getUint24, MaybePromise, toDataView } from './misc';
import { Source } from './source';
export class Reader {
@@ -80,13 +80,19 @@ export class Reader {
}
export class FileSlice {
/** The current position in the backing buffer. Do not modify directly, prefer `.skip()` instead. */
bufferPos: number;
constructor(
/** The underlying bytes backing this slice. Avoid using this directly and prefer reader functions instead. */
public readonly bytes: Uint8Array,
/** A view into the bytes backing this slice. Avoid using this directly and prefer reader functions instead. */
public readonly view: DataView,
/** The offset in "file bytes" at which `bytes` begins in the file. */
private readonly offset: number,
/** The offset in "file bytes" where this slice begins. */
public readonly start: number,
/** The offset in "file bytes" where this slice ends (exclusive). */
public readonly end: number,
) {
this.bufferPos = start - offset;
@@ -118,6 +124,7 @@ export class FileSlice {
this.bufferPos += byteCount;
}
/** Creates a new subslice of this slice whose byte range must be contained within this slice. */
slice(filePos: number, length = this.end - filePos) {
if (filePos < this.start || filePos + length > this.end) {
throw new RangeError('Slicing outside of original slice.');
@@ -157,9 +164,10 @@ export const readU16Be = (slice: FileSlice) => {
};
export const readU24Be = (slice: FileSlice) => {
const high = readU16Be(slice);
const low = readU8(slice);
return high * 0x100 + low;
const value = getUint24(slice.view, slice.bufferPos, false);
slice.bufferPos += 3;
return value;
};
export const readI16Be = (slice: FileSlice) => {
+4 -1
View File
@@ -16,6 +16,7 @@
* - For MP3 files, the metadata refers to the ID3v2 or ID3v1 tags.
* - For Ogg files, there is no global metadata so instead, the metadata refers to the combined metadata of all tracks,
* in Vorbis-style comment headers.
* - For FLAC files, the metadata lives in Vorbis style in the Vorbis comment block.
* - For WAVE files, the metadata refers to the chunks within the RIFF INFO chunk.
* - For ADTS files, there is no metadata.
*
@@ -69,7 +70,9 @@ export type MetadataTags = {
* - Ogg: The key-value string pairs from the Vorbis-style comment header (see RFC 7845, Section 5.2).
* Additionally, the `'vendor'` key refers to the vendor string within this header.
* - WAVE: The individual metadata chunks within the RIFF INFO chunk. Values are always ISO 8859-1 strings.
*/
* - FLAC: The key-value string pairs from the vorbis metadata block (see RFC 9639, Section D.2.3).
* Additionally, the `'vendor'` key refers to the vendor string within this header.
*/
raw?: Record<string, string | Uint8Array | RichImageData | null>;
};
+68
View File
@@ -0,0 +1,68 @@
import { expect, test } from 'vitest';
import { Input } from '../../src/input.js';
import { BufferSource, UrlSource } from '../../src/source.js';
import { FLAC, WAVE } from '../../src/input-format.js';
import { AudioSampleSink } from '../../src/media-sink.js';
import { assert } from '../../src/misc.js';
import { Output } from '../../src/output.js';
import { WavOutputFormat } from '../../src/output-format.js';
import { BufferTarget } from '../../src/target.js';
import { Conversion } from '../../src/conversion.js';
test('can decode samples from a FLAC file', async () => {
const input = new Input({
source: new UrlSource('/sample.flac'),
formats: [FLAC],
});
const track = await input.getPrimaryAudioTrack();
assert(track);
const sink = new AudioSampleSink(track);
const sample = await sink.getSample(1);
assert(sample);
expect(sample.timestamp).toBe(0.9287981859410431);
});
test('can convert a .flac to .wav', async () => {
const input = new Input({
source: new UrlSource('/sample.flac'),
formats: [FLAC],
});
const output = new Output({
format: new WavOutputFormat(),
target: new BufferTarget(),
});
const conversion = await Conversion.init({ input, output });
await conversion.execute();
const buffer = output.target.buffer;
assert(buffer);
const outputAsInput = new Input({
source: new BufferSource(buffer),
formats: [WAVE],
});
const inputTrack = await input.getPrimaryAudioTrack();
assert(inputTrack);
const outputTrack = await outputAsInput.getPrimaryAudioTrack();
assert(outputTrack);
const duration = await outputTrack.computeDuration();
expect(duration).toBe(19.71428571428571);
const tags = await outputAsInput.getMetadataTags();
expect(tags.raw).toEqual({
IART: 'Samples Files',
ICRD: '2020-01-01',
IGNR: 'Ambient',
INAM: 'The Happy Meeting',
IPRD: 'Samples files',
ITRK: '4',
});
expect(inputTrack.sampleRate).toBe(outputTrack.sampleRate);
expect(inputTrack.numberOfChannels).toBe(outputTrack.numberOfChannels);
expect(inputTrack.timeResolution).toBe(outputTrack.timeResolution);
});
+256
View File
@@ -0,0 +1,256 @@
import { expect, test } from 'vitest';
import path from 'node:path';
import { assert, toUint8Array } from '../../src/misc.js';
import { Input } from '../../src/input.js';
import { BufferSource, FilePathSource } from '../../src/source.js';
import { ALL_FORMATS, FLAC } from '../../src/input-format.js';
import { EncodedPacketSink } from '../../src/media-sink.js';
import { Output } from '../../src/output.js';
import { BufferTarget } from '../../src/target.js';
import { FlacOutputFormat } from '../../src/output-format.js';
import { Conversion } from '../../src/conversion.js';
const __dirname = new URL('.', import.meta.url).pathname;
test('can loop over all samples', async () => {
const filePath = path.join(__dirname, '..', 'public/sample.flac');
const input = new Input({
source: new FilePathSource(filePath),
formats: ALL_FORMATS,
});
const track = await input.getPrimaryAudioTrack();
assert(track);
expect(await track.computeDuration()).toEqual(19.71428571428571);
expect(await track.getDecoderConfig()).toEqual({
codec: 'flac',
numberOfChannels: 2,
sampleRate: 44100,
description: new Uint8Array([
102, 76, 97, 67, 128, 0, 0, 34, 16, 0, 16, 0, 0, 6, 45, 0, 37, 173, 10,
196, 66, 240, 0, 13, 68, 24, 85, 22, 231, 0, 113, 139, 185, 1, 33, 54,
155, 80, 241, 191, 203, 112,
]),
});
expect(await track.getCodecParameterString()).toEqual('flac');
expect(track.timeResolution).toEqual(44100);
expect(await input.getMimeType()).toEqual('audio/flac');
const sink = new EncodedPacketSink(track);
let samples = 0;
let lastSampleTimestamp = 0;
for await (const sample of sink.packets()) {
samples++;
lastSampleTimestamp = sample.timestamp;
if (sample.sequenceNumber === 212) {
// Last frame is a bit shorter
// due it having a custom block size and the duration not being a multiple of the frame size
expect(sample.duration).toEqual(0.023764172335600908);
} else {
expect(sample.duration).toEqual(0.09287981859410431);
}
}
expect(samples).toBe(213);
expect(lastSampleTimestamp).toBe(19.690521541950112);
});
test('can do random access', async () => {
const filePath = path.join(__dirname, '..', 'public/sample.flac');
const input = new Input({
source: new FilePathSource(filePath),
formats: ALL_FORMATS,
});
const track = await input.getPrimaryAudioTrack();
assert(track);
const packetSink = new EncodedPacketSink(track);
const packet = await packetSink.getPacket(10);
assert(packet);
expect(packet.timestamp).toBe(9.93814058956916);
expect(packet.data.byteLength).toBe(8345);
expect(packet.sequenceNumber).toBe(107);
expect(packet.duration).toBe(0.09287981859410431);
const nextPacket = await packetSink.getNextPacket(packet);
assert(nextPacket);
expect(nextPacket.timestamp).toBe(10.031020408163265);
expect(nextPacket.data.byteLength).toBe(8988);
expect(nextPacket.sequenceNumber).toBe(108);
expect(nextPacket.duration).toBe(0.09287981859410431);
const priorPacket = await packetSink.getPacket(3);
assert(priorPacket);
expect(priorPacket.timestamp).toBe(2.972154195011338);
expect(priorPacket.data.byteLength).toBe(6877);
expect(priorPacket.sequenceNumber).toBe(32);
expect(priorPacket.duration).toBe(0.09287981859410431);
});
test('can get metadata-only packets', async () => {
const filePath = path.join(__dirname, '..', 'public/sample.flac');
const input = new Input({
source: new FilePathSource(filePath),
formats: ALL_FORMATS,
});
const track = await input.getPrimaryAudioTrack();
assert(track);
const packetSink = new EncodedPacketSink(track);
const packet = await packetSink.getPacket(10, { metadataOnly: true });
assert(packet);
expect(packet.timestamp).toBe(9.93814058956916);
expect(packet.isMetadataOnly).toBe(true);
expect(packet.sequenceNumber).toBe(107);
expect(packet.duration).toBe(0.09287981859410431);
});
test('can get metadata', async () => {
const filePath = path.join(__dirname, '..', 'public/sample.flac');
const input = new Input({
source: new FilePathSource(filePath),
formats: ALL_FORMATS,
});
const { images: inputImages, ...descriptiveMetadata } = await input.getMetadataTags();
expect(inputImages![0]!.data).toHaveLength(4852);
expect(inputImages![0]!.data.slice(0, 8)).toEqual(new Uint8Array([
137, 80, 78, 71, 13, 10, 26, 10,
]));
expect(inputImages![0]!.mimeType).toBe('image/png');
expect(inputImages![0]!.description).toBe('Album cover');
expect(inputImages![0]!.kind).toBe('coverFront');
expect(descriptiveMetadata).toEqual({
title: 'The Happy Meeting',
date: new Date('2020'),
album: 'Samples files',
artist: 'Samples Files',
trackNumber: 4,
genre: 'Ambient',
raw: {
ALBUM: 'Samples files',
ARTIST: 'Samples Files',
DATE: '2020',
ENCODER: 'Lavf58.76.100',
GENRE: 'Ambient',
TITLE: 'The Happy Meeting',
TRACKNUMBER: '4',
vendor: 'Lavf58.76.100',
},
});
});
test('can re-mux a .flac', async () => {
const filePath = path.join(__dirname, '..', 'public/sample.flac');
const input = new Input({
source: new FilePathSource(filePath),
formats: ALL_FORMATS,
});
let framesWritten = 0;
const output = new Output({
format: new FlacOutputFormat({
onFrame() {
framesWritten++;
},
}),
target: new BufferTarget(),
});
const conversion = await Conversion.init({ input, output });
await conversion.execute();
expect(framesWritten).toBe(213);
const buffer = output.target.buffer;
assert(buffer);
const outputAsInput = new Input({
source: new BufferSource(buffer),
formats: [FLAC],
});
const outputTrack = await outputAsInput.getPrimaryAudioTrack();
assert(outputTrack);
const inputTrack = await input.getPrimaryAudioTrack();
assert(inputTrack);
expect(inputTrack.sampleRate).toBe(outputTrack.sampleRate);
expect(inputTrack.numberOfChannels).toBe(outputTrack.numberOfChannels);
expect(inputTrack.timeResolution).toBe(outputTrack.timeResolution);
const outputMetadataTags = await outputAsInput.getMetadataTags();
const inputMetadataTags = await input.getMetadataTags();
expect(inputMetadataTags.images).toHaveLength(1);
expect(Object.keys(outputMetadataTags)).toEqual([
'images',
'raw',
'title',
'date',
'album',
'artist',
'trackNumber',
'genre',
]);
const { images: inputImages, ...otherInputMetadataTags } = inputMetadataTags;
expect(outputMetadataTags).toEqual({
...otherInputMetadataTags,
raw: {
...otherInputMetadataTags.raw,
vendor: 'Mediabunny',
},
images: inputImages,
});
const inputPacketSink = new EncodedPacketSink(inputTrack);
const outputPacketSink = new EncodedPacketSink(outputTrack);
let packets = 0;
let timestamp = 0;
for await (const packet of outputPacketSink.packets()) {
packets++;
timestamp = packet.timestamp;
}
expect(packets).toBe(213);
expect(timestamp).toBe(19.690521541950112);
// Test that packets are byte-identical
const inputPacket = await inputPacketSink.getPacket(10);
const outputPacket = await outputPacketSink.getPacket(10);
assert(inputPacket);
assert(outputPacket);
const { data: inputPacketData, ...otherInputPacket } = inputPacket;
const { data: outputPacketData, ...otherOutputPacket } = outputPacket;
expect(otherInputPacket).toEqual(otherOutputPacket);
expect(inputPacketData).toEqual(outputPacketData);
const inputDecoderConfig = await inputTrack.getDecoderConfig();
const outputDecoderConfig = await outputTrack.getDecoderConfig();
assert(inputDecoderConfig);
assert(outputDecoderConfig);
const { description: inputDescription, ...otherInputDecoderConfig } = inputDecoderConfig;
const { description: outputDescription, ...otherOutputDecoderConfig } = outputDecoderConfig;
assert(inputDescription);
assert(outputDescription);
const inputArray = toUint8Array(inputDescription);
const outputArray = toUint8Array(outputDescription);
const inputWithoutCrc = inputArray.slice(0, -16);
const outputWithoutCrc = outputArray.slice(0, -16);
expect(inputWithoutCrc).toEqual(outputWithoutCrc);
expect(otherInputDecoderConfig).toEqual(otherOutputDecoderConfig);
});
+65 -6
View File
@@ -1,6 +1,7 @@
import { expect, test } from 'vitest';
import { Output } from '../../src/output.js';
import {
FlacOutputFormat,
MkvOutputFormat,
MovOutputFormat,
Mp3OutputFormat,
@@ -34,10 +35,18 @@ const createDummyAudioTrack = (codec: AudioCodec, output: Output) => {
data[2] = 224;
data[3] = 100;
// Opus description
const description = new Uint8Array([
79, 112, 117, 115, 72, 101, 97, 100, 1, 2, 56, 1, 68, 172, 0, 0, 0, 0, 0,
]);
const description = codec === 'flac'
? new Uint8Array([
102, 76, 97, 67, 128, 0, 0, 34, 16, 0,
16, 0, 0, 6, 45, 0, 37, 173, 10, 196,
66, 240, 0, 13, 68, 24, 85, 22, 231, 0,
113, 139, 185, 1, 33, 54, 155, 80, 241, 191,
203, 112,
])
// Opus description
: new Uint8Array([
79, 112, 117, 115, 72, 101, 97, 100, 1, 2, 56, 1, 68, 172, 0, 0, 0, 0, 0,
]);
await source.add(
new EncodedPacket(data, 'key', 0, 1),
@@ -299,7 +308,7 @@ test('Read and write metadata, Ogg', async () => {
output.setMetadataTags({
...songMetadata,
raw: {
vendor: 'mediabunny corp',
vendor: 'Mediabunny',
COMPOSER: 'Hans Zimmer',
},
});
@@ -336,7 +345,57 @@ test('Read and write metadata, Ogg', async () => {
expect(readTags.images![0]!.description).toEqual(songMetadata.images![0]!.description);
expect(readTags.images![0]!.name).toBeUndefined(); // Can't be contained in Vorbis-style metadata
expect(readTags.raw!['vendor']).toBe('mediabunny corp');
expect(readTags.raw!['vendor']).toBe('Mediabunny');
expect(readTags.raw!['COMPOSER']).toBe('Hans Zimmer');
});
test('Read and write metadata, FLAC', async () => {
const output = new Output({
format: new FlacOutputFormat(),
target: new BufferTarget(),
});
output.setMetadataTags({
...songMetadata,
raw: {
vendor: 'Mediabunny',
COMPOSER: 'Hans Zimmer',
},
});
const dummyTrack = createDummyAudioTrack('flac', output);
await output.start();
await dummyTrack.addPacket();
await output.finalize();
const input = new Input({
source: new BufferSource(output.target.buffer!),
formats: ALL_FORMATS,
});
const readTags = await input.getMetadataTags();
expect(readTags.title).toBe(songMetadata.title);
expect(readTags.description).toBe(songMetadata.description);
expect(readTags.artist).toBe(songMetadata.artist);
expect(readTags.album).toBe(songMetadata.album);
expect(readTags.albumArtist).toBe(songMetadata.albumArtist);
expect(readTags.comment).toBe(songMetadata.comment);
expect(readTags.lyrics).toBe(songMetadata.lyrics);
expect(readTags.trackNumber).toBe(songMetadata.trackNumber);
expect(readTags.tracksTotal).toBe(songMetadata.tracksTotal);
expect(readTags.discNumber).toBe(songMetadata.discNumber);
expect(readTags.discsTotal).toBe(songMetadata.discsTotal);
expect(readTags.date).toEqual(readTags.date);
expect(readTags.images).toHaveLength(1);
expect(readTags.images![0]!.data).toEqual(coverArt);
expect(readTags.images![0]!.mimeType).toEqual('image/jpeg');
expect(readTags.images![0]!.kind).toEqual('coverFront');
expect(readTags.images![0]!.description).toEqual(songMetadata.images![0]!.description);
expect(readTags.images![0]!.name).toBeUndefined(); // Can't be contained in Vorbis-style metadata
expect(readTags.raw!['vendor']).toBe('Mediabunny');
expect(readTags.raw!['COMPOSER']).toBe('Hans Zimmer');
});
Binary file not shown.
+1
View File
@@ -5,6 +5,7 @@ export default defineConfig({
browser: {
provider: 'webdriverio',
instances: [{ browser: 'chrome' }],
headless: true,
},
},
});