d426d36386
* add a test * recognize as input format * scaffold flac demuxer * implement getting metadata * Implement mime type * read all metadata + deduplicate stubs * Read first packet * copyright headers * read the first packet * Get entire first packet, work on advancing * iterate over all samples * testable with bun * stub out metadata support * parse descriptive metadata * All in 1 file seems more appropriate to the philosophy * some parameters are not needed anymore all within 1 class * skip over bytes we are sure are not the syncword * run prettier * timestamp is determined based on passed blocks, not maximumBlockSize * no binary search needed! * simplifications * Finish demuxer reading sequentially * more tests + add a file with a seektable * don't throw if (this.audioInfo.minimumBlockSize !== this.audioInfo.maximumBlockSize * Add docs * Update format compatibility table * Simplification * confirm conversion is working * Finish * Resolve TODO comment * Support images (read-only) * Returning description as Uint8Array * Cleanup of demuxer * Misc renames * Object on same line * Resolve first batch of comments * Throw errors on corrupt blocks, correctly use requestSlice() * Fix description field * Explain why last frame is a bit shorter * Add FLAC to README * Put track backings below demuxer * convert to methods * Reorder container checking * getBlockSize() -> readBlockSize() * bitStream -> bitstream * better naming for bytes * use .skip() * Don't return blockSize twice in readFlacFrameHeader * Use enum + switch to distinguish Flac block types * Use else-if * Use else-if * Compressed switch statement * readCodedNumber * Update flac-misc.ts * We don't need the bits variable at all * reorder functions in flac-demuxer * Handle gracefully not being able to load another sample * Update flac-demuxer.ts * blockingbit null * use async instead of promise.resolve * use binary search * Replace recursion with while loop * Add mutex to getPacket() * Load more data not in getPacketAtIndex, but outside * Apply suggestion from @Vanilagy Co-authored-by: David P. <[email protected]> * computeDuration() reads last packet * Update flac-demuxer.ts * share vorbis comment reading logic * reuse vorbis comment writing logic, set vendor always to "Mediabunny" * flush after writign * Use FileSlice.tempFromBytes * assert !== null * fixing nitpicks * apply suggestions * fix ogg * We are now muxing images * apply suggestion * Update src/flac/flac-muxer.ts Co-authored-by: David P. <[email protected]> * apply suggestion * readSampleRate() * seek outside writeHeader() * mention vorbis metadata + add to metadatatags comment * should be able to -> can * Add test in metadata tags * Add test for packets being byte identical after remuxing * compare to null * no casting to uint8array * make test pass * Update flac-muxer.ts * Call validateAudioChunkMetadata() and validateAndNormalizeTimestamp() * `onFrame` option * emit frames using onFrame * Run prettier over files * Fix FLAC PICTURE block logic, small other changes * Update docs * Remove .only modifier * fix remuxing and add test * Don't throw error if parsing fails in header, since hitting a syncword might just be coincidential * Fix remaining type errors --------- Co-authored-by: David P. <[email protected]>
5.3 KiB
Introduction
Mediabunny is a JavaScript library for reading, writing, and converting media files (like MP4 or WebM), directly in the browser. It aims to be a complete toolkit for high-performance media operations on the web. It's written from scratch in pure TypeScript, has zero dependencies, and is extremely tree-shakable, meaning you only include what you use. You can think of it a bit like FFmpeg, but built for the web's needs.
Features
Here's a long list of stuff this library does:
- Reading metadata from media files
- Extracting media data from media files
- Creating new media files
- Converting media files
- Hardware-accelerated decoding & encoding (via the WebCodecs API)
- Support for multiple video, audio and subtitle tracks
- Read & write support for many container formats (.mp4, .mov, .webm, .mkv, .mp3, .wav, .ogg, .aac, .flac), including variations such as MP4 with Fast Start, fragmented MP4, or streamable Matroska
- Support for 25 different codecs
- Lazy, optimized, on-demand file reading
- Input and output streaming, arbitrary file size support
- File location independence (memory, disk, network, ...)
- Utilities for compression, resizing, rotation, cropping, resampling, trimming
- Transmuxing and transcoding
- Microsecond-accurate reading and writing precision
- Efficient seeking through time
- Pipelined design for efficient hardware usage and automatic backpressure
- Custom encoder & decoder support for polyfilling
- Low- & high-level abstractions for different use cases
- Performant everything
- Node.js support
...and there's probably more.
Use cases
Mediabunny is a general-purpose toolkit and can be used in infinitely many ways. But, here are a few ideas:
- File conversion & compression
- Displaying file metadata (duration, dimensions, ...)
- Extracting thumbnails
- Creating videos in the browser
- Building a video editor
- Live recording & streaming
- Efficient, sample-accurate playback of large files via the Web Audio API
Check out the Examples page for demo implementations of many of these ideas!
Getting started
To get going with Mediabunny, here are some starting points:
- Check out Quick start for a collection of useful code snippets
- Start with Reading media files if you want to do read operations.
- Start with Writing media files if you want to do write operations.
- Start with Converting media files if you care about file conversions.
- Dive into Packets & samples for a deeper understanding of the concepts underlying this library.
Motivation
Mediabunny is the evolution of my previous libraries, mp4-muxer and webm-muxer, which were both created due to the advent of the WebCodecs API. While they fulfilled their job just fine, I saw a few painpoints:
- Lots of duplicated code between the two libraries, otherwise very similar API.
- No help with the difficulties of navigating the WebCodecs API & related browser APIs.
- "mp4-demuxer when??"
This library is the result of unifying these libraries into one, solving all the above issues, and expanding the scope. Now:
- Changing the output file format is a single-line change; the rest of the API is identical.
- Lots of abstractions on top of the WebCodecs API & browser APIs are provided.
- mp4-demuxer now.
Due to tree shaking, if you only need an MP4 or WebM muxer, this library's bundle size will still be very small.
Migration
If you're coming from mp4-muxer or webm-muxer, you should migrate to Mediabunny. For that, refer to these guides:
Technical overview
At its core, Mediabunny is a collection of multiplexers and demultiplexers, one of each for every container format. Demultiplexers stream data from sources, while multiplexers stream data to targets. Every demultiplexer is capable of extracting file metadata as well as compressed media data, while multiplexers write metadata and encoded media data into a new file.
Mediabunny then provides several wrappers around the WebCodecs API to simplify usage: for reading, it creates decoders with the correct codec configuration and efficiently decodes media data in a pipelined way. For writing, it figures out the necessary codec configuration and sets up encoders which are then used to encode raw media data, while respecting the backpressure applied by the encoder. Extracting the right decoder configuration from a media file can be tricky and sometimes involves diving into encoded media packet bitstreams.
The conversion abstraction is built on top of Mediabunny's reading and writing primitives and combines them both in a heavily-pipelined way, making sure reading and writing happen in lockstep. It also consists of a lot of conditional logic probing output track compatibility, decoding support, and finding encodable codec configurations. It makes use of the Canvas API for video processing operations, and uses a custom implementation for audio resampling and up/downmixing.