mirror of
https://github.com/arcodange-org/mediabunny.git
synced 2026-09-27 10:53:50 +02:00
Add BufferTarget.onFinalize, OutputOptions.onFinalize, ConcurrentRunner (#349)
* Add BufferTarget.onFinalize option for awaitable async action on finalize The `finalized` event fires synchronously and its return value is ignored, making it unsuitable for use cases where the muxer should wait (e.g. uploading the buffer to S3, R2, or other object stores that require a known Content-Length and therefore can't stream via StreamTarget). Adds a new `BufferTargetOptions` type with an `onFinalize` callback that the muxer awaits before resolving. Matches the existing callback pattern used by `HlsOutputFormatOptions.onSegment`, `onMaster`, etc. When used with PathedTarget, this provides proper backpressure: the next segment won't start being produced until the previous one has finished uploading, keeping memory bounded regardless of video length. - Adds `BufferTargetOptions` type, exported from the package root - `BufferTarget` constructor now accepts optional options (backward compatible) - `_finalize()` awaits `onFinalize` before emitting the `finalized` event - Adds runtime validation for non-function callbacks - Updates "Upload to a server" docs in writing-hls.md with S3 pattern - Documents `onFinalize` in writing-media-files.md BufferTarget section - Adds tests for callback invocation, async awaiting, backward compat, and validation * Add ConcurrentRunner, add OutputOptions.onFinalize, adjust docs accordingly, clean up tests * give me more control baby * forgot to tell about it * polish guide and js docs * vanilagy says remove this, docs will reveal it --------- Co-authored-by: Vanilagy <[email protected]>
This commit is contained in:
co-authored by
Vanilagy
parent
9a7120f5f4
commit
77e70f9090
@@ -115,7 +115,9 @@ await output.finalize();
|
||||
|
||||
### Upload to a server
|
||||
|
||||
This models a stream upload, where files are being uploaded *while* they are being created.
|
||||
#### Stream upload
|
||||
|
||||
This code models a stream upload, where files are being uploaded *while* they are being created:
|
||||
|
||||
```ts
|
||||
const promises: Promise<Response>[] = [];
|
||||
@@ -146,17 +148,52 @@ const output = new Output({
|
||||
return new StreamTarget(writable);
|
||||
},
|
||||
),
|
||||
onFinalize: () => Promise.all(promises),
|
||||
// ...
|
||||
});
|
||||
|
||||
// ...
|
||||
await output.finalize();
|
||||
|
||||
await Promise.all(promises);
|
||||
// All files have been uploaded to the server
|
||||
```
|
||||
|
||||
If this is too fancy, you can always use `BufferTarget` instead and upload its contents to a server in a non-streaming way after the `finalized` event.
|
||||
#### Monolithic upload
|
||||
|
||||
If streaming is not possible (e.g. when uploading to S3 via signed `PutObject`, which requires a known `Content-Length`), you can use [`BufferTarget`](../api/BufferTarget) with the [`onFinalize`](../api/BufferTargetOptions#onfinalize) option instead.
|
||||
|
||||
You could call `fetch` directly but this would halt Mediabunny's internals until the upload has completed. Instead, using a [`ConcurrentRunner`](../api/ConcurrentRunner) allows Mediabunny to keep producing data internally while the upload is in flight, while also allowing multiple concurrent uploads:
|
||||
|
||||
```ts
|
||||
import { ConcurrentRunner, ... } from 'mediabunny';
|
||||
|
||||
// This Mediabunny utility class is used to allow up to two requests
|
||||
// to run concurrently. When this number is exceeded, backpressure is
|
||||
// automatically applied internally.
|
||||
const runner = new ConcurrentRunner(2);
|
||||
|
||||
const output = new Output({
|
||||
target: new PathedTarget(
|
||||
'master.m3u8',
|
||||
({ path, mimeType }) =>
|
||||
new BufferTarget({
|
||||
onFinalize: buffer => runner.run(() =>
|
||||
fetch(`/upload?file=${encodeURIComponent(path)}`, {
|
||||
method: 'PUT',
|
||||
body: buffer,
|
||||
headers: {
|
||||
'Content-Type': mimeType,
|
||||
},
|
||||
})
|
||||
),
|
||||
}),
|
||||
),
|
||||
onFinalize: () => runner.flush(),
|
||||
// ...
|
||||
});
|
||||
|
||||
await output.finalize();
|
||||
// All files have been uploaded to the server
|
||||
```
|
||||
|
||||
## Adding tracks & media
|
||||
|
||||
|
||||
@@ -216,6 +216,10 @@ await output.finalize();
|
||||
const file = output.target.buffer; // => Uint8Array
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
An optional [`onFinalize`](../api/OutputOptions#onfinalize) callback can be provided in the output options. This function will be called at the end of `.finalize()` and can be used to do work once the output has been completed. If it returns a promise, it will be awaited by the output.
|
||||
|
||||
## Canceling an output
|
||||
|
||||
Sometimes, you may want to cancel the ongoing creation of an output file. For this, use the `cancel` method:
|
||||
@@ -288,6 +292,24 @@ output.target.buffer; // => ArrayBuffer
|
||||
|
||||
This target is a great choice for small-ish files (< 100 MB), but since all data will be kept in memory, using it for large files is suboptimal. If the output gets very large, the page might crash due to memory exhaustion. For these cases, using `StreamTarget` is recommended.
|
||||
|
||||
#### `onFinalize` callback
|
||||
|
||||
`BufferTarget` accepts an `onFinalize` option, which is called with the complete buffer once the target has been finalized. Useful for uploading the final buffer to a server or object store (e.g. S3 `PutObject`, which requires a known `Content-Length`):
|
||||
|
||||
```ts
|
||||
const output = new Output({
|
||||
target: new BufferTarget({
|
||||
onFinalize: async (buffer) => {
|
||||
await fetch('/upload', { method: 'PUT', body: buffer });
|
||||
},
|
||||
}),
|
||||
// ...
|
||||
});
|
||||
|
||||
await output.finalize();
|
||||
// The upload has completed by the time finalize resolves.
|
||||
```
|
||||
|
||||
### `StreamTarget`
|
||||
|
||||
This target passes you the data written by the `Output` in small chunks, requiring you to pipe that data elsewhere to manually assemble the final file. Example use cases include writing the file directly to disk, or uploading it to a server over the network.
|
||||
|
||||
Reference in New Issue
Block a user