Add logic for encoding video frames that change size over time (#63)

This commit is contained in:
Vanilagy
2025-08-24 16:37:03 +02:00
parent ad26edc6b5
commit 63b8190185
10 changed files with 160 additions and 44 deletions
+2
View File
@@ -141,6 +141,8 @@ If `width` or `height` is used in conjunction with `rotation`, they control the
If you want to apply max/min constraints to a video's dimensions, check out [track-specific options](#track-specific-options).
In the rare case that the input video changes size over time, the `fit` field can be used to control the size change behavior (see [`VideoEncodingConfig`](./media-sources#video-encoding-config)). When unset, the behavior is `'passThrough'`.
### Adjusting frame rate
The `frameRate` property can be used to set the frame rate of the output video in Hz. If not specified, the original input frame rate will be used (which may be variable).
+2
View File
@@ -54,6 +54,7 @@ type VideoEncodingConfig = {
hardwareAcceleration?: 'no-preference' | 'prefer-hardware' | 'prefer-software';
scalabilityMode?: string;
contentHint?: string;
sizeChangeBehavior?: 'deny' | 'passThrough' | 'fill' | 'contain' | 'cover';
onEncodedPacket?: (
packet: EncodedPacket,
@@ -73,6 +74,7 @@ type VideoEncodingConfig = {
- `hardwareAcceleration`: A hint that configures the hardware acceleration method of this codec. This is best left on `'no-preference'`.
- `scalabilityMode`: An encoding scalability mode identifier as defined by [WebRTC-SVC](https://w3c.github.io/webrtc-svc/#scalabilitymodes*).
- `contentHint`: An encoding video content hint as defined by [mst-content-hint](https://w3c.github.io/mst-content-hint/#video-content-hints).
- `sizeChangeBehavior`: Video frames may change size overtime. This field controls the behavior in case this happens. Defaults to `'deny'`.
- `onEncodedPacket`: Called for each successfully encoded packet. Useful for determining encoding progress.
- `onEncoderConfig`: Called when the internal encoder config, as used by the WebCodecs API, is created. You can use this to introspect the full codec string.
+11
View File
@@ -343,6 +343,17 @@ draw(
```
These methods behave like [drawImage](https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/drawImage) and paint the video frame at the given position with the given dimensions. This method will automatically draw the frame with the correct rotation based on its `rotation` property.
The `drawWithFit` method can be used to draw the video sample to fill an entire canvas with a specified fitting algorithm:
```ts
drawWithFit(
context: CanvasRenderingContext2D | OffscreenCanvasRenderingContext2D,
options: {
fit: 'fill' | 'contain' | 'cover';
rotation?: Rotation; // Overrides the sample's rotation
},
): void;
```
If you want to draw the raw underlying image to a canvas directly (without respecting the rotation metadata), then you can use the following method:
```ts
videoSample.toCanvasImageSource(); // => VideoFrame | OffscreenCanvas;