Skip to content

Commit 9aa7b65

Browse files
committed
Clarifications on preferred approach
1 parent 60a38cb commit 9aa7b65

File tree

1 file changed

+9
-7
lines changed

1 file changed

+9
-7
lines changed

OfflineAudioContext/explainer.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,17 +15,17 @@
1515

1616
WebAudio `OfflineAudioContext.startRendering()` allocates an `AudioBuffer` large enough to hold the entire render WebAudio graph before returning. For example, a 4 hour audio graph at 48 kHz with 4 channels will create gigabytes of in-memory float32 data in the `AudioBuffer`. This behaviour makes the API is unsuitable for very long offline renders or very large channel/length combinations. There is no simple way to chunk the output or consume it as a stream.
1717

18-
The [spec](https://webaudio.github.io/web-audio-api/#dom-offlineaudiocontext-startrendering) explicitly says at step 5: "Create a new AudioBuffer ... with ... length and sampleRate ... Assign this buffer to an internal slot" which means the API design currently mandates the full buffer allocation.
18+
The [spec](https://webaudio.github.io/web-audio-api/#dom-offlineaudiocontext-startrendering) explicitly states at step 5: "Create a new AudioBuffer ... with ... length and sampleRate ... Assign this buffer to an internal slot" which means the API design currently mandates the full buffer allocation.
1919

2020
The participants on the [GitHub discussion](https://github.com/WebAudio/web-audio-api/issues/2445) agree that incremental delivery of data is necessary. Either streaming chunks of rendered audio or dispatching data in bits rather than everything at once so that memory usage is bounded and the data can be processed/consumed as it is produced.
2121

2222
## User-Facing Problem
2323

24-
The user in this context is the web developer using the WebAudio API. Their goal is to perform media processing using the feature-rich WebAudio API without taking a dependency on a 3rd party library to render the graph in an offline context. Because the current WebAudio OfflineAudioContext API is not suitable for this use case, the developer needs to create a WASM audio processing library or use an existing 3rd party dependency to perform the workload.
24+
The user in this context is the web developer using the WebAudio API to perform media processing workflows. Ideally developers could use the feature-rich WebAudio API for realtime and faster-than-realtime processing, without taking a dependency on a 3rd party library. However, in reality, the current WebAudio OfflineAudioContext API is not suitable for faster-than-raltime processing so the developer needs to create a WASM audio processing library or use an existing 3rd party dependency to achieve this goal.
2525

2626
### Goals
2727

28-
- Allow streaming data out of an WebAudio `OfflineAudioContext.startRendering()` for rendering large WebAudio graphs faster-than-realtime
28+
- Allow streaming data out of a WebAudio `OfflineAudioContext.startRendering()` for rendering large WebAudio graphs faster-than-realtime
2929

3030
### Non-goals
3131

@@ -46,7 +46,9 @@ interface OfflineAudioContext {
4646
}
4747
```
4848

49-
In "stream" mode the implementation will not allocate a giant `AudioBuffer` upfront if `startRenderingOptions.mode = "stream"`. Instead it will render in quantums (e.g., 128 frames at a time), and enqueue chunks onto a return `ReadableStream`. [ReadableStream](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/getReader) will return a reader. `reader.read()` will resolve stream values to an AudioBuffer until it is done and when no more data is available it will set `done = true`. In this mode, the user can read chunks as they arrive and consume them for storage, transcoding via WebCodecs, sending to a server, etc. An alternative is to allow BYOB reading, in this case `reader.read()` will return a Float32Array.
49+
If the `startRendering` function is passed `{ mode = "stream" }` then it will render the audio graph in quantums (e.g., 128 frames at a time), and enqueue chunks onto a return `ReadableStream`, rather than rendering the whole graph into an `AudioBuffer` up front as it does currently. `startRendering` will return a [ReadableStream](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/getReader) promise. A `reader` can be retrieved off the `ReadableStream` for reading chunks. `reader.read()` will resolve stream values to an `AudioBuffer` until it is done. When no more data is available it will set `done = true`.
50+
51+
In this mode, the user can read chunks as they arrive and consume them for storage, transcoding via WebCodecs, sending to a server, etc. An alternative is to allow BYOB reading, in this case `reader.read()` will return a Float32Array.
5052

5153
Memory usage is bounded by the size of each chunk plus the backlog of unhandled buffers.
5254

@@ -73,11 +75,11 @@ const offlineContext = new OfflineAudioContext(...);
7375

7476
// ... build up WebAudio graph
7577

76-
const readable = await offlineContext.startRendering(options: { mode: "stream"});
77-
const stream = readable.getReader();
78+
const stream = await offlineContext.startRendering(options: { mode: "stream"});
79+
const reader = readable.getReader();
7880
while (true) {
7981
// get the next chunk of data from the stream
80-
const result = await stream.read();
82+
const result = await reader.read();
8183

8284
// the reader returns done = true when there are no more chunks to consume
8385
if (result.done) {

0 commit comments

Comments
 (0)