[Feat] Add support for terminating events in streaming APIs

[baywet]

OAI 3.2.0 added support for streaming media types via the itemSchema property, and that's a great improvement!

Observing some APIs from wild that are widely used, it seems that it's a common practice to have a "terminating event" so the server can tell the client "we're done" (in an expected way, not like disconnecting them because something is wrong).

See OpenAI which will reply back

[DONE]

Note: in that scenario it's not JSON, it's a text token.

There does not seem to be a "place" for this information today, but maybe this is something that could be added to 3.3?

[handrews]

@baywet this was discussed quite a bit when adding the SSE support. The problem is that terminating events are not part of the media type, and the Media Type Object models media types (hence he name 😜 ).

There are many ways that people make use of media types that add additional semantics, but we don't have a place for that beyond using schemas to convey constraints on the media type, which are often semantically significant.

What's not clear to me is what would even be needed here. You can already provide a distinct subschema for a "done" event, and (while this is probably not obvious to most people and requires understanding JSON Schema annotations), you could use JSON Schema annotations to detect when one is seen.

But the bigger issue here is that OpenAI's "done" sentinels are just one possible semantic convention built on top of the media type. If we want to model semantic conventions, those can't go in the media type object, unless they can be identified by a media type parameter such as a profile. Maybe. I have not thought that part through yet, TBH.

[baywet]

Thanks for all the additional context here!
When I initially started this conversation, I thought we could maybe add additional properties to the media type object like:

• terminatingItemSchema
• terminatingItemEncoding

This way people would be able to specify both. But it might be too narrow for this use case.

I'm guessing an alternative way to convey that information with 3.2.0 would be to have:

1. a oneOf itemSchema
2. a prefixEncoding??
   But that would probably be twisting the intended use around.

Can you please provide an example of what you meant by using schema annotations?

[handrews]

I'll come back and do the schema annotation thing when I have a bit more time, but really there ought to be a postfixItems JSON Schema keyword, and then we would mirror that like we mirror prefixItems. But really all that does is enforce that something is the last thing in a stream, and that would only really work if the stream is, in some HTTP sense, closed, so that we know the transmitted media type document is complete.

If what OpenAI is doing instead is leaving the stream open but using this special event value to indicate a logical end-of-stream (when the underlying stream stays open, allowing another logical sequences of events to flow if it keeps being read), then postfixItems wouldn't even be correct. In that case the [done] is not in a special position within the media type, it is instead a purely application-level semantic construct.