feat: route-level body mutation for top-level field

[xiaolin593]

Description

Support route-level body mutation for top-level field. route level body mutation is useful when have heterogeneous cloud-provider backends support same model but with slightly different request body fields to override esp OpenAI's service_tier field

Related Issues/PRs (if applicable)

Close #1481

[dosubot]

Related Documentation

Checked 1 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[codecov-commenter]

Codecov Report

❌ Patch coverage is 80.76923% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.93%. Comparing base (40fa3d6) to head (f1e54b1).

Files with missing lines	Patch %	Lines
internal/extproc/chatcompletion_processor.go	12.50%	6 Missing and 1 partial ⚠️
internal/extproc/messages_processor.go	25.00%	5 Missing and 1 partial ⚠️
internal/extproc/bodymutator/body_mutator.go	92.59%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1492      +/-   ##
==========================================
- Coverage   83.95%   83.93%   -0.02%     
==========================================
  Files         143      144       +1     
  Lines       12618    12696      +78     
==========================================
+ Hits        10594    10657      +63     
- Misses       1408     1420      +12     
- Partials      616      619       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mathetake]

I think this is not bad and looks a good start. The API still leaves a room for the future addition of more complex jsonpatch style!

cc @tiswanso do you have any comments?

[xiaolin593]

I think this is not bad and looks a good start. The API still leaves a room for the future addition of more complex jsonpatch style!

cc @tiswanso do you have any comments?

Looks like top-level field can solve the use-case cited at here #1386 (reply in thread) already?

What is the specific need for jsonpatch deep level field mutation? Wondering if we ever encounter such complex use case for ChatCompletion, Completion or other AI workload APIs.

[mathetake]

at least vendor specific fields forced at gateway would be the use case, i.e. any field in https://aigateway.envoyproxy.io/docs/capabilities/llm-integrations/vendor-specific-fields/ so the path might be nested at least not necessarily require jsonpatch though

[xiaolin593]

had to force-push due to one of commits wasn't signed.