Skip to content

sycl: unify unary kernels with a generic implementation and enable wide operator support #17213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
Nov 15, 2025
Merged

sycl: unify unary kernels with a generic implementation and enable wide operator support #17213

merged 15 commits into from
Nov 15, 2025
+2,491 −430

Conversation

@shani-f
Copy link
Contributor

@shani-f shani-f commented Nov 12, 2025

Summary

Adds a generic unary implementation for the SYCL backend, allowing many unary operators to share a single optimized execution path.
The implementation matches the behavior of the existing CPU unary kernels.

Changes

  • Added ggml_sycl_op_unary generic function
  • Updated unary dispatch in element_wise.cpp
  • Removed per-op SYCL kernels (ABS, SGN, NEG, STEP, etc.)
  • Updated documentation in:
    • docs/ops.md
    • docs/ops/SYCL.csv

Implementation

  • One templated kernel handles all unary ops
  • Supports 4-D tensors and non-contiguous views
  • Supports F16 and F32 data types
  • Uses dispatch_ggml_sycl_op_unary with parallel_for
  • Eliminates duplicated indexing logic across operators

Supported Ops

  • ABS
  • SGN
  • NEG
  • STEP
  • RELU
  • HARDSIGMOID
  • TANH
  • GELU
  • SILU
  • SIGMOID
  • HARDSWISH
  • GELU_QUICK
  • GELU_ERF
  • EXP
  • ELU

Testing

  • All supported unary ops pass test-backend-ops
  • Verified correctness on contiguous + non-contiguous tensors
  • Matches CPU results

Performance

  • Single optimized unary path for all ops
  • Reduced kernel count and maintenance complexity
  • Same SYCL scheduling style as existing ops

Compatibility

  • Works on OpenCL and Level Zero devices
  • No changes required for CPU fallback
  • Follows SYCL backend design conventions

@github-actions github-actions bot added documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Nov 12, 2025
@DajanaV DajanaV mentioned this pull request Nov 12, 2025
Closed
@shani-f
Copy link
Contributor Author

shani-f commented Nov 12, 2025

Hello @CISC @NeoZhangJianyu,
The ggml-ci-x64-cpu-low-perf test failed, but it’s CPU-only and unrelated to my SYCL changes.
What’s the next step toward merging?
Thanks!

NeoZhangJianyu
NeoZhangJianyu approved these changes Nov 13, 2025
View reviewed changes
Copy link
Collaborator

@NeoZhangJianyu NeoZhangJianyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good job!
Thank you!

@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented Nov 13, 2025

Hello @CISC @NeoZhangJianyu, The ggml-ci-x64-cpu-low-perf test failed, but it’s CPU-only and unrelated to my SYCL changes. What’s the next step toward merging? Thanks!

We could merge directly after the conflicts are fixed.

@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented Nov 14, 2025

@shani-f
Could you fix the conflicts?

Thank you!

@shani-f
Copy link
Contributor Author

shani-f commented Nov 14, 2025

Hello @NeoZhangJianyu,
Yesterday I resolved all conflicts and the branch was fully up to date.
The new conflicts came from the latest merge into master, and I’ve resolved those as well.
Thanks!

CISC
CISC reviewed Nov 14, 2025
View reviewed changes
@DajanaV DajanaV mentioned this pull request Nov 15, 2025
Open
CISC
CISC reviewed Nov 15, 2025
View reviewed changes
@CISC
Copy link
Collaborator

CISC commented Nov 15, 2025

Regen the CSV, at least TOPK_MOE should no longer be there.

@shani-f
Copy link
Contributor Author

shani-f commented Nov 15, 2025 via email •
edited
Loading

@CISC
Copy link
Collaborator

CISC commented Nov 15, 2025

Is everything correct and finalized now?

I really did mean that you should regen it, there are more ops in the CSV that would be removed if you did.

@shani-f
Copy link
Contributor Author

shani-f commented Nov 15, 2025

Hello @CISC,
I tried to regenerate the CSV, but I couldn’t find any mechanism in the current LLAMA/ggml version that actually rebuilds SYCL.csv automatically.
If you want, I can reset both ops.md and SYCL.csv to their upstream versions and leave them as-is.

Let me know what you prefer.
Thanks!

@CISC
Copy link
Collaborator

CISC commented Nov 15, 2025

I tried to regenerate the CSV, but I couldn’t find any mechanism in the current LLAMA/ggml version that actually rebuilds SYCL.csv automatically.

As explained here, simply run test-backend-ops support --output csv from your build/bin folder.

llama.cpp/docs/ops.md

Line 7 in 9ee5bdc

1. Run `test-backend-ops support --output csv` with your backend name and redirect output to a csv file in `docs/ops/` (e.g., `docs/ops/CUDA.csv`)

@shani-f
Copy link
Contributor Author

shani-f commented Nov 15, 2025

Hello @CISC,
I really hope it’s okay now.
Thank you so much for your help!

@CISC
Copy link
Collaborator

CISC commented Nov 15, 2025

Hello @CISC, I really hope it’s okay now. Thank you so much for your help!

Yes, perfect, thank you! :)

@CISC CISC merged commit 72bd732 into ggml-org:master Nov 15, 2025
72 of 73 checks passed
basnijholt pushed a commit to basnijholt/llama.cpp that referenced this pull request Nov 16, 2025
@shani-f @basnijholt
sycl : unify unary kernels with a generic implementation and enable w…
0ec9247 
…ide operator support (ggml-org#17213)

* SYCL: add generic unary op implementation for multiple ops (ABS/SGN/…); unify non-contiguous access

* SYCL: update documentation and sycl.csv to reflect new unary op support

* update ops.md after syncing SYCL.csv changes

* Fix SYCL.csv merge conflict

* Update ops.md after fixing SYCL.csv conflicts

* Fix SYCL.csv tail after merge conflict and regenerate ops.md

* Fix line endings and final newline in SYCL.csv

* Remove TOPK_MOE entries from SYCL.csv as requested

* Update ops.md after removing TOPK_MOE from SYCL.csv

* Regenerated SYCL.csv and synced ops.md with upstream

* Update ops.md using create_ops_docs.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CISC CISC CISC approved these changes

@NeoZhangJianyu NeoZhangJianyu NeoZhangJianyu approved these changes

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shani-f @NeoZhangJianyu @CISC