xma.functional.swiglu

swiglu(gate: Tensor, up: Tensor, *, kernel_backend: KernelBackend | None = None) → Tensor[source]

computes swiglu activation as up * gate * sigmoid(gate)

Parameters:

  • gate (torch.Tensor) – gate activation tensor

  • up (torch.Tensor) – up activation tensor

  • kernel_backend (KernelBackend | None) – KernelBackend

Returns:

output tensor

Return type:

Tensor