xma.functional.swiglu

swiglu(gate: Tensor, up: Tensor, *, kernel_backend: KernelBackend | None = None) → Tensor[source]

computes swiglu activation as up * gate * sigmoid(gate)

Parameters:

  • gate (torch.Tensor) – gate activation tensor

  • up (torch.Tensor) – up activation tensor

  • kernel_backend (KernelBackend | None) – KernelBackend

Returns:

output tensor

Return type:

Tensor

swiglu_packed(x: Tensor, *, kernel_backend: KernelBackend | None = None) → Tensor[source]

computes swiglu activation by splitting the tensor x into 2 parts: gate and up activations. The tensor has interleaved values of gate, up, gate, up, …

Parameters:

  • x (torch.Tensor) – input activation

  • kernel_backend (KernelBackend | None) – KernelBackend

Returns:

output tensor

Return type:

Tensor