shredx.modules.positional_encoding.PositionalEncoding

class shredx.modules.positional_encoding.PositionalEncoding(d_model: int, sequence_length: int, dropout: float, device: str = 'cpu')

Bases: Module

Sinusoidal positional encoding for transformer models.

Adds position-dependent sinusoidal signals to input embeddings to provide sequence position information. Uses sine for even dimensions and cosine for odd dimensions with exponentially decreasing frequencies.

Parameters:

d_modelint

Embedding dimension (must match input feature size).

sequence_lengthint

Maximum sequence length to encode.

dropoutfloat

Dropout probability applied after adding positional encoding.

devicestr, optional

Device on which to place the encoding buffer. Default is "cpu".

Methods

forward(x)

Add positional encoding to the input tensor and apply dropout.

Notes

Class Methods:

forward(x):

• Adds positional encoding to the input tensor and applies dropout.

• Parameters:

  • x : torch.Tensor. Input tensor of shape (batch_size, sequence_length, d_model).

• Returns:

  • torch.Tensor. Input with positional encoding added and dropout applied, same shape as input (batch_size, sequence_length, d_model).