rllm.nn.loss.SelfSupervisedVPCL

class rllm.nn.loss.SelfSupervisedVPCL(temperature: float = 10.0, base_temperature: float = 10.0, similarity: Literal['dot', 'cosine'] = 'dot', eps: float = 1e-12)

Bases: ContrastiveLoss

The self-supervised vertical-partition contrastive loss (Self-VPCL) implementation, based on the “TransTab: Learning Transferable Tabular Transformers Across Tables” paper.

In this setting, each table row is split into multiple column partitions that act as different views of the same sample. Positive pairs are formed across partitions from the same row, while negatives are formed across rows.

\[\ell(X) = - \sum_{i=1}^{B} \sum_{k=1}^{K} \sum_{k' \neq k}^{K} \log \frac{ \exp\big(\psi(\mathbf{v}_i^{k}, \mathbf{v}_i^{k'})\big) }{ \sum_{j=1}^{B}\sum_{k^{\dagger}=1}^{K} \exp\big(\psi(\mathbf{v}_i^{k}, \mathbf{v}_j^{k^{\dagger}})\big) } .\]

where \(B\) is batch size, \(K\) is partition count per sample, \(\mathbf{v}_i^k\) is the partition embedding, and \(\psi\) is the configured similarity function.

Parameters:

• temperature (float) – Temperature \(\tau\) scaling logits.

• base_temperature (float) – Reference temperature \(\tau_0\) used in the final scaling factor \(\tau / \tau_0\).

• similarity (str) – Similarity metric, either "dot" or "cosine".

• eps (float) – Numerical stability constant.

Shapes:

• input: partition embeddings \((B, K, D)\)

• output: scalar loss \(()\)

Example

>>> import torch
>>> loss_fn = SelfSupervisedVPCL()
>>> feats = torch.randn(4, 2, 8)
>>> loss_fn(feats).ndim
0

forward(features: Tensor) → Tensor

Compute self-supervised vertical-partition contrastive loss.

Parameters:

features (torch.Tensor) – Partition embeddings with shape \((B, K, D)\).

Returns:

Scalar loss tensor with shape \(()\).

Return type:

torch.Tensor

Example

>>> import torch
>>> loss_fn = SelfSupervisedVPCL()
>>> feats = torch.randn(4, 2, 8)
>>> out = loss_fn(feats)
>>> out.shape
torch.Size([])