PLMs_GNNs_in_NLP/Causformer-pytorch

• Causformer: Improving the Robustness on Spurious Correlations via Causal Attention Enforced Transformer
• • Requirements
  • Overview
  • Download datasets
  • Main contribution

Causformer: Improving the Robustness on Spurious Correlations via Causal Attention Enforced Transformer

This is the implementation of Causformers in Pytorch.

Requirements

pip install -r requirements.txt,
PyTorch, torchtext

Overview

This contains 9 python files.

• requirements.txt : The necessary tools to run this code
• AttentionVisual.py : Functions to visualize the attention as heat-maps or bipartite graphs
• datasets/ : Tool set to load the original datasets with torchtext
• optim.py : A custom optimizer (BertAdam class) adopted from Hugging Face's code
• train.py : A helper class for training and evaluation
• utils.py : Several utility functions
• catear/ : The implementation of CATEAR-enforced Transformers
• .data/ : The file of original datasets
• .vector_cache/ : The file stores GloVe-300d word embeddings and preprocessed datasets

Download datasets

Download original datasets .data.zip and unzip in .data/.

Main contribution

The CATEAR optimization is defined in the ./catear/Modelus.py line 38 - 52
The two-layer GCN updater is defined in the ./catear/gcn_layer and ./catear/Modelud.py line 35