Ghost: Token Obfuscation Against Gradient Inversion

The problem

Shared language-model gradients may expose training text. GHOST studies a defence that changes the training tokens rather than relying only on noise or pruning applied to gradients.

The method

It searches for substitute tokens that are semantically different but nearby in embedding space. A selection stage uses internal-output alignment to limit disruption to training-relevant features. This aims to weaken reconstruction of the original text while retaining useful learning signals; it does not keep every embedding, activation or gradient exactly unchanged.

Evidence and scope

The paper evaluates reconstruction and utility across classification and generation tasks, including adaptive attacks. The experiments measure both reconstruction risk and task utility. Their outcomes depend on the model, task and attack configuration; retaining useful training signals does not mean that no private information can be inferred.

GRAB describes the associated attack setting. The resource buttons above lead to the paper and original implementation.

@inproceedings{feng2026mitigating,
  author = {Feng, Xinguo and Ma, Zhongkui and Wang, Zihan and Abuadbba, Alsharif and Bai, Guangdong},
  title = {Mitigating Gradient Inversion Risks in Language Models via Token Obfuscation},
  booktitle = {Proceedings of the ACM Asia Conference on Computer and Communications Security},
  pages = {1832--1848},
  year = {2026},
  publisher = {ACM},
  doi = {10.1145/3779208.3785389}
}