What Gradients and Embeddings Can Reveal

Sharing a representation instead of raw data changes what another party sees. It does not, by itself, establish what that party can infer. This distinction connects our collaborative work on gradient inversion in language-model training with our collaborative work on embedding inversion in vector retrieval. Both lines are led by Xinguo Feng, and the project pages carry the full author lists; what follows is a reading of the three papers together rather than a claim of sole authorship.

The useful comparison is not a leaderboard of recovery rates. It is a comparison of three questions: what is observed, what is reconstructed, and what the method changes.

Start with the observation

In a training setting, the observation may be a gradient computed from a private batch. Reconstruction depends on the model, the loss, the batch, stochastic operations such as dropout, and the attacker’s side information. A gradient can lose information and still reveal information that matters. Many-to-one does not mean private: several compatible inputs may share the same sensitive attribute, and a prior may favour one input over the others.

In a retrieval setting, the observation may be a stored embedding and associated index information. The target is now a source passage rather than a training batch. The encoder, query interface, auxiliary data and visible metadata must be specified separately.

Gradient leakage has been demonstrated in both vision and language settings; a single observed gradient is not automatically safe simply because it is one observation. See Deep Leakage from Gradients.

GRAB: testing the reconstruction risk

GRAB studies gradient inversion under practical language-model training conditions. Its hybrid optimisation addresses dropout-mask uncertainty alongside token recovery, and uses discrete optimisation for token sequencing. The contribution is an attack that handles obstacles which simpler experimental settings can omit, and it recovers a significant portion of the private batch — up to a 92.9% recovery rate — under its own attack setup. See the paper.

An attack result demonstrates a capability in its evaluated setting. It does not show that every gradient reveals every token, or that an arbitrary batch can be reconstructed. When reading 92.9%, keep the batch size, model, attack knowledge and recovery metric beside the number — which is why the figure is stated here with the qualifier it needs rather than as a headline.

GHOST: changing the tokens that produce the signal

GHOST explores token obfuscation rather than only modifying the released gradients. It searches for replacement tokens that differ semantically while remaining nearby in embedding space, then selects candidates using internal-output alignment. The goal is to preserve useful training behaviour while making reconstruction of the original text harder, and the evaluation reports recovery as low as 1% across architectures from BERT to Llama, in both classification and generation, with classification F1 up to 0.92 and perplexity at 5.45. See the paper.

The distinction is between a design objective and an invariant. Nearby embeddings do not imply identical gradients, identical training trajectories, or zero information leakage. Those properties would require their own assumptions and analysis. The method’s evaluation studies utility and reconstruction against specified attacks, including adaptive ones — a 1% recovery rate against an adaptive attacker is a different statement from 1% against a fixed attack, and only the first is the claim being made.

This also does not justify saying that noise is not a defence. Properly calibrated noise can be part of a mechanism with a formal privacy guarantee: DP-SGD combines clipping, noise and privacy accounting. An empirical comparison against particular perturbation baselines is not an impossibility result for all noise-based mechanisms. Differential privacy and empirical reconstruction resistance also answer different questions.

SHAQ: changing what the index represents

SHAQ moves the question to vector retrieval. It generates several shadow queries for a document, stores their embeddings instead of directly storing that document’s embedding, and maps retrieved vectors back to documents. The motivation is specific: existing defences against embedding inversion — adding noise, or scaling the embeddings — tend to trade one problem for the other, giving limited privacy or costing retrieval utility. SHAQ’s evaluation reports a recovery rate as low as 0.2104, defending up to 19.50% more tokens than baseline defences while reaching up to 0.7967 MAP@10 with a 5.53% utility improvement. The preprint evaluates reconstruction risk and retrieval utility for this representation change.

The changed stored object matters, but it is not a zero-leakage argument. A query about a document can itself contain sensitive information, and the question of whether several queries reveal more together than separately is open rather than settled here. The generation service, document mapping, query logs and returned documents are additional surfaces whose treatment depends on deployment. These are questions to investigate, not claims that the paper’s evaluated setting necessarily exposes all of them.

Compare the settings, not just the percentages

“Recovered” needs a definition. Exact token agreement, approximate semantic recovery and retrieval of the correct document are different measurements. Likewise, a low observed recovery rate under one attack does not bound every other attack.

Privacy is not outside formal reasoning. One can state and analyse privacy properties; whether a particular project supplies such a guarantee is a separate question. Here the shared research concern is useful computation under limited disclosure, not a claim that all three works share one proof or threat model.

Where these three sit

The privacy research theme connects these projects. Their individual pages preserve the full author lists, publication state and original resources: GRAB, GHOST, SHAQ.

For a different question about controlling how AI resources are used, see where control enters the system. That is a related research concern, not a prerequisite or a claim that the mechanisms can be composed without further analysis.