What Is Text Watermarking?
Text watermarking is a technology, which allows copyright protection with the help of special metadata inserted into a copy of a text. This metadata contains details on authorship, owner status, publication date, and other info that can be extracted to avoid unauthorised distribution, counterfeiting, intellectual theft, and so on.
While the technology initially emerged in 13th century Italy, it wasn’t until 1992 when the concept of digital watermark was introduced by Andrew Tirkel and Charles Osborne. The advent of Large Language Models (LLMs) made it necessary to develop failproof watermarking techniques to prevent, among all else, academic fraud and dishonesty.
What Is the Difference between Text Watermarking and Natural Language Watermarking?
Digital watermarking refers to embedding a hidden sequence of bits that can be extracted with a key later to identify origins of the media. However, with the natural language it requires additional manipulations on the levels of:
- Surface. Manipulation of how the text looks.
- Syntax. Reordering words and phrases in the text.
- Semantics. Replacing words with synonyms, etc.
With that in mind, a protected text shouldn’t lose its original structure, meaning, and language.
Text Watermarking Approaches
To address the issue, a number of methods are suggested.
- Structural-Based Approach
The method manipulates either structure or features of the text to insert a watermark. It can include repeating letters in a word, sentence or word shifting upwards/downwards, word distance analysis, analyzing the length of preceding and following words that surround a specific keyword, and so on.
- Linguistic-Based Approach
This family of techniques focuses on syntactic and semantic components of the text that are manipulated to hide a watermark. They include grammatical changes that don’t impact the initial meaning of a writing, reshuffling of the word order, changing active clause into passive, usage of typos or acronyms, noun-verb transformation, and others.
- Image-Based Approach
If a watermark is represented as a certain symbol — a logo for example — it can be secretly inserted inside the cover text. The image of the watermark needs to be converted into text string. Then, it is aligned with the visual parameters of the text in question, so the watermark can be generated.
- LLM-Based Approach
This is a subset of methods that rely on a Large Language Model (LLM). Watermark creation includes threes stages in this case:
- Training an LLM on a specific dataset that suits the task.
- Logits generation to predict vocabulary’s probability distribution.
- Token sampling for generating a watermark token.
They include token-level, sentence-level, trigger-based, and other types of watermarking,
Text Watermarking and Natural Language Watermarking Application
- Copyright Protection
Watermarks help avoid copyright infringement since it’s easy to track down the source text. For that purpose, mostly format-based techniques are used: text color, word spacing, font manipulations, and others as they don’t alter the written content. Backdoor watermarking and word substitution are used for protecting LLMs and training datasets respectively.
- AI Generated Text Detection
It is suggested that detecting AI-produced texts may become virtually impossible in the foreseeable future. In turn, this can negatively affect academic integrity, as students and authors may automatically generate content and pass it as an original work. Watermarks covertly embedded by LLM models, can signal about the true origin of a synthetic text and prevent academic or other types of fraud.
Text Watermarking in Different Languages
The only flexible watermarking technique for multilingual content is format-based — it doesn't focus on linguistic/structural properties of a text, as they can radically differ from one language to another. For instance, an English language-based watermarking method can be adjusted to Portuguese or Spanish fairly easily as their alphabets are based on Latin.
Antispoofing
